Skip to contentSkip to conversation

Ancillary bus implementation and SOF multi-client support

PATCH In review Dave Ertman 4 participants 24 messages Oct 1, 2020 – Oct 5, 2020

Subsystems (8)
lore
Share & tools
lore
#0 lore raw

cover Ancillary bus implementation and SOF multi-client support

Brief history of Ancillary Bus
==============================
The ancillary bus code was originally submitted upstream as virtual bus, and was submitted through the netdev tree.  This process generated up to v4.  This discussion can be found here:
https://lore.kernel.org/netdev/[email protected]/T/#u

At this point, GregKH requested that we take the review and revision process to an internal mailing list and garner the buy-in of a respected kernel contributor.

The ancillary bus (then known as virtual bus) was originally submitted along with implementation code for the ice driver and irdma drive, causing the complication of also having dependencies in the rdma tree. This new submission is utilizing an ancillary bus consumer in only the sound driver tree to create the initial implementation and a single user.

Since implementation work has started on this patch set, there have been multiple inquiries about the time frame of its completion. It appears that there will be numerous consumers of this functionality.

The process of internal review and implementation using the sound drivers generated 19 internal versions.  The changes, including the name change from virtual bus to ancillary bus, from these versions can be summarized as the following:

- Fixed compilation and checkpatch errors
- Improved documentation to address the motivation for virtual bus.
- Renamed virtual bus to ancillary bus
- increased maximum device name size
- Correct order in Kconfig and Makefile
- removed the mid-layer adev->release layer for device unregister
- pushed adev->id management to parent driver
- all error paths out of ancillary_device_register return error code
- all error paths out of ancillary_device_register use put_device
- added adev->name element
- modname in register cannot be NULL
- added KBUILD_MODNAME as prefix for match_name
- push adev->id responsibility to registering driver
- uevent now parses adev->dev name
- match_id function now parses adev->dev name
- changed drivers probe function to also take an ancillary_device_id param
- split ancillary_device_register into device_initialize and device_add
- adjusted what is done in device_initialize and device_add
- change adev to ancildev and adrv to ancildrv
- change adev to ancildev in documentation

This submission is the first time that this patch set will be sent to the alsa-devel mailing list, so it is currently being submitted as version 1.

==========================

Introduces the ancillary bus implementation along with the example usage in the Sound Open Firmware(SOF) audio driver.

In some subsystems, the functionality of the core device (PCI/ACPI/other) may be too complex for a single device to be managed as a monolithic block or a part of the functionality might need to be exposed to a different subsystem. Splitting the functionality into smaller orthogonal devices makes it easier to manage data, power management and domain-specific communication with the hardware. Also, common ancillary_device functionality across primary devices can be handled by a common ancillary_device. A key requirement for such a split is that there is no dependency on a physical bus, device, register accesses or regmap support. These individual devices split from the core cannot live on the platform bus as they are not physical devices that are controlled by DT/ACPI. The same argument applies for not using MFD in this scenario as it relies on individual function devices being physical devices that are DT enumerated.

An example for this kind of requirement is the audio subsystem where a single IP handles multiple entities such as HDMI, Soundwire, local devices such as mics/speakers etc. The split for the core's functionality can be arbitrary or be defined by the DSP firmware topology and include hooks for test/debug. This allows for the audio core device to be minimal and tightly coupled with handling the hardware-specific logic and communication.

The ancillary bus is intended to be minimal, generic and avoid domain-specific assumptions. Each ancillary bus device represents a part of its parent functionality. The generic behavior can be extended and specialized as needed by encapsulating an ancillary bus device within other domain-specific structures and the use of .ops callbacks.

The SOF driver adopts the ancillary bus for implementing the multi-client support. A client in the context of the SOF driver represents a part of the core device's functionality. It is not a physical device but rather an ancillary device that needs to communicate with the DSP via IPCs. With multi-client support,the sound card can be separated into multiple orthogonal ancillary devices for local devices (mic/speakers etc), HDMI, sensing, probes, debug etc. In this series, we demonstrate the usage of the ancillary bus with the help of the IPC test client which is used for testing the serialization of IPCs when multiple clients talk to the DSP at the same time.

Dave Ertman (1):
Add ancillary bus support

Fred Oh (1):
ASoC: SOF: debug: Remove IPC flood test support in SOF core

Ranjani Sridharan (4):
ASoC: SOF: Introduce descriptors for SOF client
ASoC: SOF: Create client driver for IPC test
ASoC: SOF: ops: Add ops for client registration
ASoC: SOF: Intel: Define ops for client registration

 Documentation/driver-api/ancillary_bus.rst | 230 +++++++++++++++
 Documentation/driver-api/index.rst         |   1 +
 drivers/bus/Kconfig                        |   3 +
 drivers/bus/Makefile                       |   3 +
 drivers/bus/ancillary.c                    | 191 +++++++++++++
 include/linux/ancillary_bus.h              |  58 ++++
 include/linux/mod_devicetable.h            |   8 +
 scripts/mod/devicetable-offsets.c          |   3 +
 scripts/mod/file2alias.c                   |   8 +
 sound/soc/sof/Kconfig                      |  29 +-
 sound/soc/sof/Makefile                     |   7 +
 sound/soc/sof/core.c                       |  12 +
 sound/soc/sof/debug.c                      | 230 ---------------
 sound/soc/sof/intel/Kconfig                |   9 +
 sound/soc/sof/intel/Makefile               |   3 +
 sound/soc/sof/intel/apl.c                  |  18 ++
 sound/soc/sof/intel/bdw.c                  |  18 ++
 sound/soc/sof/intel/byt.c                  |  22 ++
 sound/soc/sof/intel/cnl.c                  |  18 ++
 sound/soc/sof/intel/intel-client.c         |  49 ++++
 sound/soc/sof/intel/intel-client.h         |  26 ++
 sound/soc/sof/ops.h                        |  14 +
 sound/soc/sof/sof-client.c                 | 117 ++++++++
 sound/soc/sof/sof-client.h                 |  65 +++++
 sound/soc/sof/sof-ipc-test-client.c        | 314 +++++++++++++++++++++
 sound/soc/sof/sof-priv.h                   |  16 +-
 26 files changed, 1233 insertions(+), 239 deletions(-)
 create mode 100644 Documentation/driver-api/ancillary_bus.rst
 create mode 100644 drivers/bus/ancillary.c
 create mode 100644 include/linux/ancillary_bus.h
 create mode 100644 sound/soc/sof/intel/intel-client.c
 create mode 100644 sound/soc/sof/intel/intel-client.h
 create mode 100644 sound/soc/sof/sof-client.c
 create mode 100644 sound/soc/sof/sof-client.h
 create mode 100644 sound/soc/sof/sof-ipc-test-client.c
Signature · 1 line
2.26.2
#1 lore raw

PATCH 1/6 Add ancillary bus support

Add support for the Ancillary Bus, ancillary_device and ancillary_driver. It enables drivers to create an ancillary_device and bind an ancillary_driver to it.

The bus supports probe/remove shutdown and suspend/resume callbacks. Each ancillary_device has a unique string based id; driver binds to an ancillary_device based on this id through the bus.

---
 Documentation/driver-api/ancillary_bus.rst | 230 +++++++++++++++++++++
 Documentation/driver-api/index.rst         |   1 +
 drivers/bus/Kconfig                        |   3 +
 drivers/bus/Makefile                       |   3 +
 drivers/bus/ancillary.c                    | 191 +++++++++++++++++
 include/linux/ancillary_bus.h              |  58 ++++++
 include/linux/mod_devicetable.h            |   8 +
 scripts/mod/devicetable-offsets.c          |   3 +
 scripts/mod/file2alias.c                   |   8 +
 9 files changed, 505 insertions(+)
 create mode 100644 Documentation/driver-api/ancillary_bus.rst
 create mode 100644 drivers/bus/ancillary.c
 create mode 100644 include/linux/ancillary_bus.h
Documentation/driver-api/ancillary_bus.rst new file +230 -0
new file mode 100644
index 000000000000..0a11979aa927
@@ -0,0 +1,230 @@
+.. SPDX-License-Identifier: GPL-2.0-only
+
+=============
+Ancillary Bus
+=============
+
+In some subsystems, the functionality of the core device (PCI/ACPI/other) is
+too complex for a single device to be managed as a monolithic block or a part of
+the functionality needs to be exposed to a different subsystem.  Splitting the
+functionality into smaller orthogonal devices would make it easier to manage
+data, power management and domain-specific interaction with the hardware. A key
+requirement for such a split is that there is no dependency on a physical bus,
+device, register accesses or regmap support. These individual devices split from
+the core cannot live on the platform bus as they are not physical devices that
+are controlled by DT/ACPI. The same argument applies for not using MFD in this
+scenario as MFD relies on individual function devices being physical devices
+that are DT enumerated.
+
+An example for this kind of requirement is the audio subsystem where a single
+IP is handling multiple entities such as HDMI, Soundwire, local devices such as
+mics/speakers etc. The split for the core's functionality can be arbitrary or
+be defined by the DSP firmware topology and include hooks for test/debug. This
+allows for the audio core device to be minimal and focused on hardware-specific
+control and communication.
+
+The ancillary bus is intended to be minimal, generic and avoid domain-specific
+assumptions. Each ancillary_device represents a part of its parent
+functionality. The generic behavior can be extended and specialized as needed
+by encapsulating an ancillary_device within other domain-specific structures and
+the use of .ops callbacks. Devices on the ancillary bus do not share any
+structures and the use of a communication channel with the parent is
+domain-specific.
+
+When Should the Ancillary Bus Be Used
+=====================================
+
+The ancillary bus is to be used when a driver and one or more kernel modules,
+who share a common header file with the driver, need a mechanism to connect and
+provide access to a shared object allocated by the ancillary_device's
+registering driver.  The registering driver for the ancillary_device(s) and the
+kernel module(s) registering ancillary_drivers can be from the same subsystem,
+or from multiple subsystems.
+
+The emphasis here is on a common generic interface that keeps subsystem
+customization out of the bus infrastructure.
+
+One example could be a multi-port PCI network device that is rdma-capable and
+needs to export this functionality and attach to an rdma driver in another
+subsystem.  The PCI driver will allocate and register an ancillary_device for
+each physical function on the NIC.  The rdma driver will register an
+ancillary_driver that will be matched with and probed for each of these
+ancillary_devices.  This will give the rdma driver access to the shared data/ops
+in the PCI drivers shared object to establish a connection with the PCI driver.
+
+Another use case is for the a PCI device to be split out into multiple sub
+functions.  For each sub function an ancillary_device will be created.  A PCI
+sub function driver will bind to such devices that will create its own one or
+more class devices.  A PCI sub function ancillary device will likely be
+contained in a struct with additional attributes such as user defined sub
+function number and optional attributes such as resources and a link to the
+parent device.  These attributes could be used by systemd/udev; and hence should
+be initialized before a driver binds to an ancillary_device.
+
+Ancillary Device
+================
+
+An ancillary_device is created and registered to represent a part of its parent
+device's functionality. It is given a name that, combined with the registering
+drivers KBUILD_MODNAME, creates a match_name that is used for driver binding,
+and an id that combined with the match_name provide a unique name to register
+with the bus subsystem.
+
+Registering an ancillary_device is a two-step process.  First you must call
+ancillary_device_initialize(), which will check several aspects of the
+ancillary_device struct and perform a device_initialize().  After this step
+completes, any error state must have a call to put_device() in its resolution
+path.  The second step in registering an ancillary_device is to perform a call
+to ancillary_device_add(), which will set the name of the device and add the
+device to the bus.
+
+To unregister an ancillary_device, just a call to ancillary_device_unregister()
+is used.  This will perform both a device_del() and a put_device().
+
+.. code-block:: c
+
+	struct ancillary_device {
+		struct device dev;
+                const char *name;
+		u32 id;
+	};
+
+If two ancillary_devices both with a match_name "mod.foo" are registered onto
+the bus, they must have unique id values (e.g. "x" and "y") so that the
+registered devices names will be "mod.foo.x" and "mod.foo.y".  If match_name +
+id are not unique, then the device_add will fail and generate an error message.
+
+The ancillary_device.dev.type.release or ancillary_device.dev.release must be
+populated with a non-NULL pointer to successfully register the ancillary_device.
+
+The ancillary_device.dev.parent must also be populated.
+
+Ancillary Device Memory Model and Lifespan
+------------------------------------------
+
+When a kernel driver registers an ancillary_device on the ancillary bus, we will
+use the nomenclature to refer to this kernel driver as a registering driver.  It
+is the entity that will allocate memory for the ancillary_device and register it
+on the ancillary bus.  It is important to note that, as opposed to the platform
+bus, the registering driver is wholly responsible for the management for the
+memory used for the driver object.
+
+A parent object, defined in the shared header file, will contain the
+ancillary_device.  It will also contain a pointer to the shared object(s), which
+will also be defined in the shared header.  Both the parent object and the
+shared object(s) will be allocated by the registering driver.  This layout
+allows the ancillary_driver's registering module to perform a container_of()
+call to go from the pointer to the ancillary_device, that is passed during the
+call to the ancillary_driver's probe function, up to the parent object, and then
+have access to the shared object(s).
+
+The memory for the ancillary_device will be freed only in its release()
+callback flow as defined by its registering driver.
+
+The memory for the shared object(s) must have a lifespan equal to, or greater
+than, the lifespan of the memory for the ancillary_device.  The ancillary_driver
+should only consider that this shared object is valid as long as the
+ancillary_device is still registered on the ancillary bus.  It is up to the
+registering driver to manage (e.g. free or keep available) the memory for the
+shared object beyond the life of the ancillary_device.
+
+Registering driver must unregister all ancillary devices before its registering
+parent device's remove() is completed.
+
+Ancillary Drivers
+=================
+
+Ancillary drivers follow the standard driver model convention, where
+discovery/enumeration is handled by the core, and drivers
+provide probe() and remove() methods. They support power management
+and shutdown notifications using the standard conventions.
+
+.. code-block:: c
+
+	struct ancillary_driver {
+		int (*probe)(struct ancillary_device *,
+                             const struct ancillary_device_id *id);
+		int (*remove)(struct ancillary_device *);
+		void (*shutdown)(struct ancillary_device *);
+		int (*suspend)(struct ancillary_device *, pm_message_t);
+		int (*resume)(struct ancillary_device *);
+		struct device_driver driver;
+		const struct ancillary_device_id *id_table;
+	};
+
+Ancillary drivers register themselves with the bus by calling
+ancillary_driver_register(). The id_table contains the match_names of ancillary
+devices that a driver can bind with.
+
+Example Usage
+=============
+
+Ancillary devices are created and registered by a subsystem-level core device
+that needs to break up its functionality into smaller fragments. One way to
+extend the scope of an ancillary_device would be to encapsulate it within a
+domain-specific structure defined by the parent device. This structure contains
+the ancillary_device and any associated shared data/callbacks needed to
+establish the connection with the parent.
+
+An example would be:
+
+.. code-block:: c
+
+        struct foo {
+		struct ancillary_device ancildev;
+		void (*connect)(struct ancillary_device *ancildev);
+		void (*disconnect)(struct ancillary_device *ancildev);
+		void *data;
+        };
+
+The parent device would then register the ancillary_device by calling
+ancillary_device_initialize(), and then ancillary_device_add(), with the pointer
+to the ancildev member of the above structure. The parent would provide a name
+for the ancillary_device that, combined with the parent's KBUILD_MODNAME, will
+create a match_name that will be used for matching and binding with a driver.
+
+Whenever an ancillary_driver is registered, based on the match_name, the
+ancillary_driver's probe() is invoked for the matching devices.  The
+ancillary_driver can also be encapsulated inside custom drivers that make the
+core device's functionality extensible by adding additional domain-specific ops
+as follows:
+
+.. code-block:: c
+
+	struct my_ops {
+		void (*send)(struct ancillary_device *ancildev);
+		void (*receive)(struct ancillary_device *ancildev);
+	};
+
+
+	struct my_driver {
+		struct ancillary_driver ancillary_drv;
+		const struct my_ops ops;
+	};
+
+An example of this type of usage would be:
+
+.. code-block:: c
+
+	const struct ancillary_device_id my_ancillary_id_table[] = {
+		{ .name = "foo_mod.foo_dev" },
+		{ },
+	};
+
+	const struct my_ops my_custom_ops = {
+		.send = my_tx,
+		.receive = my_rx,
+	};
+
+	const struct my_driver my_drv = {
+		.ancillary_drv = {
+			.driver = {
+				.name = "myancillarydrv",
+			},
+			.id_table = my_ancillary_id_table,
+			.probe = my_probe,
+			.remove = my_remove,
+			.shutdown = my_shutdown,
+		},
+		.ops = my_custom_ops,
+	};
Documentation/driver-api/index.rst +1 -0
index 5ef2cfe3a16b..9584ac2ed1f5 100644
@@ -74,6 +74,7 @@ available subsections can be seen below.
    thermal/index
    fpga/index
    acpi/index
+   ancillary_bus
    backlight/lp855x-driver.rst
    connector
    console
drivers/bus/Kconfig +3 -0
index 0c262c2aeaf2..ba82a045b847 100644
@@ -5,6 +5,9 @@
 
 menu "Bus devices"
 
+config ANCILLARY_BUS
+       tristate
+
 config ARM_CCI
 	bool
 
drivers/bus/Makefile +3 -0
index 397e35392bff..1fd238094543 100644
@@ -3,6 +3,9 @@
 # Makefile for the bus drivers.
 #
 
+#Ancillary bus driver
+obj-$(CONFIG_ANCILLARY_BUS)	+= ancillary.o
+
 # Interconnect bus drivers for ARM platforms
 obj-$(CONFIG_ARM_CCI)		+= arm-cci.o
 obj-$(CONFIG_ARM_INTEGRATOR_LM)	+= arm-integrator-lm.o
drivers/bus/ancillary.c new file +191 -0

191 lines not on this page · raw message

include/linux/ancillary_bus.h new file +58 -0

58 lines not on this page · raw message

include/linux/mod_devicetable.h +8 -0
index 5b08a473cdba..7d596dc30833 100644
@@ -838,4 +838,12 @@ struct mhi_device_id {
 	kernel_ulong_t driver_data;
 };
 
+#define ANCILLARY_NAME_SIZE 32
+#define ANCILLARY_MODULE_PREFIX "ancillary:"
+
+struct ancillary_device_id {
+	char name[ANCILLARY_NAME_SIZE];
+	kernel_ulong_t driver_data;
+};
+
 #endif /* LINUX_MOD_DEVICETABLE_H */
scripts/mod/devicetable-offsets.c +3 -0
index 27007c18e754..79e37c4c25b3 100644
@@ -243,5 +243,8 @@ int main(void)
 	DEVID(mhi_device_id);
 	DEVID_FIELD(mhi_device_id, chan);
 
+	DEVID(ancillary_device_id);
+	DEVID_FIELD(ancillary_device_id, name);
+
 	return 0;
 }
scripts/mod/file2alias.c +8 -0
index 2417dd1dee33..99c4fcd82bf3 100644
@@ -1364,6 +1364,13 @@ static int do_mhi_entry(const char *filename, void *symval, char *alias)
 {
 	DEF_FIELD_ADDR(symval, mhi_device_id, chan);
 	sprintf(alias, MHI_DEVICE_MODALIAS_FMT, *chan);
+	return 1;
+}
+
+static int do_ancillary_entry(const char *filename, void *symval, char *alias)
+{
+	DEF_FIELD_ADDR(symval, ancillary_device_id, name);
+	sprintf(alias, ANCILLARY_MODULE_PREFIX "%s", *name);
 
 	return 1;
 }
@@ -1442,6 +1449,7 @@ static const struct devtable devtable[] = {
 	{"tee", SIZE_tee_client_device_id, do_tee_entry},
 	{"wmi", SIZE_wmi_device_id, do_wmi_entry},
 	{"mhi", SIZE_mhi_device_id, do_mhi_entry},
+	{"ancillary", SIZE_ancillary_device_id, do_ancillary_entry},
 };
 
 /* Create MODULE_ALIAS() statements.
Signature · 1 line
2.26.2
#2 lore raw

Re: [PATCH 1/6] Add ancillary bus support

Quoted 112 lines · Dave Ertman

Add support for the Ancillary Bus, ancillary_device and ancillary_driver. It enables drivers to create an ancillary_device and bind an ancillary_driver to it.

The bus supports probe/remove shutdown and suspend/resume callbacks. Each ancillary_device has a unique string based id; driver binds to an ancillary_device based on this id through the bus.

---
 Documentation/driver-api/ancillary_bus.rst | 230 +++++++++++++++++++++
 Documentation/driver-api/index.rst         |   1 +
 drivers/bus/Kconfig                        |   3 +
 drivers/bus/Makefile                       |   3 +
 drivers/bus/ancillary.c                    | 191 +++++++++++++++++
 include/linux/ancillary_bus.h              |  58 ++++++
 include/linux/mod_devicetable.h            |   8 +
 scripts/mod/devicetable-offsets.c          |   3 +
 scripts/mod/file2alias.c                   |   8 +
 9 files changed, 505 insertions(+)
 create mode 100644 Documentation/driver-api/ancillary_bus.rst
 create mode 100644 drivers/bus/ancillary.c
 create mode 100644 include/linux/ancillary_bus.h
Documentation/driver-api/ancillary_bus.rst new file +72 -0
new file mode 100644
index 000000000000..0a11979aa927
@@ -0,0 +1,230 @@
+.. SPDX-License-Identifier: GPL-2.0-only
+
+=============
+Ancillary Bus
+=============
+
+In some subsystems, the functionality of the core device (PCI/ACPI/other) is
+too complex for a single device to be managed as a monolithic block or a part of
+the functionality needs to be exposed to a different subsystem.  Splitting the
+functionality into smaller orthogonal devices would make it easier to manage
+data, power management and domain-specific interaction with the hardware. A key
+requirement for such a split is that there is no dependency on a physical bus,
+device, register accesses or regmap support. These individual devices split from
+the core cannot live on the platform bus as they are not physical devices that
+are controlled by DT/ACPI. The same argument applies for not using MFD in this
+scenario as MFD relies on individual function devices being physical devices
+that are DT enumerated.
+
+An example for this kind of requirement is the audio subsystem where a single
+IP is handling multiple entities such as HDMI, Soundwire, local devices such as
+mics/speakers etc. The split for the core's functionality can be arbitrary or
+be defined by the DSP firmware topology and include hooks for test/debug. This
+allows for the audio core device to be minimal and focused on hardware-specific
+control and communication.
+
+The ancillary bus is intended to be minimal, generic and avoid domain-specific
+assumptions. Each ancillary_device represents a part of its parent
+functionality. The generic behavior can be extended and specialized as needed
+by encapsulating an ancillary_device within other domain-specific structures and
+the use of .ops callbacks. Devices on the ancillary bus do not share any
+structures and the use of a communication channel with the parent is
+domain-specific.
+
+When Should the Ancillary Bus Be Used
+=====================================
+
+The ancillary bus is to be used when a driver and one or more kernel modules,
+who share a common header file with the driver, need a mechanism to connect and
+provide access to a shared object allocated by the ancillary_device's
+registering driver.  The registering driver for the ancillary_device(s) and the
+kernel module(s) registering ancillary_drivers can be from the same subsystem,
+or from multiple subsystems.
+
+The emphasis here is on a common generic interface that keeps subsystem
+customization out of the bus infrastructure.
+
+One example could be a multi-port PCI network device that is rdma-capable and
+needs to export this functionality and attach to an rdma driver in another
+subsystem.  The PCI driver will allocate and register an ancillary_device for
+each physical function on the NIC.  The rdma driver will register an
+ancillary_driver that will be matched with and probed for each of these
+ancillary_devices.  This will give the rdma driver access to the shared data/ops
+in the PCI drivers shared object to establish a connection with the PCI driver.
+
+Another use case is for the a PCI device to be split out into multiple sub
+functions.  For each sub function an ancillary_device will be created.  A PCI
+sub function driver will bind to such devices that will create its own one or
+more class devices.  A PCI sub function ancillary device will likely be
+contained in a struct with additional attributes such as user defined sub
+function number and optional attributes such as resources and a link to the
+parent device.  These attributes could be used by systemd/udev; and hence should
+be initialized before a driver binds to an ancillary_device.
+
+Ancillary Device
+================
+
+An ancillary_device is created and registered to represent a part of its parent
+device's functionality. It is given a name that, combined with the registering
+drivers KBUILD_MODNAME, creates a match_name that is used for driver binding,
+and an id that combined with the match_name provide a unique name to register
+with the bus subsystem.
+

Documentation/driver-api/ancillary_bus.rst line 82

Quoted 10 lines
Documentation/driver-api/ancillary_bus.rst new file +10 -0
+Registering an ancillary_device is a two-step process.  First you must call
+ancillary_device_initialize(), which will check several aspects of the
+ancillary_device struct and perform a device_initialize().  After this step
+completes, any error state must have a call to put_device() in its resolution
+path.  The second step in registering an ancillary_device is to perform a call
+to ancillary_device_add(), which will set the name of the device and add the
+device to the bus.
+
+To unregister an ancillary_device, just a call to ancillary_device_unregister()
+is used.  This will perform both a device_del() and a put_device().

Why did you chose ancillary_device_initialize() and not ancillary_device_register() to be paired with ancillary_device_unregister()?

Thanks

#3 lore raw

RE: [PATCH 1/6] Add ancillary bus support

Quoted 166 lines

-----Original Message-----
From: Leon Romanovsky <[email protected]>
Sent: Thursday, October 1, 2020 1:32 AM
To: Ertman, David M <[email protected]>
Cc: [email protected]
Subject: Re: [PATCH 1/6] Add ancillary bus support

Quoted 19 lines · Dave Ertman

Add support for the Ancillary Bus, ancillary_device and ancillary_driver. It enables drivers to create an ancillary_device and bind an ancillary_driver to it.

The bus supports probe/remove shutdown and suspend/resume callbacks. Each ancillary_device has a unique string based id; driver binds to an ancillary_device based on this id through the bus.

Quoted 2 lines
---
 Documentation/driver-api/ancillary_bus.rst | 230

+++++++++++++++++++++

Quoted 13 lines
 Documentation/driver-api/index.rst         |   1 +
 drivers/bus/Kconfig                        |   3 +
 drivers/bus/Makefile                       |   3 +
 drivers/bus/ancillary.c                    | 191 +++++++++++++++++
 include/linux/ancillary_bus.h              |  58 ++++++
 include/linux/mod_devicetable.h            |   8 +
 scripts/mod/devicetable-offsets.c          |   3 +
 scripts/mod/file2alias.c                   |   8 +
 9 files changed, 505 insertions(+)
 create mode 100644 Documentation/driver-api/ancillary_bus.rst
 create mode 100644 drivers/bus/ancillary.c
 create mode 100644 include/linux/ancillary_bus.h

Review

Quoted 1 line
(quoted diff) +0 -0

b/Documentation/driver-api/ancillary_bus.rst

Documentation/driver-api/ancillary_bus.rst line 7

Quoted 12 lines

new file mode 100644
index 000000000000..0a11979aa927

Documentation/driver-api/ancillary_bus.rst new file +7 -0
@@ -0,0 +1,230 @@
+.. SPDX-License-Identifier: GPL-2.0-only
+
+=============
+Ancillary Bus
+=============
+
+In some subsystems, the functionality of the core device

(PCI/ACPI/other) is

Documentation/driver-api/ancillary_bus.rst line 8

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+too complex for a single device to be managed as a monolithic block or a

part of

Documentation/driver-api/ancillary_bus.rst line 9

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+the functionality needs to be exposed to a different subsystem.  Splitting

the

Documentation/driver-api/ancillary_bus.rst line 10

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+functionality into smaller orthogonal devices would make it easier to

manage

Documentation/driver-api/ancillary_bus.rst line 11

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+data, power management and domain-specific interaction with the

hardware. A key

Documentation/driver-api/ancillary_bus.rst line 12

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+requirement for such a split is that there is no dependency on a physical

bus,

Documentation/driver-api/ancillary_bus.rst line 13

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+device, register accesses or regmap support. These individual devices split

from

Documentation/driver-api/ancillary_bus.rst line 14

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+the core cannot live on the platform bus as they are not physical devices

that

Documentation/driver-api/ancillary_bus.rst line 15

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+are controlled by DT/ACPI. The same argument applies for not using MFD

in this

Documentation/driver-api/ancillary_bus.rst line 16

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+scenario as MFD relies on individual function devices being physical

devices

Documentation/driver-api/ancillary_bus.rst

Quoted 3 lines
Documentation/driver-api/ancillary_bus.rst new file +3 -0
+that are DT enumerated.
+
+An example for this kind of requirement is the audio subsystem where a

single

Documentation/driver-api/ancillary_bus.rst line 20

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+IP is handling multiple entities such as HDMI, Soundwire, local devices

such as

Documentation/driver-api/ancillary_bus.rst line 22

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+mics/speakers etc. The split for the core's functionality can be arbitrary or
+be defined by the DSP firmware topology and include hooks for

test/debug. This

Documentation/driver-api/ancillary_bus.rst line 23

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+allows for the audio core device to be minimal and focused on hardware-

specific

Documentation/driver-api/ancillary_bus.rst

Quoted 3 lines
Documentation/driver-api/ancillary_bus.rst new file +3 -0
+control and communication.
+
+The ancillary bus is intended to be minimal, generic and avoid domain-

specific

Documentation/driver-api/ancillary_bus.rst line 28

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+assumptions. Each ancillary_device represents a part of its parent
+functionality. The generic behavior can be extended and specialized as

needed

Documentation/driver-api/ancillary_bus.rst line 29

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+by encapsulating an ancillary_device within other domain-specific

structures and

Documentation/driver-api/ancillary_bus.rst

Quoted 8 lines
Documentation/driver-api/ancillary_bus.rst new file +8 -0
+the use of .ops callbacks. Devices on the ancillary bus do not share any
+structures and the use of a communication channel with the parent is
+domain-specific.
+
+When Should the Ancillary Bus Be Used
+=====================================
+
+The ancillary bus is to be used when a driver and one or more kernel

modules,

Documentation/driver-api/ancillary_bus.rst line 38

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+who share a common header file with the driver, need a mechanism to

connect and

Documentation/driver-api/ancillary_bus.rst line 40

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+provide access to a shared object allocated by the ancillary_device's
+registering driver.  The registering driver for the ancillary_device(s) and

the

Documentation/driver-api/ancillary_bus.rst line 41

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+kernel module(s) registering ancillary_drivers can be from the same

subsystem,

Documentation/driver-api/ancillary_bus.rst

Quoted 3 lines
Documentation/driver-api/ancillary_bus.rst new file +3 -0
+or from multiple subsystems.
+
+The emphasis here is on a common generic interface that keeps

subsystem

Documentation/driver-api/ancillary_bus.rst

Quoted 3 lines
Documentation/driver-api/ancillary_bus.rst new file +3 -0
+customization out of the bus infrastructure.
+
+One example could be a multi-port PCI network device that is rdma-

capable and

Documentation/driver-api/ancillary_bus.rst

Quoted 5 lines
Documentation/driver-api/ancillary_bus.rst new file +5 -0
+needs to export this functionality and attach to an rdma driver in another
+subsystem.  The PCI driver will allocate and register an ancillary_device for
+each physical function on the NIC.  The rdma driver will register an
+ancillary_driver that will be matched with and probed for each of these
+ancillary_devices.  This will give the rdma driver access to the shared

data/ops

Documentation/driver-api/ancillary_bus.rst line 53

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+in the PCI drivers shared object to establish a connection with the PCI

driver.

Documentation/driver-api/ancillary_bus.rst

Quoted 7 lines
Documentation/driver-api/ancillary_bus.rst new file +7 -0
+
+Another use case is for the a PCI device to be split out into multiple sub
+functions.  For each sub function an ancillary_device will be created.  A PCI
+sub function driver will bind to such devices that will create its own one or
+more class devices.  A PCI sub function ancillary device will likely be
+contained in a struct with additional attributes such as user defined sub
+function number and optional attributes such as resources and a link to

the

Documentation/driver-api/ancillary_bus.rst line 61

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+parent device.  These attributes could be used by systemd/udev; and

hence should

Documentation/driver-api/ancillary_bus.rst

Quoted 6 lines
Documentation/driver-api/ancillary_bus.rst new file +6 -0
+be initialized before a driver binds to an ancillary_device.
+
+Ancillary Device
+================
+
+An ancillary_device is created and registered to represent a part of its

parent

Documentation/driver-api/ancillary_bus.rst line 68

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+device's functionality. It is given a name that, combined with the

registering

Documentation/driver-api/ancillary_bus.rst line 69

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+drivers KBUILD_MODNAME, creates a match_name that is used for driver

binding,

Documentation/driver-api/ancillary_bus.rst line 70

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+and an id that combined with the match_name provide a unique name to

register

Documentation/driver-api/ancillary_bus.rst

Quoted 6 lines
Documentation/driver-api/ancillary_bus.rst new file +6 -0
+with the bus subsystem.
+
+Registering an ancillary_device is a two-step process.  First you must call
+ancillary_device_initialize(), which will check several aspects of the
+ancillary_device struct and perform a device_initialize().  After this step
+completes, any error state must have a call to put_device() in its

resolution

Documentation/driver-api/ancillary_bus.rst line 77

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+path.  The second step in registering an ancillary_device is to perform a

call

Documentation/driver-api/ancillary_bus.rst line 78

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+to ancillary_device_add(), which will set the name of the device and add

the

Documentation/driver-api/ancillary_bus.rst

Quoted 3 lines
Documentation/driver-api/ancillary_bus.rst new file +3 -0
+device to the bus.
+
+To unregister an ancillary_device, just a call to

ancillary_device_unregister()

Quoted 6 lines

Documentation/driver-api/ancillary_bus.rst line 82

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+is used.  This will perform both a device_del() and a put_device().

Why did you chose ancillary_device_initialize() and not ancillary_device_register() to be paired with ancillary_device_unregister()?

Thanks

We originally had a single call to ancillary_device_register() that paired with unregister, but there was an ask to separate the register into an initialize and add to make the error condition unwind more compartimentalized.

-DaveE

#4 lore raw

Re: [PATCH 1/6] Add ancillary bus support

Quoted 168 lines · David M
Quoted 168 lines

-----Original Message-----
From: Leon Romanovsky <[email protected]>
Sent: Thursday, October 1, 2020 1:32 AM
To: Ertman, David M <[email protected]>
Cc: [email protected]
Subject: Re: [PATCH 1/6] Add ancillary bus support

Quoted 19 lines · Dave Ertman

Add support for the Ancillary Bus, ancillary_device and ancillary_driver. It enables drivers to create an ancillary_device and bind an ancillary_driver to it.

The bus supports probe/remove shutdown and suspend/resume callbacks. Each ancillary_device has a unique string based id; driver binds to an ancillary_device based on this id through the bus.

Quoted 2 lines
---
 Documentation/driver-api/ancillary_bus.rst | 230

+++++++++++++++++++++

Quoted 13 lines
 Documentation/driver-api/index.rst         |   1 +
 drivers/bus/Kconfig                        |   3 +
 drivers/bus/Makefile                       |   3 +
 drivers/bus/ancillary.c                    | 191 +++++++++++++++++
 include/linux/ancillary_bus.h              |  58 ++++++
 include/linux/mod_devicetable.h            |   8 +
 scripts/mod/devicetable-offsets.c          |   3 +
 scripts/mod/file2alias.c                   |   8 +
 9 files changed, 505 insertions(+)
 create mode 100644 Documentation/driver-api/ancillary_bus.rst
 create mode 100644 drivers/bus/ancillary.c
 create mode 100644 include/linux/ancillary_bus.h

Review

Quoted 1 line
(quoted diff) +0 -0

b/Documentation/driver-api/ancillary_bus.rst

Documentation/driver-api/ancillary_bus.rst line 7

Quoted 12 lines

new file mode 100644
index 000000000000..0a11979aa927

Documentation/driver-api/ancillary_bus.rst new file +7 -0
@@ -0,0 +1,230 @@
+.. SPDX-License-Identifier: GPL-2.0-only
+
+=============
+Ancillary Bus
+=============
+
+In some subsystems, the functionality of the core device

(PCI/ACPI/other) is

Documentation/driver-api/ancillary_bus.rst line 8

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+too complex for a single device to be managed as a monolithic block or a

part of

Documentation/driver-api/ancillary_bus.rst line 9

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+the functionality needs to be exposed to a different subsystem.  Splitting

the

Documentation/driver-api/ancillary_bus.rst line 10

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+functionality into smaller orthogonal devices would make it easier to

manage

Documentation/driver-api/ancillary_bus.rst line 11

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+data, power management and domain-specific interaction with the

hardware. A key

Documentation/driver-api/ancillary_bus.rst line 12

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+requirement for such a split is that there is no dependency on a physical

bus,

Documentation/driver-api/ancillary_bus.rst line 13

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+device, register accesses or regmap support. These individual devices split

from

Documentation/driver-api/ancillary_bus.rst line 14

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+the core cannot live on the platform bus as they are not physical devices

that

Documentation/driver-api/ancillary_bus.rst line 15

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+are controlled by DT/ACPI. The same argument applies for not using MFD

in this

Documentation/driver-api/ancillary_bus.rst line 16

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+scenario as MFD relies on individual function devices being physical

devices

Documentation/driver-api/ancillary_bus.rst

Quoted 3 lines
Documentation/driver-api/ancillary_bus.rst new file +3 -0
+that are DT enumerated.
+
+An example for this kind of requirement is the audio subsystem where a

single

Documentation/driver-api/ancillary_bus.rst line 20

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+IP is handling multiple entities such as HDMI, Soundwire, local devices

such as

Documentation/driver-api/ancillary_bus.rst line 22

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+mics/speakers etc. The split for the core's functionality can be arbitrary or
+be defined by the DSP firmware topology and include hooks for

test/debug. This

Documentation/driver-api/ancillary_bus.rst line 23

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+allows for the audio core device to be minimal and focused on hardware-

specific

Documentation/driver-api/ancillary_bus.rst

Quoted 3 lines
Documentation/driver-api/ancillary_bus.rst new file +3 -0
+control and communication.
+
+The ancillary bus is intended to be minimal, generic and avoid domain-

specific

Documentation/driver-api/ancillary_bus.rst line 28

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+assumptions. Each ancillary_device represents a part of its parent
+functionality. The generic behavior can be extended and specialized as

needed

Documentation/driver-api/ancillary_bus.rst line 29

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+by encapsulating an ancillary_device within other domain-specific

structures and

Documentation/driver-api/ancillary_bus.rst

Quoted 8 lines
Documentation/driver-api/ancillary_bus.rst new file +8 -0
+the use of .ops callbacks. Devices on the ancillary bus do not share any
+structures and the use of a communication channel with the parent is
+domain-specific.
+
+When Should the Ancillary Bus Be Used
+=====================================
+
+The ancillary bus is to be used when a driver and one or more kernel

modules,

Documentation/driver-api/ancillary_bus.rst line 38

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+who share a common header file with the driver, need a mechanism to

connect and

Documentation/driver-api/ancillary_bus.rst line 40

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+provide access to a shared object allocated by the ancillary_device's
+registering driver.  The registering driver for the ancillary_device(s) and

the

Documentation/driver-api/ancillary_bus.rst line 41

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+kernel module(s) registering ancillary_drivers can be from the same

subsystem,

Documentation/driver-api/ancillary_bus.rst

Quoted 3 lines
Documentation/driver-api/ancillary_bus.rst new file +3 -0
+or from multiple subsystems.
+
+The emphasis here is on a common generic interface that keeps

subsystem

Documentation/driver-api/ancillary_bus.rst

Quoted 3 lines
Documentation/driver-api/ancillary_bus.rst new file +3 -0
+customization out of the bus infrastructure.
+
+One example could be a multi-port PCI network device that is rdma-

capable and

Documentation/driver-api/ancillary_bus.rst

Quoted 5 lines
Documentation/driver-api/ancillary_bus.rst new file +5 -0
+needs to export this functionality and attach to an rdma driver in another
+subsystem.  The PCI driver will allocate and register an ancillary_device for
+each physical function on the NIC.  The rdma driver will register an
+ancillary_driver that will be matched with and probed for each of these
+ancillary_devices.  This will give the rdma driver access to the shared

data/ops

Documentation/driver-api/ancillary_bus.rst line 53

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+in the PCI drivers shared object to establish a connection with the PCI

driver.

Documentation/driver-api/ancillary_bus.rst

Quoted 7 lines
Documentation/driver-api/ancillary_bus.rst new file +7 -0
+
+Another use case is for the a PCI device to be split out into multiple sub
+functions.  For each sub function an ancillary_device will be created.  A PCI
+sub function driver will bind to such devices that will create its own one or
+more class devices.  A PCI sub function ancillary device will likely be
+contained in a struct with additional attributes such as user defined sub
+function number and optional attributes such as resources and a link to

the

Documentation/driver-api/ancillary_bus.rst line 61

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+parent device.  These attributes could be used by systemd/udev; and

hence should

Documentation/driver-api/ancillary_bus.rst

Quoted 6 lines
Documentation/driver-api/ancillary_bus.rst new file +6 -0
+be initialized before a driver binds to an ancillary_device.
+
+Ancillary Device
+================
+
+An ancillary_device is created and registered to represent a part of its

parent

Documentation/driver-api/ancillary_bus.rst line 68

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+device's functionality. It is given a name that, combined with the

registering

Documentation/driver-api/ancillary_bus.rst line 69

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+drivers KBUILD_MODNAME, creates a match_name that is used for driver

binding,

Documentation/driver-api/ancillary_bus.rst line 70

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+and an id that combined with the match_name provide a unique name to

register

Documentation/driver-api/ancillary_bus.rst

Quoted 6 lines
Documentation/driver-api/ancillary_bus.rst new file +6 -0
+with the bus subsystem.
+
+Registering an ancillary_device is a two-step process.  First you must call
+ancillary_device_initialize(), which will check several aspects of the
+ancillary_device struct and perform a device_initialize().  After this step
+completes, any error state must have a call to put_device() in its

resolution

Documentation/driver-api/ancillary_bus.rst line 77

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+path.  The second step in registering an ancillary_device is to perform a

call

Documentation/driver-api/ancillary_bus.rst line 78

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+to ancillary_device_add(), which will set the name of the device and add

the

Documentation/driver-api/ancillary_bus.rst

Quoted 3 lines
Documentation/driver-api/ancillary_bus.rst new file +3 -0
+device to the bus.
+
+To unregister an ancillary_device, just a call to

ancillary_device_unregister()

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+is used.  This will perform both a device_del() and a put_device().
Quoted 8 lines
Quoted 4 lines

Why did you chose ancillary_device_initialize() and not ancillary_device_register() to be paired with ancillary_device_unregister()?

Thanks

We originally had a single call to ancillary_device_register() that paired with unregister, but there was an ask to separate the register into an initialize and add to make the error condition unwind more compartimentalized.

It is correct thing to separate, but I would expect:
ancillary_device_register() ancillary_device_add()

vs.
ancillary_device_unregister()

It is not a big deal, just curious.

The much more big deal is that I'm required to create 1-to-1 mapping between device and driver, and I can't connect all my different modules to one xxx_core.pf.y device in N-to-1 mapping. "N" represents different protocols (IB, ETH, SCSI) and "1" is one PCI core.

Thanks

Quoted 2 lines

-DaveE

#5 lore raw

RE: [PATCH 1/6] Add ancillary bus support

Quoted 215 lines

From: Leon Romanovsky <[email protected]>
Sent: Thursday, October 1, 2020 11:10 PM

Quoted 7 lines · David M
Quoted 7 lines

-----Original Message-----
From: Leon Romanovsky <[email protected]>
Sent: Thursday, October 1, 2020 1:32 AM
To: Ertman, David M <[email protected]>
Cc: [email protected]
Subject: Re: [PATCH 1/6] Add ancillary bus support

Quoted 6 lines
Quoted 6 lines · Dave Ertman
Quoted 5 lines

Add support for the Ancillary Bus, ancillary_device and ancillary_driver. It enables drivers to create an ancillary_device and bind an ancillary_driver to it.

The bus supports probe/remove shutdown and suspend/resume

callbacks.

Quoted 183 lines
Quoted 183 lines
Quoted 12 lines

Each ancillary_device has a unique string based id; driver binds to an ancillary_device based on this id through the bus.

Co-developed-by Ranjani Sridharan

<[email protected]>
Signed-off-by: Ranjani Sridharan
<[email protected]>

Reviewed-by Pierre-Louis Bossart
Quoted 7 lines

<[email protected]>

---
 Documentation/driver-api/ancillary_bus.rst | 230

+++++++++++++++++++++

Quoted 11 lines
 Documentation/driver-api/index.rst         |   1 +
 drivers/bus/Kconfig                        |   3 +
 drivers/bus/Makefile                       |   3 +
 drivers/bus/ancillary.c                    | 191 +++++++++++++++++
 include/linux/ancillary_bus.h              |  58 ++++++
 include/linux/mod_devicetable.h            |   8 +
 scripts/mod/devicetable-offsets.c          |   3 +
 scripts/mod/file2alias.c                   |   8 +
 9 files changed, 505 insertions(+)  create mode 100644
Documentation/driver-api/ancillary_bus.rst
 create mode 100644 drivers/bus/ancillary.c  create mode 100644

Review

Quoted 3 lines

include/linux/ancillary_bus.h

(quoted diff) +0 -0

b/Documentation/driver-api/ancillary_bus.rst

Documentation/driver-api/ancillary_bus.rst line 7

Quoted 12 lines

new file mode 100644
index 000000000000..0a11979aa927

Documentation/driver-api/ancillary_bus.rst new file +7 -0
@@ -0,0 +1,230 @@
+.. SPDX-License-Identifier: GPL-2.0-only
+
+=============
+Ancillary Bus
+=============
+
+In some subsystems, the functionality of the core device

(PCI/ACPI/other) is

Documentation/driver-api/ancillary_bus.rst line 9

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+too complex for a single device to be managed as a monolithic
+block or a

part of

Documentation/driver-api/ancillary_bus.rst line 11

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+the functionality needs to be exposed to a different subsystem.
+Splitting

the

Documentation/driver-api/ancillary_bus.rst line 13

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+functionality into smaller orthogonal devices would make it
+easier to

manage

Documentation/driver-api/ancillary_bus.rst line 14

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+data, power management and domain-specific interaction with the

hardware. A key

Documentation/driver-api/ancillary_bus.rst line 16

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+requirement for such a split is that there is no dependency on a
+physical

bus,

Documentation/driver-api/ancillary_bus.rst line 18

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+device, register accesses or regmap support. These individual
+devices split

from

Documentation/driver-api/ancillary_bus.rst line 20

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+the core cannot live on the platform bus as they are not physical
+devices

that

Documentation/driver-api/ancillary_bus.rst line 22

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+are controlled by DT/ACPI. The same argument applies for not
+using MFD

in this

Documentation/driver-api/ancillary_bus.rst line 24

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+scenario as MFD relies on individual function devices being
+physical

devices

Documentation/driver-api/ancillary_bus.rst

Quoted 4 lines
Documentation/driver-api/ancillary_bus.rst new file +4 -0
+that are DT enumerated.
+
+An example for this kind of requirement is the audio subsystem
+where a

single

Documentation/driver-api/ancillary_bus.rst line 30

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+IP is handling multiple entities such as HDMI, Soundwire, local
+devices

such as

Documentation/driver-api/ancillary_bus.rst line 33

Quoted 3 lines
Documentation/driver-api/ancillary_bus.rst new file +3 -0
+mics/speakers etc. The split for the core's functionality can be
+arbitrary or be defined by the DSP firmware topology and include
+hooks for

test/debug. This

Documentation/driver-api/ancillary_bus.rst line 35

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+allows for the audio core device to be minimal and focused on
+hardware-

specific

Documentation/driver-api/ancillary_bus.rst

Quoted 4 lines
Documentation/driver-api/ancillary_bus.rst new file +4 -0
+control and communication.
+
+The ancillary bus is intended to be minimal, generic and avoid
+domain-

specific

Documentation/driver-api/ancillary_bus.rst line 42

Quoted 3 lines
Documentation/driver-api/ancillary_bus.rst new file +3 -0
+assumptions. Each ancillary_device represents a part of its
+parent functionality. The generic behavior can be extended and
+specialized as

needed

Documentation/driver-api/ancillary_bus.rst line 43

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+by encapsulating an ancillary_device within other domain-specific

structures and

Documentation/driver-api/ancillary_bus.rst line 52

Quoted 9 lines
Documentation/driver-api/ancillary_bus.rst new file +9 -0
+the use of .ops callbacks. Devices on the ancillary bus do not
+share any structures and the use of a communication channel with
+the parent is domain-specific.
+
+When Should the Ancillary Bus Be Used
+=====================================
+
+The ancillary bus is to be used when a driver and one or more
+kernel

modules,

Documentation/driver-api/ancillary_bus.rst line 54

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+who share a common header file with the driver, need a mechanism
+to

connect and

Documentation/driver-api/ancillary_bus.rst line 57

Quoted 3 lines
Documentation/driver-api/ancillary_bus.rst new file +3 -0
+provide access to a shared object allocated by the
+ancillary_device's registering driver.  The registering driver
+for the ancillary_device(s) and

the

Documentation/driver-api/ancillary_bus.rst line 59

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+kernel module(s) registering ancillary_drivers can be from the
+same

subsystem,

Documentation/driver-api/ancillary_bus.rst

Quoted 3 lines
Documentation/driver-api/ancillary_bus.rst new file +3 -0
+or from multiple subsystems.
+
+The emphasis here is on a common generic interface that keeps

subsystem

Documentation/driver-api/ancillary_bus.rst

Quoted 4 lines
Documentation/driver-api/ancillary_bus.rst new file +4 -0
+customization out of the bus infrastructure.
+
+One example could be a multi-port PCI network device that is
+rdma-

capable and

Documentation/driver-api/ancillary_bus.rst line 72

Quoted 6 lines
Documentation/driver-api/ancillary_bus.rst new file +6 -0
+needs to export this functionality and attach to an rdma driver
+in another subsystem.  The PCI driver will allocate and register
+an ancillary_device for each physical function on the NIC.  The
+rdma driver will register an ancillary_driver that will be
+matched with and probed for each of these ancillary_devices.
+This will give the rdma driver access to the shared

data/ops

Documentation/driver-api/ancillary_bus.rst line 74

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+in the PCI drivers shared object to establish a connection with
+the PCI

driver.

Documentation/driver-api/ancillary_bus.rst line 83

Quoted 9 lines
Documentation/driver-api/ancillary_bus.rst new file +9 -0
+
+Another use case is for the a PCI device to be split out into
+multiple sub functions.  For each sub function an
+ancillary_device will be created.  A PCI sub function driver will
+bind to such devices that will create its own one or more class
+devices.  A PCI sub function ancillary device will likely be
+contained in a struct with additional attributes such as user
+defined sub function number and optional attributes such as
+resources and a link to

the

Documentation/driver-api/ancillary_bus.rst line 85

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+parent device.  These attributes could be used by systemd/udev;
+and

hence should

Documentation/driver-api/ancillary_bus.rst

Quoted 7 lines
Documentation/driver-api/ancillary_bus.rst new file +7 -0
+be initialized before a driver binds to an ancillary_device.
+
+Ancillary Device
+================
+
+An ancillary_device is created and registered to represent a part
+of its

parent

Documentation/driver-api/ancillary_bus.rst line 94

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+device's functionality. It is given a name that, combined with
+the

registering

Documentation/driver-api/ancillary_bus.rst line 96

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+drivers KBUILD_MODNAME, creates a match_name that is used for
+driver

binding,

Documentation/driver-api/ancillary_bus.rst line 98

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+and an id that combined with the match_name provide a unique name
+to

register

Documentation/driver-api/ancillary_bus.rst line 105

Quoted 7 lines
Documentation/driver-api/ancillary_bus.rst new file +7 -0
+with the bus subsystem.
+
+Registering an ancillary_device is a two-step process.  First you
+must call ancillary_device_initialize(), which will check several
+aspects of the ancillary_device struct and perform a
+device_initialize().  After this step completes, any error state
+must have a call to put_device() in its

resolution

Documentation/driver-api/ancillary_bus.rst line 107

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+path.  The second step in registering an ancillary_device is to
+perform a

call

Documentation/driver-api/ancillary_bus.rst line 109

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+to ancillary_device_add(), which will set the name of the device
+and add

the

Quoted 8 lines
Quoted 8 lines

Documentation/driver-api/ancillary_bus.rst

Quoted 3 lines
Documentation/driver-api/ancillary_bus.rst new file +3 -0
+device to the bus.
+
+To unregister an ancillary_device, just a call to

ancillary_device_unregister()

Documentation/driver-api/ancillary_bus.rst line 82

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+is used.  This will perform both a device_del() and a put_device().

Why did you chose ancillary_device_initialize() and not ancillary_device_register() to be paired with

ancillary_device_unregister()?

Quoted 3 lines
Quoted 2 lines

Thanks

Quoted 9 lines
Quoted 3 lines

We originally had a single call to ancillary_device_register() that paired with unregister, but there was an ask to separate the register into an initialize and add to make the error condition unwind more

compartimentalized.

It is correct thing to separate, but I would expect:
ancillary_device_register() ancillary_device_add()

device_initialize(), device_add() and device_unregister() is the pattern widely followed in the core.

Quoted 9 lines

vs.
ancillary_device_unregister()

It is not a big deal, just curious.

The much more big deal is that I'm required to create 1-to-1 mapping between device and driver, and I can't connect all my different modules to one xxx_core.pf.y device in N-to-1 mapping. "N" represents different protocols (IB, ETH, SCSI) and "1" is one PCI core.

For one mlx5 (pf/vf/sf) device, there are three class erivers (ib, vdpa, en).

So there should be one ida allocate per mlx5 device type.

Static const mlx5_adev_names[MLX5_INTERFACE_PROTOCOL_MAX] = {
	"rdma",
	"eth",
	"vdpa"
};

Something like for current mlx5_register_device(), mlx5_register_device()

{
	id = ida_alloc(0, UINT_MAX, GFP_KERNEL);

	for (i = 0; I < MLX5_INTERFACE_PROTOCOL_MAX; i++) {
		adev = kzalloc(sizeof(*adev), GFP_KERNEL);
		adev.name = mlx5_adev_names[i];
		adev.id = ret;
		adev.dev.parent = mlx5_core_dev->device;
		adev->coredev = mlx5_core_dev;
		ret = ancillary_device_initialize(&adev);	
		ret = ancillary_device_register(adev);
}

This will create 3 ancillary devices for each PCI PF/VF/SF. mlx5_core.rdma.1
mlx5_core.eth.1
mlx5_core.vdpa.1

and mlx5_ib driver will do

ancillary_driver_register()
{
For ID of mlx5_core.rdma.
}

mlx5_vdpa driver does,

ancillary_driver_register()
{
For ID of mlx5_core.vdpa
}

This is uniform for pf/vf/sf for one or more all protocols.

#6 lore raw

Re: [PATCH 1/6] Add ancillary bus support

Quoted 218 lines · Parav Pandit
Quoted 218 lines

From: Leon Romanovsky <[email protected]>
Sent: Thursday, October 1, 2020 11:10 PM

Quoted 7 lines · David M
Quoted 7 lines

-----Original Message-----
From: Leon Romanovsky <[email protected]>
Sent: Thursday, October 1, 2020 1:32 AM
To: Ertman, David M <[email protected]>
Cc: [email protected]
Subject: Re: [PATCH 1/6] Add ancillary bus support

Quoted 6 lines
Quoted 6 lines · Dave Ertman
Quoted 5 lines

Add support for the Ancillary Bus, ancillary_device and ancillary_driver. It enables drivers to create an ancillary_device and bind an ancillary_driver to it.

The bus supports probe/remove shutdown and suspend/resume

callbacks.

Quoted 183 lines
Quoted 183 lines
Quoted 12 lines

Each ancillary_device has a unique string based id; driver binds to an ancillary_device based on this id through the bus.

Co-developed-by Ranjani Sridharan

<[email protected]>
Signed-off-by: Ranjani Sridharan
<[email protected]>

Reviewed-by Pierre-Louis Bossart
Quoted 7 lines

<[email protected]>

---
 Documentation/driver-api/ancillary_bus.rst | 230

+++++++++++++++++++++

Quoted 11 lines
 Documentation/driver-api/index.rst         |   1 +
 drivers/bus/Kconfig                        |   3 +
 drivers/bus/Makefile                       |   3 +
 drivers/bus/ancillary.c                    | 191 +++++++++++++++++
 include/linux/ancillary_bus.h              |  58 ++++++
 include/linux/mod_devicetable.h            |   8 +
 scripts/mod/devicetable-offsets.c          |   3 +
 scripts/mod/file2alias.c                   |   8 +
 9 files changed, 505 insertions(+)  create mode 100644
Documentation/driver-api/ancillary_bus.rst
 create mode 100644 drivers/bus/ancillary.c  create mode 100644

Review

Quoted 3 lines

include/linux/ancillary_bus.h

(quoted diff) +0 -0

b/Documentation/driver-api/ancillary_bus.rst

Documentation/driver-api/ancillary_bus.rst line 7

Quoted 12 lines

new file mode 100644
index 000000000000..0a11979aa927

Documentation/driver-api/ancillary_bus.rst new file +7 -0
@@ -0,0 +1,230 @@
+.. SPDX-License-Identifier: GPL-2.0-only
+
+=============
+Ancillary Bus
+=============
+
+In some subsystems, the functionality of the core device

(PCI/ACPI/other) is

Documentation/driver-api/ancillary_bus.rst line 9

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+too complex for a single device to be managed as a monolithic
+block or a

part of

Documentation/driver-api/ancillary_bus.rst line 11

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+the functionality needs to be exposed to a different subsystem.
+Splitting

the

Documentation/driver-api/ancillary_bus.rst line 13

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+functionality into smaller orthogonal devices would make it
+easier to

manage

Documentation/driver-api/ancillary_bus.rst line 14

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+data, power management and domain-specific interaction with the

hardware. A key

Documentation/driver-api/ancillary_bus.rst line 16

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+requirement for such a split is that there is no dependency on a
+physical

bus,

Documentation/driver-api/ancillary_bus.rst line 18

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+device, register accesses or regmap support. These individual
+devices split

from

Documentation/driver-api/ancillary_bus.rst line 20

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+the core cannot live on the platform bus as they are not physical
+devices

that

Documentation/driver-api/ancillary_bus.rst line 22

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+are controlled by DT/ACPI. The same argument applies for not
+using MFD

in this

Documentation/driver-api/ancillary_bus.rst line 24

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+scenario as MFD relies on individual function devices being
+physical

devices

Documentation/driver-api/ancillary_bus.rst

Quoted 4 lines
Documentation/driver-api/ancillary_bus.rst new file +4 -0
+that are DT enumerated.
+
+An example for this kind of requirement is the audio subsystem
+where a

single

Documentation/driver-api/ancillary_bus.rst line 30

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+IP is handling multiple entities such as HDMI, Soundwire, local
+devices

such as

Documentation/driver-api/ancillary_bus.rst line 33

Quoted 3 lines
Documentation/driver-api/ancillary_bus.rst new file +3 -0
+mics/speakers etc. The split for the core's functionality can be
+arbitrary or be defined by the DSP firmware topology and include
+hooks for

test/debug. This

Documentation/driver-api/ancillary_bus.rst line 35

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+allows for the audio core device to be minimal and focused on
+hardware-

specific

Documentation/driver-api/ancillary_bus.rst

Quoted 4 lines
Documentation/driver-api/ancillary_bus.rst new file +4 -0
+control and communication.
+
+The ancillary bus is intended to be minimal, generic and avoid
+domain-

specific

Documentation/driver-api/ancillary_bus.rst line 42

Quoted 3 lines
Documentation/driver-api/ancillary_bus.rst new file +3 -0
+assumptions. Each ancillary_device represents a part of its
+parent functionality. The generic behavior can be extended and
+specialized as

needed

Documentation/driver-api/ancillary_bus.rst line 43

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+by encapsulating an ancillary_device within other domain-specific

structures and

Documentation/driver-api/ancillary_bus.rst line 52

Quoted 9 lines
Documentation/driver-api/ancillary_bus.rst new file +9 -0
+the use of .ops callbacks. Devices on the ancillary bus do not
+share any structures and the use of a communication channel with
+the parent is domain-specific.
+
+When Should the Ancillary Bus Be Used
+=====================================
+
+The ancillary bus is to be used when a driver and one or more
+kernel

modules,

Documentation/driver-api/ancillary_bus.rst line 54

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+who share a common header file with the driver, need a mechanism
+to

connect and

Documentation/driver-api/ancillary_bus.rst line 57

Quoted 3 lines
Documentation/driver-api/ancillary_bus.rst new file +3 -0
+provide access to a shared object allocated by the
+ancillary_device's registering driver.  The registering driver
+for the ancillary_device(s) and

the

Documentation/driver-api/ancillary_bus.rst line 59

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+kernel module(s) registering ancillary_drivers can be from the
+same

subsystem,

Documentation/driver-api/ancillary_bus.rst

Quoted 3 lines
Documentation/driver-api/ancillary_bus.rst new file +3 -0
+or from multiple subsystems.
+
+The emphasis here is on a common generic interface that keeps

subsystem

Documentation/driver-api/ancillary_bus.rst

Quoted 4 lines
Documentation/driver-api/ancillary_bus.rst new file +4 -0
+customization out of the bus infrastructure.
+
+One example could be a multi-port PCI network device that is
+rdma-

capable and

Documentation/driver-api/ancillary_bus.rst line 72

Quoted 6 lines
Documentation/driver-api/ancillary_bus.rst new file +6 -0
+needs to export this functionality and attach to an rdma driver
+in another subsystem.  The PCI driver will allocate and register
+an ancillary_device for each physical function on the NIC.  The
+rdma driver will register an ancillary_driver that will be
+matched with and probed for each of these ancillary_devices.
+This will give the rdma driver access to the shared

data/ops

Documentation/driver-api/ancillary_bus.rst line 74

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+in the PCI drivers shared object to establish a connection with
+the PCI

driver.

Documentation/driver-api/ancillary_bus.rst line 83

Quoted 9 lines
Documentation/driver-api/ancillary_bus.rst new file +9 -0
+
+Another use case is for the a PCI device to be split out into
+multiple sub functions.  For each sub function an
+ancillary_device will be created.  A PCI sub function driver will
+bind to such devices that will create its own one or more class
+devices.  A PCI sub function ancillary device will likely be
+contained in a struct with additional attributes such as user
+defined sub function number and optional attributes such as
+resources and a link to

the

Documentation/driver-api/ancillary_bus.rst line 85

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+parent device.  These attributes could be used by systemd/udev;
+and

hence should

Documentation/driver-api/ancillary_bus.rst

Quoted 7 lines
Documentation/driver-api/ancillary_bus.rst new file +7 -0
+be initialized before a driver binds to an ancillary_device.
+
+Ancillary Device
+================
+
+An ancillary_device is created and registered to represent a part
+of its

parent

Documentation/driver-api/ancillary_bus.rst line 94

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+device's functionality. It is given a name that, combined with
+the

registering

Documentation/driver-api/ancillary_bus.rst line 96

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+drivers KBUILD_MODNAME, creates a match_name that is used for
+driver

binding,

Documentation/driver-api/ancillary_bus.rst line 98

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+and an id that combined with the match_name provide a unique name
+to

register

Documentation/driver-api/ancillary_bus.rst line 105

Quoted 7 lines
Documentation/driver-api/ancillary_bus.rst new file +7 -0
+with the bus subsystem.
+
+Registering an ancillary_device is a two-step process.  First you
+must call ancillary_device_initialize(), which will check several
+aspects of the ancillary_device struct and perform a
+device_initialize().  After this step completes, any error state
+must have a call to put_device() in its

resolution

Documentation/driver-api/ancillary_bus.rst line 107

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+path.  The second step in registering an ancillary_device is to
+perform a

call

Documentation/driver-api/ancillary_bus.rst line 109

Quoted 2 lines
Documentation/driver-api/ancillary_bus.rst new file +2 -0
+to ancillary_device_add(), which will set the name of the device
+and add

the

Quoted 8 lines
Quoted 8 lines

Documentation/driver-api/ancillary_bus.rst

Quoted 3 lines
Documentation/driver-api/ancillary_bus.rst new file +3 -0
+device to the bus.
+
+To unregister an ancillary_device, just a call to

ancillary_device_unregister()

Documentation/driver-api/ancillary_bus.rst line 82

Quoted 1 line
Documentation/driver-api/ancillary_bus.rst new file +1 -0
+is used.  This will perform both a device_del() and a put_device().

Why did you chose ancillary_device_initialize() and not ancillary_device_register() to be paired with

ancillary_device_unregister()?

Quoted 6 lines
Quoted 2 lines

Thanks

We originally had a single call to ancillary_device_register() that paired with unregister, but there was an ask to separate the register into an initialize and add to make the error condition unwind more

Quoted 7 lines
Quoted 6 lines

compartimentalized.

It is correct thing to separate, but I would expect:
ancillary_device_register() ancillary_device_add()

device_initialize(), device_add() and device_unregister() is the pattern widely followed in the core.

It doesn't mean that I need to agree with that, right?

Quoted 15 lines
Quoted 10 lines

vs.
ancillary_device_unregister()

It is not a big deal, just curious.

The much more big deal is that I'm required to create 1-to-1 mapping between device and driver, and I can't connect all my different modules to one xxx_core.pf.y device in N-to-1 mapping. "N" represents different protocols (IB, ETH, SCSI) and "1" is one PCI core.

For one mlx5 (pf/vf/sf) device, there are three class erivers (ib, vdpa, en).

So there should be one ida allocate per mlx5 device type.

Quoted 7 lines
Static const mlx5_adev_names[MLX5_INTERFACE_PROTOCOL_MAX] = {
	"rdma",
	"eth",
	"vdpa"
};

Something like for current mlx5_register_device(),

I know it and already implemented it, this is why I'm saying that it is not what I would expect from the implementation.

It is wrong create mlx5_core.rdma.1 device that is equal to mlx5_core.eth.1 just to connect our mlx5_ib.ko to it, while documentation explains about creating single PCI code device and other drivers connect to it.

Thanks

Quoted 34 lines

mlx5_register_device()

{
	id = ida_alloc(0, UINT_MAX, GFP_KERNEL);

	for (i = 0; I < MLX5_INTERFACE_PROTOCOL_MAX; i++) {
		adev = kzalloc(sizeof(*adev), GFP_KERNEL);
		adev.name = mlx5_adev_names[i];
		adev.id = ret;
		adev.dev.parent = mlx5_core_dev->device;
		adev->coredev = mlx5_core_dev;
		ret = ancillary_device_initialize(&adev);
		ret = ancillary_device_register(adev);
}

This will create 3 ancillary devices for each PCI PF/VF/SF. mlx5_core.rdma.1
mlx5_core.eth.1
mlx5_core.vdpa.1

and mlx5_ib driver will do

ancillary_driver_register()
{
For ID of mlx5_core.rdma.
}

mlx5_vdpa driver does,

ancillary_driver_register()
{
For ID of mlx5_core.vdpa
}

This is uniform for pf/vf/sf for one or more all protocols.

#7 lore raw

RE: [PATCH 1/6] Add ancillary bus support

Quoted 35 lines

From: Leon Romanovsky <[email protected]>
Sent: Friday, October 2, 2020 1:02 AM

Quoted 30 lines
Quoted 30 lines
Quoted 12 lines
Quoted 12 lines

Review

Quoted 5 lines
(quoted diff) +5 -0
+Registering an ancillary_device is a two-step process.  First
+you must call ancillary_device_initialize(), which will check
+several aspects of the ancillary_device struct and perform a
+device_initialize().  After this step completes, any error
+state must have a call to put_device() in its

resolution

Review

Quoted 2 lines
(quoted diff) +2 -0
+path.  The second step in registering an ancillary_device is
+to perform a

call

Review

Quoted 2 lines
(quoted diff) +2 -0
+to ancillary_device_add(), which will set the name of the
+device and add

the

Quoted 8 lines
Quoted 8 lines

Documentation/driver-api/ancillary_bus.rst

Quoted 3 lines
Documentation/driver-api/ancillary_bus.rst new file +3 -0
+device to the bus.
+
+To unregister an ancillary_device, just a call to

ancillary_device_unregister()

Quoted 1 line

+is used. This will perform both a device_del() and a put_device().

Why did you chose ancillary_device_initialize() and not ancillary_device_register() to be paired with

ancillary_device_unregister()?

Quoted 7 lines
Quoted 2 lines

Thanks

We originally had a single call to ancillary_device_register() that paired with unregister, but there was an ask to separate the register into an initialize and add to make the error condition unwind more

compartimentalized.

Quoted 9 lines
Quoted 5 lines
Quoted 4 lines

It is correct thing to separate, but I would expect:
ancillary_device_register() ancillary_device_add()

device_initialize(), device_add() and device_unregister() is the pattern widely

followed in the core.

It doesn't mean that I need to agree with that, right?

Right. May be I misunderstood your comment where you said "I expect device_register() and device_add()" in response to "device_initialize() and device_add". I interpreted your comment as to replace initialize() with register().
Because that is odd naming and completely out of sync from the core APIs.

A helper like below that wraps initialize() and add() is buggy, because when register() returns an error, it doesn't know if should do kfree() or put_device().

ancillary_device_register(adev)
{
ret = ancillary_device_initialize();
if (ret)

    return ret;

ret = ancillary_device_add();
if (ret)

    put_device(dev);

return ret;
}

Quoted 21 lines
Quoted 21 lines
Quoted 10 lines

vs.
ancillary_device_unregister()

It is not a big deal, just curious.

The much more big deal is that I'm required to create 1-to-1 mapping between device and driver, and I can't connect all my different modules to one xxx_core.pf.y device in N-to-1 mapping. "N" represents different protocols (IB, ETH, SCSI) and "1" is one PCI core.

For one mlx5 (pf/vf/sf) device, there are three class erivers (ib, vdpa, en).

So there should be one ida allocate per mlx5 device type.

Static const mlx5_adev_names[MLX5_INTERFACE_PROTOCOL_MAX] = {
	"rdma",
	"eth",
	"vdpa"
};
Quoted 7 lines
Quoted 1 line

Something like for current mlx5_register_device(),

I know it and already implemented it, this is why I'm saying that it is not what I would expect from the implementation.

It is wrong create mlx5_core.rdma.1 device that is equal to mlx5_core.eth.1 just to connect our mlx5_ib.ko to it, while documentation explains about creating

Ancillary bus's documentation? If so, it should be corrected. Do you have specific text snippet to point to that should be fixed?

For purpose of matching service functionality, for each different class (ib, eth, vdpa) there is one ancillary device created. What exactly is wrong here? (and why is it wrong now and not in previous version of the RFC?)

Do you want to create one device and 3 drivers to bind to it? If you think that way, may be pci core should be extended to support that, ancillary bus may not be required. But that is different solution than what is being proposed here.
Not sure I understand your comment about wrong create mlx5_core.rdma1 equal to core.eth.1", because they are not equal.
To begin with, they have different id for matching service, and in future it should be extended to pass 'only' needed info.
mlx5_core exposing the 'whole' mlx5_core_dev to other drivers doesn't look correct.

Quoted 38 lines

single PCI code device and other drivers connect to it.

Thanks

Quoted 33 lines

mlx5_register_device()

{
	id = ida_alloc(0, UINT_MAX, GFP_KERNEL);

	for (i = 0; I < MLX5_INTERFACE_PROTOCOL_MAX; i++) {
		adev = kzalloc(sizeof(*adev), GFP_KERNEL);
		adev.name = mlx5_adev_names[i];
		adev.id = ret;
		adev.dev.parent = mlx5_core_dev->device;
		adev->coredev = mlx5_core_dev;
		ret = ancillary_device_initialize(&adev);
		ret = ancillary_device_register(adev); }

This will create 3 ancillary devices for each PCI PF/VF/SF. mlx5_core.rdma.1
mlx5_core.eth.1
mlx5_core.vdpa.1

and mlx5_ib driver will do

ancillary_driver_register()
{
For ID of mlx5_core.rdma.
}

mlx5_vdpa driver does,

ancillary_driver_register()
{
For ID of mlx5_core.vdpa
}

This is uniform for pf/vf/sf for one or more all protocols.

#8 lore raw

Re: [PATCH 1/6] Add ancillary bus support

Quoted 42 lines · Parav Pandit
Quoted 42 lines

From: Leon Romanovsky <[email protected]>
Sent: Friday, October 2, 2020 1:02 AM

Quoted 28 lines
Quoted 28 lines
Quoted 12 lines
Quoted 12 lines

Review

Quoted 5 lines
(quoted diff) +5 -0
+Registering an ancillary_device is a two-step process.  First
+you must call ancillary_device_initialize(), which will check
+several aspects of the ancillary_device struct and perform a
+device_initialize().  After this step completes, any error
+state must have a call to put_device() in its

resolution

Review

Quoted 2 lines
(quoted diff) +2 -0
+path.  The second step in registering an ancillary_device is
+to perform a

call

Review

Quoted 2 lines
(quoted diff) +2 -0
+to ancillary_device_add(), which will set the name of the
+device and add

the

Quoted 8 lines
Quoted 8 lines

Review

Quoted 3 lines
(quoted diff) +3 -0
+device to the bus.
+
+To unregister an ancillary_device, just a call to

ancillary_device_unregister()

Quoted 1 line

+is used. This will perform both a device_del() and a put_device().

Why did you chose ancillary_device_initialize() and not ancillary_device_register() to be paired with

ancillary_device_unregister()?

Quoted 7 lines
Quoted 2 lines

Thanks

We originally had a single call to ancillary_device_register() that paired with unregister, but there was an ask to separate the register into an initialize and add to make the error condition unwind more

Quoted 7 lines
Quoted 6 lines

compartimentalized.

It is correct thing to separate, but I would expect:
ancillary_device_register() ancillary_device_add()

device_initialize(), device_add() and device_unregister() is the pattern widely

followed in the core.

Quoted 7 lines
Quoted 2 lines

It doesn't mean that I need to agree with that, right?

Right. May be I misunderstood your comment where you said "I expect device_register() and device_add()" in response to "device_initialize() and device_add". I interpreted your comment as to replace initialize() with register().
Because that is odd naming and completely out of sync from the core APIs.

A helper like below that wraps initialize() and add() is buggy, because when register() returns an error, it doesn't know if should do kfree() or put_device().

I wrote above (14 lines above this line) that I understand and support the request to separate init and add parts. There is only one thing that I didn't like that we have _unregister() but don't have _register(). It is not a big deal.

Quoted 36 lines

ancillary_device_register(adev)
{
ret = ancillary_device_initialize();
if (ret)

    return ret;

ret = ancillary_device_add();
if (ret)

    put_device(dev);

return ret;
}

Quoted 23 lines
Quoted 22 lines
Quoted 10 lines

vs.
ancillary_device_unregister()

It is not a big deal, just curious.

The much more big deal is that I'm required to create 1-to-1 mapping between device and driver, and I can't connect all my different modules to one xxx_core.pf.y device in N-to-1 mapping. "N" represents different protocols (IB, ETH, SCSI) and "1" is one PCI core.

For one mlx5 (pf/vf/sf) device, there are three class erivers (ib, vdpa, en).

So there should be one ida allocate per mlx5 device type.

Static const mlx5_adev_names[MLX5_INTERFACE_PROTOCOL_MAX] = {
	"rdma",
	"eth",
	"vdpa"
};

Something like for current mlx5_register_device(),

Quoted 8 lines
Quoted 5 lines

I know it and already implemented it, this is why I'm saying that it is not what I would expect from the implementation.

It is wrong create mlx5_core.rdma.1 device that is equal to mlx5_core.eth.1 just to connect our mlx5_ib.ko to it, while documentation explains about creating

Ancillary bus's documentation? If so, it should be corrected. Do you have specific text snippet to point to that should be fixed?

+One example could be a multi-port PCI network device that is rdma-capable and +needs to export this functionality and attach to an rdma driver in another +subsystem. The PCI driver will allocate and register an ancillary_device for +each physical function on the NIC. The rdma driver will register an +ancillary_driver that will be matched with and probed for each of these +ancillary_devices. This will give the rdma driver access to the shared data/ops +in the PCI drivers shared object to establish a connection with the PCI driver.

Quoted 3 lines

For purpose of matching service functionality, for each different class (ib, eth, vdpa) there is one ancillary device created. What exactly is wrong here? (and why is it wrong now and not in previous version of the RFC?)

Here the example of two real systems, see how many links we created to same mlx5_core PCI logic. I imagine that Qlogic and Chelsio drivers will look even worse, because they spread over more subsystems than mlx5.

This is first time when I see real implementation of real device.

System with 2 IB and 1 RoCE cards:
[leonro@vm ~]$ ls -l /sys/bus/ancillary/devices/
mlx5_core.eth.0 -> ../../../devices/pci0000:00/0000:00:09.0/mlx5_core.eth.0 mlx5_core.eth.1 -> ../../../devices/pci0000:00/0000:00:0a.0/mlx5_core.eth.1 mlx5_core.eth.2 -> ../../../devices/pci0000:00/0000:00:0b.0/mlx5_core.eth.2 mlx5_core.ib.0 -> ../../../devices/pci0000:00/0000:00:09.0/mlx5_core.ib.0 mlx5_core.ib.1 -> ../../../devices/pci0000:00/0000:00:0a.0/mlx5_core.ib.1 mlx5_core.ib.2 -> ../../../devices/pci0000:00/0000:00:0b.0/mlx5_core.ib.2 mlx5_core.vdpa.0 -> ../../../devices/pci0000:00/0000:00:09.0/mlx5_core.ib.0 mlx5_core.vdpa.1 -> ../../../devices/pci0000:00/0000:00:0a.0/mlx5_core.ib.1 mlx5_core.vdpa.2 -> ../../../devices/pci0000:00/0000:00:0b.0/mlx5_core.ib.2
[leonro@vm ~]$ rdma dev
0: ibp0s9: node_type ca fw 4.6.9999 node_guid 5254:00c0:fe12:3455 sys_image_guid 5254:00c0:fe12:3455 1: ibp0s10: node_type ca fw 4.6.9999 node_guid 5254:00c0:fe12:3456 sys_image_guid 5254:00c0:fe12:3456 2: rdmap0s11: node_type ca fw 4.6.9999 node_guid 5254:00c0:fe12:3457 sys_image_guid 5254:00c0:fe12:3457

System with RoCE SR-IOV card with 4 VFs:
[leonro@vm ~]$ ls -l /sys/bus/ancillary/devices/
mlx5_core.eth.0 -> ../../../devices/pci0000:00/0000:00:09.0/0000:01:00.0/mlx5_core.eth.0 mlx5_core.eth.1 -> ../../../devices/pci0000:00/0000:00:09.0/0000:01:00.1/mlx5_core.eth.1 mlx5_core.eth.2 -> ../../../devices/pci0000:00/0000:00:09.0/0000:01:00.2/mlx5_core.eth.2 mlx5_core.eth.3 -> ../../../devices/pci0000:00/0000:00:09.0/0000:01:00.3/mlx5_core.eth.3 mlx5_core.eth.4 -> ../../../devices/pci0000:00/0000:00:09.0/0000:01:00.4/mlx5_core.eth.4 mlx5_core.ib.0 -> ../../../devices/pci0000:00/0000:00:09.0/0000:01:00.0/mlx5_core.ib.0 mlx5_core.ib.1 -> ../../../devices/pci0000:00/0000:00:09.0/0000:01:00.1/mlx5_core.ib.1 mlx5_core.ib.2 -> ../../../devices/pci0000:00/0000:00:09.0/0000:01:00.2/mlx5_core.ib.2 mlx5_core.ib.3 -> ../../../devices/pci0000:00/0000:00:09.0/0000:01:00.3/mlx5_core.ib.3 mlx5_core.ib.4 -> ../../../devices/pci0000:00/0000:00:09.0/0000:01:00.4/mlx5_core.ib.4 mlx5_core.vdpa.0 -> ../../../devices/pci0000:00/0000:00:09.0/0000:01:00.0/mlx5_core.ib.0 mlx5_core.vdpa.1 -> ../../../devices/pci0000:00/0000:00:09.0/0000:01:00.1/mlx5_core.ib.1 mlx5_core.vdpa.2 -> ../../../devices/pci0000:00/0000:00:09.0/0000:01:00.2/mlx5_core.ib.2 mlx5_core.vdpa.3 -> ../../../devices/pci0000:00/0000:00:09.0/0000:01:00.3/mlx5_core.ib.3 mlx5_core.vdpa.4 -> ../../../devices/pci0000:00/0000:00:09.0/0000:01:00.4/mlx5_core.ib.4
[leonro@vm ~]$ rdma dev
0: rocep1s0f0: node_type ca fw 4.6.9999 node_guid 5254:00c0:fe12:3455 sys_image_guid 5254:00c0:fe12:3455 1: rocep1s0f0v0: node_type ca fw 4.6.9999 node_guid 0000:0000:0000:0000 sys_image_guid 5254:00c0:fe12:3456 2: rocep1s0f0v1: node_type ca fw 4.6.9999 node_guid 0000:0000:0000:0000 sys_image_guid 5254:00c0:fe12:3457 3: rocep1s0f0v2: node_type ca fw 4.6.9999 node_guid 0000:0000:0000:0000 sys_image_guid 5254:00c0:fe12:3458 4: rocep1s0f0v3: node_type ca fw 4.6.9999 node_guid 0000:0000:0000:0000 sys_image_guid 5254:00c0:fe12:3459

Quoted 3 lines

Do you want to create one device and 3 drivers to bind to it? If you think that way, may be pci core should be extended to support that, ancillary bus may not be required. But that is different solution than what is being proposed here.

This is my expectation, I see this virtual bus as glue logic for the drivers between different subsystems.

Quoted 3 lines

Not sure I understand your comment about wrong create mlx5_core.rdma1 equal to core.eth.1", because they are not equal. To begin with, they have different id for matching service, and in future it should be extended to pass 'only' needed info. mlx5_core exposing the 'whole' mlx5_core_dev to other drivers doesn't look correct.

Maybe, what I would like to see is:
[leonro@vm ~]$ ls -l /sys/bus/ancillary/devices/
mlx5_core.pf.0 -> ../../../devices/pci0000:00/0000:00:09.0/0000:01:00.0/mlx5_core.pf.0 mlx5_core.vf.0 -> ../../../devices/pci0000:00/0000:00:09.0/0000:01:00.1/mlx5_core.vf.0 mlx5_core.vf.1 -> ../../../devices/pci0000:00/0000:00:09.0/0000:01:00.2/mlx5_core.vf.1 mlx5_core.vf.2 -> ../../../devices/pci0000:00/0000:00:09.0/0000:01:00.3/mlx5_core.vf.2 mlx5_core.vf.3 -> ../../../devices/pci0000:00/0000:00:09.0/0000:01:00.4/mlx5_core.vf.3

The mlx5_ib, mlx5_vdpa and mlx5_en will connect to one of mlx5_core.XX.YY devices.

Thanks

#9 lore raw

RE: [PATCH 1/6] Add ancillary bus support

Quoted 133 lines

From: Leon Romanovsky <[email protected]>
Sent: Friday, October 2, 2020 11:50 AM

Quoted 15 lines · Parav Pandit
Quoted 15 lines

From: Leon Romanovsky <[email protected]>
Sent: Friday, October 2, 2020 1:02 AM

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

Review

Quoted 6 lines
(quoted diff) +6 -0
+Registering an ancillary_device is a two-step process.
+First you must call ancillary_device_initialize(), which
+will check several aspects of the ancillary_device struct
+and perform a device_initialize().  After this step
+completes, any error state must have a call to
+put_device() in its

resolution

Review

Quoted 2 lines
(quoted diff) +2 -0
+path.  The second step in registering an ancillary_device
+is to perform a

call

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+to ancillary_device_add(), which will set the name of the
+device and add

the

Review

Quoted 3 lines
(quoted diff) +3 -0
+device to the bus.
+
+To unregister an ancillary_device, just a call to

ancillary_device_unregister()

Quoted 1 line

+is used. This will perform both a device_del() and a

put_device().

Quoted 17 lines
Quoted 17 lines
Quoted 17 lines
Quoted 17 lines
Quoted 3 lines
Quoted 3 lines

Why did you chose ancillary_device_initialize() and not ancillary_device_register() to be paired with

ancillary_device_unregister()?

Quoted 7 lines
Quoted 2 lines

Thanks

We originally had a single call to ancillary_device_register() that paired with unregister, but there was an ask to separate the register into an initialize and add to make the error condition unwind more

compartimentalized.

It is correct thing to separate, but I would expect:
ancillary_device_register() ancillary_device_add()

Quoted 7 lines
Quoted 6 lines
Quoted 2 lines

device_initialize(), device_add() and device_unregister() is the pattern widely

followed in the core.

It doesn't mean that I need to agree with that, right?

Right. May be I misunderstood your comment where you said "I expect

device_register() and device_add()" in response to "device_initialize() and device_add".

Quoted 4 lines

I interpreted your comment as to replace initialize() with register(). Because that is odd naming and completely out of sync from the core APIs.

A helper like below that wraps initialize() and add() is buggy, because when

register() returns an error, it doesn't know if should do kfree() or put_device().

I wrote above (14 lines above this line) that I understand and support the request to separate init and add parts. There is only one thing that I didn't like that we have _unregister() but don't have _register().
It is not a big deal.

Quoted 5 lines

ancillary_device_register(adev)
{
ret = ancillary_device_initialize();
if (ret)

Quoted 20 lines
    return ret;

ret = ancillary_device_add();
if (ret)

    put_device(dev);

return ret;
}

Quoted 12 lines
Quoted 12 lines
Quoted 10 lines

vs.
ancillary_device_unregister()

It is not a big deal, just curious.

The much more big deal is that I'm required to create 1-to-1 mapping between device and driver, and I can't connect all my different modules to one xxx_core.pf.y device in N-to-1 mapping. "N" represents different protocols (IB, ETH, SCSI) and "1" is one PCI core.

For one mlx5 (pf/vf/sf) device, there are three class erivers (ib, vdpa,

en).

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

So there should be one ida allocate per mlx5 device type.

Static const mlx5_adev_names[MLX5_INTERFACE_PROTOCOL_MAX] =

{

Quoted 16 lines
Quoted 5 lines
Quoted 5 lines
	"rdma",
	"eth",
	"vdpa"
};
Quoted 8 lines
Quoted 1 line

Something like for current mlx5_register_device(),

I know it and already implemented it, this is why I'm saying that it is not what I would expect from the implementation.

It is wrong create mlx5_core.rdma.1 device that is equal to mlx5_core.eth.1 just to connect our mlx5_ib.ko to it, while documentation explains about creating

Ancillary bus's documentation? If so, it should be corrected. Do you have specific text snippet to point to that should be fixed?

(quoted diff) +7 -0
+One example could be a multi-port PCI network device that is
+rdma-capable and needs to export this functionality and attach to an
+rdma driver in another subsystem.  The PCI driver will allocate and
+register an ancillary_device for each physical function on the NIC.
+The rdma driver will register an ancillary_driver that will be matched
+with and probed for each of these ancillary_devices.  This will give
+the rdma driver access to the shared data/ops in the PCI drivers shared

object to establish a connection with the PCI driver.

Quoted 2 lines

For purpose of matching service functionality, for each different class (ib,

eth, vdpa) there is one ancillary device created.

Quoted 2 lines

What exactly is wrong here? (and why is it wrong now and not in previous version of the RFC?)

Here the example of two real systems, see how many links we created to same mlx5_core PCI logic. I imagine that Qlogic and Chelsio drivers will look even worse, because they spread over more subsystems than mlx5.

Quoted 7 lines

This is first time when I see real implementation of real device.

System with 2 IB and 1 RoCE cards:
[leonro@vm ~]$ ls -l /sys/bus/ancillary/devices/
mlx5_core.eth.0 -> ../../../devices/pci0000:00/0000:00:09.0/mlx5_core.eth.0 mlx5_core.eth.1 -> ../../../devices/pci0000:00/0000:00:0a.0/mlx5_core.eth.1 mlx5_core.eth.2 -> ../../../devices/pci0000:00/0000:00:0b.0/mlx5_core.eth.2

This gives you the ability to not load the netdevice and rdma device of a VF and only load the vdpa device. These are real use case that users have asked for.
In use case one, they are only interested in rdma device.
In second use case only vdpa device.
How shall one achieve that without spinning of the device for each class?

Quoted 29 lines

mlx5_core.ib.0 -> ../../../devices/pci0000:00/0000:00:09.0/mlx5_core.ib.0 mlx5_core.ib.1 -> ../../../devices/pci0000:00/0000:00:0a.0/mlx5_core.ib.1 mlx5_core.ib.2 -> ../../../devices/pci0000:00/0000:00:0b.0/mlx5_core.ib.2 mlx5_core.vdpa.0 -> ../../../devices/pci0000:00/0000:00:09.0/mlx5_core.ib.0 mlx5_core.vdpa.1 -> ../../../devices/pci0000:00/0000:00:0a.0/mlx5_core.ib.1 mlx5_core.vdpa.2 -> ../../../devices/pci0000:00/0000:00:0b.0/mlx5_core.ib.2
[leonro@vm ~]$ rdma dev
0: ibp0s9: node_type ca fw 4.6.9999 node_guid 5254:00c0:fe12:3455 sys_image_guid 5254:00c0:fe12:3455
1: ibp0s10: node_type ca fw 4.6.9999 node_guid 5254:00c0:fe12:3456 sys_image_guid 5254:00c0:fe12:3456
2: rdmap0s11: node_type ca fw 4.6.9999 node_guid 5254:00c0:fe12:3457 sys_image_guid 5254:00c0:fe12:3457

System with RoCE SR-IOV card with 4 VFs:

Maybe, what I would like to see is:
[leonro@vm ~]$ ls -l /sys/bus/ancillary/devices/
mlx5_core.pf.0 ->
../../../devices/pci0000:00/0000:00:09.0/0000:01:00.0/mlx5_core.pf.0
mlx5_core.vf.0 ->
../../../devices/pci0000:00/0000:00:09.0/0000:01:00.1/mlx5_core.vf.0
mlx5_core.vf.1 ->
../../../devices/pci0000:00/0000:00:09.0/0000:01:00.2/mlx5_core.vf.1
mlx5_core.vf.2 ->
../../../devices/pci0000:00/0000:00:09.0/0000:01:00.3/mlx5_core.vf.2
mlx5_core.vf.3 ->
../../../devices/pci0000:00/0000:00:09.0/0000:01:00.4/mlx5_core.vf.3

Quoted 2 lines

The mlx5_ib, mlx5_vdpa and mlx5_en will connect to one of mlx5_core.XX.YY devices.

In that case, I really don't see the need of creating ancillary device at all.
A generic wrapper around blocking_notifier_chain_register() with a notion of id, and some well defined structure as library can serve the purpose. But it will miss out power suspend() resume() hooks, which we get for free now using ancillary device, in addition to the ability to selectively load only few class device. Each 'struct device's is close to 744 bytes in size in 5.9 kernel.
Is so many 'struct device' of ancillary type a real problem, that aims to address these use cases?

Quoted 2 lines

Thanks

#10 lore raw

Re: [PATCH 1/6] Add ancillary bus support

Quoted 140 lines · Parav Pandit
Quoted 133 lines

From: Leon Romanovsky <[email protected]>
Sent: Friday, October 2, 2020 11:50 AM

Quoted 15 lines · Parav Pandit
Quoted 15 lines

From: Leon Romanovsky <[email protected]>
Sent: Friday, October 2, 2020 1:02 AM

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

Review

Quoted 6 lines
(quoted diff) +6 -0
+Registering an ancillary_device is a two-step process.
+First you must call ancillary_device_initialize(), which
+will check several aspects of the ancillary_device struct
+and perform a device_initialize().  After this step
+completes, any error state must have a call to
+put_device() in its

resolution

Review

Quoted 2 lines
(quoted diff) +2 -0
+path.  The second step in registering an ancillary_device
+is to perform a

call

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+to ancillary_device_add(), which will set the name of the
+device and add

the

Review

Quoted 3 lines
(quoted diff) +3 -0
+device to the bus.
+
+To unregister an ancillary_device, just a call to

ancillary_device_unregister()

Quoted 1 line

+is used. This will perform both a device_del() and a

put_device().

Quoted 17 lines
Quoted 17 lines
Quoted 17 lines
Quoted 17 lines
Quoted 3 lines
Quoted 3 lines

Why did you chose ancillary_device_initialize() and not ancillary_device_register() to be paired with

ancillary_device_unregister()?

Quoted 7 lines
Quoted 2 lines

Thanks

We originally had a single call to ancillary_device_register() that paired with unregister, but there was an ask to separate the register into an initialize and add to make the error condition unwind more

compartimentalized.

It is correct thing to separate, but I would expect:
ancillary_device_register() ancillary_device_add()

Quoted 7 lines
Quoted 6 lines
Quoted 2 lines

device_initialize(), device_add() and device_unregister() is the pattern widely

followed in the core.

It doesn't mean that I need to agree with that, right?

Right. May be I misunderstood your comment where you said "I expect

device_register() and device_add()" in response to "device_initialize() and device_add".

Quoted 4 lines

I interpreted your comment as to replace initialize() with register(). Because that is odd naming and completely out of sync from the core APIs.

A helper like below that wraps initialize() and add() is buggy, because when

register() returns an error, it doesn't know if should do kfree() or put_device().

I wrote above (14 lines above this line) that I understand and support the request to separate init and add parts. There is only one thing that I didn't like that we have _unregister() but don't have _register().
It is not a big deal.

Quoted 5 lines

ancillary_device_register(adev)
{
ret = ancillary_device_initialize();
if (ret)

Quoted 20 lines
    return ret;

ret = ancillary_device_add();
if (ret)

    put_device(dev);

return ret;
}

Quoted 12 lines
Quoted 12 lines
Quoted 10 lines

vs.
ancillary_device_unregister()

It is not a big deal, just curious.

The much more big deal is that I'm required to create 1-to-1 mapping between device and driver, and I can't connect all my different modules to one xxx_core.pf.y device in N-to-1 mapping. "N" represents different protocols (IB, ETH, SCSI) and "1" is one PCI core.

For one mlx5 (pf/vf/sf) device, there are three class erivers (ib, vdpa,

en).

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

So there should be one ida allocate per mlx5 device type.

Static const mlx5_adev_names[MLX5_INTERFACE_PROTOCOL_MAX] =

{

Quoted 16 lines
Quoted 5 lines
Quoted 5 lines
	"rdma",
	"eth",
	"vdpa"
};
Quoted 8 lines
Quoted 1 line

Something like for current mlx5_register_device(),

I know it and already implemented it, this is why I'm saying that it is not what I would expect from the implementation.

It is wrong create mlx5_core.rdma.1 device that is equal to mlx5_core.eth.1 just to connect our mlx5_ib.ko to it, while documentation explains about creating

Ancillary bus's documentation? If so, it should be corrected. Do you have specific text snippet to point to that should be fixed?

(quoted diff) +7 -0
+One example could be a multi-port PCI network device that is
+rdma-capable and needs to export this functionality and attach to an
+rdma driver in another subsystem.  The PCI driver will allocate and
+register an ancillary_device for each physical function on the NIC.
+The rdma driver will register an ancillary_driver that will be matched
+with and probed for each of these ancillary_devices.  This will give
+the rdma driver access to the shared data/ops in the PCI drivers shared

object to establish a connection with the PCI driver.

Quoted 2 lines

For purpose of matching service functionality, for each different class (ib,

eth, vdpa) there is one ancillary device created.

Quoted 2 lines

What exactly is wrong here? (and why is it wrong now and not in previous version of the RFC?)

Here the example of two real systems, see how many links we created to same mlx5_core PCI logic. I imagine that Qlogic and Chelsio drivers will look even worse, because they spread over more subsystems than mlx5.

Quoted 7 lines

This is first time when I see real implementation of real device.

System with 2 IB and 1 RoCE cards:
[leonro@vm ~]$ ls -l /sys/bus/ancillary/devices/
mlx5_core.eth.0 -> ../../../devices/pci0000:00/0000:00:09.0/mlx5_core.eth.0 mlx5_core.eth.1 -> ../../../devices/pci0000:00/0000:00:0a.0/mlx5_core.eth.1 mlx5_core.eth.2 -> ../../../devices/pci0000:00/0000:00:0b.0/mlx5_core.eth.2

Quoted 5 lines

This gives you the ability to not load the netdevice and rdma device of a VF and only load the vdpa device. These are real use case that users have asked for.
In use case one, they are only interested in rdma device.
In second use case only vdpa device.
How shall one achieve that without spinning of the device for each class?

Why will it be different if ancillary device is small PCI core? If you want RDMA, you will use specific ancillary driver that connects to that small PCI logic.

Quoted 30 lines
Quoted 30 lines

mlx5_core.ib.0 -> ../../../devices/pci0000:00/0000:00:09.0/mlx5_core.ib.0 mlx5_core.ib.1 -> ../../../devices/pci0000:00/0000:00:0a.0/mlx5_core.ib.1 mlx5_core.ib.2 -> ../../../devices/pci0000:00/0000:00:0b.0/mlx5_core.ib.2 mlx5_core.vdpa.0 -> ../../../devices/pci0000:00/0000:00:09.0/mlx5_core.ib.0 mlx5_core.vdpa.1 -> ../../../devices/pci0000:00/0000:00:0a.0/mlx5_core.ib.1 mlx5_core.vdpa.2 -> ../../../devices/pci0000:00/0000:00:0b.0/mlx5_core.ib.2
[leonro@vm ~]$ rdma dev
0: ibp0s9: node_type ca fw 4.6.9999 node_guid 5254:00c0:fe12:3455 sys_image_guid 5254:00c0:fe12:3455
1: ibp0s10: node_type ca fw 4.6.9999 node_guid 5254:00c0:fe12:3456 sys_image_guid 5254:00c0:fe12:3456
2: rdmap0s11: node_type ca fw 4.6.9999 node_guid 5254:00c0:fe12:3457 sys_image_guid 5254:00c0:fe12:3457

System with RoCE SR-IOV card with 4 VFs:

Maybe, what I would like to see is:
[leonro@vm ~]$ ls -l /sys/bus/ancillary/devices/
mlx5_core.pf.0 ->
../../../devices/pci0000:00/0000:00:09.0/0000:01:00.0/mlx5_core.pf.0
mlx5_core.vf.0 ->
../../../devices/pci0000:00/0000:00:09.0/0000:01:00.1/mlx5_core.vf.0
mlx5_core.vf.1 ->
../../../devices/pci0000:00/0000:00:09.0/0000:01:00.2/mlx5_core.vf.1
mlx5_core.vf.2 ->
../../../devices/pci0000:00/0000:00:09.0/0000:01:00.3/mlx5_core.vf.2
mlx5_core.vf.3 ->
../../../devices/pci0000:00/0000:00:09.0/0000:01:00.4/mlx5_core.vf.3

Quoted 4 lines
Quoted 2 lines

The mlx5_ib, mlx5_vdpa and mlx5_en will connect to one of mlx5_core.XX.YY devices.

In that case, I really don't see the need of creating ancillary device at all.
A generic wrapper around blocking_notifier_chain_register() with a notion of id, and some well defined structure as library can serve the purpose.

Ancillary bus provides generic implementation instead of existing copy/paste code.

Quoted 1 line

But it will miss out power suspend() resume() hooks, which we get for free now using ancillary device, in addition to the ability to selectively load only few class device.

While I'm interested to load specific device and limit supported classes later and not vice-versa as it is now.

Quoted 2 lines

Each 'struct device's is close to 744 bytes in size in 5.9 kernel.
Is so many 'struct device' of ancillary type a real problem, that aims to address these use cases?

Being nice to the users and provide clean abstraction are important goals too.

Thanks

Quoted 3 lines
Quoted 3 lines

Thanks

#11 lore raw

RE: [PATCH 1/6] Add ancillary bus support

Quoted 4 lines

From: Leon Romanovsky <[email protected]>
Sent: Friday, October 2, 2020 4:44 PM

[..]

Quoted 11 lines
Quoted 2 lines
Quoted 1 line

../../../devices/pci0000:00/0000:00:0b.0/mlx5_core.eth.2

This gives you the ability to not load the netdevice and rdma device of a VF

and only load the vdpa device.

Quoted 4 lines

These are real use case that users have asked for.
In use case one, they are only interested in rdma device.
In second use case only vdpa device.
How shall one achieve that without spinning of the device for each class?

Why will it be different if ancillary device is small PCI core? If you want RDMA, you will use specific ancillary driver that connects to that small PCI logic.

I didn't follow, wwhat is PCI core and PCI logic in this context?

Not sure if you understood the use case.
Let me try again.
Let say there are 4 VFs enabled.
User would not like to create netdev for 3 VFs (0 to 2) ; user only wants rdma device for these VFs 0 to 2. User wants only vdpa device for 4th VF.
User doesn't want to create rdma device and netdevice for the 4th VF.
How one shall achieve this?
It is easily achievable with current ancillary device instantiation per class proposal.

Quoted 1 line

Being nice to the users and provide clean abstraction are important goals too.

Which part of this makes not_nice_to_users and what is not abstracted. I lost you.

#12 lore raw

Re: [PATCH 1/6] Add ancillary bus support

Quoted 14 lines · Parav Pandit
Quoted 4 lines

From: Leon Romanovsky <[email protected]>
Sent: Friday, October 2, 2020 4:44 PM

[..]

Quoted 8 lines
Quoted 2 lines
Quoted 1 line

../../../devices/pci0000:00/0000:00:0b.0/mlx5_core.eth.2

This gives you the ability to not load the netdevice and rdma device of a VF

and only load the vdpa device.

Quoted 4 lines

These are real use case that users have asked for.
In use case one, they are only interested in rdma device.
In second use case only vdpa device.
How shall one achieve that without spinning of the device for each class?

Quoted 5 lines
Quoted 3 lines

Why will it be different if ancillary device is small PCI core? If you want RDMA, you will use specific ancillary driver that connects to that small PCI logic.

I didn't follow, wwhat is PCI core and PCI logic in this context?

mlx5_core is PCI core/logic - ancillary device
mlx5_ib/mlx5_en/mlx5_vdpa - ancillary drivers

Quoted 8 lines

Not sure if you understood the use case.
Let me try again.
Let say there are 4 VFs enabled.
User would not like to create netdev for 3 VFs (0 to 2) ; user only wants rdma device for these VFs 0 to 2. User wants only vdpa device for 4th VF.
User doesn't want to create rdma device and netdevice for the 4th VF.
How one shall achieve this?

It depends on how N-to-1 bus will be implemented. For example, devlink already allows to disable RoCE on specific function, nothing prohibits to extend it to support other classes.

Quoted 1 line

It is easily achievable with current ancillary device instantiation per class proposal.

It is byproduct of 1-to-1 connection and not specific design decision.

Quoted 4 lines
Quoted 2 lines

Being nice to the users and provide clean abstraction are important goals too.

Which part of this makes not_nice_to_users and what is not abstracted. I lost you.

Your use case perfectly presented not_nice_to_users thing. Users are interested to work on functions (VF/PF/SF) and configure them without need to dig into the sysfs directories to connect ancillary classes and their indexes to the real functions.

Thanks

#13 lore raw

RE: [PATCH 1/6] Add ancillary bus support

Quoted 36 lines

From: Leon Romanovsky <[email protected]>
Sent: Friday, October 2, 2020 5:16 PM

Quoted 6 lines · Parav Pandit
Quoted 4 lines

From: Leon Romanovsky <[email protected]>
Sent: Friday, October 2, 2020 4:44 PM

[..]

Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
Quoted 1 line

../../../devices/pci0000:00/0000:00:0b.0/mlx5_core.eth.2

This gives you the ability to not load the netdevice and rdma device of a VF

and only load the vdpa device.

Quoted 4 lines

These are real use case that users have asked for.
In use case one, they are only interested in rdma device. In second use case only vdpa device.
How shall one achieve that without spinning of the device for each

class?

Quoted 6 lines
Quoted 4 lines

Why will it be different if ancillary device is small PCI core? If you want RDMA, you will use specific ancillary driver that connects to that small PCI logic.

I didn't follow, wwhat is PCI core and PCI logic in this context?

mlx5_core is PCI core/logic - ancillary device mlx5_ib/mlx5_en/mlx5_vdpa - ancillary drivers

Quoted 5 lines

Not sure if you understood the use case.
Let me try again.
Let say there are 4 VFs enabled.
User would not like to create netdev for 3 VFs (0 to 2) ; user only wants

Quoted 8 lines

rdma device for these VFs 0 to 2.

Quoted 3 lines

User wants only vdpa device for 4th VF.
User doesn't want to create rdma device and netdevice for the 4th VF. How one shall achieve this?

It depends on how N-to-1 bus will be implemented. For example, devlink already allows to disable RoCE on specific function, nothing prohibits to extend it to support other classes.

Sure. Please go through the our internal RFC dated 7/7/20 with subject "devlink device params to disable rdma, netdev" which exactly achieves similar.

And recent 3rd internal RFC "devlink to set driver parameters" discussion with Eli, Jason, Jiri and others that describes to have one ancillary_device_per_class instead of devlink.

Lets discuss it offline as we have multiple proposals floating.

Quoted 18 lines
Quoted 2 lines

It is easily achievable with current ancillary device instantiation per class

proposal.

It is byproduct of 1-to-1 connection and not specific design decision.

Quoted 2 lines
Quoted 2 lines

Being nice to the users and provide clean abstraction are important goals

too.

Quoted 2 lines

Which part of this makes not_nice_to_users and what is not abstracted. I lost you.

Your use case perfectly presented not_nice_to_users thing. Users are interested to work on functions (VF/PF/SF) and configure them without need to dig into the sysfs directories to connect ancillary classes and their indexes to the real functions.

Thanks

#14 lore raw

PATCH 2/6 ASoC: SOF: Introduce descriptors for SOF client

In reply to Dave Ertman

From: Ranjani Sridharan <[email protected]>

A client in the SOF (Sound Open Firmware) context is a device that needs to communicate with the DSP via IPC messages. The SOF core is responsible for serializing the IPC messages to the DSP from the different clients. One example of an SOF client would be an IPC test client that floods the DSP with test IPC messages to validate if the serialization works as expected. Multi-client support will also add the ability to split the existing audio cards into multiple ones, so as to e.g. to deal with HDMI with a dedicated client instead of adding HDMI to all cards.

This patch introduces descriptors for SOF client driver and SOF client device along with APIs for registering and unregistering a SOF client driver, sending IPCs from a client device and accessing the SOF core debugfs root entry.

Along with this, add a couple of new members to struct snd_sof_dev that will be used for maintaining the list of clients.

---
 sound/soc/sof/Kconfig      |  19 ++++++
 sound/soc/sof/Makefile     |   3 +
 sound/soc/sof/core.c       |   2 +
 sound/soc/sof/sof-client.c | 117 +++++++++++++++++++++++++++++++++++++
 sound/soc/sof/sof-client.h |  65 +++++++++++++++++++++
 sound/soc/sof/sof-priv.h   |   6 ++
 6 files changed, 212 insertions(+)
 create mode 100644 sound/soc/sof/sof-client.c
 create mode 100644 sound/soc/sof/sof-client.h
sound/soc/sof/Kconfig +19 -0
index 4dda4b62509f..cea7efedafef 100644
@@ -50,6 +50,24 @@ config SND_SOC_SOF_DEBUG_PROBES
 	  Say Y if you want to enable probes.
 	  If unsure, select "N".
 
+config SND_SOC_SOF_CLIENT
+	tristate
+	select ANCILLARY_BUS
+	help
+	  This option is not user-selectable but automagically handled by
+	  'select' statements at a higher level
+
+config SND_SOC_SOF_CLIENT_SUPPORT
+	bool "SOF enable clients"
+	depends on SND_SOC_SOF
+	help
+	  This adds support for ancillary client devices to separate out the debug
+	  functionality for IPC tests, probes etc. into separate devices. This
+	  option would also allow adding client devices based on DSP FW
+	  capabilities and ACPI/OF device information.
+	  Say Y if you want to enable clients with SOF.
+	  If unsure select "N".
+
 config SND_SOC_SOF_DEVELOPER_SUPPORT
 	bool "SOF developer options support"
 	depends on EXPERT
@@ -186,6 +204,7 @@ endif ## SND_SOC_SOF_DEVELOPER_SUPPORT
 
 config SND_SOC_SOF
 	tristate
+	select SND_SOC_SOF_CLIENT if SND_SOC_SOF_CLIENT_SUPPORT
 	select SND_SOC_TOPOLOGY
 	select SND_SOC_SOF_NOCODEC if SND_SOC_SOF_NOCODEC_SUPPORT
 	help
sound/soc/sof/Makefile +3 -0
index 05718dfe6cd2..5e46f25a3851 100644
@@ -2,6 +2,7 @@
 
 snd-sof-objs := core.o ops.o loader.o ipc.o pcm.o pm.o debug.o topology.o\
 		control.o trace.o utils.o sof-audio.o
+snd-sof-client-objs := sof-client.o
 snd-sof-$(CONFIG_SND_SOC_SOF_DEBUG_PROBES) += probe.o compress.o
 
 snd-sof-pci-objs := sof-pci-dev.o
@@ -18,6 +19,8 @@ obj-$(CONFIG_SND_SOC_SOF_ACPI) += snd-sof-acpi.o
 obj-$(CONFIG_SND_SOC_SOF_OF) += snd-sof-of.o
 obj-$(CONFIG_SND_SOC_SOF_PCI) += snd-sof-pci.o
 
+obj-$(CONFIG_SND_SOC_SOF_CLIENT) += snd-sof-client.o
+
 obj-$(CONFIG_SND_SOC_SOF_INTEL_TOPLEVEL) += intel/
 obj-$(CONFIG_SND_SOC_SOF_IMX_TOPLEVEL) += imx/
 obj-$(CONFIG_SND_SOC_SOF_XTENSA) += xtensa/
sound/soc/sof/core.c +2 -0
index adc7c37145d6..72a97219395f 100644
@@ -314,8 +314,10 @@ int snd_sof_device_probe(struct device *dev, struct snd_sof_pdata *plat_data)
 	INIT_LIST_HEAD(&sdev->widget_list);
 	INIT_LIST_HEAD(&sdev->dai_list);
 	INIT_LIST_HEAD(&sdev->route_list);
+	INIT_LIST_HEAD(&sdev->client_list);
 	spin_lock_init(&sdev->ipc_lock);
 	spin_lock_init(&sdev->hw_lock);
+	mutex_init(&sdev->client_mutex);
 
 	if (IS_ENABLED(CONFIG_SND_SOC_SOF_PROBE_WORK_QUEUE))
 		INIT_WORK(&sdev->probe_work, sof_probe_work);
sound/soc/sof/sof-client.c new file +117 -0

117 lines not on this page · raw message

sound/soc/sof/sof-client.h new file +65 -0

65 lines not on this page · raw message

sound/soc/sof/sof-priv.h +6 -0
index 64f28e082049..043fcec5a288 100644
@@ -438,6 +438,12 @@ struct snd_sof_dev {
 
 	bool msi_enabled;
 
+	/* list of client devices */
+	struct list_head client_list;
+
+	/* mutex to protect client list */
+	struct mutex client_mutex;
+
 	void *private;			/* core does not touch this */
 };
 
Signature · 1 line
2.26.2
#15 lore raw

PATCH 3/6 ASoC: SOF: Create client driver for IPC test

In reply to Dave Ertman

From: Ranjani Sridharan <[email protected]>

Create an SOF client driver for IPC flood test. This driver is used to set up the debugfs entries and the read/write ops for initiating the IPC flood test that would be used to measure the min/max/avg response times for sending IPCs to the DSP.

---
 sound/soc/sof/Kconfig               |  10 +
 sound/soc/sof/Makefile              |   4 +
 sound/soc/sof/sof-ipc-test-client.c | 314 ++++++++++++++++++++++++++++
 3 files changed, 328 insertions(+)
 create mode 100644 sound/soc/sof/sof-ipc-test-client.c
sound/soc/sof/Kconfig +10 -0
index cea7efedafef..55a2a20c3ec9 100644
@@ -190,6 +190,16 @@ config SND_SOC_SOF_DEBUG_IPC_FLOOD_TEST
 	  Say Y if you want to enable IPC flood test.
 	  If unsure, select "N".
 
+config SND_SOC_SOF_DEBUG_IPC_FLOOD_TEST_CLIENT
+	tristate "SOF enable IPC flood test client"
+	depends on SND_SOC_SOF_CLIENT
+	help
+	  This option enables a separate client device for IPC flood test
+	  which can be used to flood the DSP with test IPCs and gather stats
+	  about response times.
+	  Say Y if you want to enable IPC flood test.
+	  If unsure, select "N".
+
 config SND_SOC_SOF_DEBUG_RETAIN_DSP_CONTEXT
 	bool "SOF retain DSP context on any FW exceptions"
 	help
sound/soc/sof/Makefile +4 -0
index 5e46f25a3851..baa93fe2cc9a 100644
@@ -9,6 +9,8 @@ snd-sof-pci-objs := sof-pci-dev.o
 snd-sof-acpi-objs := sof-acpi-dev.o
 snd-sof-of-objs := sof-of-dev.o
 
+snd-sof-ipc-test-objs := sof-ipc-test-client.o
+
 snd-sof-nocodec-objs := nocodec.o
 
 obj-$(CONFIG_SND_SOC_SOF) += snd-sof.o
@@ -21,6 +23,8 @@ obj-$(CONFIG_SND_SOC_SOF_PCI) += snd-sof-pci.o
 
 obj-$(CONFIG_SND_SOC_SOF_CLIENT) += snd-sof-client.o
 
+obj-$(CONFIG_SND_SOC_SOF_DEBUG_IPC_FLOOD_TEST_CLIENT) += snd-sof-ipc-test.o
+
 obj-$(CONFIG_SND_SOC_SOF_INTEL_TOPLEVEL) += intel/
 obj-$(CONFIG_SND_SOC_SOF_IMX_TOPLEVEL) += imx/
 obj-$(CONFIG_SND_SOC_SOF_XTENSA) += xtensa/
sound/soc/sof/sof-ipc-test-client.c new file +314 -0

314 lines not on this page · raw message

Signature · 1 line
2.26.2
#16 lore raw

PATCH 4/6 ASoC: SOF: ops: Add ops for client registration

In reply to Dave Ertman

From: Ranjani Sridharan <[email protected]>

Add new ops for registering/unregistering clients based on DSP capabilities and/or DT information.

---
 sound/soc/sof/core.c     | 10 ++++++++++
 sound/soc/sof/ops.h      | 14 ++++++++++++++
 sound/soc/sof/sof-priv.h |  4 ++++
 3 files changed, 28 insertions(+)
sound/soc/sof/core.c +10 -0
index 72a97219395f..ddb9a12d5aac 100644
@@ -246,8 +246,17 @@ static int sof_probe_continue(struct snd_sof_dev *sdev)
 	if (plat_data->sof_probe_complete)
 		plat_data->sof_probe_complete(sdev->dev);
 
+	/* If registering certain clients fails, unregister the previously registered clients. */
+	ret = snd_sof_register_clients(sdev);
+	if (ret < 0) {
+		dev_err(sdev->dev, "error: failed to register clients %d\n", ret);
+		goto client_reg_err;
+	}
+
 	return 0;
 
+client_reg_err:
+	snd_sof_unregister_clients(sdev);
 fw_trace_err:
 	snd_sof_free_trace(sdev);
 fw_run_err:
@@ -356,6 +365,7 @@ int snd_sof_device_remove(struct device *dev)
 			dev_warn(dev, "error: %d failed to prepare DSP for device removal",
 				 ret);
 
+		snd_sof_unregister_clients(sdev);
 		snd_sof_fw_unload(sdev);
 		snd_sof_ipc_free(sdev);
 		snd_sof_free_debug(sdev);
sound/soc/sof/ops.h +14 -0
index b21632f5511a..0e5660d7a2fd 100644
@@ -470,6 +470,20 @@ snd_sof_set_mach_params(const struct snd_soc_acpi_mach *mach,
 		sof_ops(sdev)->set_mach_params(mach, dev);
 }
 
+static inline int snd_sof_register_clients(struct snd_sof_dev *sdev)
+{
+	if (sof_ops(sdev) && sof_ops(sdev)->register_clients)
+		return sof_ops(sdev)->register_clients(sdev);
+
+	return 0;
+}
+
+static inline void snd_sof_unregister_clients(struct snd_sof_dev *sdev)
+{
+	if (sof_ops(sdev) && sof_ops(sdev)->unregister_clients)
+		return sof_ops(sdev)->unregister_clients(sdev);
+}
+
 static inline const struct snd_sof_dsp_ops
 *sof_get_ops(const struct sof_dev_desc *d,
 	     const struct sof_ops_table mach_ops[], int asize)
sound/soc/sof/sof-priv.h +4 -0
index 043fcec5a288..151614224f47 100644
@@ -249,6 +249,10 @@ struct snd_sof_dsp_ops {
 	void (*set_mach_params)(const struct snd_soc_acpi_mach *mach,
 				struct device *dev); /* optional */
 
+	/* client ops */
+	int (*register_clients)(struct snd_sof_dev *sdev); /* optional */
+	void (*unregister_clients)(struct snd_sof_dev *sdev); /* optional */
+
 	/* DAI ops */
 	struct snd_soc_dai_driver *drv;
 	int num_drv;
Signature · 1 line
2.26.2
#17 lore raw

PATCH 5/6 ASoC: SOF: Intel: Define ops for client registration

In reply to Dave Ertman

From: Ranjani Sridharan <[email protected]>

Define client ops for Intel platforms. For now, we only add 2 IPC test clients that will be used for run tandem IPC flood tests for.

For ACPI platforms, change the Kconfig to select SND_SOC_SOF_PROBE_WORK_QUEUE to allow the ancillary driver to probe when the client is registered.

---
 sound/soc/sof/intel/Kconfig        |  9 ++++++
 sound/soc/sof/intel/Makefile       |  3 ++
 sound/soc/sof/intel/apl.c          | 18 +++++++++++
 sound/soc/sof/intel/bdw.c          | 18 +++++++++++
 sound/soc/sof/intel/byt.c          | 22 ++++++++++++++
 sound/soc/sof/intel/cnl.c          | 18 +++++++++++
 sound/soc/sof/intel/intel-client.c | 49 ++++++++++++++++++++++++++++++
 sound/soc/sof/intel/intel-client.h | 26 ++++++++++++++++
 8 files changed, 163 insertions(+)
 create mode 100644 sound/soc/sof/intel/intel-client.c
 create mode 100644 sound/soc/sof/intel/intel-client.h
sound/soc/sof/intel/Kconfig +9 -0
index 3aaf25e4f766..28aba42f4658 100644
@@ -13,6 +13,8 @@ config SND_SOC_SOF_INTEL_ACPI
 	def_tristate SND_SOC_SOF_ACPI
 	select SND_SOC_SOF_BAYTRAIL  if SND_SOC_SOF_BAYTRAIL_SUPPORT
 	select SND_SOC_SOF_BROADWELL if SND_SOC_SOF_BROADWELL_SUPPORT
+	select SND_SOC_SOF_PROBE_WORK_QUEUE if SND_SOC_SOF_CLIENT
+	select SND_SOC_SOF_INTEL_CLIENT if SND_SOC_SOF_CLIENT
 	help
 	  This option is not user-selectable but automagically handled by
 	  'select' statements at a higher level
@@ -29,6 +31,7 @@ config SND_SOC_SOF_INTEL_PCI
 	select SND_SOC_SOF_TIGERLAKE   if SND_SOC_SOF_TIGERLAKE_SUPPORT
 	select SND_SOC_SOF_ELKHARTLAKE if SND_SOC_SOF_ELKHARTLAKE_SUPPORT
 	select SND_SOC_SOF_JASPERLAKE  if SND_SOC_SOF_JASPERLAKE_SUPPORT
+	select SND_SOC_SOF_INTEL_CLIENT if SND_SOC_SOF_CLIENT
 	help
 	  This option is not user-selectable but automagically handled by
 	  'select' statements at a higher level
@@ -57,6 +60,12 @@ config SND_SOC_SOF_INTEL_COMMON
 	  This option is not user-selectable but automagically handled by
 	  'select' statements at a higher level
 
+config SND_SOC_SOF_INTEL_CLIENT
+	tristate
+	help
+	  This option is not user-selectable but automagically handled by
+	  'select' statements at a higher level
+
 if SND_SOC_SOF_INTEL_ACPI
 
 config SND_SOC_SOF_BAYTRAIL_SUPPORT
sound/soc/sof/intel/Makefile +3 -0
index f7e9358f1f06..50e40caaa787 100644
@@ -5,6 +5,8 @@ snd-sof-intel-bdw-objs := bdw.o
 
 snd-sof-intel-ipc-objs := intel-ipc.o
 
+snd-sof-intel-client-objs := intel-client.o
+
 snd-sof-intel-hda-common-objs := hda.o hda-loader.o hda-stream.o hda-trace.o \
 				 hda-dsp.o hda-ipc.o hda-ctrl.o hda-pcm.o \
 				 hda-dai.o hda-bus.o \
@@ -18,3 +20,4 @@ obj-$(CONFIG_SND_SOC_SOF_BROADWELL) += snd-sof-intel-bdw.o
 obj-$(CONFIG_SND_SOC_SOF_INTEL_HIFI_EP_IPC) += snd-sof-intel-ipc.o
 obj-$(CONFIG_SND_SOC_SOF_HDA_COMMON) += snd-sof-intel-hda-common.o
 obj-$(CONFIG_SND_SOC_SOF_HDA) += snd-sof-intel-hda.o
+obj-$(CONFIG_SND_SOC_SOF_INTEL_CLIENT) += snd-sof-intel-client.o
sound/soc/sof/intel/apl.c +18 -0
index 9e29d4fd393a..b31353b1a3ea 100644
@@ -15,9 +15,12 @@
  * Hardware interface for audio DSP on Apollolake and GeminiLake
  */
 
+#include <linux/list.h>
 #include "../sof-priv.h"
 #include "hda.h"
 #include "../sof-audio.h"
+#include "../sof-client.h"
+#include "intel-client.h"
 
 static const struct snd_sof_debugfs_map apl_dsp_debugfs[] = {
 	{"hda", HDA_DSP_HDA_BAR, 0, 0x4000, SOF_DEBUGFS_ACCESS_ALWAYS},
@@ -25,6 +28,16 @@ static const struct snd_sof_debugfs_map apl_dsp_debugfs[] = {
 	{"dsp", HDA_DSP_BAR,  0, 0x10000, SOF_DEBUGFS_ACCESS_ALWAYS},
 };
 
+static int apl_register_clients(struct snd_sof_dev *sdev)
+{
+	return intel_register_ipc_test_clients(sdev);
+}
+
+static void apl_unregister_clients(struct snd_sof_dev *sdev)
+{
+	intel_unregister_ipc_test_clients(sdev);
+}
+
 /* apollolake ops */
 const struct snd_sof_dsp_ops sof_apl_ops = {
 	/* probe and remove */
@@ -101,6 +114,10 @@ const struct snd_sof_dsp_ops sof_apl_ops = {
 	.trace_release = hda_dsp_trace_release,
 	.trace_trigger = hda_dsp_trace_trigger,
 
+	/* client ops */
+	.register_clients = apl_register_clients,
+	.unregister_clients = apl_unregister_clients,
+
 	/* DAI drivers */
 	.drv		= skl_dai,
 	.num_drv	= SOF_SKL_NUM_DAIS,
@@ -140,3 +157,4 @@ const struct sof_intel_dsp_desc apl_chip_info = {
 	.ssp_base_offset = APL_SSP_BASE_OFFSET,
 };
 EXPORT_SYMBOL_NS(apl_chip_info, SND_SOC_SOF_INTEL_HDA_COMMON);
+MODULE_IMPORT_NS(SND_SOC_SOF_INTEL_CLIENT);
sound/soc/sof/intel/bdw.c +18 -0
index 99fd0bd7276e..b14026c5fa97 100644
@@ -12,12 +12,15 @@
  * Hardware interface for audio DSP on Broadwell
  */
 
+#include <linux/list.h>
 #include <linux/module.h>
 #include <sound/sof.h>
 #include <sound/sof/xtensa.h>
 #include "../ops.h"
 #include "shim.h"
 #include "../sof-audio.h"
+#include "../sof-client.h"
+#include "intel-client.h"
 
 /* BARs */
 #define BDW_DSP_BAR 0
@@ -563,6 +566,16 @@ static void bdw_set_mach_params(const struct snd_soc_acpi_mach *mach,
 	mach_params->platform = dev_name(dev);
 }
 
+static int bdw_register_clients(struct snd_sof_dev *sdev)
+{
+	return intel_register_ipc_test_clients(sdev);
+}
+
+static void bdw_unregister_clients(struct snd_sof_dev *sdev)
+{
+	intel_unregister_ipc_test_clients(sdev);
+}
+
 /* Broadwell DAIs */
 static struct snd_soc_dai_driver bdw_dai[] = {
 {
@@ -638,6 +651,10 @@ const struct snd_sof_dsp_ops sof_bdw_ops = {
 	/*Firmware loading */
 	.load_firmware	= snd_sof_load_firmware_memcpy,
 
+	/* client ops */
+	.register_clients = bdw_register_clients,
+	.unregister_clients = bdw_unregister_clients,
+
 	/* DAI drivers */
 	.drv = bdw_dai,
 	.num_drv = ARRAY_SIZE(bdw_dai),
@@ -662,3 +679,4 @@ EXPORT_SYMBOL_NS(bdw_chip_info, SND_SOC_SOF_BROADWELL);
 MODULE_LICENSE("Dual BSD/GPL");
 MODULE_IMPORT_NS(SND_SOC_SOF_INTEL_HIFI_EP_IPC);
 MODULE_IMPORT_NS(SND_SOC_SOF_XTENSA);
+MODULE_IMPORT_NS(SND_SOC_SOF_INTEL_CLIENT);
sound/soc/sof/intel/byt.c +22 -0
index 49f67f1b94e0..8951f756d078 100644
@@ -12,13 +12,16 @@
  * Hardware interface for audio DSP on Baytrail, Braswell and Cherrytrail.
  */
 
+#include <linux/list.h>
 #include <linux/module.h>
 #include <sound/sof.h>
 #include <sound/sof/xtensa.h>
 #include "../ops.h"
 #include "shim.h"
 #include "../sof-audio.h"
+#include "../sof-client.h"
 #include "../../intel/common/soc-intel-quirks.h"
+#include "intel-client.h"
 
 /* DSP memories */
 #define IRAM_OFFSET		0x0C0000
@@ -821,6 +824,16 @@ static int byt_acpi_probe(struct snd_sof_dev *sdev)
 	return ret;
 }
 
+static int byt_register_clients(struct snd_sof_dev *sdev)
+{
+	return intel_register_ipc_test_clients(sdev);
+}
+
+static void byt_unregister_clients(struct snd_sof_dev *sdev)
+{
+	intel_unregister_ipc_test_clients(sdev);
+}
+
 /* baytrail ops */
 const struct snd_sof_dsp_ops sof_byt_ops = {
 	/* device init */
@@ -879,6 +892,10 @@ const struct snd_sof_dsp_ops sof_byt_ops = {
 	.suspend = byt_suspend,
 	.resume = byt_resume,
 
+	/* client ops */
+	.register_clients = byt_register_clients,
+	.unregister_clients = byt_unregister_clients,
+
 	/* DAI drivers */
 	.drv = byt_dai,
 	.num_drv = 3, /* we have only 3 SSPs on byt*/
@@ -958,6 +975,10 @@ const struct snd_sof_dsp_ops sof_cht_ops = {
 	.suspend = byt_suspend,
 	.resume = byt_resume,
 
+	/* client ops */
+	.register_clients = byt_register_clients,
+	.unregister_clients = byt_unregister_clients,
+
 	/* DAI drivers */
 	.drv = byt_dai,
 	/* all 6 SSPs may be available for cherrytrail */
@@ -985,3 +1006,4 @@ EXPORT_SYMBOL_NS(cht_chip_info, SND_SOC_SOF_BAYTRAIL);
 MODULE_LICENSE("Dual BSD/GPL");
 MODULE_IMPORT_NS(SND_SOC_SOF_INTEL_HIFI_EP_IPC);
 MODULE_IMPORT_NS(SND_SOC_SOF_XTENSA);
+MODULE_IMPORT_NS(SND_SOC_SOF_INTEL_CLIENT);
sound/soc/sof/intel/cnl.c +18 -0
index 16db0f50d139..5d7c2a667798 100644
@@ -15,10 +15,13 @@
  * Hardware interface for audio DSP on Cannonlake.
  */
 
+#include <linux/list.h>
 #include "../ops.h"
 #include "hda.h"
 #include "hda-ipc.h"
 #include "../sof-audio.h"
+#include "../sof-client.h"
+#include "intel-client.h"
 
 static const struct snd_sof_debugfs_map cnl_dsp_debugfs[] = {
 	{"hda", HDA_DSP_HDA_BAR, 0, 0x4000, SOF_DEBUGFS_ACCESS_ALWAYS},
@@ -231,6 +234,16 @@ static void cnl_ipc_dump(struct snd_sof_dev *sdev)
 		hipcida, hipctdr, hipcctl);
 }
 
+static int cnl_register_clients(struct snd_sof_dev *sdev)
+{
+	return intel_register_ipc_test_clients(sdev);
+}
+
+static void cnl_unregister_clients(struct snd_sof_dev *sdev)
+{
+	intel_unregister_ipc_test_clients(sdev);
+}
+
 /* cannonlake ops */
 const struct snd_sof_dsp_ops sof_cnl_ops = {
 	/* probe and remove */
@@ -307,6 +320,10 @@ const struct snd_sof_dsp_ops sof_cnl_ops = {
 	.trace_release = hda_dsp_trace_release,
 	.trace_trigger = hda_dsp_trace_trigger,
 
+	/* client ops */
+	.register_clients = cnl_register_clients,
+	.unregister_clients = cnl_unregister_clients,
+
 	/* DAI drivers */
 	.drv		= skl_dai,
 	.num_drv	= SOF_SKL_NUM_DAIS,
@@ -417,3 +434,4 @@ const struct sof_intel_dsp_desc jsl_chip_info = {
 	.ssp_base_offset = CNL_SSP_BASE_OFFSET,
 };
 EXPORT_SYMBOL_NS(jsl_chip_info, SND_SOC_SOF_INTEL_HDA_COMMON);
+MODULE_IMPORT_NS(SND_SOC_SOF_INTEL_CLIENT);
sound/soc/sof/intel/intel-client.c new file +49 -0

49 lines not on this page · raw message

sound/soc/sof/intel/intel-client.h new file +26 -0
new file mode 100644
index 000000000000..49b2c6c0dcc4
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: (GPL-2.0-only OR BSD-3-Clause) */
+/*
+ * This file is provided under a dual BSD/GPLv2 license.  When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * Copyright(c) 2020 Intel Corporation. All rights reserved.
+ *
+ * Author: Ranjani Sridharan <[email protected]>
+ */
+
+#ifndef __INTEL_CLIENT_H
+#define __INTEL_CLIENT_H
+
+#if IS_ENABLED(CONFIG_SND_SOC_SOF_DEBUG_IPC_FLOOD_TEST_CLIENT)
+int intel_register_ipc_test_clients(struct snd_sof_dev *sdev);
+void intel_unregister_ipc_test_clients(struct snd_sof_dev *sdev);
+#else
+static inline int intel_register_ipc_test_clients(struct snd_sof_dev *sdev)
+{
+	return 0;
+}
+
+static void intel_unregister_ipc_test_clients(struct snd_sof_dev *sdev) {}
+#endif
+
+#endif
Signature · 1 line
2.26.2
#18 lore raw

PATCH 6/6 ASoC: SOF: debug: Remove IPC flood test support in SOF core

In reply to Dave Ertman

From: Fred Oh <[email protected]>

Remove the IPC flood test support in the SOF core as it is now added in the IPC flood test client.

---
 sound/soc/sof/Kconfig    |   8 --
 sound/soc/sof/debug.c    | 230 ---------------------------------------
 sound/soc/sof/sof-priv.h |   6 +-
 3 files changed, 1 insertion(+), 243 deletions(-)
sound/soc/sof/Kconfig +0 -8
index 55a2a20c3ec9..4046e96eed92 100644
@@ -182,14 +182,6 @@ config SND_SOC_SOF_DEBUG_ENABLE_FIRMWARE_TRACE
 	  module parameter (similar to dynamic debug)
 	  If unsure, select "N".
 
-config SND_SOC_SOF_DEBUG_IPC_FLOOD_TEST
-	bool "SOF enable IPC flood test"
-	help
-	  This option enables the IPC flood test which can be used to flood
-	  the DSP with test IPCs and gather stats about response times.
-	  Say Y if you want to enable IPC flood test.
-	  If unsure, select "N".
-
 config SND_SOC_SOF_DEBUG_IPC_FLOOD_TEST_CLIENT
 	tristate "SOF enable IPC flood test client"
 	depends on SND_SOC_SOF_CLIENT
sound/soc/sof/debug.c +0 -230

264 lines not on this page · raw message

sound/soc/sof/sof-priv.h +1 -5
index 151614224f47..ece5fce97460 100644
@@ -50,10 +50,6 @@ extern int sof_core_debug;
 #define SOF_FORMATS (SNDRV_PCM_FMTBIT_S16_LE | SNDRV_PCM_FMTBIT_S24_LE | \
 	SNDRV_PCM_FMTBIT_S32_LE | SNDRV_PCM_FMTBIT_FLOAT)
 
-#define ENABLE_DEBUGFS_CACHEBUF \
-	(IS_ENABLED(CONFIG_SND_SOC_SOF_DEBUG_ENABLE_DEBUGFS_CACHE) || \
-	 IS_ENABLED(CONFIG_SND_SOC_SOF_DEBUG_IPC_FLOOD_TEST))
-
 /* DSP power state */
 enum sof_dsp_power_states {
 	SOF_DSP_PM_D0,
@@ -298,7 +294,7 @@ struct snd_sof_dfsentry {
 	 * or if it is accessible only when the DSP is in D0.
 	 */
 	enum sof_debugfs_access_type access_type;
-#if ENABLE_DEBUGFS_CACHEBUF
+#if IS_ENABLED(CONFIG_SND_SOC_SOF_DEBUG_ENABLE_DEBUGFS_CACHE)
 	char *cache_buf; /* buffer to cache the contents of debugfs memory */
 #endif
 	struct snd_sof_dev *sdev;
Signature · 1 line
2.26.2
#19 lore raw
In reply to Dave Ertman

Hi Dave,

I don't know why did you send this series separately to every mailing list, but it is not correct thing to do.

RDMA ML and discussion:
https://lore.kernel.org/linux-rdma/[email protected]/T/#t
Netdev ML (completely separated):
https://lore.kernel.org/netdev/[email protected]/
Alsa ML (separated too):
https://lore.kernel.org/alsa-devel/[email protected]/

Thanks

Quoted 148 lines · Dave Ertman

Brief history of Ancillary Bus
==============================
The ancillary bus code was originally submitted upstream as virtual bus, and was submitted through the netdev tree.  This process generated up to v4.  This discussion can be found here:
https://lore.kernel.org/netdev/[email protected]/T/#u

At this point, GregKH requested that we take the review and revision process to an internal mailing list and garner the buy-in of a respected kernel contributor.

The ancillary bus (then known as virtual bus) was originally submitted along with implementation code for the ice driver and irdma drive, causing the complication of also having dependencies in the rdma tree. This new submission is utilizing an ancillary bus consumer in only the sound driver tree to create the initial implementation and a single user.

Since implementation work has started on this patch set, there have been multiple inquiries about the time frame of its completion. It appears that there will be numerous consumers of this functionality.

The process of internal review and implementation using the sound drivers generated 19 internal versions.  The changes, including the name change from virtual bus to ancillary bus, from these versions can be summarized as the following:

- Fixed compilation and checkpatch errors
- Improved documentation to address the motivation for virtual bus.
- Renamed virtual bus to ancillary bus
- increased maximum device name size
- Correct order in Kconfig and Makefile
- removed the mid-layer adev->release layer for device unregister
- pushed adev->id management to parent driver
- all error paths out of ancillary_device_register return error code
- all error paths out of ancillary_device_register use put_device
- added adev->name element
- modname in register cannot be NULL
- added KBUILD_MODNAME as prefix for match_name
- push adev->id responsibility to registering driver
- uevent now parses adev->dev name
- match_id function now parses adev->dev name
- changed drivers probe function to also take an ancillary_device_id param
- split ancillary_device_register into device_initialize and device_add
- adjusted what is done in device_initialize and device_add
- change adev to ancildev and adrv to ancildrv
- change adev to ancildev in documentation

This submission is the first time that this patch set will be sent to the alsa-devel mailing list, so it is currently being submitted as version 1.

==========================

Introduces the ancillary bus implementation along with the example usage in the Sound Open Firmware(SOF) audio driver.

In some subsystems, the functionality of the core device (PCI/ACPI/other) may be too complex for a single device to be managed as a monolithic block or a part of the functionality might need to be exposed to a different subsystem. Splitting the functionality into smaller orthogonal devices makes it easier to manage data, power management and domain-specific communication with the hardware. Also, common ancillary_device functionality across primary devices can be handled by a common ancillary_device. A key requirement for such a split is that there is no dependency on a physical bus, device, register accesses or regmap support. These individual devices split from the core cannot live on the platform bus as they are not physical devices that are controlled by DT/ACPI. The same argument applies for not using MFD in this scenario as it relies on individual function devices being physical devices that are DT enumerated.

An example for this kind of requirement is the audio subsystem where a single IP handles multiple entities such as HDMI, Soundwire, local devices such as mics/speakers etc. The split for the core's functionality can be arbitrary or be defined by the DSP firmware topology and include hooks for test/debug. This allows for the audio core device to be minimal and tightly coupled with handling the hardware-specific logic and communication.

The ancillary bus is intended to be minimal, generic and avoid domain-specific assumptions. Each ancillary bus device represents a part of its parent functionality. The generic behavior can be extended and specialized as needed by encapsulating an ancillary bus device within other domain-specific structures and the use of .ops callbacks.

The SOF driver adopts the ancillary bus for implementing the multi-client support. A client in the context of the SOF driver represents a part of the core device's functionality. It is not a physical device but rather an ancillary device that needs to communicate with the DSP via IPCs. With multi-client support,the sound card can be separated into multiple orthogonal ancillary devices for local devices (mic/speakers etc), HDMI, sensing, probes, debug etc. In this series, we demonstrate the usage of the ancillary bus with the help of the IPC test client which is used for testing the serialization of IPCs when multiple clients talk to the DSP at the same time.

Dave Ertman (1):
Add ancillary bus support

Fred Oh (1):
ASoC: SOF: debug: Remove IPC flood test support in SOF core

Ranjani Sridharan (4):
ASoC: SOF: Introduce descriptors for SOF client
ASoC: SOF: Create client driver for IPC test
ASoC: SOF: ops: Add ops for client registration
ASoC: SOF: Intel: Define ops for client registration

 Documentation/driver-api/ancillary_bus.rst | 230 +++++++++++++++
 Documentation/driver-api/index.rst         |   1 +
 drivers/bus/Kconfig                        |   3 +
 drivers/bus/Makefile                       |   3 +
 drivers/bus/ancillary.c                    | 191 +++++++++++++
 include/linux/ancillary_bus.h              |  58 ++++
 include/linux/mod_devicetable.h            |   8 +
 scripts/mod/devicetable-offsets.c          |   3 +
 scripts/mod/file2alias.c                   |   8 +
 sound/soc/sof/Kconfig                      |  29 +-
 sound/soc/sof/Makefile                     |   7 +
 sound/soc/sof/core.c                       |  12 +
 sound/soc/sof/debug.c                      | 230 ---------------
 sound/soc/sof/intel/Kconfig                |   9 +
 sound/soc/sof/intel/Makefile               |   3 +
 sound/soc/sof/intel/apl.c                  |  18 ++
 sound/soc/sof/intel/bdw.c                  |  18 ++
 sound/soc/sof/intel/byt.c                  |  22 ++
 sound/soc/sof/intel/cnl.c                  |  18 ++
 sound/soc/sof/intel/intel-client.c         |  49 ++++
 sound/soc/sof/intel/intel-client.h         |  26 ++
 sound/soc/sof/ops.h                        |  14 +
 sound/soc/sof/sof-client.c                 | 117 ++++++++
 sound/soc/sof/sof-client.h                 |  65 +++++
 sound/soc/sof/sof-ipc-test-client.c        | 314 +++++++++++++++++++++
 sound/soc/sof/sof-priv.h                   |  16 +-
 26 files changed, 1233 insertions(+), 239 deletions(-)
 create mode 100644 Documentation/driver-api/ancillary_bus.rst
 create mode 100644 drivers/bus/ancillary.c
 create mode 100644 include/linux/ancillary_bus.h
 create mode 100644 sound/soc/sof/intel/intel-client.c
 create mode 100644 sound/soc/sof/intel/intel-client.h
 create mode 100644 sound/soc/sof/sof-client.c
 create mode 100644 sound/soc/sof/sof-client.h
 create mode 100644 sound/soc/sof/sof-ipc-test-client.c

--
2.26.2

#20 lore raw
Quoted 11 lines · Leon Romanovsky

Hi Dave,

I don't know why did you send this series separately to every mailing list, but it is not correct thing to do.

RDMA ML and discussion:
https://lore.kernel.org/linux-rdma/[email protected]/T/#t
Netdev ML (completely separated):
https://lore.kernel.org/netdev/[email protected]/
Alsa ML (separated too):
https://lore.kernel.org/alsa-devel/[email protected]/

Seems like the goal was to spread it around to different places so that no one could strongly object or review it :(

greg k-h

#21 lore raw
Quoted 13 lines · Greg KH
Quoted 11 lines · Leon Romanovsky

Hi Dave,

I don't know why did you send this series separately to every mailing list, but it is not correct thing to do.

RDMA ML and discussion:
https://lore.kernel.org/linux-rdma/[email protected]/T/#t
Netdev ML (completely separated):
https://lore.kernel.org/netdev/[email protected]/
Alsa ML (separated too):
https://lore.kernel.org/alsa-devel/[email protected]/

Quoted 2 lines

Seems like the goal was to spread it around to different places so that no one could strongly object or review it :(

It took me time to realize why I was alone expressing my thoughts :).

BTW, I'm looking on ALSA thread and happy to see that people didn't like "ancillary" name. It is far from intuitive name for any non-English speaker.

Thanks

Quoted 2 lines

greg k-h

#22 lore raw
Quoted 14 lines · Leon Romanovsky
Quoted 13 lines · Greg KH
Quoted 11 lines · Leon Romanovsky

Hi Dave,

I don't know why did you send this series separately to every mailing list, but it is not correct thing to do.

RDMA ML and discussion:
https://lore.kernel.org/linux-rdma/[email protected]/T/#t
Netdev ML (completely separated):
https://lore.kernel.org/netdev/[email protected]/
Alsa ML (separated too):
https://lore.kernel.org/alsa-devel/[email protected]/

Quoted 7 lines
Quoted 2 lines

Seems like the goal was to spread it around to different places so that no one could strongly object or review it :(

It took me time to realize why I was alone expressing my thoughts :).

BTW, I'm looking on ALSA thread and happy to see that people didn't like "ancillary" name. It is far from intuitive name for any non-English speaker.

It's not intuitive for a native-english speaker either :)

greg k-h

#23 lore raw
In reply to Greg KH
Quoted 22 lines

-----Original Message-----
From: Greg KH <[email protected]>
Sent: Saturday, October 3, 2020 2:11 AM
To: Leon Romanovsky <[email protected]>
Cc: Ertman, David M <[email protected]>; linux- [email protected]; linux-netdev <[email protected]>; alsa- [email protected]
Subject: Re: [PATCH 0/6] Ancillary bus implementation and SOF multi-client support

Quoted 7 lines · Leon Romanovsky

Hi Dave,

I don't know why did you send this series separately to every mailing list, but it is not correct thing to do.

RDMA ML and discussion:
https://lore.kernel.org/linux-rdma/20201001050534.890666-1-

[email protected]/T/#t

Quoted 2 lines

Netdev ML (completely separated):
https://lore.kernel.org/netdev/20201001050851.890722-1-

[email protected]/

Quoted 8 lines
Quoted 2 lines

Alsa ML (separated too):
https://lore.kernel.org/alsa-devel/20200930225051.889607-1-

[email protected]/

Seems like the goal was to spread it around to different places so that no one could strongly object or review it :(

greg k-h

This was my first time sending a patchset to more than one mailing list and I screwed it up.

I will be sending a v2 soon (Monday maybe?) and I will be sending it to all CC's and mailing list in one send so that everyone will see everyone's response.

Sorry for the mistake.

-DaveE.