Skip to contentSkip to conversation

add support for CDX bus

PATCH In review Nipun Gupta 4 participants 23 messages Jan 17, 2023 – Jan 19, 2023

Subsystems (4)
lore
Share & tools
lore

Nipun Gupta

author maintainer
#0 lore raw

cover add support for CDX bus

Introduces AMD CDX bus, which provides a mechanism to discover/rescan CDX devices. The CDX devices are memory mapped on system bus for embedded CPUs.

CDX controller interacts with the firmware to query different CDX devices present in the Fabric and expose them to the Linux host on CDX bus.

This patch series:
- Introduces the CDX bus and CDX devices.
- Device tree binding for CDX controller
- Support for CDX bus in arm-smmu-v3 driver
- Add MCDI (Management CPU Driver Interface) as a protocol for communication with RPU Firmware
- Support RPMSg channel for Firmware communication

MSI patches for CDX are not added in this series as it's support is being revisited as per patch series:
https://lore.kernel.org/all/[email protected]/
It will be added as separate patches.

RFC changes with stubs were submitted at:
https://lore.kernel.org/linux-arm-kernel/[email protected]/

Abhijit Gangurde (1):
bus/cdx: add rpmsg communication channel for CDX

Nipun Gupta (6):
bus/cdx: add the cdx bus driver
iommu/arm-smmu-v3: support ops registration for CDX bus
dt-bindings: bus: add CDX bus controller device tree bindings bus/cdx: add MCDI protocol interface for firmware interaction bus/cdx: add cdx controller
bus/cdx: add device attributes

 Documentation/ABI/testing/sysfs-bus-cdx       |  46 +
 .../bindings/bus/xlnx,cdxbus-controller.yaml  |  68 ++
 MAINTAINERS                                   |   8 +
 drivers/bus/Kconfig                           |   1 +
 drivers/bus/Makefile                          |   2 +
 drivers/bus/cdx/Kconfig                       |  16 +
 drivers/bus/cdx/Makefile                      |   8 +
 drivers/bus/cdx/cdx.c                         | 577 +++++++++++
 drivers/bus/cdx/cdx.h                         |  62 ++
 drivers/bus/cdx/controller/Kconfig            |  30 +
 drivers/bus/cdx/controller/Makefile           |   9 +
 drivers/bus/cdx/controller/bitfield.h         |  88 ++
 drivers/bus/cdx/controller/cdx_controller.c   | 282 ++++++
 drivers/bus/cdx/controller/cdx_controller.h   |  30 +
 drivers/bus/cdx/controller/cdx_rpmsg.c        | 222 +++++
 drivers/bus/cdx/controller/mc_cdx_pcol.h      | 707 ++++++++++++++
 drivers/bus/cdx/controller/mcdi.c             | 918 ++++++++++++++++++
 drivers/bus/cdx/controller/mcdi.h             | 259 +++++
 drivers/bus/cdx/controller/mcdi_functions.c   | 139 +++
 drivers/bus/cdx/controller/mcdi_functions.h   |  61 ++
 drivers/iommu/iommu.c                         |   4 +
 include/linux/cdx/cdx_bus.h                   | 176 ++++
 include/linux/mod_devicetable.h               |  15 +
 scripts/mod/devicetable-offsets.c             |   4 +
 scripts/mod/file2alias.c                      |  12 +
 25 files changed, 3744 insertions(+)
 create mode 100644 Documentation/ABI/testing/sysfs-bus-cdx
 create mode 100644 Documentation/devicetree/bindings/bus/xlnx,cdxbus-controller.yaml
 create mode 100644 drivers/bus/cdx/Kconfig
 create mode 100644 drivers/bus/cdx/Makefile
 create mode 100644 drivers/bus/cdx/cdx.c
 create mode 100644 drivers/bus/cdx/cdx.h
 create mode 100644 drivers/bus/cdx/controller/Kconfig
 create mode 100644 drivers/bus/cdx/controller/Makefile
 create mode 100644 drivers/bus/cdx/controller/bitfield.h
 create mode 100644 drivers/bus/cdx/controller/cdx_controller.c
 create mode 100644 drivers/bus/cdx/controller/cdx_controller.h
 create mode 100644 drivers/bus/cdx/controller/cdx_rpmsg.c
 create mode 100644 drivers/bus/cdx/controller/mc_cdx_pcol.h
 create mode 100644 drivers/bus/cdx/controller/mcdi.c
 create mode 100644 drivers/bus/cdx/controller/mcdi.h
 create mode 100644 drivers/bus/cdx/controller/mcdi_functions.c
 create mode 100644 drivers/bus/cdx/controller/mcdi_functions.h
 create mode 100644 include/linux/cdx/cdx_bus.h
Signature · 1 line
2.17.1
author maintainer
#1 lore raw

PATCH 1/19 bus/cdx: add the cdx bus driver

Introduce AMD CDX bus, which provides a mechanism for scanning and probing CDX devices. These devices are memory mapped on system bus for Application Processors(APUs).

CDX devices can be changed dynamically in the Fabric and CDX bus interacts with CDX controller to rescan the bus and rediscover the devices.

---
 Documentation/ABI/testing/sysfs-bus-cdx |  12 +
 MAINTAINERS                             |   7 +
 drivers/bus/Kconfig                     |   1 +
 drivers/bus/Makefile                    |   2 +
 drivers/bus/cdx/Kconfig                 |  14 +
 drivers/bus/cdx/Makefile                |   8 +
 drivers/bus/cdx/cdx.c                   | 433 ++++++++++++++++++++++++
 drivers/bus/cdx/cdx.h                   |  62 ++++
 include/linux/cdx/cdx_bus.h             | 153 +++++++++
 include/linux/mod_devicetable.h         |  15 +
 scripts/mod/devicetable-offsets.c       |   4 +
 scripts/mod/file2alias.c                |  12 +
 12 files changed, 723 insertions(+)
 create mode 100644 Documentation/ABI/testing/sysfs-bus-cdx
 create mode 100644 drivers/bus/cdx/Kconfig
 create mode 100644 drivers/bus/cdx/Makefile
 create mode 100644 drivers/bus/cdx/cdx.c
 create mode 100644 drivers/bus/cdx/cdx.h
 create mode 100644 include/linux/cdx/cdx_bus.h
Documentation/ABI/testing/sysfs-bus-cdx new file +12 -0
new file mode 100644
index 000000000000..8c2425fdb6d9
@@ -0,0 +1,12 @@
+What:		/sys/bus/cdx/rescan
+Date:		January 2023
+Contact:	[email protected]
+Description:
+		Writing a non-zero value to this file would cause rescan
+		of the bus and devices on the CDX bus. Any new devices would
+		be scanned and added to the list of linux devices and any
+		devices removed are also deleted from linux.
+
+                For example::
+
+		  # echo 1 > /sys/bus/cdx/rescan
MAINTAINERS +7 -0
index 1c83c26e9f68..25c76cf27f21 100644
@@ -962,6 +962,13 @@ Q:	https://patchwork.kernel.org/project/linux-rdma/list/
 F:	drivers/infiniband/hw/efa/
 F:	include/uapi/rdma/efa-abi.h
 
+AMD CDX BUS DRIVER
+M:	Nipun Gupta <[email protected]>
+M:	Nikhil Agarwal <[email protected]>
+S:	Maintained
+F:	drivers/bus/cdx/*
+F:	include/linux/cdx/*
+
 AMD CRYPTOGRAPHIC COPROCESSOR (CCP) DRIVER
 M:	Tom Lendacky <[email protected]>
 M:	John Allen <[email protected]>
drivers/bus/Kconfig +1 -0
index 7bfe998f3514..b0324efb9a6a 100644
@@ -251,5 +251,6 @@ config DA8XX_MSTPRI
 
 source "drivers/bus/fsl-mc/Kconfig"
 source "drivers/bus/mhi/Kconfig"
+source "drivers/bus/cdx/Kconfig"
 
 endmenu
drivers/bus/Makefile +2 -0
index d90eed189a65..9e4d09789a81 100644
@@ -20,6 +20,8 @@ obj-$(CONFIG_INTEL_IXP4XX_EB)	+= intel-ixp4xx-eb.o
 obj-$(CONFIG_MIPS_CDMM)		+= mips_cdmm.o
 obj-$(CONFIG_MVEBU_MBUS) 	+= mvebu-mbus.o
 
+obj-$(CONFIG_CDX_BUS)		+= cdx/
+
 # Interconnect bus driver for OMAP SoCs.
 obj-$(CONFIG_OMAP_INTERCONNECT)	+= omap_l3_smx.o omap_l3_noc.o
 
drivers/bus/cdx/Kconfig new file +14 -0
new file mode 100644
index 000000000000..54e0623ebcff
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# CDX bus configuration
+#
+# Copyright (C) 2022-2023, Advanced Micro Devices, Inc.
+#
+
+config CDX_BUS
+	bool "CDX Bus driver"
+	help
+	  Driver to enable CDX Bus. CDX bus provides a mechanism for
+	  scanning and probing of CDX devices. CDX devices are memory
+	  mapped on system bus for embedded CPUs. CDX bus uses CDX
+	  controller and firmware to scan the CDX devices.
drivers/bus/cdx/Makefile new file +8 -0
new file mode 100644
index 000000000000..c5feb11fc718
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for CDX
+#
+# Copyright (C) 2022-2023, Advanced Micro Devices, Inc.
+#
+
+obj-$(CONFIG_CDX_BUS) += cdx.o
drivers/bus/cdx/cdx.c new file +433 -0

433 lines not on this page · raw message

drivers/bus/cdx/cdx.h new file +62 -0

62 lines not on this page · raw message

include/linux/cdx/cdx_bus.h new file +153 -0

153 lines not on this page · raw message

include/linux/mod_devicetable.h +15 -0
index 549590e9c644..23445b163c2a 100644
@@ -911,4 +911,19 @@ struct ishtp_device_id {
 	kernel_ulong_t driver_data;
 };
 
+/**
+ * struct cdx_device_id - CDX device identifier
+ * @vendor: Vendor ID
+ * @device: Device ID
+ * @override_only: Match only when dev->driver_override is this driver.
+ *
+ * Type of entries in the "device Id" table for CDX devices supported by
+ * a CDX device driver.
+ */
+struct cdx_device_id {
+	__u16 vendor;
+	__u16 device;
+	__u32 override_only;
+};
+
 #endif /* LINUX_MOD_DEVICETABLE_H */
scripts/mod/devicetable-offsets.c +4 -0
index c0d3bcb99138..62dc988df84d 100644
@@ -262,5 +262,9 @@ int main(void)
 	DEVID(ishtp_device_id);
 	DEVID_FIELD(ishtp_device_id, guid);
 
+	DEVID(cdx_device_id);
+	DEVID_FIELD(cdx_device_id, vendor);
+	DEVID_FIELD(cdx_device_id, device);
+
 	return 0;
 }
scripts/mod/file2alias.c +12 -0
index 91c2e7ba5e52..28da34ba4359 100644
@@ -1452,6 +1452,17 @@ static int do_dfl_entry(const char *filename, void *symval, char *alias)
 	return 1;
 }
 
+/* Looks like: cdx:vNdN */
+static int do_cdx_entry(const char *filename, void *symval,
+			char *alias)
+{
+	DEF_FIELD(symval, cdx_device_id, vendor);
+	DEF_FIELD(symval, cdx_device_id, device);
+
+	sprintf(alias, "cdx:v%08Xd%08Xd", vendor, device);
+	return 1;
+}
+
 /* Does namelen bytes of name exactly match the symbol? */
 static bool sym_is(const char *name, unsigned namelen, const char *symbol)
 {
@@ -1531,6 +1542,7 @@ static const struct devtable devtable[] = {
 	{"ssam", SIZE_ssam_device_id, do_ssam_entry},
 	{"dfl", SIZE_dfl_device_id, do_dfl_entry},
 	{"ishtp", SIZE_ishtp_device_id, do_ishtp_entry},
+	{"cdx", SIZE_cdx_device_id, do_cdx_entry},
 };
 
 /* Create MODULE_ALIAS() statements.
Signature · 1 line
2.17.1
#2 lore raw

Re: [PATCH 01/19] bus/cdx: add the cdx bus driver

Quoted 10 lines · Nipun Gupta

Introduce AMD CDX bus, which provides a mechanism for scanning and probing CDX devices. These devices are memory mapped on system bus for Application Processors(APUs).

CDX devices can be changed dynamically in the Fabric and CDX bus interacts with CDX controller to rescan the bus and rediscover the devices.

First off, very nice job, I didn't find any obvious issues with this integration into the driver core.

That being said, why do you want this in drivers/bus/? Why not drivers/cdx/ ?

One minor comment to make the code smaller:

Quoted 11 lines
drivers/bus/cdx/cdx.c new file +11 -0
+static int get_free_index(void)
+{
+	unsigned long id_map;
+	unsigned long mask;
+	int index = 0;
+
+	mask  = (1UL << MAX_CDX_CONTROLLERS) - 1;
+retry:
+	id_map = cdx_controller_id_map[0];
+	if ((id_map & mask) == mask)
+		return -ENOSPC;

drivers/bus/cdx/cdx.c line 394

Quoted 10 lines
drivers/bus/cdx/cdx.c new file +10 -0
+
+	index = ffz(id_map);
+	if (index >= MAX_CDX_CONTROLLERS)
+		return -ENOSPC;
+
+	if (test_and_set_bit(index, &cdx_controller_id_map[0]))
+		goto retry;
+
+	return index;
+}

Why not just use the idr/ida structure instead? That will handle all of that logic for you and get rid of your bit twiddling.

Quoted 6 lines
drivers/bus/cdx/cdx.h new file +6 -0
+/**
+ * struct cdx_dev_params - CDX device parameters
+ * @cdx: CDX controller associated with the device
+ * @parent: Associated CDX controller
+ * @vendor: Vendor ID for CDX device
+ * @device: Device ID for CDX device

drivers/bus/cdx/cdx.h line 28

Quoted 10 lines
drivers/bus/cdx/cdx.h new file +10 -0
+ * @bus_num: Bus number for this CDX device
+ * @dev_num: Device number for this device
+ * @res: array of MMIO region entries
+ * @res_count: number of valid MMIO regions
+ * @req_id: Requestor ID associated with CDX device
+ */
+struct cdx_dev_params {
+	struct cdx_controller *cdx;
+	u16 vendor;
+	u16 device;

Are these in little endian format in memory? Or native? Or something else?

thanks,

greg k-h

author maintainer
#3 lore raw

RE: [PATCH 01/19] bus/cdx: add the cdx bus driver

[AMD Official Use Only - General]

Quoted 31 lines

-----Original Message-----
From: Greg KH <[email protected]>
Sent: Tuesday, January 17, 2023 7:37 PM
To: Gupta, Nipun <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux-arm- [email protected]; [email protected]; linux- [email protected]; [email protected]; [email protected]; Anand, Harpreet <[email protected]>; Agarwal, Nikhil <[email protected]>; Simek, Michal <[email protected]>; git (AMD-Xilinx) <[email protected]>
Subject: Re: [PATCH 01/19] bus/cdx: add the cdx bus driver

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

Quoted 8 lines · Nipun Gupta

Introduce AMD CDX bus, which provides a mechanism for scanning and probing CDX devices. These devices are memory mapped on system bus for Application Processors(APUs).

CDX devices can be changed dynamically in the Fabric and CDX bus interacts with CDX controller to rescan the bus and rediscover the devices.

Quoted 8 lines
Quoted 2 lines

First off, very nice job, I didn't find any obvious issues with this integration into the driver core.

That being said, why do you want this in drivers/bus/? Why not drivers/cdx/ ?

Thanks, Greg, for taking time out for review and providing your valuable feedback. We do not have strong affiliation to drivers/bus/cdx so will move it to drivers/cdx in the next spin.

Quoted 19 lines

One minor comment to make the code smaller:

Quoted 16 lines
drivers/bus/cdx/cdx.c new file +16 -0
+static int get_free_index(void)
+{
+     unsigned long id_map;
+     unsigned long mask;
+     int index = 0;
+
+     mask  = (1UL << MAX_CDX_CONTROLLERS) - 1;
+retry:
+     id_map = cdx_controller_id_map[0];
+     if ((id_map & mask) == mask)
+             return -ENOSPC;
+
+     index = ffz(id_map);
+     if (index >= MAX_CDX_CONTROLLERS)
+             return -ENOSPC;
+
Quoted 8 lines

drivers/bus/cdx/cdx.c line 394

Quoted 5 lines
drivers/bus/cdx/cdx.c new file +5 -0
+     if (test_and_set_bit(index, &cdx_controller_id_map[0]))
+             goto retry;
+
+     return index;
+}

Why not just use the idr/ida structure instead? That will handle all of that logic for you and get rid of your bit twiddling.

Agree. Using idr/ida seems more appropriate. Will update the code accordingly.

Quoted 12 lines
Quoted 12 lines

+/**
+ * struct cdx_dev_params - CDX device parameters
+ * @cdx: CDX controller associated with the device
+ * @parent: Associated CDX controller
+ * @vendor: Vendor ID for CDX device
+ * @device: Device ID for CDX device
+ * @bus_num: Bus number for this CDX device
+ * @dev_num: Device number for this device
+ * @res: array of MMIO region entries
+ * @res_count: number of valid MMIO regions
+ * @req_id: Requestor ID associated with CDX device

Quoted 8 lines

drivers/bus/cdx/cdx.h line 28

Quoted 5 lines

+ */

drivers/bus/cdx/cdx.h new file +4 -0
+struct cdx_dev_params {
+     struct cdx_controller *cdx;
+     u16 vendor;
+     u16 device;

Are these in little endian format in memory? Or native? Or something else?

While reading from the hardware, these values are little-endian; and they are converted to CPU endianness by the controller code using le_to_cpu32() and then passed as CPU endian while registering the CDX device.

Thanks,
Nipun

Quoted 4 lines

thanks,

greg k-h

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

#4 lore raw

Re: [PATCH 01/19] bus/cdx: add the cdx bus driver

In reply to Nipun Gupta

Hi--

Quoted 42 lines · Nipun Gupta

Introduce AMD CDX bus, which provides a mechanism for scanning and probing CDX devices. These devices are memory mapped on system bus for Application Processors(APUs).

CDX devices can be changed dynamically in the Fabric and CDX bus interacts with CDX controller to rescan the bus and rediscover the devices.

---
 Documentation/ABI/testing/sysfs-bus-cdx |  12 +
 MAINTAINERS                             |   7 +
 drivers/bus/Kconfig                     |   1 +
 drivers/bus/Makefile                    |   2 +
 drivers/bus/cdx/Kconfig                 |  14 +
 drivers/bus/cdx/Makefile                |   8 +
 drivers/bus/cdx/cdx.c                   | 433 ++++++++++++++++++++++++
 drivers/bus/cdx/cdx.h                   |  62 ++++
 include/linux/cdx/cdx_bus.h             | 153 +++++++++
 include/linux/mod_devicetable.h         |  15 +
 scripts/mod/devicetable-offsets.c       |   4 +
 scripts/mod/file2alias.c                |  12 +
 12 files changed, 723 insertions(+)
 create mode 100644 Documentation/ABI/testing/sysfs-bus-cdx
 create mode 100644 drivers/bus/cdx/Kconfig
 create mode 100644 drivers/bus/cdx/Makefile
 create mode 100644 drivers/bus/cdx/cdx.c
 create mode 100644 drivers/bus/cdx/cdx.h
 create mode 100644 include/linux/cdx/cdx_bus.h
drivers/bus/cdx/Kconfig new file +4 -0
new file mode 100644
index 000000000000..54e0623ebcff
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# CDX bus configuration
+#

drivers/bus/cdx/Kconfig line 14

Quoted 10 lines
drivers/bus/cdx/Kconfig new file +10 -0
+# Copyright (C) 2022-2023, Advanced Micro Devices, Inc.
+#
+
+config CDX_BUS
+	bool "CDX Bus driver"
+	help
+	  Driver to enable CDX Bus. CDX bus provides a mechanism for
+	  scanning and probing of CDX devices. CDX devices are memory
+	  mapped on system bus for embedded CPUs. CDX bus uses CDX
+	  controller and firmware to scan the CDX devices.

Would you mind telling people who use 'make *config' what CDX means, either in the bool prompt string or in the help text?

Quoted 9 lines

+/**
+ * cdx_unregister_device - Unregister a CDX device
+ * @dev: CDX device
+ * @data: This is always passed as NULL, and is not used in this API,

+ *	  but is required here as the bus_for_each_dev() API expects
+ *	  the passed function (cdx_unregister_device) to have this
+ *	  as an argument.

+ *
+ * @return: -errno on failure, 0 on success.

The syntax (or spelling) for a function's return value in kernel-doc is just:
* Return: -errno on failure, 0 on success.

That should be changed in many places throughout.

Thanks.

Signature · 6 lines
~Randy

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
author maintainer
#5 lore raw

RE: [PATCH 01/19] bus/cdx: add the cdx bus driver

[AMD Official Use Only - General]

Quoted 73 lines

-----Original Message-----
From: Randy Dunlap <[email protected]>
Sent: Tuesday, January 17, 2023 10:52 PM
To: Gupta, Nipun <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux-arm- [email protected]; [email protected]; linux- [email protected]; [email protected]
Cc: [email protected]; Anand, Harpreet <[email protected]>; Agarwal, Nikhil <[email protected]>; Simek, Michal <[email protected]>; git (AMD-Xilinx) <[email protected]>
Subject: Re: [PATCH 01/19] bus/cdx: add the cdx bus driver

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

Hi--

Quoted 47 lines · Nipun Gupta

Introduce AMD CDX bus, which provides a mechanism for scanning and probing CDX devices. These devices are memory mapped on system bus for Application Processors(APUs).

CDX devices can be changed dynamically in the Fabric and CDX bus interacts with CDX controller to rescan the bus and rediscover the devices.

---
 Documentation/ABI/testing/sysfs-bus-cdx |  12 +
 MAINTAINERS                             |   7 +
 drivers/bus/Kconfig                     |   1 +
 drivers/bus/Makefile                    |   2 +
 drivers/bus/cdx/Kconfig                 |  14 +
 drivers/bus/cdx/Makefile                |   8 +
 drivers/bus/cdx/cdx.c                   | 433 ++++++++++++++++++++++++
 drivers/bus/cdx/cdx.h                   |  62 ++++
 include/linux/cdx/cdx_bus.h             | 153 +++++++++
 include/linux/mod_devicetable.h         |  15 +
 scripts/mod/devicetable-offsets.c       |   4 +
 scripts/mod/file2alias.c                |  12 +
 12 files changed, 723 insertions(+)
 create mode 100644 Documentation/ABI/testing/sysfs-bus-cdx
 create mode 100644 drivers/bus/cdx/Kconfig
 create mode 100644 drivers/bus/cdx/Makefile
 create mode 100644 drivers/bus/cdx/cdx.c
 create mode 100644 drivers/bus/cdx/cdx.h
 create mode 100644 include/linux/cdx/cdx_bus.h
drivers/bus/cdx/Kconfig new file +9 -0
new file mode 100644
index 000000000000..54e0623ebcff
@@ -0,0 +1,14 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# CDX bus configuration
+#
+# Copyright (C) 2022-2023, Advanced Micro Devices, Inc.
+#
+
+config CDX_BUS
+     bool "CDX Bus driver"
Quoted 8 lines

drivers/bus/cdx/Kconfig line 14

Quoted 5 lines
drivers/bus/cdx/Kconfig new file +5 -0
+     help
+       Driver to enable CDX Bus. CDX bus provides a mechanism for
+       scanning and probing of CDX devices. CDX devices are memory
+       mapped on system bus for embedded CPUs. CDX bus uses CDX
+       controller and firmware to scan the CDX devices.

Would you mind telling people who use 'make *config' what CDX means, either in the bool prompt string or in the help text?

Sure. Will update the help section for this.

Quoted 12 lines
Quoted 11 lines

+/**
+ * cdx_unregister_device - Unregister a CDX device
+ * @dev: CDX device
+ * @data: This is always passed as NULL, and is not used in this API,

+ *     but is required here as the bus_for_each_dev() API expects
+ *     the passed function (cdx_unregister_device) to have this
+ *     as an argument.

+ *
+ * @return: -errno on failure, 0 on success.

Quoted 4 lines

The syntax (or spelling) for a function's return value in kernel-doc is just:
* Return: -errno on failure, 0 on success.

That should be changed in many places throughout.

Agree. Will update at all places.

Thanks,
Nipun

Quoted 5 lines

Thanks.

--
~Randy

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

author maintainer
#6 lore raw

PATCH 2/19 iommu/arm-smmu-v3: support ops registration for CDX bus

In reply to Nipun Gupta

With new CDX bus supported for AMD FPGA devices on ARM platform, the bus requires registration for the SMMU v3 driver.

---
 drivers/iommu/iommu.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/iommu/iommu.c +4 -0
index 834e6ecf3e51..6a28ef25f337 100644
@@ -28,6 +28,7 @@
 #include <linux/fsl/mc.h>
 #include <linux/module.h>
 #include <linux/cc_platform.h>
+#include <linux/cdx/cdx_bus.h>
 #include <trace/events/iommu.h>
 #include <linux/sched/mm.h>
 #include <linux/msi.h>
@@ -129,6 +130,9 @@ static struct bus_type * const iommu_buses[] = {
 #ifdef CONFIG_TEGRA_HOST1X_CONTEXT_BUS
 	&host1x_context_device_bus_type,
 #endif
+#ifdef CONFIG_CDX_BUS
+	&cdx_bus_type,
+#endif
 };
 
 /*
Signature · 1 line
2.17.1
author maintainer
#7 lore raw

PATCH 3/19 dt-bindings: bus: add CDX bus controller device tree bindings

In reply to Nipun Gupta

Add device tree bindings for CDX bus controller.

---
 .../bindings/bus/xlnx,cdxbus-controller.yaml  | 68 +++++++++++++++++++
 MAINTAINERS                                   |  1 +
 2 files changed, 69 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/bus/xlnx,cdxbus-controller.yaml
Documentation/devicetree/bindings/bus/xlnx,cdxbus-controller.yaml new file +68 -0
new file mode 100644
index 000000000000..b2f186864021
@@ -0,0 +1,68 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/bus/xlnx,cdxbus-controller.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: AMD CDX bus controller
+
+description: |
+  CDX bus controller for AMD devices is implemented to dynamically
+  detect CDX bus and devices on these bus using the firmware.
+  The CDX bus manages multiple FPGA based hardware devices, which
+  can support network, crypto or any other specialized type of
+  devices. These FPGA based devices can be added/modified dynamically
+  on run-time.
+
+  All devices on the CDX bus will have a unique streamid (for IOMMU)
+  and a unique device ID (for MSI) corresponding to a requestor ID
+  (one to one associated with the device). The streamid and deviceid
+  are used to configure SMMU and GIC-ITS respectively.
+
+  iommu-map property is used to define the set of stream ids
+  corresponding to each device and the associated IOMMU.
+
+  The MSI writes are accompanied by sideband data (Device ID).
+  The msi-map property is used to associate the devices with the
+  device ID as well as the associated ITS controller.
+
+  rproc property (xlnx,rproc) is used to identify the remote processor
+  with which APU (Application Processor Unit) interacts to find out
+  the bus and device configuration.
+
+maintainers:
+  - Nipun Gupta <[email protected]>
+  - Nikhil Agarwal <[email protected]>
+
+properties:
+  compatible:
+    const: xlnx,cdxbus-controller
+
+  iommu-map: true
+
+  msi-map: true
+
+  xlnx,rproc:
+    $ref: /schemas/types.yaml#/definitions/phandle
+    description:
+      phandle to the remoteproc_r5 rproc node using which APU interacts
+      with remote processor.
+
+required:
+  - compatible
+  - iommu-map
+  - msi-map
+  - xlnx,rproc
+
+additionalProperties: false
+
+examples:
+  - |
+    cdxbus-controller {
+        compatible = "xlnx,cdxbus-controller";
+        /* define map for RIDs 250-259 */
+        iommu-map = <250 &smmu 250 10>;
+        /* define msi map for RIDs 250-259 */
+        msi-map = <250 &its 250 10>;
+        xlnx,rproc = <&remoteproc_r5>;
+    };
MAINTAINERS +1 -0
index 25c76cf27f21..b43242546b17 100644
@@ -966,6 +966,7 @@ AMD CDX BUS DRIVER
 M:	Nipun Gupta <[email protected]>
 M:	Nikhil Agarwal <[email protected]>
 S:	Maintained
+F:	Documentation/devicetree/bindings/bus/xlnx,cdxbus-controller.yaml
 F:	drivers/bus/cdx/*
 F:	include/linux/cdx/*
 
Signature · 1 line
2.17.1
#8 lore raw

Re: [PATCH 03/19] dt-bindings: bus: add CDX bus controller device tree bindings

Quoted 1 line · Nipun Gupta

Add device tree bindings for CDX bus controller.

Subject: drop second/last, redundant "device tree bindings". The "dt-bindings" prefix is already stating that these are bindings.

Quoted 43 lines
---
 .../bindings/bus/xlnx,cdxbus-controller.yaml  | 68 +++++++++++++++++++
 MAINTAINERS                                   |  1 +
 2 files changed, 69 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/bus/xlnx,cdxbus-controller.yaml
Documentation/devicetree/bindings/bus/xlnx,cdxbus-controller.yaml new file +29 -0
new file mode 100644
index 000000000000..b2f186864021
@@ -0,0 +1,68 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/bus/xlnx,cdxbus-controller.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: AMD CDX bus controller
+
+description: |
+  CDX bus controller for AMD devices is implemented to dynamically
+  detect CDX bus and devices on these bus using the firmware.
+  The CDX bus manages multiple FPGA based hardware devices, which
+  can support network, crypto or any other specialized type of
+  devices. These FPGA based devices can be added/modified dynamically
+  on run-time.
+
+  All devices on the CDX bus will have a unique streamid (for IOMMU)
+  and a unique device ID (for MSI) corresponding to a requestor ID
+  (one to one associated with the device). The streamid and deviceid
+  are used to configure SMMU and GIC-ITS respectively.
+
+  iommu-map property is used to define the set of stream ids
+  corresponding to each device and the associated IOMMU.
+
+  The MSI writes are accompanied by sideband data (Device ID).
+  The msi-map property is used to associate the devices with the
+  device ID as well as the associated ITS controller.
+
+  rproc property (xlnx,rproc) is used to identify the remote processor

Documentation/devicetree/bindings/bus/xlnx,cdxbus-controller.yaml line 39

Quoted 10 lines
Documentation/devicetree/bindings/bus/xlnx,cdxbus-controller.yaml new file +10 -0
+  with which APU (Application Processor Unit) interacts to find out
+  the bus and device configuration.
+
+maintainers:
+  - Nipun Gupta <[email protected]>
+  - Nikhil Agarwal <[email protected]>
+
+properties:
+  compatible:
+    const: xlnx,cdxbus-controller

This misses SoC specific compatible. Drop "bus" - redundant. I would also say - drop controller - do you see any other devices with such compatible naming? Use naming consistent with other devices in the kernel. Just open some controllers - SPI, I2C etc. and look there.

Documentation/devicetree/bindings/bus/xlnx,cdxbus-controller.yaml line 41

Quoted 2 lines
Documentation/devicetree/bindings/bus/xlnx,cdxbus-controller.yaml new file +2 -0
+
+  iommu-map: true

No mask?

Quoted 10 lines
Documentation/devicetree/bindings/bus/xlnx,cdxbus-controller.yaml new file +10 -0
+
+  msi-map: true
+
+  xlnx,rproc:
+    $ref: /schemas/types.yaml#/definitions/phandle
+    description:
+      phandle to the remoteproc_r5 rproc node using which APU interacts
+      with remote processor.
+
+required:

Documentation/devicetree/bindings/bus/xlnx,cdxbus-controller.yaml line 61

Quoted 10 lines
Documentation/devicetree/bindings/bus/xlnx,cdxbus-controller.yaml new file +10 -0
+  - compatible
+  - iommu-map
+  - msi-map
+  - xlnx,rproc
+
+additionalProperties: false
+
+examples:
+  - |
+    cdxbus-controller {

Node names should be generic, so just cdx.
https://devicetree-specification.readthedocs.io/en/latest/chapter2-devicetree-basics.html#generic-names-recommendation

Documentation/devicetree/bindings/bus/xlnx,cdxbus-controller.yaml line 64

Quoted 3 lines
Documentation/devicetree/bindings/bus/xlnx,cdxbus-controller.yaml new file +3 -0
+        compatible = "xlnx,cdxbus-controller";
+        /* define map for RIDs 250-259 */
+        iommu-map = <250 &smmu 250 10>;

Best regards,
Krzysztof

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

author maintainer
#9 lore raw

RE: [PATCH 03/19] dt-bindings: bus: add CDX bus controller device tree bindings

[AMD Official Use Only - General]

Quoted 24 lines

-----Original Message-----
From: Krzysztof Kozlowski <[email protected]>
Sent: Tuesday, January 17, 2023 11:24 PM
To: Gupta, Nipun <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux-arm- [email protected]; [email protected]; linux- [email protected]; [email protected]
Cc: [email protected]; Anand, Harpreet <[email protected]>; Agarwal, Nikhil <[email protected]>; Simek, Michal <[email protected]>; git (AMD-Xilinx) <[email protected]>
Subject: Re: [PATCH 03/19] dt-bindings: bus: add CDX bus controller device tree bindings

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

Quoted 5 lines
Quoted 1 line · Nipun Gupta

Add device tree bindings for CDX bus controller.

Subject: drop second/last, redundant "device tree bindings". The "dt-bindings" prefix is already stating that these are bindings.

Agree. Will update.

Quoted 54 lines
Quoted 8 lines
---
 .../bindings/bus/xlnx,cdxbus-controller.yaml  | 68 +++++++++++++++++++
 MAINTAINERS                                   |  1 +
 2 files changed, 69 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/bus/xlnx,cdxbus-

controller.yaml

Review

Quoted 2 lines
(quoted diff) +0 -0

controller.yaml b/Documentation/devicetree/bindings/bus/xlnx,cdxbus- controller.yaml

Quoted 41 lines

new file mode 100644
index 000000000000..b2f186864021

Documentation/devicetree/bindings/bus/xlnx,cdxbus-controller.yaml new file +36 -0
@@ -0,0 +1,68 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/bus/xlnx,cdxbus-controller.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: AMD CDX bus controller
+
+description: |
+  CDX bus controller for AMD devices is implemented to dynamically
+  detect CDX bus and devices on these bus using the firmware.
+  The CDX bus manages multiple FPGA based hardware devices, which
+  can support network, crypto or any other specialized type of
+  devices. These FPGA based devices can be added/modified dynamically
+  on run-time.
+
+  All devices on the CDX bus will have a unique streamid (for IOMMU)
+  and a unique device ID (for MSI) corresponding to a requestor ID
+  (one to one associated with the device). The streamid and deviceid
+  are used to configure SMMU and GIC-ITS respectively.
+
+  iommu-map property is used to define the set of stream ids
+  corresponding to each device and the associated IOMMU.
+
+  The MSI writes are accompanied by sideband data (Device ID).
+  The msi-map property is used to associate the devices with the
+  device ID as well as the associated ITS controller.
+
+  rproc property (xlnx,rproc) is used to identify the remote processor
+  with which APU (Application Processor Unit) interacts to find out
+  the bus and device configuration.
+
+maintainers:
+  - Nipun Gupta <[email protected]>
+  - Nikhil Agarwal <[email protected]>
+
Quoted 8 lines

Documentation/devicetree/bindings/bus/xlnx,cdxbus-controller.yaml line 39

Quoted 3 lines
Documentation/devicetree/bindings/bus/xlnx,cdxbus-controller.yaml new file +3 -0
+properties:
+  compatible:
+    const: xlnx,cdxbus-controller

This misses SoC specific compatible. Drop "bus" - redundant. I would also say - drop controller - do you see any other devices with such compatible naming? Use naming consistent with other devices in the kernel. Just open some controllers - SPI, I2C etc. and look there.

Makes sense. Will use "xlnx,cdx" in compatible.

Quoted 5 lines

Documentation/devicetree/bindings/bus/xlnx,cdxbus-controller.yaml line 41

Quoted 3 lines
Documentation/devicetree/bindings/bus/xlnx,cdxbus-controller.yaml new file +2 -0
+
+  iommu-map: true

No mask?

Currently there is no use for iommu-map-mask as RID's are one to one mapped stream ID's, so we have not added this optional property.

Quoted 17 lines
Quoted 17 lines
Documentation/devicetree/bindings/bus/xlnx,cdxbus-controller.yaml new file +16 -0
+
+  msi-map: true
+
+  xlnx,rproc:
+    $ref: /schemas/types.yaml#/definitions/phandle
+    description:
+      phandle to the remoteproc_r5 rproc node using which APU interacts
+      with remote processor.
+
+required:
+  - compatible
+  - iommu-map
+  - msi-map
+  - xlnx,rproc
+
+additionalProperties: false
Quoted 8 lines

Documentation/devicetree/bindings/bus/xlnx,cdxbus-controller.yaml line 61

Quoted 4 lines
Documentation/devicetree/bindings/bus/xlnx,cdxbus-controller.yaml new file +4 -0
+
+examples:
+  - |
+    cdxbus-controller {

Node names should be generic, so just cdx.
https://devicetree-specification.readthedocs.io/en/latest/chapter2-devicetree-
basics.html#generic-names-recommendation

Sure. Will update to cdx.

Thanks,
Nipun

Quoted 7 lines

Documentation/devicetree/bindings/bus/xlnx,cdxbus-controller.yaml line 64

Quoted 4 lines
Documentation/devicetree/bindings/bus/xlnx,cdxbus-controller.yaml new file +3 -0
+        compatible = "xlnx,cdxbus-controller";
+        /* define map for RIDs 250-259 */
+        iommu-map = <250 &smmu 250 10>;

Best regards,
Krzysztof

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

#10 lore raw

Re: [PATCH 03/19] dt-bindings: bus: add CDX bus controller device tree bindings

Quoted 1 line · Nipun

[AMD Official Use Only - General]

Fix your email client. This is not helping us. What shall I do with it?

Quoted 3 lines

(...)

Quoted 10 lines
Quoted 8 lines

Documentation/devicetree/bindings/bus/xlnx,cdxbus-controller.yaml line 39

Quoted 3 lines
Documentation/devicetree/bindings/bus/xlnx,cdxbus-controller.yaml new file +3 -0
+properties:
+  compatible:
+    const: xlnx,cdxbus-controller

This misses SoC specific compatible. Drop "bus" - redundant. I would also say - drop controller - do you see any other devices with such compatible naming? Use naming consistent with other devices in the kernel. Just open some controllers - SPI, I2C etc. and look there.

Makes sense. Will use "xlnx,cdx" in compatible.

No, this still misses device specific compatible. You did ignored half of my comment now.

Best regards,
Krzysztof

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

author maintainer
#11 lore raw

RE: [PATCH 03/19] dt-bindings: bus: add CDX bus controller device tree bindings

Quoted 21 lines

-----Original Message-----
From: Krzysztof Kozlowski <[email protected]>
Sent: Wednesday, January 18, 2023 6:13 PM
To: Gupta, Nipun <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux-arm- [email protected]; [email protected]; linux- [email protected]; [email protected]
Cc: [email protected]; Anand, Harpreet <[email protected]>; Agarwal, Nikhil <[email protected]>; Simek, Michal <[email protected]>; git (AMD-Xilinx) <[email protected]>
Subject: Re: [PATCH 03/19] dt-bindings: bus: add CDX bus controller device tree bindings

(...)

Quoted 9 lines
Quoted 9 lines
Quoted 8 lines

Documentation/devicetree/bindings/bus/xlnx,cdxbus-controller.yaml line 39

Quoted 3 lines
Documentation/devicetree/bindings/bus/xlnx,cdxbus-controller.yaml new file +3 -0
+properties:
+  compatible:
+    const: xlnx,cdxbus-controller

This misses SoC specific compatible. Drop "bus" - redundant. I would also say - drop controller - do you see any other devices with such compatible naming? Use naming consistent with other devices in the kernel. Just open some controllers - SPI, I2C etc. and look there.

Quoted 4 lines
Quoted 1 line

Makes sense. Will use "xlnx,cdx" in compatible.

No, this still misses device specific compatible. You did ignored half of my comment now.

Sorry I missed that part. Will rename this to "xlnx,versal-cdx".

Thanks,
Nipun

Quoted 4 lines

Best regards,
Krzysztof

author maintainer
#12 lore raw

PATCH 4/19 bus/cdx: add MCDI protocol interface for firmware interaction

In reply to Nipun Gupta

The MCDI (Management CPU Driver Interface) is used as a protocol to communicate with the RPU firmware. It has pre-defined set of messages for different message exchanges between APU and RPU.

---
 drivers/bus/cdx/Kconfig                  |   2 +
 drivers/bus/cdx/Makefile                 |   2 +-
 drivers/bus/cdx/controller/Kconfig       |  20 +
 drivers/bus/cdx/controller/Makefile      |   9 +
 drivers/bus/cdx/controller/bitfield.h    |  88 +++
 drivers/bus/cdx/controller/mc_cdx_pcol.h | 590 +++++++++++++++
 drivers/bus/cdx/controller/mcdi.c        | 918 +++++++++++++++++++++++
 drivers/bus/cdx/controller/mcdi.h        | 250 ++++++
 8 files changed, 1878 insertions(+), 1 deletion(-)
 create mode 100644 drivers/bus/cdx/controller/Kconfig
 create mode 100644 drivers/bus/cdx/controller/Makefile
 create mode 100644 drivers/bus/cdx/controller/bitfield.h
 create mode 100644 drivers/bus/cdx/controller/mc_cdx_pcol.h
 create mode 100644 drivers/bus/cdx/controller/mcdi.c
 create mode 100644 drivers/bus/cdx/controller/mcdi.h
drivers/bus/cdx/Kconfig +2 -0
index 54e0623ebcff..a7e474d621e0 100644
@@ -12,3 +12,5 @@ config CDX_BUS
 	  scanning and probing of CDX devices. CDX devices are memory
 	  mapped on system bus for embedded CPUs. CDX bus uses CDX
 	  controller and firmware to scan the CDX devices.
+
+source "drivers/bus/cdx/controller/Kconfig"
drivers/bus/cdx/Makefile +1 -1
index c5feb11fc718..0324e4914f6e 100644
@@ -5,4 +5,4 @@
 # Copyright (C) 2022-2023, Advanced Micro Devices, Inc.
 #
 
-obj-$(CONFIG_CDX_BUS) += cdx.o
+obj-$(CONFIG_CDX_BUS) += cdx.o controller/
drivers/bus/cdx/controller/Kconfig new file +20 -0
new file mode 100644
index 000000000000..785c71063b2a
@@ -0,0 +1,20 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# CDX controller configuration
+#
+# Copyright (C) 2022-2023, Advanced Micro Devices, Inc.
+#
+
+if CDX_BUS
+
+config MCDI_LOGGING
+	bool "MCDI Logging for the CDX controller"
+	depends on CDX_CONTROLLER
+	help
+	  Enable MCDI Logging for
+	  the CDX Controller for debug
+	  purpose.
+
+	  If unsure, say N.
+
+endif
drivers/bus/cdx/controller/Makefile new file +9 -0
new file mode 100644
index 000000000000..0ce200678eda
@@ -0,0 +1,9 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Makefile for CDX controller drivers
+#
+# Copyright (C) 2022-2023, Advanced Micro Devices, Inc.
+#
+
+obj-$(CONFIG_CDX_CONTROLLER) += cdx-controller.o
+cdx-controller-objs := mcdi.o
drivers/bus/cdx/controller/bitfield.h new file +88 -0

88 lines not on this page · raw message

drivers/bus/cdx/controller/mc_cdx_pcol.h new file +590 -0

590 lines not on this page · raw message

drivers/bus/cdx/controller/mcdi.c new file +918 -0

918 lines not on this page · raw message

drivers/bus/cdx/controller/mcdi.h new file +250 -0

250 lines not on this page · raw message

Signature · 1 line
2.17.1
#13 lore raw

Re: [PATCH 04/19] bus/cdx: add MCDI protocol interface for firmware interaction

Quoted 10 lines · Nipun Gupta
drivers/bus/cdx/controller/mc_cdx_pcol.h new file +10 -0
+/** Request/Response structure */
+#define MCDI_HEADER_OFST 0
+#define MCDI_HEADER_CODE_LBN 0
+#define MCDI_HEADER_CODE_WIDTH 7
+#define MCDI_HEADER_RESYNC_LBN 7
+#define MCDI_HEADER_RESYNC_WIDTH 1
+#define MCDI_HEADER_DATALEN_LBN 8
+#define MCDI_HEADER_DATALEN_WIDTH 8
+#define MCDI_HEADER_SEQ_LBN 16
+#define MCDI_HEADER_SEQ_WIDTH 4

drivers/bus/cdx/controller/mc_cdx_pcol.h line 63

Quoted 10 lines
drivers/bus/cdx/controller/mc_cdx_pcol.h new file +10 -0
+#define MCDI_HEADER_RSVD_LBN 20
+#define MCDI_HEADER_RSVD_WIDTH 1
+#define MCDI_HEADER_NOT_EPOCH_LBN 21
+#define MCDI_HEADER_NOT_EPOCH_WIDTH 1
+#define MCDI_HEADER_ERROR_LBN 22
+#define MCDI_HEADER_ERROR_WIDTH 1
+#define MCDI_HEADER_RESPONSE_LBN 23
+#define MCDI_HEADER_RESPONSE_WIDTH 1
+#define MCDI_HEADER_XFLAGS_LBN 24
+#define MCDI_HEADER_XFLAGS_WIDTH 8

<snip>

This whole file could use some tabs to align all of the values for the defines to make it readable. Any chance of doing that?

thanks,

greg k-h

author maintainer
#14 lore raw

RE: [PATCH 04/19] bus/cdx: add MCDI protocol interface for firmware interaction

[AMD Official Use Only - General]

Quoted 47 lines

-----Original Message-----
From: Greg KH <[email protected]>
Sent: Tuesday, January 17, 2023 7:38 PM
To: Gupta, Nipun <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux-arm- [email protected]; [email protected]; linux- [email protected]; [email protected]; [email protected]; Anand, Harpreet <[email protected]>; Agarwal, Nikhil <[email protected]>; Simek, Michal <[email protected]>; git (AMD-Xilinx) <[email protected]>
Subject: Re: [PATCH 04/19] bus/cdx: add MCDI protocol interface for firmware interaction

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

Quoted 20 lines · Nipun Gupta
drivers/bus/cdx/controller/mc_cdx_pcol.h new file +20 -0
+/** Request/Response structure */
+#define MCDI_HEADER_OFST 0
+#define MCDI_HEADER_CODE_LBN 0
+#define MCDI_HEADER_CODE_WIDTH 7
+#define MCDI_HEADER_RESYNC_LBN 7
+#define MCDI_HEADER_RESYNC_WIDTH 1
+#define MCDI_HEADER_DATALEN_LBN 8
+#define MCDI_HEADER_DATALEN_WIDTH 8
+#define MCDI_HEADER_SEQ_LBN 16
+#define MCDI_HEADER_SEQ_WIDTH 4
+#define MCDI_HEADER_RSVD_LBN 20
+#define MCDI_HEADER_RSVD_WIDTH 1
+#define MCDI_HEADER_NOT_EPOCH_LBN 21
+#define MCDI_HEADER_NOT_EPOCH_WIDTH 1
+#define MCDI_HEADER_ERROR_LBN 22
+#define MCDI_HEADER_ERROR_WIDTH 1
+#define MCDI_HEADER_RESPONSE_LBN 23
+#define MCDI_HEADER_RESPONSE_WIDTH 1
+#define MCDI_HEADER_XFLAGS_LBN 24
+#define MCDI_HEADER_XFLAGS_WIDTH 8

<snip>

Quoted 2 lines

This whole file could use some tabs to align all of the values for the defines to make it readable. Any chance of doing that?

Yes sure. Will update this.

Quoted 4 lines

thanks,

greg k-h

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

author maintainer
#15 lore raw

PATCH 5/19 bus/cdx: add cdx controller

In reply to Nipun Gupta

CDX controller uses MCDI interface as a protocol to communicate with the RPU firmware and registers the detected CDX devices on the CDX bus. It also uses RPMsg as the communication channel with the Firmware.

---
 drivers/bus/cdx/controller/Kconfig          |   9 +
 drivers/bus/cdx/controller/Makefile         |   2 +-
 drivers/bus/cdx/controller/cdx_controller.c | 243 ++++++++++++++++++++
 drivers/bus/cdx/controller/mcdi_functions.c | 125 ++++++++++
 drivers/bus/cdx/controller/mcdi_functions.h |  50 ++++
 5 files changed, 428 insertions(+), 1 deletion(-)
 create mode 100644 drivers/bus/cdx/controller/cdx_controller.c
 create mode 100644 drivers/bus/cdx/controller/mcdi_functions.c
 create mode 100644 drivers/bus/cdx/controller/mcdi_functions.h
drivers/bus/cdx/controller/Kconfig +9 -0
index 785c71063b2a..17f9c6be2fe1 100644
@@ -7,6 +7,15 @@
 
 if CDX_BUS
 
+config CDX_CONTROLLER
+	tristate "CDX bus controller"
+	help
+	  CDX controller drives the CDX bus. It interacts with
+	  firmware to get the hardware devices and registers with
+	  the CDX bus. Say Y to enable the CDX hardware driver.
+
+	  If unsure, say N.
+
 config MCDI_LOGGING
 	bool "MCDI Logging for the CDX controller"
 	depends on CDX_CONTROLLER
drivers/bus/cdx/controller/Makefile +1 -1
index 0ce200678eda..f7437c882cc9 100644
@@ -6,4 +6,4 @@
 #
 
 obj-$(CONFIG_CDX_CONTROLLER) += cdx-controller.o
-cdx-controller-objs := mcdi.o
+cdx-controller-objs := cdx_controller.o mcdi.o mcdi_functions.o
drivers/bus/cdx/controller/cdx_controller.c new file +243 -0

243 lines not on this page · raw message

drivers/bus/cdx/controller/mcdi_functions.c new file +125 -0

125 lines not on this page · raw message

drivers/bus/cdx/controller/mcdi_functions.h new file +50 -0

50 lines not on this page · raw message

Signature · 1 line
2.17.1
#16 lore raw

Re: [PATCH 05/19] bus/cdx: add cdx controller

drivers/bus/cdx/controller/cdx_controller.c line 3

Quoted 6 lines · Nipun Gupta
drivers/bus/cdx/controller/cdx_controller.c new file +3 -0
@@ -0,0 +1,243 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Platform driver for CDX bus controller.

Why is this a platform driver? Shouldn't it also be on some type of bus so that you can find it?

drivers/bus/cdx/controller/cdx_controller.c line 240

Quoted 1 line
drivers/bus/cdx/controller/cdx_controller.c new file +1 -0
+MODULE_VERSION("1.0");

There's never need for any module versions once the code is in the kernel tree as then they make no sense at all. Please drop them from this series.

thanks,

greg k-h

author maintainer
#17 lore raw

RE: [PATCH 05/19] bus/cdx: add cdx controller

[AMD Official Use Only - General]

Quoted 23 lines

-----Original Message-----
From: Greg KH <[email protected]>
Sent: Tuesday, January 17, 2023 7:40 PM
To: Gupta, Nipun <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux-arm- [email protected]; [email protected]; linux- [email protected]; [email protected]; [email protected]; Anand, Harpreet <[email protected]>; Agarwal, Nikhil <[email protected]>; Simek, Michal <[email protected]>; git (AMD-Xilinx) <[email protected]>
Subject: Re: [PATCH 05/19] bus/cdx: add cdx controller

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

On Tue, Jan 17, 2023 at 07:11:37PM +0530, Nipun Gupta wrote:

Quoted 9 lines

drivers/bus/cdx/controller/cdx_controller.c line 3

Quoted 6 lines
drivers/bus/cdx/controller/cdx_controller.c new file +3 -0
@@ -0,0 +1,243 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Platform driver for CDX bus controller.

Why is this a platform driver? Shouldn't it also be on some type of bus so that you can find it?

This is host controller for CDX bus similar to PCI controller which is also on the platform bus.
Since CDX bus controller is based on communication with RPU firmware we need to have references to remoteproc device in CDX controller node to use the RPMsg device.

Quoted 6 lines

drivers/bus/cdx/controller/cdx_controller.c line 240

Quoted 2 lines
drivers/bus/cdx/controller/cdx_controller.c new file +1 -0
+MODULE_VERSION("1.0");

There's never need for any module versions once the code is in the kernel tree as then they make no sense at all. Please drop them from this series.

Sure. Will remove.

Quoted 4 lines

thanks,

greg k-h

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

author maintainer
#18 lore raw

PATCH 6/19 bus/cdx: add rpmsg communication channel for CDX

In reply to Nipun Gupta

From: Abhijit Gangurde <[email protected]>

RPMsg is used as a transport communication channel. This change introduces RPMsg driver and integrates it with the CDX controller.

---
 drivers/bus/cdx/controller/Kconfig          |   1 +
 drivers/bus/cdx/controller/Makefile         |   2 +-
 drivers/bus/cdx/controller/cdx_controller.c |  36 +++-
 drivers/bus/cdx/controller/cdx_controller.h |  30 +++
 drivers/bus/cdx/controller/cdx_rpmsg.c      | 222 ++++++++++++++++++++
 drivers/bus/cdx/controller/mcdi.h           |   9 +
 6 files changed, 291 insertions(+), 9 deletions(-)
 create mode 100644 drivers/bus/cdx/controller/cdx_controller.h
 create mode 100644 drivers/bus/cdx/controller/cdx_rpmsg.c
drivers/bus/cdx/controller/Kconfig +1 -0
index 17f9c6be2fe1..aea3ac86d3aa 100644
@@ -9,6 +9,7 @@ if CDX_BUS
 
 config CDX_CONTROLLER
 	tristate "CDX bus controller"
+	select RPMSG
 	help
 	  CDX controller drives the CDX bus. It interacts with
 	  firmware to get the hardware devices and registers with
drivers/bus/cdx/controller/Makefile +1 -1
index f7437c882cc9..f071be411d96 100644
@@ -6,4 +6,4 @@
 #
 
 obj-$(CONFIG_CDX_CONTROLLER) += cdx-controller.o
-cdx-controller-objs := cdx_controller.o mcdi.o mcdi_functions.o
+cdx-controller-objs := cdx_controller.o cdx_rpmsg.o mcdi.o mcdi_functions.o
drivers/bus/cdx/controller/cdx_controller.c +28 -8
index 9b910c9cb31e..b62a6d3b7bd4 100644
@@ -8,6 +8,7 @@
 #include <linux/of_platform.h>
 #include <linux/cdx/cdx_bus.h>
 
+#include "cdx_controller.h"
 #include "../cdx.h"
 #include "mcdi_functions.h"
 #include "mcdi.h"
@@ -53,20 +54,14 @@ static void cdx_mcdi_request(struct cdx_mcdi *cdx, u8 bufid,
 			     const struct cdx_dword *hdr, size_t hdr_len,
 			     const struct cdx_dword *sdu, size_t sdu_len)
 {
-	/*
-	 * This will get updated by rpmsg APIs, with RPMSG introduction
-	 * in CDX controller as a transport layer.
-	 */
+	cdx_rpmsg_send(cdx, hdr, hdr_len, sdu, sdu_len);
 }
 
 static void cdx_mcdi_read_response(struct cdx_mcdi *cdx_mcdi, u8 bufid,
 				   struct cdx_dword *outbuf, size_t offset,
 				   size_t outlen)
 {
-	/*
-	 * This will get updated by rpmsg APIs, with RPMSG introduction
-	 * in CDX controller as a transport layer.
-	 */
+	cdx_rpmsg_read_resp(cdx_mcdi, outbuf, offset, outlen);
 }
 
 static const struct cdx_mcdi_ops mcdi_ops = {
@@ -77,6 +72,19 @@ static const struct cdx_mcdi_ops mcdi_ops = {
 	.mcdi_put_buf = cdx_mcdi_put_buf,
 };
 
+void cdx_rpmsg_post_probe(struct cdx_controller *cdx)
+{
+	/* Register CDX controller with CDX bus driver */
+	if (cdx_register_controller(cdx))
+		dev_err(cdx->dev, "Failed to register CDX controller\n");
+}
+
+void cdx_rpmsg_pre_remove(struct cdx_controller *cdx)
+{
+	cdx_unregister_controller(cdx);
+	cdx_mcdi_wait_for_quiescence(cdx->priv, MCDI_RPC_TIMEOUT);
+}
+
 static int cdx_scan_devices(struct cdx_controller *cdx)
 {
 	struct cdx_mcdi *cdx_mcdi = cdx->priv;
@@ -174,9 +182,19 @@ static int xlnx_cdx_probe(struct platform_device *pdev)
 		goto cdx_mcdi_buf_fail;
 	}
 
+	ret = cdx_setup_rpmsg(pdev);
+	if (ret) {
+		if (ret != -EPROBE_DEFER)
+			dev_err(&pdev->dev, "Failed to register CDX RPMsg transport\n");
+		goto cdx_rpmsg_fail;
+	}
+
 	dev_info(&pdev->dev, "Successfully registered CDX controller with RPMsg as transport\n");
 	return 0;
 
+cdx_rpmsg_fail:
+	kfree(cdx_mcdi->mcdi_buf);
+	cdx_mcdi->mcdi_buf = NULL;
 cdx_mcdi_buf_fail:
 	kfree(cdx);
 cdx_alloc_fail:
@@ -192,6 +210,8 @@ static int xlnx_cdx_remove(struct platform_device *pdev)
 	struct cdx_controller *cdx = platform_get_drvdata(pdev);
 	struct cdx_mcdi *cdx_mcdi = cdx->priv;
 
+	cdx_destroy_rpmsg(pdev);
+
 	kfree(cdx_mcdi->mcdi_buf);
 	kfree(cdx);
 
drivers/bus/cdx/controller/cdx_controller.h new file +30 -0
new file mode 100644
index 000000000000..43b7c742df87
@@ -0,0 +1,30 @@
+/* SPDX-License-Identifier: GPL-2.0
+ *
+ * Header file for the CDX Controller
+ *
+ * Copyright (C) 2022-2023, Advanced Micro Devices, Inc.
+ */
+
+#ifndef _CDX_CONTROLLER_H_
+#define _CDX_CONTROLLER_H_
+
+#include <linux/cdx/cdx_bus.h>
+#include "mcdi_functions.h"
+
+void cdx_rpmsg_post_probe(struct cdx_controller *cdx);
+
+void cdx_rpmsg_pre_remove(struct cdx_controller *cdx);
+
+int cdx_rpmsg_send(struct cdx_mcdi *cdx_mcdi,
+		   const struct cdx_dword *hdr, size_t hdr_len,
+		   const struct cdx_dword *sdu, size_t sdu_len);
+
+void cdx_rpmsg_read_resp(struct cdx_mcdi *cdx_mcdi,
+			 struct cdx_dword *outbuf, size_t offset,
+			 size_t outlen);
+
+int cdx_setup_rpmsg(struct platform_device *pdev);
+
+void cdx_destroy_rpmsg(struct platform_device *pdev);
+
+#endif /* _CDX_CONT_PRIV_H_ */
drivers/bus/cdx/controller/cdx_rpmsg.c new file +222 -0

222 lines not on this page · raw message

drivers/bus/cdx/controller/mcdi.h +9 -0
index a2ee8821b6fe..54e1b27c2a53 100644
@@ -64,6 +64,10 @@ enum cdx_mcdi_cmd_state {
  * @mcdi_buf_use: Stores MCDI buffer usage
  * @mcdi_ops: MCDI operations
  * @mcdi_buf: MCDI receive buffer
+ * @r5_rproc : R5 Remoteproc device handle
+ * @rpdev: RPMsg device
+ * @ept: RPMsg endpoint
+ * @work: Post probe work
  */
 struct cdx_mcdi {
 	/* MCDI interface */
@@ -71,6 +75,11 @@ struct cdx_mcdi {
 	unsigned long mcdi_buf_use;
 	const struct cdx_mcdi_ops *mcdi_ops;
 	void *mcdi_buf;
+
+	struct rproc *r5_rproc;
+	struct rpmsg_device *rpdev;
+	struct rpmsg_endpoint *ept;
+	struct work_struct work;
 };
 
 struct cdx_mcdi_ops {
Signature · 1 line
2.17.1
author maintainer
#19 lore raw

PATCH 7/19 bus/cdx: add device attributes

In reply to Nipun Gupta

Create sysfs entry for CDX devices.

Sysfs entries provided in each of the CDX device detected by the CDX controller
- vendor id
- device id
- remove
- reset of the device.
- driver override

---
 Documentation/ABI/testing/sysfs-bus-cdx     |  34 +++++
 drivers/bus/cdx/cdx.c                       | 144 ++++++++++++++++++++
 drivers/bus/cdx/controller/cdx_controller.c |  19 +++
 drivers/bus/cdx/controller/mcdi_functions.c |  14 ++
 drivers/bus/cdx/controller/mcdi_functions.h |  11 ++
 include/linux/cdx/cdx_bus.h                 |  23 ++++
 6 files changed, 245 insertions(+)
Documentation/ABI/testing/sysfs-bus-cdx +34 -0
index 8c2425fdb6d9..1e0fdce18cde 100644
@@ -10,3 +10,37 @@ Description:
                 For example::
 
 		  # echo 1 > /sys/bus/cdx/rescan
+
+What:		/sys/bus/cdx/devices/.../vendor
+Date:		January 2023
+Contact:	[email protected]
+Description:
+		Vendor ID for this CDX device
+
+What:		/sys/bus/cdx/devices/.../device
+Date:		January 2023
+Contact:	[email protected]
+Description:
+		Device ID for this CDX device
+
+What:		/sys/bus/cdx/devices/.../reset
+Date:		January 2023
+Contact:	[email protected]
+Description:
+		Writing a non-zero value to this file would reset the
+		CDX device
+
+                For example::
+
+		  # echo 1 > /sys/bus/cdx/.../reset
+
+What:		/sys/bus/cdx/devices/.../remove
+Date:		January 2023
+Contact:	[email protected]
+Description:
+		Writing a non-zero value to this file would remove the
+		corrosponding device from the CDX bus
+
+		For example::
+
+		# echo 1 > /sys/bus/cdx/devices/.../remove
drivers/bus/cdx/cdx.c +144 -0

162 lines not on this page · raw message

drivers/bus/cdx/controller/cdx_controller.c +19 -0
index b62a6d3b7bd4..dbcd236b360d 100644
@@ -85,6 +85,24 @@ void cdx_rpmsg_pre_remove(struct cdx_controller *cdx)
 	cdx_mcdi_wait_for_quiescence(cdx->priv, MCDI_RPC_TIMEOUT);
 }
 
+static int cdx_configure_device(struct cdx_controller *cdx,
+				u8 bus_num, u8 dev_num,
+				struct cdx_device_config *dev_config)
+{
+	int ret = 0;
+
+	switch (dev_config->type) {
+	case CDX_DEV_RESET_CONF:
+		ret = cdx_mcdi_reset_device(cdx->priv, bus_num, dev_num);
+		break;
+	default:
+		dev_err(cdx->dev, "Invalid device configuration flag\n");
+		ret = -EINVAL;
+	}
+
+	return ret;
+}
+
 static int cdx_scan_devices(struct cdx_controller *cdx)
 {
 	struct cdx_mcdi *cdx_mcdi = cdx->priv;
@@ -144,6 +162,7 @@ static int cdx_scan_devices(struct cdx_controller *cdx)
 
 static struct cdx_ops cdx_ops = {
 	.scan		= cdx_scan_devices,
+	.dev_configure	= cdx_configure_device,
 };
 
 static int xlnx_cdx_probe(struct platform_device *pdev)
drivers/bus/cdx/controller/mcdi_functions.c +14 -0
index 3940a2c7919c..673b3896411e 100644
@@ -123,3 +123,17 @@ int cdx_mcdi_get_dev_config(struct cdx_mcdi *cdx,
 
 	return 0;
 }
+
+int cdx_mcdi_reset_device(struct cdx_mcdi *cdx, u8 bus_num, u8 dev_num)
+{
+	MCDI_DECLARE_BUF(inbuf, MC_CMD_CDX_DEVICE_RESET_IN_LEN);
+	int rc;
+
+	MCDI_SET_DWORD(inbuf, CDX_DEVICE_RESET_IN_BUS, bus_num);
+	MCDI_SET_DWORD(inbuf, CDX_DEVICE_RESET_IN_DEVICE, dev_num);
+
+	rc = cdx_mcdi_rpc(cdx, MC_CMD_CDX_DEVICE_RESET, inbuf, sizeof(inbuf),
+			  NULL, 0, NULL);
+
+	return rc;
+}
drivers/bus/cdx/controller/mcdi_functions.h +11 -0
index 97dfde18592f..54d6017ab7d1 100644
@@ -47,4 +47,15 @@ int cdx_mcdi_get_dev_config(struct cdx_mcdi *cdx,
 			    u8 bus_num, u8 dev_num,
 			    struct cdx_dev_params *dev_params);
 
+/**
+ * cdx_mcdi_reset_device - Reset cdx device represented by bus_num:dev_num
+ * @cdx: pointer to MCDI interface.
+ * @bus_num: Bus number.
+ * @dev_num: Device number.
+ *
+ * @return: 0 on success, <0 on failure
+ */
+int cdx_mcdi_reset_device(struct cdx_mcdi *cdx,
+			  u8 bus_num, u8 dev_num);
+
 #endif /* CDX_MCDI_FUNCTIONS_H */
include/linux/cdx/cdx_bus.h +23 -0
index 1d28f11da5e7..9f055739ed30 100644
@@ -21,8 +21,20 @@
 /* Forward declaration for CDX controller */
 struct cdx_controller;
 
+enum {
+	CDX_DEV_RESET_CONF,
+};
+
+struct cdx_device_config {
+	u8 type;
+};
+
 typedef int (*cdx_scan_cb)(struct cdx_controller *cdx);
 
+typedef int (*cdx_dev_configure_cb)(struct cdx_controller *cdx,
+				    u8 bus_num, u8 dev_num,
+				    struct cdx_device_config *dev_config);
+
 /**
  * CDX_DEVICE_DRIVER_OVERRIDE - macro used to describe a CDX device with
  *                              override_only flags.
@@ -39,9 +51,12 @@ typedef int (*cdx_scan_cb)(struct cdx_controller *cdx);
 /**
  * struct cdx_ops - Callbacks supported by CDX controller.
  * @scan: scan the devices on the controller
+ * @dev_configure: configuration like reset, master_enable,
+ *		   msi_config etc for a CDX device
  */
 struct cdx_ops {
 	cdx_scan_cb scan;
+	cdx_dev_configure_cb dev_configure;
 };
 
 /**
@@ -150,4 +165,12 @@ void cdx_driver_unregister(struct cdx_driver *cdx_driver);
 
 extern struct bus_type cdx_bus_type;
 
+/**
+ * cdx_dev_reset - Reset CDX device
+ * @dev: device pointer
+ *
+ * @return: 0 for success, -errno on failure
+ */
+int cdx_dev_reset(struct device *dev);
+
 #endif /* _CDX_BUS_H_ */
Signature · 1 line
2.17.1
#20 lore raw

Re: [PATCH 07/19] bus/cdx: add device attributes

Quoted 27 lines · Nipun Gupta

Create sysfs entry for CDX devices.

Sysfs entries provided in each of the CDX device detected by the CDX controller
- vendor id
- device id
- remove
- reset of the device.
- driver override

---
 Documentation/ABI/testing/sysfs-bus-cdx     |  34 +++++
 drivers/bus/cdx/cdx.c                       | 144 ++++++++++++++++++++
 drivers/bus/cdx/controller/cdx_controller.c |  19 +++
 drivers/bus/cdx/controller/mcdi_functions.c |  14 ++
 drivers/bus/cdx/controller/mcdi_functions.h |  11 ++
 include/linux/cdx/cdx_bus.h                 |  23 ++++
 6 files changed, 245 insertions(+)
Documentation/ABI/testing/sysfs-bus-cdx +0 -0
index 8c2425fdb6d9..1e0fdce18cde 100644

Documentation/ABI/testing/sysfs-bus-cdx Description: · line 18

Quoted 9 lines
Documentation/ABI/testing/sysfs-bus-cdx +6 -0
@@ -10,3 +10,37 @@ Description:
                 For example::
 
 		  # echo 1 > /sys/bus/cdx/rescan
+
+What:		/sys/bus/cdx/devices/.../vendor
+Date:		January 2023
+Contact:	[email protected]
+Description:
+		Vendor ID for this CDX device

What format is this in? How big is it? Examples?

Documentation/ABI/testing/sysfs-bus-cdx Description: · line 24

Quoted 6 lines
Documentation/ABI/testing/sysfs-bus-cdx +6 -0
+
+What:		/sys/bus/cdx/devices/.../device
+Date:		January 2023
+Contact:	[email protected]
+Description:
+		Device ID for this CDX device

Same here, be specific.

Documentation/ABI/testing/sysfs-bus-cdx Description: · line 35

Quoted 11 lines
Documentation/ABI/testing/sysfs-bus-cdx +11 -0
+
+What:		/sys/bus/cdx/devices/.../reset
+Date:		January 2023
+Contact:	[email protected]
+Description:
+		Writing a non-zero value to this file would reset the
+		CDX device
+
+                For example::
+
+		  # echo 1 > /sys/bus/cdx/.../reset

Will that remove the device from the driver too?

Again, more documentation please.

Documentation/ABI/testing/sysfs-bus-cdx Description: · line 46

Quoted 11 lines
Documentation/ABI/testing/sysfs-bus-cdx +11 -0
+
+What:		/sys/bus/cdx/devices/.../remove
+Date:		January 2023
+Contact:	[email protected]
+Description:
+		Writing a non-zero value to this file would remove the
+		corrosponding device from the CDX bus
+
+		For example::
+
+		# echo 1 > /sys/bus/cdx/devices/.../remove

Why would you want to remove a device from the bus like this?

Quoted 7 lines
drivers/bus/cdx/cdx.c +0 -0
index 2737470f08d3..1372d8dcaa4c 100644
@@ -72,6 +72,38 @@ static DECLARE_BITMAP(cdx_controller_id_map, MAX_CDX_CONTROLLERS);
 /* List of CDX controllers registered with the CDX bus */
 static LIST_HEAD(cdx_controllers);

drivers/bus/cdx/cdx.c static DECLARE_BITMAP(cdx_controller_id_map, MAX_CDX_CONTROLLERS); · line 83

Quoted 10 lines
drivers/bus/cdx/cdx.c +9 -0
 
+/**
+ * reset_cdx_device - Reset a CDX device
+ * @dev: CDX device
+ * @data: This is always passed as NULL, and is not used in this API,
+ *	  but is required here as the bus_for_each_dev() API expects
+ *	  the passed function (reset_cdx_device) to have this
+ *	  as an argument.
+ *
+ * @return: -errno on failure, 0 on success.

I recommend this function actually be the one without the data pointer, that way you don't get confused here.

Quoted 12 lines
drivers/bus/cdx/cdx.c +12 -0
+ */
+static int reset_cdx_device(struct device *dev, void *data)
+{
+	struct cdx_device *cdx_dev = to_cdx_device(dev);
+	struct cdx_controller *cdx = cdx_dev->cdx;
+	struct cdx_device_config dev_config;
+	int ret;
+
+	dev_config.type = CDX_DEV_RESET_CONF;
+	ret = cdx->ops->dev_configure(cdx, cdx_dev->bus_num,
+				      cdx_dev->dev_num, &dev_config);
+	if (ret)

drivers/bus/cdx/cdx.c static DECLARE_BITMAP(cdx_controller_id_map, MAX_CDX_CONTROLLERS); · line 105

Quoted 10 lines
drivers/bus/cdx/cdx.c +10 -0
+		dev_err(dev, "cdx device reset failed\n");
+
+	return ret;
+}
+
+int cdx_dev_reset(struct device *dev)
+{
+	return reset_cdx_device(dev, NULL);
+}
+EXPORT_SYMBOL_GPL(cdx_dev_reset);

Remove the indirection as mentioned above please.

Otherwise, very minor comments, nice work.

greg k-h

author maintainer
#21 lore raw

RE: [PATCH 07/19] bus/cdx: add device attributes

[AMD Official Use Only - General]

Quoted 53 lines

-----Original Message-----
From: Greg KH <[email protected]>
Sent: Tuesday, January 17, 2023 7:44 PM
To: Gupta, Nipun <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; linux-arm- [email protected]; [email protected]; linux- [email protected]; [email protected]; [email protected]; Anand, Harpreet <[email protected]>; Agarwal, Nikhil <[email protected]>; Simek, Michal <[email protected]>; git (AMD-Xilinx) <[email protected]>
Subject: Re: [PATCH 07/19] bus/cdx: add device attributes

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

Quoted 22 lines · Nipun Gupta

Create sysfs entry for CDX devices.

Sysfs entries provided in each of the CDX device detected by the CDX controller
- vendor id
- device id
- remove
- reset of the device.
- driver override

---
 Documentation/ABI/testing/sysfs-bus-cdx     |  34 +++++
 drivers/bus/cdx/cdx.c                       | 144 ++++++++++++++++++++
 drivers/bus/cdx/controller/cdx_controller.c |  19 +++
 drivers/bus/cdx/controller/mcdi_functions.c |  14 ++
 drivers/bus/cdx/controller/mcdi_functions.h |  11 ++
 include/linux/cdx/cdx_bus.h                 |  23 ++++
 6 files changed, 245 insertions(+)

Review

Quoted 1 line
(quoted diff) +0 -0

b/Documentation/ABI/testing/sysfs-bus-cdx

Quoted 6 lines

index 8c2425fdb6d9..1e0fdce18cde 100644

Documentation/ABI/testing/sysfs-bus-cdx +0 -0
@@ -10,3 +10,37 @@ Description:
                 For example::
 
Quoted 9 lines

Documentation/ABI/testing/sysfs-bus-cdx Description: · line 18

Quoted 7 lines
Documentation/ABI/testing/sysfs-bus-cdx +6 -0
                # echo 1 > /sys/bus/cdx/rescan
+
+What:                /sys/bus/cdx/devices/.../vendor
+Date:                January 2023
+Contact:     [email protected]
+Description:
+             Vendor ID for this CDX device

What format is this in? How big is it? Examples?

This is 16 bits. Agree need to add more info on this and other params. Will update in next version.

Quoted 10 lines

Documentation/ABI/testing/sysfs-bus-cdx Description: · line 24

Quoted 7 lines
Documentation/ABI/testing/sysfs-bus-cdx +6 -0
+
+What:                /sys/bus/cdx/devices/.../device
+Date:                January 2023
+Contact:     [email protected]
+Description:
+             Device ID for this CDX device

Same here, be specific.

Quoted 13 lines

Documentation/ABI/testing/sysfs-bus-cdx Description: · line 35

Quoted 11 lines
Documentation/ABI/testing/sysfs-bus-cdx +11 -0
+
+What:                /sys/bus/cdx/devices/.../reset
+Date:                January 2023
+Contact:     [email protected]
+Description:
+             Writing a non-zero value to this file would reset the
+             CDX device
+
+                For example::
+
+               # echo 1 > /sys/bus/cdx/.../reset

Will that remove the device from the driver too?

No, it will not remove the device from the driver. We would Introduce reset_done and reset_prepare callbacks which would inform the device driver about reset in next spin.

Quoted 2 lines

Again, more documentation please.

Sure.

Quoted 14 lines

Documentation/ABI/testing/sysfs-bus-cdx Description: · line 46

Quoted 12 lines
Documentation/ABI/testing/sysfs-bus-cdx +11 -0
+
+What:                /sys/bus/cdx/devices/.../remove
+Date:                January 2023
+Contact:     [email protected]
+Description:
+             Writing a non-zero value to this file would remove the
+             corrosponding device from the CDX bus
+
+             For example::
+
+             # echo 1 > /sys/bus/cdx/devices/.../remove

Why would you want to remove a device from the bus like this?

This is required to prepare the bus for applying new fabric configuration (pseudo hot plug). So that driver does not access the device while it is being reconfigured.

Quoted 20 lines

drivers/bus/cdx/cdx.c static DECLARE_BITMAP(cdx_controller_id_map,

Quoted 6 lines
drivers/bus/cdx/cdx.c +0 -0
index 2737470f08d3..1372d8dcaa4c 100644
@@ -72,6 +72,38 @@ static DECLARE_BITMAP(cdx_controller_id_map,

MAX_CDX_CONTROLLERS);

drivers/bus/cdx/cdx.c static DECLARE_BITMAP(cdx_controller_id_map, MAX_CDX_CONTROLLERS); · line 83

Quoted 12 lines
drivers/bus/cdx/cdx.c +9 -0
 /* List of CDX controllers registered with the CDX bus */
 static LIST_HEAD(cdx_controllers);
 
+/**
+ * reset_cdx_device - Reset a CDX device
+ * @dev: CDX device
+ * @data: This is always passed as NULL, and is not used in this API,
+ *     but is required here as the bus_for_each_dev() API expects
+ *     the passed function (reset_cdx_device) to have this
+ *     as an argument.
+ *
+ * @return: -errno on failure, 0 on success.
Quoted 2 lines

I recommend this function actually be the one without the data pointer, that way you don't get confused here.

Agree will update this.

Quoted 26 lines
Quoted 13 lines
drivers/bus/cdx/cdx.c +12 -0
+ */
+static int reset_cdx_device(struct device *dev, void *data)
+{
+     struct cdx_device *cdx_dev = to_cdx_device(dev);
+     struct cdx_controller *cdx = cdx_dev->cdx;
+     struct cdx_device_config dev_config;
+     int ret;
+
+     dev_config.type = CDX_DEV_RESET_CONF;
+     ret = cdx->ops->dev_configure(cdx, cdx_dev->bus_num,
+                                   cdx_dev->dev_num, &dev_config);
+     if (ret)

drivers/bus/cdx/cdx.c static DECLARE_BITMAP(cdx_controller_id_map, MAX_CDX_CONTROLLERS); · line 105

Quoted 10 lines
drivers/bus/cdx/cdx.c +10 -0
+             dev_err(dev, "cdx device reset failed\n");
+
+     return ret;
+}
+
+int cdx_dev_reset(struct device *dev)
+{
+     return reset_cdx_device(dev, NULL);
+}
+EXPORT_SYMBOL_GPL(cdx_dev_reset);

Remove the indirection as mentioned above please.

Quoted 3 lines

Otherwise, very minor comments, nice work.

greg k-h

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

#22 lore raw
In reply to Nipun Gupta
Quoted 16 lines · Nipun Gupta

Introduces AMD CDX bus, which provides a mechanism to discover/rescan CDX devices. The CDX devices are memory mapped on system bus for embedded CPUs.

CDX controller interacts with the firmware to query different CDX devices present in the Fabric and expose them to the Linux host on CDX bus.

This patch series:
- Introduces the CDX bus and CDX devices.
- Device tree binding for CDX controller
- Support for CDX bus in arm-smmu-v3 driver
- Add MCDI (Management CPU Driver Interface) as a protocol for communication with RPU Firmware
- Support RPMSg channel for Firmware communication

Quoted 7 lines

MSI patches for CDX are not added in this series as it's support is being revisited as per patch series:
https://lore.kernel.org/all/[email protected]/
It will be added as separate patches.

RFC changes with stubs were submitted at:
https://lore.kernel.org/linux-arm-kernel/[email protected]/

This is v5 then. Please mark it appropriately, otherwise it looks like it is first submission.

Best regards,
Krzysztof

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel