Skip to contentSkip to conversation

pds_core driver

PATCH Superseded Shannon Nelson 2 participants 17 messages Apr 18, 2023

Subsystems (4)

v10 of 11 Superseded Latest is v11 →

lore
Share & tools
lore
#0 lore raw

v10 cover pds_core driver

Summary:
--------
This patchset implements a new driver for use with the AMD/Pensando Distributed Services Card (DSC), intended to provide core configuration services through the auxiliary_bus and through a couple of EXPORTed functions for use initially in VFio and vDPA feature specific drivers.

To keep this patchset to a manageable size, the pds_vdpa and pds_vfio drivers have been split out into their own patchsets to be reviewed separately.

Detail:
-------
AMD/Pensando is making available a new set of devices for supporting vDPA, VFio, and potentially other features in the Distributed Services Card (DSC). These features are implemented through a PF that serves as a Core device for controlling and configuring its VF devices. These VF devices have separate drivers that use the auxiliary_bus to work through the Core device as the control path.

Currently, the DSC supports standard ethernet operations using the ionic driver. This is not replaced by the Core-based devices - these new devices are in addition to the existing Ethernet device. Typical DSC configurations will include both PDS devices and Ionic Eth devices. However, there is a potential future path for ethernet services to come through this device as well.

The Core device is a new PCI PF/VF device managed by a new driver 'pds_core'. The PF device has access to an admin queue for configuring the services used by the VFs, and sets up auxiliary_bus devices for each vDPA VF for communicating with the drivers for the vDPA devices. The VFs may be for VFio or vDPA, and other services in the future; these VF types are selected as part of the DSC internal FW configurations, which is out of the scope of this patchset.

When the vDPA support set is enabled in the core PF through its devlink param, auxiliary_bus devices are created for each VF that supports the feature. The vDPA driver then connects to and uses this auxiliary_device to do control path configuration through the PF device. This can then be used with the vdpa kernel module to provide devices for virtio_vdpa kernel module for host interfaces, or vhost_vdpa kernel module for interfaces exported into your favorite VM.

A cheap ASCII diagram of a vDPA instance looks something like this:

                                ,----------.
                                |   vdpa   |
                                '----------'
                                  |     ||
                                 ctl   data
                                  |     ||
                          .----------.  ||
                          | pds_vdpa |  ||
                          '----------'  ||
                               |        ||
                       pds_core.vDPA.1  ||
                               |        ||
                    .---------------.   ||
                    |   pds_core    |   ||
                    '---------------'   ||
                        ||         ||   ||
                      09:00.0      09:00.1  
        == PCI ============================================
                        ||            ||
                   .----------.   .----------.
            ,------|    PF    |---|    VF    |-------,
            |      '----------'   '----------'       |
            |                  DSC                   |
            |                                        |
            ------------------------------------------

Changes:
v10:
- remove CONFIG_DEBUG_FS guard static inline stuff
- remove unnecessary 0 and null initializations
- verify in driver load that PDS_CORE_DRV_NAME matches KBUILD_MODNAME
- remove debugfs irqs_show(), redundant with /proc
- return -ENOMEM if intr_info = kcalloc() fails
- move the status code enum into pds_core_if.h as part of API definition
- fix up one place in pdsc_devcmd_wait() we're using the status codes where we could use the errno
- remove redundant calls to flush_workqueue()
- grab config_lock before testing state bits in pdsc_fw_reporter_diagnose()
- change pdsc_color_match() to return bool
- remove useless VIF setup loop and just setup vDPA services for now
- remove pf pointer from struct padev and have clients use pci_physfn()
- drop use of "vf" in auxdev.c function names, make more generic
- remove last of client ops struct and simply export the functions
- drop [email protected] from MAINTAINERS and add new include dir
- include dynamic_debug.h in adminq.c to protect dynamic_hex_dump()
- fixed fw_slot type from u8 to int for handling error returns
- fixed comment spelling
- changed void arg in pdsc_adminq_post() to struct pdsc *

v9:

- change pdsc field name id to uid to clarify the unique id used for aux device
- remove unnecessary pf->state and other checks in aux device creation
- hardcode fw slotnames for devlink info, don't use strings from FW
- handle errors from PDS_CORE_CMD_INIT devcmd call
- tighten up health thread use of config_lock
- remove pdsc_queue_health_check() layer over queuing health check
- start pds_core.rst file in first patch, add to it incrementally
- give more user interaction info in commit messages
- removed a few more extraneous includes

v8:

- fixed deadlock problem, use devl_health_reporter_destroy() when devlink is locked
- don't clear client_id until after auxiliary_device_uninit()

v7:

- use explicit devlink locking and devl_* APIs
- move some of devlink setup logic into probe and remove
- use debugfs_create_u{type}() for state and queue head and tail
- add include for linux/vmalloc.h

v6:

- removed version.h include noticed by kernel test robot's version check

- fixed up the more egregious checkpatch line length complaints
- make sure pdsc_auxbus_dev_register() checks padev pointer errcode

v5:

- added devlink health reporter for FW issues
- removed asic_type, asic_rev, serial_num, fw_version from debugfs as they are available through other means
- trimed OS info in pdsc_identify(), we don't need to send that much info to the FW
- removed reg/unreg from auxbus client API, they are now in the core when VF is started
- removed need for pdsc definition in client by simplifying the padev to only carry struct pci_dev pointers rather than full struct pdsc to the pf and vf
- removed the unused pdsc argument in pdsc_notify()
- moved include/linux/pds/pds_core.h to driver/../pds_core/core.h
- restored a few pds_core_if.h interface values and structs that are shared with FW source
- moved final config_lock unlock to before tear down of timer and workqueue to be sure there are no deadlocks while waiting for any stragglers
- changed use of PAGE_SIZE to local PDS_PAGE_SIZE to keep with FW layout needs without regard to kernel PAGE_SIZE configuration
- removed the redundant *adminqcq argument from pdsc_adminq_post()

v4:

- reworked to attach to both Core PF and vDPA VF PCI devices
- now creates auxiliary_device as part of each VF PCI probe, removes them on PCI remove
- auxiliary devices now use simple unique id rather than PCI address for identifier
- replaced home-grown event publishing with kernel-based notifier service
- dropped live_migration parameter, not needed when not creating aux device for it
- replaced devm_* functions with traditional interfaces
- added MAINTAINERS entry
- removed lingering traces of set/get_vf attribute adminq commands
- trimmed some include lists
- cleaned a kernel test robot complaint about a stray unused variable

        Link: https://lore.kernel.org/oe-kbuild-all/[email protected]/

v3:

- changed names from "pensando" to "amd" and updated copyright strings
- dropped the DEVLINK_PARAM_GENERIC_ID_FW_BANK for future development
- changed the auxiliary device creation to be triggered by the PCI bus event BOUND_DRIVER, and torn down at UNBIND_DRIVER in order to properly handle users using the sysfs bind/unbind functions
- dropped some noisy log messages
- rebased to current net-next

RFC to v2:

- added separate devlink param patches for DEVLINK_PARAM_GENERIC_ID_ENABLE_MIGRATION and DEVLINK_PARAM_GENERIC_ID_FW_BANK, and dropped the driver specific implementations
- updated descriptions for the new devlink parameters
- dropped netdev support
- dropped vDPA patches, will followup later
- separated fw update and fw bank select into their own patches

RFC:

Shannon Nelson (14):
pds_core: initial framework for pds_core PF driver
pds_core: add devcmd device interfaces
pds_core: health timer and workqueue
pds_core: add devlink health facilities
pds_core: set up device and adminq
pds_core: Add adminq processing and commands
pds_core: add FW update feature to devlink
pds_core: set up the VIF definitions and defaults
pds_core: add initial VF device handling
pds_core: add auxiliary_bus devices
pds_core: devlink params for enabling VIF support
pds_core: add the aux client API
pds_core: publish events to the clients
pds_core: Kconfig and pds_core.rst

 .../device_drivers/ethernet/amd/pds_core.rst  | 139 ++++
 .../device_drivers/ethernet/index.rst         |   1 +
 MAINTAINERS                                   |   9 +
 drivers/net/ethernet/amd/Kconfig              |  12 +
 drivers/net/ethernet/amd/Makefile             |   1 +
 drivers/net/ethernet/amd/pds_core/Makefile    |  14 +
 drivers/net/ethernet/amd/pds_core/adminq.c    | 290 ++++++++
 drivers/net/ethernet/amd/pds_core/auxbus.c    | 264 +++++++
 drivers/net/ethernet/amd/pds_core/core.c      | 597 ++++++++++++++++
 drivers/net/ethernet/amd/pds_core/core.h      | 312 +++++++++
 drivers/net/ethernet/amd/pds_core/debugfs.c   | 170 +++++
 drivers/net/ethernet/amd/pds_core/dev.c       | 351 ++++++++++
 drivers/net/ethernet/amd/pds_core/devlink.c   | 183 +++++
 drivers/net/ethernet/amd/pds_core/fw.c        | 194 ++++++
 drivers/net/ethernet/amd/pds_core/main.c      | 475 +++++++++++++
 include/linux/pds/pds_adminq.h                | 647 ++++++++++++++++++
 include/linux/pds/pds_auxbus.h                |  20 +
 include/linux/pds/pds_common.h                |  68 ++
 include/linux/pds/pds_core_if.h               | 571 ++++++++++++++++
 include/linux/pds/pds_intr.h                  | 163 +++++
 20 files changed, 4481 insertions(+)
 create mode 100644 Documentation/networking/device_drivers/ethernet/amd/pds_core.rst
 create mode 100644 drivers/net/ethernet/amd/pds_core/Makefile
 create mode 100644 drivers/net/ethernet/amd/pds_core/adminq.c
 create mode 100644 drivers/net/ethernet/amd/pds_core/auxbus.c
 create mode 100644 drivers/net/ethernet/amd/pds_core/core.c
 create mode 100644 drivers/net/ethernet/amd/pds_core/core.h
 create mode 100644 drivers/net/ethernet/amd/pds_core/debugfs.c
 create mode 100644 drivers/net/ethernet/amd/pds_core/dev.c
 create mode 100644 drivers/net/ethernet/amd/pds_core/devlink.c
 create mode 100644 drivers/net/ethernet/amd/pds_core/fw.c
 create mode 100644 drivers/net/ethernet/amd/pds_core/main.c
 create mode 100644 include/linux/pds/pds_adminq.h
 create mode 100644 include/linux/pds/pds_auxbus.h
 create mode 100644 include/linux/pds/pds_common.h
 create mode 100644 include/linux/pds/pds_core_if.h
 create mode 100644 include/linux/pds/pds_intr.h
Signature · 1 line
2.17.1
#1 lore raw

PATCH v10 1/14 pds_core: initial framework for pds_core PF driver

This is the initial PCI driver framework for the new pds_core device driver and its family of devices. This does the very basics of registering for the new PF PCI device 1dd8:100c, setting up debugfs entries, and registering with devlink.

---
 .../device_drivers/ethernet/amd/pds_core.rst  |  35 ++
 .../device_drivers/ethernet/index.rst         |   1 +
 drivers/net/ethernet/amd/pds_core/Makefile    |   8 +
 drivers/net/ethernet/amd/pds_core/core.h      |  56 ++
 drivers/net/ethernet/amd/pds_core/debugfs.c   |  31 +
 drivers/net/ethernet/amd/pds_core/main.c      | 277 +++++++++
 include/linux/pds/pds_common.h                |  14 +
 include/linux/pds/pds_core_if.h               | 571 ++++++++++++++++++
 8 files changed, 993 insertions(+)
 create mode 100644 Documentation/networking/device_drivers/ethernet/amd/pds_core.rst
 create mode 100644 drivers/net/ethernet/amd/pds_core/Makefile
 create mode 100644 drivers/net/ethernet/amd/pds_core/core.h
 create mode 100644 drivers/net/ethernet/amd/pds_core/debugfs.c
 create mode 100644 drivers/net/ethernet/amd/pds_core/main.c
 create mode 100644 include/linux/pds/pds_common.h
 create mode 100644 include/linux/pds/pds_core_if.h
Documentation/networking/device_drivers/ethernet/amd/pds_core.rst new file +35 -0
new file mode 100644
index 000000000000..99a70026f1bc
@@ -0,0 +1,35 @@
+.. SPDX-License-Identifier: GPL-2.0+
+
+========================================================
+Linux Driver for the AMD/Pensando(R) DSC adapter family
+========================================================
+
+Copyright(c) 2023 Advanced Micro Devices, Inc
+
+Identifying the Adapter
+=======================
+
+To find if one or more AMD/Pensando PCI Core devices are installed on the
+host, check for the PCI devices::
+
+  # lspci -d 1dd8:100c
+  b5:00.0 Processing accelerators: Pensando Systems Device 100c
+  b6:00.0 Processing accelerators: Pensando Systems Device 100c
+
+If such devices are listed as above, then the pds_core.ko driver should find
+and configure them for use.  There should be log entries in the kernel
+messages such as these::
+
+  $ dmesg | grep pds_core
+  pds_core 0000:b5:00.0: 252.048 Gb/s available PCIe bandwidth (16.0 GT/s PCIe x16 link)
+  pds_core 0000:b5:00.0: FW: 1.60.0-73
+  pds_core 0000:b6:00.0: 252.048 Gb/s available PCIe bandwidth (16.0 GT/s PCIe x16 link)
+  pds_core 0000:b6:00.0: FW: 1.60.0-73
+
+Support
+=======
+
+For general Linux networking support, please use the netdev mailing
+list, which is monitored by AMD/Pensando personnel::
+
+  [email protected]
Documentation/networking/device_drivers/ethernet/index.rst +1 -0
index 6e9e7012d000..417ca514a4d0 100644
@@ -14,6 +14,7 @@ Contents:
    3com/vortex
    amazon/ena
    altera/altera_tse
+   amd/pds_core
    aquantia/atlantic
    chelsio/cxgb
    cirrus/cs89x0
drivers/net/ethernet/amd/pds_core/Makefile new file +8 -0
new file mode 100644
index 000000000000..de3bf1d1886c
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2023 Advanced Micro Devices, Inc.
+
+obj-$(CONFIG_PDS_CORE) := pds_core.o
+
+pds_core-y := main.o
+
+pds_core-$(CONFIG_DEBUG_FS) += debugfs.o
drivers/net/ethernet/amd/pds_core/core.h new file +56 -0

56 lines not on this page · raw message

drivers/net/ethernet/amd/pds_core/debugfs.c new file +31 -0
new file mode 100644
index 000000000000..b2f7cb795c20
@@ -0,0 +1,31 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright(c) 2023 Advanced Micro Devices, Inc */
+
+#include <linux/pci.h>
+
+#include "core.h"
+
+static struct dentry *pdsc_dir;
+
+void pdsc_debugfs_create(void)
+{
+	pdsc_dir = debugfs_create_dir(PDS_CORE_DRV_NAME, NULL);
+}
+
+void pdsc_debugfs_destroy(void)
+{
+	debugfs_remove_recursive(pdsc_dir);
+}
+
+void pdsc_debugfs_add_dev(struct pdsc *pdsc)
+{
+	pdsc->dentry = debugfs_create_dir(pci_name(pdsc->pdev), pdsc_dir);
+
+	debugfs_create_ulong("state", 0400, pdsc->dentry, &pdsc->state);
+}
+
+void pdsc_debugfs_del_dev(struct pdsc *pdsc)
+{
+	debugfs_remove_recursive(pdsc->dentry);
+	pdsc->dentry = NULL;
+}
drivers/net/ethernet/amd/pds_core/main.c new file +277 -0

277 lines not on this page · raw message

include/linux/pds/pds_common.h new file +14 -0
new file mode 100644
index 000000000000..bd041a5170a6
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: (GPL-2.0 OR Linux-OpenIB) OR BSD-2-Clause */
+/* Copyright(c) 2023 Advanced Micro Devices, Inc. */
+
+#ifndef _PDS_COMMON_H_
+#define _PDS_COMMON_H_
+
+#define PDS_CORE_DRV_NAME			"pds_core"
+
+/* the device's internal addressing uses up to 52 bits */
+#define PDS_CORE_ADDR_LEN	52
+#define PDS_CORE_ADDR_MASK	(BIT_ULL(PDS_ADDR_LEN) - 1)
+#define PDS_PAGE_SIZE		4096
+
+#endif /* _PDS_COMMON_H_ */
include/linux/pds/pds_core_if.h new file +571 -0

571 lines not on this page · raw message

Signature · 1 line
2.17.1
#2 lore raw

PATCH v10 2/14 pds_core: add devcmd device interfaces

In reply to Shannon Nelson

The devcmd interface is the basic connection to the device through the PCI BAR for low level identification and command services. This does the early device initialization and finds the identity data, and adds devcmd routines to be used by later driver bits.

---
 drivers/net/ethernet/amd/pds_core/Makefile  |   4 +-
 drivers/net/ethernet/amd/pds_core/core.c    |  36 ++
 drivers/net/ethernet/amd/pds_core/core.h    |  50 +++
 drivers/net/ethernet/amd/pds_core/debugfs.c |  38 +++
 drivers/net/ethernet/amd/pds_core/dev.c     | 348 ++++++++++++++++++++
 drivers/net/ethernet/amd/pds_core/main.c    |  33 +-
 include/linux/pds/pds_common.h              |  30 ++
 include/linux/pds/pds_intr.h                | 163 +++++++++
 8 files changed, 699 insertions(+), 3 deletions(-)
 create mode 100644 drivers/net/ethernet/amd/pds_core/core.c
 create mode 100644 drivers/net/ethernet/amd/pds_core/dev.c
 create mode 100644 include/linux/pds/pds_intr.h
drivers/net/ethernet/amd/pds_core/Makefile +3 -1
index de3bf1d1886c..95a6c31e92d2 100644
@@ -3,6 +3,8 @@
 
 obj-$(CONFIG_PDS_CORE) := pds_core.o
 
-pds_core-y := main.o
+pds_core-y := main.o \
+	      dev.o \
+	      core.o
 
 pds_core-$(CONFIG_DEBUG_FS) += debugfs.o
drivers/net/ethernet/amd/pds_core/core.c new file +36 -0
new file mode 100644
index 000000000000..80d2ecb045df
@@ -0,0 +1,36 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright(c) 2023 Advanced Micro Devices, Inc */
+
+#include "core.h"
+
+int pdsc_setup(struct pdsc *pdsc, bool init)
+{
+	int err = 0;
+
+	if (init)
+		err = pdsc_dev_init(pdsc);
+	else
+		err = pdsc_dev_reinit(pdsc);
+	if (err)
+		return err;
+
+	clear_bit(PDSC_S_FW_DEAD, &pdsc->state);
+	return 0;
+}
+
+void pdsc_teardown(struct pdsc *pdsc, bool removing)
+{
+	pdsc_devcmd_reset(pdsc);
+
+	if (removing) {
+		kfree(pdsc->intr_info);
+		pdsc->intr_info = NULL;
+	}
+
+	if (pdsc->kern_dbpage) {
+		iounmap(pdsc->kern_dbpage);
+		pdsc->kern_dbpage = NULL;
+	}
+
+	set_bit(PDSC_S_FW_DEAD, &pdsc->state);
+}
drivers/net/ethernet/amd/pds_core/core.h +50 -0

76 lines not on this page · raw message

drivers/net/ethernet/amd/pds_core/debugfs.c +38 -0
index b2f7cb795c20..601431b41abb 100644
@@ -29,3 +29,41 @@ void pdsc_debugfs_del_dev(struct pdsc *pdsc)
 	debugfs_remove_recursive(pdsc->dentry);
 	pdsc->dentry = NULL;
 }
+
+static int identity_show(struct seq_file *seq, void *v)
+{
+	struct pdsc *pdsc = seq->private;
+	struct pds_core_dev_identity *ident;
+	int vt;
+
+	ident = &pdsc->dev_ident;
+
+	seq_printf(seq, "fw_heartbeat:     0x%x\n",
+		   ioread32(&pdsc->info_regs->fw_heartbeat));
+
+	seq_printf(seq, "nlifs:            %d\n",
+		   le32_to_cpu(ident->nlifs));
+	seq_printf(seq, "nintrs:           %d\n",
+		   le32_to_cpu(ident->nintrs));
+	seq_printf(seq, "ndbpgs_per_lif:   %d\n",
+		   le32_to_cpu(ident->ndbpgs_per_lif));
+	seq_printf(seq, "intr_coal_mult:   %d\n",
+		   le32_to_cpu(ident->intr_coal_mult));
+	seq_printf(seq, "intr_coal_div:    %d\n",
+		   le32_to_cpu(ident->intr_coal_div));
+
+	seq_puts(seq, "vif_types:        ");
+	for (vt = 0; vt < PDS_DEV_TYPE_MAX; vt++)
+		seq_printf(seq, "%d ",
+			   le16_to_cpu(pdsc->dev_ident.vif_types[vt]));
+	seq_puts(seq, "\n");
+
+	return 0;
+}
+DEFINE_SHOW_ATTRIBUTE(identity);
+
+void pdsc_debugfs_add_ident(struct pdsc *pdsc)
+{
+	debugfs_create_file("identity", 0400, pdsc->dentry,
+			    pdsc, &identity_fops);
+}
drivers/net/ethernet/amd/pds_core/dev.c new file +348 -0

348 lines not on this page · raw message

drivers/net/ethernet/amd/pds_core/main.c +31 -2
index c2b12f226959..09afb069dcb3 100644
@@ -137,6 +137,18 @@ static int pdsc_init_pf(struct pdsc *pdsc)
 	if (err)
 		goto err_out_release_regions;
 
+	mutex_init(&pdsc->devcmd_lock);
+	mutex_init(&pdsc->config_lock);
+
+	mutex_lock(&pdsc->config_lock);
+	set_bit(PDSC_S_FW_DEAD, &pdsc->state);
+
+	err = pdsc_setup(pdsc, PDSC_SETUP_INIT);
+	if (err)
+		goto err_out_unmap_bars;
+
+	mutex_unlock(&pdsc->config_lock);
+
 	dl = priv_to_devlink(pdsc);
 	devl_lock(dl);
 	devl_register(dl);
@@ -144,6 +156,12 @@ static int pdsc_init_pf(struct pdsc *pdsc)
 
 	return 0;
 
+err_out_unmap_bars:
+	mutex_unlock(&pdsc->config_lock);
+	mutex_destroy(&pdsc->config_lock);
+	mutex_destroy(&pdsc->devcmd_lock);
+	pci_free_irq_vectors(pdsc->pdev);
+	pdsc_unmap_bars(pdsc);
 err_out_release_regions:
 	pci_release_regions(pdsc->pdev);
 
@@ -240,8 +258,19 @@ static void pdsc_remove(struct pci_dev *pdev)
 	devl_unregister(dl);
 	devl_unlock(dl);
 
-	pdsc_unmap_bars(pdsc);
-	pci_release_regions(pdev);
+	if (!pdev->is_virtfn) {
+		mutex_lock(&pdsc->config_lock);
+		set_bit(PDSC_S_STOPPING_DRIVER, &pdsc->state);
+
+		pdsc_teardown(pdsc, PDSC_TEARDOWN_REMOVING);
+		mutex_unlock(&pdsc->config_lock);
+		mutex_destroy(&pdsc->config_lock);
+		mutex_destroy(&pdsc->devcmd_lock);
+
+		pci_free_irq_vectors(pdev);
+		pdsc_unmap_bars(pdsc);
+		pci_release_regions(pdev);
+	}
 
 	pci_clear_master(pdev);
 	pci_disable_device(pdev);
include/linux/pds/pds_common.h +30 -0
index bd041a5170a6..f0798ce01acf 100644
@@ -11,4 +11,34 @@
 #define PDS_CORE_ADDR_MASK	(BIT_ULL(PDS_ADDR_LEN) - 1)
 #define PDS_PAGE_SIZE		4096
 
+enum pds_core_driver_type {
+	PDS_DRIVER_LINUX   = 1,
+	PDS_DRIVER_WIN     = 2,
+	PDS_DRIVER_DPDK    = 3,
+	PDS_DRIVER_FREEBSD = 4,
+	PDS_DRIVER_IPXE    = 5,
+	PDS_DRIVER_ESXI    = 6,
+};
+
+#define PDS_CORE_IFNAMSIZ		16
+
+/**
+ * enum pds_core_logical_qtype - Logical Queue Types
+ * @PDS_CORE_QTYPE_ADMINQ:    Administrative Queue
+ * @PDS_CORE_QTYPE_NOTIFYQ:   Notify Queue
+ * @PDS_CORE_QTYPE_RXQ:       Receive Queue
+ * @PDS_CORE_QTYPE_TXQ:       Transmit Queue
+ * @PDS_CORE_QTYPE_EQ:        Event Queue
+ * @PDS_CORE_QTYPE_MAX:       Max queue type supported
+ */
+enum pds_core_logical_qtype {
+	PDS_CORE_QTYPE_ADMINQ  = 0,
+	PDS_CORE_QTYPE_NOTIFYQ = 1,
+	PDS_CORE_QTYPE_RXQ     = 2,
+	PDS_CORE_QTYPE_TXQ     = 3,
+	PDS_CORE_QTYPE_EQ      = 4,
+
+	PDS_CORE_QTYPE_MAX     = 16   /* don't change - used in struct size */
+};
+
 #endif /* _PDS_COMMON_H_ */
include/linux/pds/pds_intr.h new file +163 -0

163 lines not on this page · raw message

Signature · 1 line
2.17.1
#3 lore raw

PATCH v10 3/14 pds_core: health timer and workqueue

In reply to Shannon Nelson

Add in the periodic health check and the related workqueue, as well as the handlers for when a FW reset is seen.

The firmware is polled every 5 seconds to be sure that it is still alive and that the FW generation didn't change.

The alive check looks to see that the PCI bus is still readable and the fw_status still has the RUNNING bit on. If not alive, the driver stops activity and tears things down. When the FW recovers and the alive check again succeeds, the driver sets back up for activity.

The generation check looks at the fw_generation to see if it has changed, which can happen if the FW crashed and recovered or was updated in between health checks. If changed, the driver counts that as though the alive test failed and forces the fw_down/fw_up cycle.

---
 drivers/net/ethernet/amd/pds_core/core.c | 61 ++++++++++++++++++++++++
 drivers/net/ethernet/amd/pds_core/core.h |  8 ++++
 drivers/net/ethernet/amd/pds_core/dev.c  |  3 ++
 drivers/net/ethernet/amd/pds_core/main.c | 31 ++++++++++++
 4 files changed, 103 insertions(+)
drivers/net/ethernet/amd/pds_core/core.c +61 -0
index 80d2ecb045df..701d27471858 100644
@@ -34,3 +34,64 @@ void pdsc_teardown(struct pdsc *pdsc, bool removing)
 
 	set_bit(PDSC_S_FW_DEAD, &pdsc->state);
 }
+
+static void pdsc_fw_down(struct pdsc *pdsc)
+{
+	if (test_and_set_bit(PDSC_S_FW_DEAD, &pdsc->state)) {
+		dev_err(pdsc->dev, "%s: already happening\n", __func__);
+		return;
+	}
+
+	pdsc_teardown(pdsc, PDSC_TEARDOWN_RECOVERY);
+}
+
+static void pdsc_fw_up(struct pdsc *pdsc)
+{
+	int err;
+
+	if (!test_bit(PDSC_S_FW_DEAD, &pdsc->state)) {
+		dev_err(pdsc->dev, "%s: fw not dead\n", __func__);
+		return;
+	}
+
+	err = pdsc_setup(pdsc, PDSC_SETUP_RECOVERY);
+	if (err)
+		goto err_out;
+
+	return;
+
+err_out:
+	pdsc_teardown(pdsc, PDSC_TEARDOWN_RECOVERY);
+}
+
+void pdsc_health_thread(struct work_struct *work)
+{
+	struct pdsc *pdsc = container_of(work, struct pdsc, health_work);
+	unsigned long mask;
+	bool healthy;
+
+	mutex_lock(&pdsc->config_lock);
+
+	/* Don't do a check when in a transition state */
+	mask = BIT_ULL(PDSC_S_INITING_DRIVER) |
+	       BIT_ULL(PDSC_S_STOPPING_DRIVER);
+	if (pdsc->state & mask)
+		goto out_unlock;
+
+	healthy = pdsc_is_fw_good(pdsc);
+	dev_dbg(pdsc->dev, "%s: health %d fw_status %#02x fw_heartbeat %d\n",
+		__func__, healthy, pdsc->fw_status, pdsc->last_hb);
+
+	if (test_bit(PDSC_S_FW_DEAD, &pdsc->state)) {
+		if (healthy)
+			pdsc_fw_up(pdsc);
+	} else {
+		if (!healthy)
+			pdsc_fw_down(pdsc);
+	}
+
+	pdsc->fw_generation = pdsc->fw_status & PDS_CORE_FW_STS_F_GENERATION;
+
+out_unlock:
+	mutex_unlock(&pdsc->config_lock);
+}
drivers/net/ethernet/amd/pds_core/core.h +8 -0
index fcf6c6545c49..83c528a2a131 100644
@@ -12,6 +12,8 @@
 #include <linux/pds/pds_intr.h>
 
 #define PDSC_DRV_DESCRIPTION	"AMD/Pensando Core Driver"
+
+#define PDSC_WATCHDOG_SECS	5
 #define PDSC_TEARDOWN_RECOVERY	false
 #define PDSC_TEARDOWN_REMOVING	true
 #define PDSC_SETUP_RECOVERY	false
@@ -63,12 +65,17 @@ struct pdsc {
 	u8 fw_generation;
 	unsigned long last_fw_time;
 	u32 last_hb;
+	struct timer_list wdtimer;
+	unsigned int wdtimer_period;
+	struct work_struct health_work;
 
 	struct pdsc_devinfo dev_info;
 	struct pds_core_dev_identity dev_ident;
 	unsigned int nintrs;
 	struct pdsc_intr_info *intr_info;	/* array of nintrs elements */
 
+	struct workqueue_struct *wq;
+
 	unsigned int devcmd_timeout;
 	struct mutex devcmd_lock;	/* lock for dev_cmd operations */
 	struct mutex config_lock;	/* lock for configuration operations */
@@ -102,5 +109,6 @@ int pdsc_dev_init(struct pdsc *pdsc);
 
 int pdsc_setup(struct pdsc *pdsc, bool init);
 void pdsc_teardown(struct pdsc *pdsc, bool removing);
+void pdsc_health_thread(struct work_struct *work);
 
 #endif /* _PDSC_H_ */
drivers/net/ethernet/amd/pds_core/dev.c +3 -0
index f082d69c5128..f7c597ea5daf 100644
@@ -177,6 +177,9 @@ int pdsc_devcmd_locked(struct pdsc *pdsc, union pds_core_dev_cmd *cmd,
 	err = pdsc_devcmd_wait(pdsc, max_seconds);
 	memcpy_fromio(comp, &pdsc->cmd_regs->comp, sizeof(*comp));
 
+	if (err == -ENXIO || err == -ETIMEDOUT)
+		queue_work(pdsc->wq, &pdsc->health_work);
+
 	return err;
 }
 
drivers/net/ethernet/amd/pds_core/main.c +31 -0
index 09afb069dcb3..c9fbf1d374a7 100644
@@ -20,6 +20,17 @@ static const struct pci_device_id pdsc_id_table[] = {
 };
 MODULE_DEVICE_TABLE(pci, pdsc_id_table);
 
+static void pdsc_wdtimer_cb(struct timer_list *t)
+{
+	struct pdsc *pdsc = from_timer(pdsc, t, wdtimer);
+
+	dev_dbg(pdsc->dev, "%s: jiffies %ld\n", __func__, jiffies);
+	mod_timer(&pdsc->wdtimer,
+		  round_jiffies(jiffies + pdsc->wdtimer_period));
+
+	queue_work(pdsc->wq, &pdsc->health_work);
+}
+
 static void pdsc_unmap_bars(struct pdsc *pdsc)
 {
 	struct pdsc_dev_bar *bars = pdsc->bars;
@@ -119,8 +130,11 @@ static int pdsc_init_vf(struct pdsc *vf)
 	return -1;
 }
 
+#define PDSC_WQ_NAME_LEN 24
+
 static int pdsc_init_pf(struct pdsc *pdsc)
 {
+	char wq_name[PDSC_WQ_NAME_LEN];
 	struct devlink *dl;
 	int err;
 
@@ -137,6 +151,13 @@ static int pdsc_init_pf(struct pdsc *pdsc)
 	if (err)
 		goto err_out_release_regions;
 
+	/* General workqueue and timer, but don't start timer yet */
+	snprintf(wq_name, sizeof(wq_name), "%s.%d", PDS_CORE_DRV_NAME, pdsc->uid);
+	pdsc->wq = create_singlethread_workqueue(wq_name);
+	INIT_WORK(&pdsc->health_work, pdsc_health_thread);
+	timer_setup(&pdsc->wdtimer, pdsc_wdtimer_cb, 0);
+	pdsc->wdtimer_period = PDSC_WATCHDOG_SECS * HZ;
+
 	mutex_init(&pdsc->devcmd_lock);
 	mutex_init(&pdsc->config_lock);
 
@@ -154,10 +175,16 @@ static int pdsc_init_pf(struct pdsc *pdsc)
 	devl_register(dl);
 	devl_unlock(dl);
 
+	/* Lastly, start the health check timer */
+	mod_timer(&pdsc->wdtimer, round_jiffies(jiffies + pdsc->wdtimer_period));
+
 	return 0;
 
 err_out_unmap_bars:
 	mutex_unlock(&pdsc->config_lock);
+	del_timer_sync(&pdsc->wdtimer);
+	if (pdsc->wq)
+		destroy_workqueue(pdsc->wq);
 	mutex_destroy(&pdsc->config_lock);
 	mutex_destroy(&pdsc->devcmd_lock);
 	pci_free_irq_vectors(pdsc->pdev);
@@ -259,6 +286,10 @@ static void pdsc_remove(struct pci_dev *pdev)
 	devl_unlock(dl);
 
 	if (!pdev->is_virtfn) {
+		del_timer_sync(&pdsc->wdtimer);
+		if (pdsc->wq)
+			destroy_workqueue(pdsc->wq);
+
 		mutex_lock(&pdsc->config_lock);
 		set_bit(PDSC_S_STOPPING_DRIVER, &pdsc->state);
 
Signature · 1 line
2.17.1
#4 lore raw

PATCH v10 4/14 pds_core: add devlink health facilities

In reply to Shannon Nelson

Add devlink health reporting on top of our fw watchdog.

Example:

  # devlink health show pci/0000:2b:00.0 reporter fw
  pci/0000:2b:00.0:
    reporter fw
      state healthy error 0 recover 0
  # devlink health diagnose pci/0000:2b:00.0 reporter fw
   Status: healthy State: 1 Generation: 0 Recoveries: 0
---
 .../device_drivers/ethernet/amd/pds_core.rst  | 12 ++++++
 drivers/net/ethernet/amd/pds_core/Makefile    |  1 +
 drivers/net/ethernet/amd/pds_core/core.c      |  8 +++-
 drivers/net/ethernet/amd/pds_core/core.h      |  6 +++
 drivers/net/ethernet/amd/pds_core/devlink.c   | 40 +++++++++++++++++++
 drivers/net/ethernet/amd/pds_core/main.c      | 22 ++++++++++
 6 files changed, 88 insertions(+), 1 deletion(-)
 create mode 100644 drivers/net/ethernet/amd/pds_core/devlink.c
Documentation/networking/device_drivers/ethernet/amd/pds_core.rst +12 -0
index 99a70026f1bc..5b88173a20ff 100644
@@ -26,6 +26,18 @@ messages such as these::
   pds_core 0000:b6:00.0: 252.048 Gb/s available PCIe bandwidth (16.0 GT/s PCIe x16 link)
   pds_core 0000:b6:00.0: FW: 1.60.0-73
 
+Health Reporters
+================
+
+The driver supports a devlink health reporter for FW status::
+
+  # devlink health show pci/0000:2b:00.0 reporter fw
+  pci/0000:2b:00.0:
+    reporter fw
+      state healthy error 0 recover 0
+  # devlink health diagnose pci/0000:2b:00.0 reporter fw
+   Status: healthy State: 1 Generation: 0 Recoveries: 0
+
 Support
 =======
 
drivers/net/ethernet/amd/pds_core/Makefile +1 -0
index 95a6c31e92d2..eaca8557ba66 100644
@@ -4,6 +4,7 @@
 obj-$(CONFIG_PDS_CORE) := pds_core.o
 
 pds_core-y := main.o \
+	      devlink.o \
 	      dev.o \
 	      core.o
 
drivers/net/ethernet/amd/pds_core/core.c +7 -1
index 701d27471858..ab8531386226 100644
@@ -5,7 +5,7 @@
 
 int pdsc_setup(struct pdsc *pdsc, bool init)
 {
-	int err = 0;
+	int err;
 
 	if (init)
 		err = pdsc_dev_init(pdsc);
@@ -42,6 +42,8 @@ static void pdsc_fw_down(struct pdsc *pdsc)
 		return;
 	}
 
+	devlink_health_report(pdsc->fw_reporter, "FW down reported", pdsc);
+
 	pdsc_teardown(pdsc, PDSC_TEARDOWN_RECOVERY);
 }
 
@@ -58,6 +60,10 @@ static void pdsc_fw_up(struct pdsc *pdsc)
 	if (err)
 		goto err_out;
 
+	pdsc->fw_recoveries++;
+	devlink_health_reporter_state_update(pdsc->fw_reporter,
+					     DEVLINK_HEALTH_REPORTER_STATE_HEALTHY);
+
 	return;
 
 err_out:
drivers/net/ethernet/amd/pds_core/core.h +6 -0
index 83c528a2a131..32aa38c40024 100644
@@ -68,6 +68,8 @@ struct pdsc {
 	struct timer_list wdtimer;
 	unsigned int wdtimer_period;
 	struct work_struct health_work;
+	struct devlink_health_reporter *fw_reporter;
+	u32 fw_recoveries;
 
 	struct pdsc_devinfo dev_info;
 	struct pds_core_dev_identity dev_ident;
@@ -88,6 +90,10 @@ struct pdsc {
 	u64 __iomem *kern_dbpage;
 };
 
+int pdsc_fw_reporter_diagnose(struct devlink_health_reporter *reporter,
+			      struct devlink_fmsg *fmsg,
+			      struct netlink_ext_ack *extack);
+
 void pdsc_debugfs_create(void);
 void pdsc_debugfs_destroy(void);
 void pdsc_debugfs_add_dev(struct pdsc *pdsc);
drivers/net/ethernet/amd/pds_core/devlink.c new file +40 -0
new file mode 100644
index 000000000000..3b05b1af65d1
@@ -0,0 +1,40 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright(c) 2023 Advanced Micro Devices, Inc */
+
+#include "core.h"
+
+int pdsc_fw_reporter_diagnose(struct devlink_health_reporter *reporter,
+			      struct devlink_fmsg *fmsg,
+			      struct netlink_ext_ack *extack)
+{
+	struct pdsc *pdsc = devlink_health_reporter_priv(reporter);
+	int err;
+
+	mutex_lock(&pdsc->config_lock);
+
+	if (test_bit(PDSC_S_FW_DEAD, &pdsc->state))
+		err = devlink_fmsg_string_pair_put(fmsg, "Status", "dead");
+	else if (!pdsc_is_fw_good(pdsc))
+		err = devlink_fmsg_string_pair_put(fmsg, "Status", "unhealthy");
+	else
+		err = devlink_fmsg_string_pair_put(fmsg, "Status", "healthy");
+
+	mutex_unlock(&pdsc->config_lock);
+
+	if (err)
+		return err;
+
+	err = devlink_fmsg_u32_pair_put(fmsg, "State",
+					pdsc->fw_status &
+						~PDS_CORE_FW_STS_F_GENERATION);
+	if (err)
+		return err;
+
+	err = devlink_fmsg_u32_pair_put(fmsg, "Generation",
+					pdsc->fw_generation >> 4);
+	if (err)
+		return err;
+
+	return devlink_fmsg_u32_pair_put(fmsg, "Recoveries",
+					 pdsc->fw_recoveries);
+}
drivers/net/ethernet/amd/pds_core/main.c +22 -0
index c9fbf1d374a7..54f3aed7adb1 100644
@@ -130,10 +130,16 @@ static int pdsc_init_vf(struct pdsc *vf)
 	return -1;
 }
 
+static const struct devlink_health_reporter_ops pdsc_fw_reporter_ops = {
+	.name = "fw",
+	.diagnose = pdsc_fw_reporter_diagnose,
+};
+
 #define PDSC_WQ_NAME_LEN 24
 
 static int pdsc_init_pf(struct pdsc *pdsc)
 {
+	struct devlink_health_reporter *hr;
 	char wq_name[PDSC_WQ_NAME_LEN];
 	struct devlink *dl;
 	int err;
@@ -172,6 +178,16 @@ static int pdsc_init_pf(struct pdsc *pdsc)
 
 	dl = priv_to_devlink(pdsc);
 	devl_lock(dl);
+
+	hr = devl_health_reporter_create(dl, &pdsc_fw_reporter_ops, 0, pdsc);
+	if (IS_ERR(hr)) {
+		dev_warn(pdsc->dev, "Failed to create fw reporter: %pe\n", hr);
+		err = PTR_ERR(hr);
+		devl_unlock(dl);
+		goto err_out_teardown;
+	}
+	pdsc->fw_reporter = hr;
+
 	devl_register(dl);
 	devl_unlock(dl);
 
@@ -180,6 +196,8 @@ static int pdsc_init_pf(struct pdsc *pdsc)
 
 	return 0;
 
+err_out_teardown:
+	pdsc_teardown(pdsc, PDSC_TEARDOWN_REMOVING);
 err_out_unmap_bars:
 	mutex_unlock(&pdsc->config_lock);
 	del_timer_sync(&pdsc->wdtimer);
@@ -283,6 +301,10 @@ static void pdsc_remove(struct pci_dev *pdev)
 	dl = priv_to_devlink(pdsc);
 	devl_lock(dl);
 	devl_unregister(dl);
+	if (pdsc->fw_reporter) {
+		devl_health_reporter_destroy(pdsc->fw_reporter);
+		pdsc->fw_reporter = NULL;
+	}
 	devl_unlock(dl);
 
 	if (!pdev->is_virtfn) {
Signature · 1 line
2.17.1
#5 lore raw

PATCH v10 5/14 pds_core: set up device and adminq

In reply to Shannon Nelson

Set up the basic adminq and notifyq queue structures. These are used mostly by the client drivers for feature configuration. These are essentially the same adminq and notifyq as in the ionic driver.

Part of this includes querying for device identity and FW information, so we can make that available to devlink dev info.

  $ devlink dev info pci/0000:b5:00.0
  pci/0000:b5:00.0:
    driver pds_core
    serial_number FLM18420073
    versions:
        fixed:
          asic.id 0x0
          asic.rev 0x0
        running:
          fw 1.51.0-73
        stored:
          fw.goldfw 1.15.9-C-22
          fw.mainfwa 1.60.0-73
          fw.mainfwb 1.60.0-57
---
 .../device_drivers/ethernet/amd/pds_core.rst  |  47 ++
 drivers/net/ethernet/amd/pds_core/core.c      | 431 +++++++++++-
 drivers/net/ethernet/amd/pds_core/core.h      | 151 +++++
 drivers/net/ethernet/amd/pds_core/debugfs.c   |  77 +++
 drivers/net/ethernet/amd/pds_core/devlink.c   |  61 ++
 drivers/net/ethernet/amd/pds_core/main.c      |  17 +-
 include/linux/pds/pds_adminq.h                | 638 ++++++++++++++++++
 7 files changed, 1418 insertions(+), 4 deletions(-)
 create mode 100644 include/linux/pds/pds_adminq.h
Documentation/networking/device_drivers/ethernet/amd/pds_core.rst +47 -0
index 5b88173a20ff..a48eafb3d0d3 100644
@@ -26,6 +26,53 @@ messages such as these::
   pds_core 0000:b6:00.0: 252.048 Gb/s available PCIe bandwidth (16.0 GT/s PCIe x16 link)
   pds_core 0000:b6:00.0: FW: 1.60.0-73
 
+Driver and firmware version information can be gathered with devlink::
+
+  $ devlink dev info pci/0000:b5:00.0
+  pci/0000:b5:00.0:
+    driver pds_core
+    serial_number FLM18420073
+    versions:
+        fixed:
+          asic.id 0x0
+          asic.rev 0x0
+        running:
+          fw 1.51.0-73
+        stored:
+          fw.goldfw 1.15.9-C-22
+          fw.mainfwa 1.60.0-73
+          fw.mainfwb 1.60.0-57
+
+Info versions
+=============
+
+The ``pds_core`` driver reports the following versions
+
+.. list-table:: devlink info versions implemented
+   :widths: 5 5 90
+
+   * - Name
+     - Type
+     - Description
+   * - ``fw``
+     - running
+     - Version of firmware running on the device
+   * - ``fw.goldfw``
+     - stored
+     - Version of firmware stored in the goldfw slot
+   * - ``fw.mainfwa``
+     - stored
+     - Version of firmware stored in the mainfwa slot
+   * - ``fw.mainfwb``
+     - stored
+     - Version of firmware stored in the mainfwb slot
+   * - ``asic.id``
+     - fixed
+     - The ASIC type for this device
+   * - ``asic.rev``
+     - fixed
+     - The revision of the ASIC for this device
+
 Health Reporters
 ================
 
drivers/net/ethernet/amd/pds_core/core.c +428 -3

473 lines not on this page · raw message

drivers/net/ethernet/amd/pds_core/core.h +151 -0

205 lines not on this page · raw message

drivers/net/ethernet/amd/pds_core/debugfs.c +77 -0

80 lines not on this page · raw message

drivers/net/ethernet/amd/pds_core/devlink.c +61 -0

67 lines not on this page · raw message

drivers/net/ethernet/amd/pds_core/main.c +16 -1
index 54f3aed7adb1..eaff311d4a10 100644
@@ -125,6 +125,13 @@ static int pdsc_map_bars(struct pdsc *pdsc)
 	return err;
 }
 
+void __iomem *pdsc_map_dbpage(struct pdsc *pdsc, int page_num)
+{
+	return pci_iomap_range(pdsc->pdev,
+			       pdsc->bars[PDS_CORE_PCI_BAR_DBELL].res_index,
+			       (u64)page_num << PAGE_SHIFT, PAGE_SIZE);
+}
+
 static int pdsc_init_vf(struct pdsc *vf)
 {
 	return -1;
@@ -166,6 +173,7 @@ static int pdsc_init_pf(struct pdsc *pdsc)
 
 	mutex_init(&pdsc->devcmd_lock);
 	mutex_init(&pdsc->config_lock);
+	spin_lock_init(&pdsc->adminq_lock);
 
 	mutex_lock(&pdsc->config_lock);
 	set_bit(PDSC_S_FW_DEAD, &pdsc->state);
@@ -173,6 +181,9 @@ static int pdsc_init_pf(struct pdsc *pdsc)
 	err = pdsc_setup(pdsc, PDSC_SETUP_INIT);
 	if (err)
 		goto err_out_unmap_bars;
+	err = pdsc_start(pdsc);
+	if (err)
+		goto err_out_teardown;
 
 	mutex_unlock(&pdsc->config_lock);
 
@@ -184,7 +195,7 @@ static int pdsc_init_pf(struct pdsc *pdsc)
 		dev_warn(pdsc->dev, "Failed to create fw reporter: %pe\n", hr);
 		err = PTR_ERR(hr);
 		devl_unlock(dl);
-		goto err_out_teardown;
+		goto err_out_stop;
 	}
 	pdsc->fw_reporter = hr;
 
@@ -196,6 +207,8 @@ static int pdsc_init_pf(struct pdsc *pdsc)
 
 	return 0;
 
+err_out_stop:
+	pdsc_stop(pdsc);
 err_out_teardown:
 	pdsc_teardown(pdsc, PDSC_TEARDOWN_REMOVING);
 err_out_unmap_bars:
@@ -214,6 +227,7 @@ static int pdsc_init_pf(struct pdsc *pdsc)
 }
 
 static const struct devlink_ops pdsc_dl_ops = {
+	.info_get	= pdsc_dl_info_get,
 };
 
 static const struct devlink_ops pdsc_dl_vf_ops = {
@@ -315,6 +329,7 @@ static void pdsc_remove(struct pci_dev *pdev)
 		mutex_lock(&pdsc->config_lock);
 		set_bit(PDSC_S_STOPPING_DRIVER, &pdsc->state);
 
+		pdsc_stop(pdsc);
 		pdsc_teardown(pdsc, PDSC_TEARDOWN_REMOVING);
 		mutex_unlock(&pdsc->config_lock);
 		mutex_destroy(&pdsc->config_lock);
include/linux/pds/pds_adminq.h new file +638 -0

638 lines not on this page · raw message

Signature · 1 line
2.17.1
#6 lore raw

PATCH v10 6/14 pds_core: Add adminq processing and commands

In reply to Shannon Nelson

Add the service routines for submitting and processing the adminq messages and for handling notifyq events.

---
 drivers/net/ethernet/amd/pds_core/Makefile |   1 +
 drivers/net/ethernet/amd/pds_core/adminq.c | 288 +++++++++++++++++++++
 drivers/net/ethernet/amd/pds_core/core.c   |  11 -
 include/linux/pds/pds_adminq.h             |  11 +-
 4 files changed, 299 insertions(+), 12 deletions(-)
 create mode 100644 drivers/net/ethernet/amd/pds_core/adminq.c
drivers/net/ethernet/amd/pds_core/Makefile +1 -0
index eaca8557ba66..ef76dcd7fccd 100644
@@ -6,6 +6,7 @@ obj-$(CONFIG_PDS_CORE) := pds_core.o
 pds_core-y := main.o \
 	      devlink.o \
 	      dev.o \
+	      adminq.o \
 	      core.o
 
 pds_core-$(CONFIG_DEBUG_FS) += debugfs.o
drivers/net/ethernet/amd/pds_core/adminq.c new file +288 -0

288 lines not on this page · raw message

drivers/net/ethernet/amd/pds_core/core.c +0 -11
index 8c0dbdb5efc5..59daf8a67ac6 100644
@@ -6,17 +6,6 @@
 
 #include "core.h"
 
-void pdsc_work_thread(struct work_struct *work)
-{
-	/* stub */
-}
-
-irqreturn_t pdsc_adminq_isr(int irq, void *data)
-{
-	/* stub */
-	return IRQ_HANDLED;
-}
-
 void pdsc_intr_free(struct pdsc *pdsc, int index)
 {
 	struct pdsc_intr_info *intr_info;
include/linux/pds/pds_adminq.h +10 -1
index dd5fbe3ee141..98a60ce87b92 100644
@@ -4,6 +4,8 @@
 #ifndef _PDS_CORE_ADMINQ_H_
 #define _PDS_CORE_ADMINQ_H_
 
+#define PDSC_ADMINQ_MAX_POLL_INTERVAL	256
+
 enum pds_core_adminq_flags {
 	PDS_AQ_FLAG_FASTPOLL	= BIT(1),	/* completion poll at 1ms */
 };
@@ -631,8 +633,15 @@ static_assert(sizeof(union pds_core_notifyq_comp) == 64);
  * where the meaning alternates between '1' and '0' for alternating
  * passes through the completion descriptor ring.
  */
-static inline u8 pdsc_color_match(u8 color, u8 done_color)
+static inline bool pdsc_color_match(u8 color, bool done_color)
 {
 	return (!!(color & PDS_COMP_COLOR_MASK)) == done_color;
 }
+
+struct pdsc;
+int pdsc_adminq_post(struct pdsc *pdsc,
+		     union pds_core_adminq_cmd *cmd,
+		     union pds_core_adminq_comp *comp,
+		     bool fast_poll);
+
 #endif /* _PDS_CORE_ADMINQ_H_ */
Signature · 1 line
2.17.1
#7 lore raw

PATCH v10 7/14 pds_core: add FW update feature to devlink

In reply to Shannon Nelson

Add in the support for doing firmware updates. Of the two main banks available, a and b, this updates the one not in use and then selects it for the next boot.

Example:

    devlink dev flash pci/0000:b2:00.0 \
	    file pensando/dsc_fw_1.63.0-22.tar
---
 .../device_drivers/ethernet/amd/pds_core.rst  |  10 +
 drivers/net/ethernet/amd/pds_core/Makefile    |   3 +-
 drivers/net/ethernet/amd/pds_core/core.h      |   5 +
 drivers/net/ethernet/amd/pds_core/devlink.c   |   9 +
 drivers/net/ethernet/amd/pds_core/fw.c        | 194 ++++++++++++++++++
 drivers/net/ethernet/amd/pds_core/main.c      |   1 +
 6 files changed, 221 insertions(+), 1 deletion(-)
 create mode 100644 drivers/net/ethernet/amd/pds_core/fw.c
Documentation/networking/device_drivers/ethernet/amd/pds_core.rst +10 -0
index a48eafb3d0d3..932ac03a3359 100644
@@ -73,6 +73,16 @@ The ``pds_core`` driver reports the following versions
      - fixed
      - The revision of the ASIC for this device
 
+Firmware Management
+===================
+
+The ``flash`` command can update a the DSC firmware.  The downloaded firmware
+will be saved into either of firmware bank 1 or bank 2, whichever is not
+currently in use, and that bank will used for the next boot::
+
+  # devlink dev flash pci/0000:b5:00.0 \
+            file pensando/dsc_fw_1.63.0-22.tar
+
 Health Reporters
 ================
 
drivers/net/ethernet/amd/pds_core/Makefile +2 -1
index ef76dcd7fccd..6d1d6c58a1fa 100644
@@ -7,6 +7,7 @@ pds_core-y := main.o \
 	      devlink.o \
 	      dev.o \
 	      adminq.o \
-	      core.o
+	      core.o \
+	      fw.o
 
 pds_core-$(CONFIG_DEBUG_FS) += debugfs.o
drivers/net/ethernet/amd/pds_core/core.h +5 -0
index 7adbd10cb6d9..7eb02b359f3a 100644
@@ -226,6 +226,9 @@ int pdsc_fw_reporter_diagnose(struct devlink_health_reporter *reporter,
 			      struct netlink_ext_ack *extack);
 int pdsc_dl_info_get(struct devlink *dl, struct devlink_info_req *req,
 		     struct netlink_ext_ack *extack);
+int pdsc_dl_flash_update(struct devlink *dl,
+			 struct devlink_flash_update_params *params,
+			 struct netlink_ext_ack *extack);
 
 void __iomem *pdsc_map_dbpage(struct pdsc *pdsc, int page_num);
 
@@ -268,4 +271,6 @@ void pdsc_process_adminq(struct pdsc_qcq *qcq);
 void pdsc_work_thread(struct work_struct *work);
 irqreturn_t pdsc_adminq_isr(int irq, void *data);
 
+int pdsc_firmware_update(struct pdsc *pdsc, const struct firmware *fw,
+			 struct netlink_ext_ack *extack);
 #endif /* _PDSC_H_ */
drivers/net/ethernet/amd/pds_core/devlink.c +9 -0
index 89f23d620d1d..f91d65cc78b5 100644
@@ -3,6 +3,15 @@
 
 #include "core.h"
 
+int pdsc_dl_flash_update(struct devlink *dl,
+			 struct devlink_flash_update_params *params,
+			 struct netlink_ext_ack *extack)
+{
+	struct pdsc *pdsc = devlink_priv(dl);
+
+	return pdsc_firmware_update(pdsc, params->fw, extack);
+}
+
 static char *fw_slotnames[] = {
 	"fw.goldfw",
 	"fw.mainfwa",
drivers/net/ethernet/amd/pds_core/fw.c new file +194 -0

194 lines not on this page · raw message

drivers/net/ethernet/amd/pds_core/main.c +1 -0
index eaff311d4a10..54aaf213679f 100644
@@ -228,6 +228,7 @@ static int pdsc_init_pf(struct pdsc *pdsc)
 
 static const struct devlink_ops pdsc_dl_ops = {
 	.info_get	= pdsc_dl_info_get,
+	.flash_update	= pdsc_dl_flash_update,
 };
 
 static const struct devlink_ops pdsc_dl_vf_ops = {
Signature · 1 line
2.17.1
#8 lore raw

PATCH v10 8/14 pds_core: set up the VIF definitions and defaults

In reply to Shannon Nelson

The Virtual Interfaces (VIFs) supported by the DSC's configuration (vDPA, Eth, RDMA, etc) are reported in the dev_ident struct and made visible in debugfs. At this point only vDPA is supported in this driver so we only setup devices for that feature.

---
 drivers/net/ethernet/amd/pds_core/core.c    | 48 +++++++++++++++++++++
 drivers/net/ethernet/amd/pds_core/core.h    | 11 +++++
 drivers/net/ethernet/amd/pds_core/debugfs.c | 24 +++++++++++
 include/linux/pds/pds_common.h              | 19 ++++++++
 4 files changed, 102 insertions(+)
drivers/net/ethernet/amd/pds_core/core.c +48 -0
index 59daf8a67ac6..b2fca3b99f02 100644
@@ -346,6 +346,43 @@ static int pdsc_core_init(struct pdsc *pdsc)
 	return err;
 }
 
+static struct pdsc_viftype pdsc_viftype_defaults[] = {
+	[PDS_DEV_TYPE_VDPA] = { .name = PDS_DEV_TYPE_VDPA_STR,
+				.vif_id = PDS_DEV_TYPE_VDPA,
+				.dl_id = DEVLINK_PARAM_GENERIC_ID_ENABLE_VNET },
+	[PDS_DEV_TYPE_MAX] = {}
+};
+
+static int pdsc_viftypes_init(struct pdsc *pdsc)
+{
+	enum pds_core_vif_types vt;
+
+	pdsc->viftype_status = kzalloc(sizeof(pdsc_viftype_defaults),
+				       GFP_KERNEL);
+	if (!pdsc->viftype_status)
+		return -ENOMEM;
+
+	for (vt = 0; vt < PDS_DEV_TYPE_MAX; vt++) {
+		bool vt_support;
+
+		if (!pdsc_viftype_defaults[vt].name)
+			continue;
+
+		/* Grab the defaults */
+		pdsc->viftype_status[vt] = pdsc_viftype_defaults[vt];
+
+		/* See what the Core device has for support */
+		vt_support = !!le16_to_cpu(pdsc->dev_ident.vif_types[vt]);
+		dev_dbg(pdsc->dev, "VIF %s is %ssupported\n",
+			pdsc->viftype_status[vt].name,
+			vt_support ? "" : "not ");
+
+		pdsc->viftype_status[vt].supported = vt_support;
+	}
+
+	return 0;
+}
+
 int pdsc_setup(struct pdsc *pdsc, bool init)
 {
 	int numdescs;
@@ -388,6 +425,14 @@ int pdsc_setup(struct pdsc *pdsc, bool init)
 	if (err)
 		goto err_out_teardown;
 
+	/* Set up the VIFs */
+	err = pdsc_viftypes_init(pdsc);
+	if (err)
+		goto err_out_teardown;
+
+	if (init)
+		pdsc_debugfs_add_viftype(pdsc);
+
 	clear_bit(PDSC_S_FW_DEAD, &pdsc->state);
 	return 0;
 
@@ -404,6 +449,9 @@ void pdsc_teardown(struct pdsc *pdsc, bool removing)
 	pdsc_qcq_free(pdsc, &pdsc->notifyqcq);
 	pdsc_qcq_free(pdsc, &pdsc->adminqcq);
 
+	kfree(pdsc->viftype_status);
+	pdsc->viftype_status = NULL;
+
 	if (pdsc->intr_info) {
 		for (i = 0; i < pdsc->nintrs; i++)
 			pdsc_intr_free(pdsc, i);
drivers/net/ethernet/amd/pds_core/core.h +11 -0
index 7eb02b359f3a..ac0480d7f0f1 100644
@@ -123,6 +123,15 @@ struct pdsc_qcq {
 	struct dentry *dentry;
 };
 
+struct pdsc_viftype {
+	char *name;
+	bool supported;
+	bool enabled;
+	int dl_id;
+	int vif_id;
+	struct pds_auxiliary_dev *padev;
+};
+
 /* No state flags set means we are in a steady running state */
 enum pdsc_state_flags {
 	PDSC_S_FW_DEAD,		    /* stopped, wait on startup or recovery */
@@ -174,6 +183,7 @@ struct pdsc {
 	struct pdsc_qcq adminqcq;
 	struct pdsc_qcq notifyqcq;
 	u64 last_eid;
+	struct pdsc_viftype *viftype_status;
 };
 
 /** enum pds_core_dbell_bits - bitwise composition of dbell values.
@@ -237,6 +247,7 @@ void pdsc_debugfs_destroy(void);
 void pdsc_debugfs_add_dev(struct pdsc *pdsc);
 void pdsc_debugfs_del_dev(struct pdsc *pdsc);
 void pdsc_debugfs_add_ident(struct pdsc *pdsc);
+void pdsc_debugfs_add_viftype(struct pdsc *pdsc);
 void pdsc_debugfs_add_irqs(struct pdsc *pdsc);
 void pdsc_debugfs_add_qcq(struct pdsc *pdsc, struct pdsc_qcq *qcq);
 void pdsc_debugfs_del_qcq(struct pdsc_qcq *qcq);
drivers/net/ethernet/amd/pds_core/debugfs.c +24 -0
index b83e5016644b..8ec392299b7d 100644
@@ -68,6 +68,30 @@ void pdsc_debugfs_add_ident(struct pdsc *pdsc)
 			    pdsc, &identity_fops);
 }
 
+static int viftype_show(struct seq_file *seq, void *v)
+{
+	struct pdsc *pdsc = seq->private;
+	int vt;
+
+	for (vt = 0; vt < PDS_DEV_TYPE_MAX; vt++) {
+		if (!pdsc->viftype_status[vt].name)
+			continue;
+
+		seq_printf(seq, "%s\t%d supported %d enabled\n",
+			   pdsc->viftype_status[vt].name,
+			   pdsc->viftype_status[vt].supported,
+			   pdsc->viftype_status[vt].enabled);
+	}
+	return 0;
+}
+DEFINE_SHOW_ATTRIBUTE(viftype);
+
+void pdsc_debugfs_add_viftype(struct pdsc *pdsc)
+{
+	debugfs_create_file("viftypes", 0400, pdsc->dentry,
+			    pdsc, &viftype_fops);
+}
+
 static const struct debugfs_reg32 intr_ctrl_regs[] = {
 	{ .name = "coal_init", .offset = 0, },
 	{ .name = "mask", .offset = 4, },
include/linux/pds/pds_common.h +19 -0
index f0798ce01acf..b2be14ebadb6 100644
@@ -20,6 +20,25 @@ enum pds_core_driver_type {
 	PDS_DRIVER_ESXI    = 6,
 };
 
+enum pds_core_vif_types {
+	PDS_DEV_TYPE_CORE	= 0,
+	PDS_DEV_TYPE_VDPA	= 1,
+	PDS_DEV_TYPE_VFIO	= 2,
+	PDS_DEV_TYPE_ETH	= 3,
+	PDS_DEV_TYPE_RDMA	= 4,
+	PDS_DEV_TYPE_LM		= 5,
+
+	/* new ones added before this line */
+	PDS_DEV_TYPE_MAX	= 16   /* don't change - used in struct size */
+};
+
+#define PDS_DEV_TYPE_CORE_STR	"Core"
+#define PDS_DEV_TYPE_VDPA_STR	"vDPA"
+#define PDS_DEV_TYPE_VFIO_STR	"VFio"
+#define PDS_DEV_TYPE_ETH_STR	"Eth"
+#define PDS_DEV_TYPE_RDMA_STR	"RDMA"
+#define PDS_DEV_TYPE_LM_STR	"LM"
+
 #define PDS_CORE_IFNAMSIZ		16
 
 /**
Signature · 1 line
2.17.1
#9 lore raw

PATCH v10 9/14 pds_core: add initial VF device handling

In reply to Shannon Nelson

This is the initial VF PCI driver framework for the new pds_vdpa VF device, which will work in conjunction with an auxiliary_bus client of the pds_core driver. This does the very basics of registering for the new VF device, setting up debugfs entries, and registering with devlink.

---
 drivers/net/ethernet/amd/pds_core/core.h |  9 +++++
 drivers/net/ethernet/amd/pds_core/main.c | 48 +++++++++++++++++++++++-
 2 files changed, 56 insertions(+), 1 deletion(-)
drivers/net/ethernet/amd/pds_core/core.h +9 -0
index ac0480d7f0f1..1ec8784773f1 100644
@@ -30,6 +30,12 @@ struct pdsc_dev_bar {
 	int res_index;
 };
 
+struct pdsc_vf {
+	struct pds_auxiliary_dev *padev;
+	u16     index;
+	__le16  vif_types[PDS_DEV_TYPE_MAX];
+};
+
 struct pdsc_devinfo {
 	u8 asic_type;
 	u8 asic_rev;
@@ -147,6 +153,9 @@ struct pdsc {
 	struct dentry *dentry;
 	struct device *dev;
 	struct pdsc_dev_bar bars[PDS_CORE_BARS_MAX];
+	struct pdsc_vf *vfs;
+	int num_vfs;
+	int vf_id;
 	int hw_index;
 	int uid;
 
drivers/net/ethernet/amd/pds_core/main.c +47 -1

74 lines not on this page · raw message

Signature · 1 line
2.17.1
#10 lore raw

PATCH v10 10/14 pds_core: add auxiliary_bus devices

In reply to Shannon Nelson

An auxiliary_bus device is created for each vDPA type VF at VF probe and destroyed at VF remove. The aux device name comes from the driver name + VIF type + the unique id assigned at PCI probe. The VFs are always removed on PF remove, so there should be no issues with VFs trying to access missing PF structures.

The auxiliary_device names will look like "pds_core.vDPA.nn" where 'nn' is the VF's uid.

---
 drivers/net/ethernet/amd/pds_core/Makefile |   1 +
 drivers/net/ethernet/amd/pds_core/auxbus.c | 115 +++++++++++++++++++++
 drivers/net/ethernet/amd/pds_core/core.h   |   6 ++
 drivers/net/ethernet/amd/pds_core/main.c   |  36 ++++++-
 include/linux/pds/pds_auxbus.h             |  14 +++
 include/linux/pds/pds_common.h             |   1 +
 6 files changed, 171 insertions(+), 2 deletions(-)
 create mode 100644 drivers/net/ethernet/amd/pds_core/auxbus.c
 create mode 100644 include/linux/pds/pds_auxbus.h
drivers/net/ethernet/amd/pds_core/Makefile +1 -0
index 6d1d6c58a1fa..0abc33ce826c 100644
@@ -5,6 +5,7 @@ obj-$(CONFIG_PDS_CORE) := pds_core.o
 
 pds_core-y := main.o \
 	      devlink.o \
+	      auxbus.o \
 	      dev.o \
 	      adminq.o \
 	      core.o \
drivers/net/ethernet/amd/pds_core/auxbus.c new file +115 -0

115 lines not on this page · raw message

drivers/net/ethernet/amd/pds_core/core.h +6 -0
index 1ec8784773f1..36099d3ac3dd 100644
@@ -30,8 +30,11 @@ struct pdsc_dev_bar {
 	int res_index;
 };
 
+struct pdsc;
+
 struct pdsc_vf {
 	struct pds_auxiliary_dev *padev;
+	struct pdsc *vf;
 	u16     index;
 	__le16  vif_types[PDS_DEV_TYPE_MAX];
 };
@@ -287,6 +290,9 @@ int pdsc_start(struct pdsc *pdsc);
 void pdsc_stop(struct pdsc *pdsc);
 void pdsc_health_thread(struct work_struct *work);
 
+int pdsc_auxbus_dev_add(struct pdsc *cf, struct pdsc *pf);
+int pdsc_auxbus_dev_del(struct pdsc *cf, struct pdsc *pf);
+
 void pdsc_process_adminq(struct pdsc_qcq *qcq);
 void pdsc_work_thread(struct work_struct *work);
 irqreturn_t pdsc_adminq_isr(int irq, void *data);
drivers/net/ethernet/amd/pds_core/main.c +34 -2
index 511cb91a295e..b848f3360fe2 100644
@@ -169,6 +169,12 @@ static int pdsc_sriov_configure(struct pci_dev *pdev, int num_vfs)
 static int pdsc_init_vf(struct pdsc *vf)
 {
 	struct devlink *dl;
+	struct pdsc *pf;
+	int err;
+
+	pf = pdsc_get_pf_struct(vf->pdev);
+	if (IS_ERR_OR_NULL(pf))
+		return PTR_ERR(pf) ?: -1;
 
 	vf->vf_id = pci_iov_vf_id(vf->pdev);
 
@@ -177,7 +183,15 @@ static int pdsc_init_vf(struct pdsc *vf)
 	devl_register(dl);
 	devl_unlock(dl);
 
-	return 0;
+	pf->vfs[vf->vf_id].vf = vf;
+	err = pdsc_auxbus_dev_add(vf, pf);
+	if (err) {
+		devl_lock(dl);
+		devl_unregister(dl);
+		devl_unlock(dl);
+	}
+
+	return err;
 }
 
 static const struct devlink_health_reporter_ops pdsc_fw_reporter_ops = {
@@ -365,7 +379,19 @@ static void pdsc_remove(struct pci_dev *pdev)
 	}
 	devl_unlock(dl);
 
-	if (!pdev->is_virtfn) {
+	if (pdev->is_virtfn) {
+		struct pdsc *pf;
+
+		pf = pdsc_get_pf_struct(pdsc->pdev);
+		if (!IS_ERR(pf)) {
+			pdsc_auxbus_dev_del(pdsc, pf);
+			pf->vfs[pdsc->vf_id].vf = NULL;
+		}
+	} else {
+		/* Remove the VFs and their aux_bus connections before other
+		 * cleanup so that the clients can use the AdminQ to cleanly
+		 * shut themselves down.
+		 */
 		pdsc_sriov_configure(pdev, 0);
 
 		del_timer_sync(&pdsc->wdtimer);
@@ -402,6 +428,12 @@ static struct pci_driver pdsc_driver = {
 	.sriov_configure = pdsc_sriov_configure,
 };
 
+void *pdsc_get_pf_struct(struct pci_dev *vf_pdev)
+{
+	return pci_iov_get_pf_drvdata(vf_pdev, &pdsc_driver);
+}
+EXPORT_SYMBOL_GPL(pdsc_get_pf_struct);
+
 static int __init pdsc_init_module(void)
 {
 	if (strcmp(KBUILD_MODNAME, PDS_CORE_DRV_NAME))
include/linux/pds/pds_auxbus.h new file +14 -0
new file mode 100644
index 000000000000..493f75b1995e
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2023 Advanced Micro Devices, Inc */
+
+#ifndef _PDSC_AUXBUS_H_
+#define _PDSC_AUXBUS_H_
+
+#include <linux/auxiliary_bus.h>
+
+struct pds_auxiliary_dev {
+	struct auxiliary_device aux_dev;
+	struct pci_dev *vf_pdev;
+	u16 client_id;
+};
+#endif /* _PDSC_AUXBUS_H_ */
include/linux/pds/pds_common.h +1 -0
index b2be14ebadb6..961b3d02c69f 100644
@@ -60,4 +60,5 @@ enum pds_core_logical_qtype {
 	PDS_CORE_QTYPE_MAX     = 16   /* don't change - used in struct size */
 };
 
+void *pdsc_get_pf_struct(struct pci_dev *vf_pdev);
 #endif /* _PDS_COMMON_H_ */
Signature · 1 line
2.17.1
#11 lore raw

PATCH v10 11/14 pds_core: devlink params for enabling VIF support

In reply to Shannon Nelson

Add the devlink parameter switches so the user can enable the features supported by the VFs. The only feature supported at the moment is vDPA.

Example:
devlink dev param set pci/0000:2b:00.0 \
name enable_vnet cmode runtime value true

---
 .../device_drivers/ethernet/amd/pds_core.rst  | 19 +++++
 drivers/net/ethernet/amd/pds_core/core.h      |  7 ++
 drivers/net/ethernet/amd/pds_core/devlink.c   | 73 +++++++++++++++++++
 drivers/net/ethernet/amd/pds_core/main.c      | 34 +++++++--
 4 files changed, 127 insertions(+), 6 deletions(-)
Documentation/networking/device_drivers/ethernet/amd/pds_core.rst +19 -0
index 932ac03a3359..b9f310de862e 100644
@@ -73,6 +73,25 @@ The ``pds_core`` driver reports the following versions
      - fixed
      - The revision of the ASIC for this device
 
+Parameters
+==========
+
+The ``pds_core`` driver implements the following generic
+parameters for controlling the functionality to be made available
+as auxiliary_bus devices.
+
+.. list-table:: Generic parameters implemented
+   :widths: 5 5 8 82
+
+   * - Name
+     - Mode
+     - Type
+     - Description
+   * - ``enable_vnet``
+     - runtime
+     - Boolean
+     - Enables vDPA functionality through an auxiliary_bus device
+
 Firmware Management
 ===================
 
drivers/net/ethernet/amd/pds_core/core.h +7 -0
index 36099d3ac3dd..9e01a9ee6868 100644
@@ -251,6 +251,13 @@ int pdsc_dl_info_get(struct devlink *dl, struct devlink_info_req *req,
 int pdsc_dl_flash_update(struct devlink *dl,
 			 struct devlink_flash_update_params *params,
 			 struct netlink_ext_ack *extack);
+int pdsc_dl_enable_get(struct devlink *dl, u32 id,
+		       struct devlink_param_gset_ctx *ctx);
+int pdsc_dl_enable_set(struct devlink *dl, u32 id,
+		       struct devlink_param_gset_ctx *ctx);
+int pdsc_dl_enable_validate(struct devlink *dl, u32 id,
+			    union devlink_param_value val,
+			    struct netlink_ext_ack *extack);
 
 void __iomem *pdsc_map_dbpage(struct pdsc *pdsc, int page_num);
 
drivers/net/ethernet/amd/pds_core/devlink.c +73 -0

79 lines not on this page · raw message

drivers/net/ethernet/amd/pds_core/main.c +28 -6
index b848f3360fe2..e2d14b1ca471 100644
@@ -199,6 +199,14 @@ static const struct devlink_health_reporter_ops pdsc_fw_reporter_ops = {
 	.diagnose = pdsc_fw_reporter_diagnose,
 };
 
+static const struct devlink_param pdsc_dl_params[] = {
+	DEVLINK_PARAM_GENERIC(ENABLE_VNET,
+			      BIT(DEVLINK_PARAM_CMODE_RUNTIME),
+			      pdsc_dl_enable_get,
+			      pdsc_dl_enable_set,
+			      pdsc_dl_enable_validate),
+};
+
 #define PDSC_WQ_NAME_LEN 24
 
 static int pdsc_init_pf(struct pdsc *pdsc)
@@ -246,13 +254,19 @@ static int pdsc_init_pf(struct pdsc *pdsc)
 
 	dl = priv_to_devlink(pdsc);
 	devl_lock(dl);
+	err = devl_params_register(dl, pdsc_dl_params,
+				   ARRAY_SIZE(pdsc_dl_params));
+	if (err) {
+		dev_warn(pdsc->dev, "Failed to register devlink params: %pe\n",
+			 ERR_PTR(err));
+		goto err_out_unlock_dl;
+	}
 
 	hr = devl_health_reporter_create(dl, &pdsc_fw_reporter_ops, 0, pdsc);
 	if (IS_ERR(hr)) {
 		dev_warn(pdsc->dev, "Failed to create fw reporter: %pe\n", hr);
 		err = PTR_ERR(hr);
-		devl_unlock(dl);
-		goto err_out_stop;
+		goto err_out_unreg_params;
 	}
 	pdsc->fw_reporter = hr;
 
@@ -264,7 +278,11 @@ static int pdsc_init_pf(struct pdsc *pdsc)
 
 	return 0;
 
-err_out_stop:
+err_out_unreg_params:
+	devl_params_unregister(dl, pdsc_dl_params,
+			       ARRAY_SIZE(pdsc_dl_params));
+err_out_unlock_dl:
+	devl_unlock(dl);
 	pdsc_stop(pdsc);
 err_out_teardown:
 	pdsc_teardown(pdsc, PDSC_TEARDOWN_REMOVING);
@@ -373,9 +391,13 @@ static void pdsc_remove(struct pci_dev *pdev)
 	dl = priv_to_devlink(pdsc);
 	devl_lock(dl);
 	devl_unregister(dl);
-	if (pdsc->fw_reporter) {
-		devl_health_reporter_destroy(pdsc->fw_reporter);
-		pdsc->fw_reporter = NULL;
+	if (!pdev->is_virtfn) {
+		if (pdsc->fw_reporter) {
+			devl_health_reporter_destroy(pdsc->fw_reporter);
+			pdsc->fw_reporter = NULL;
+		}
+		devl_params_unregister(dl, pdsc_dl_params,
+				       ARRAY_SIZE(pdsc_dl_params));
 	}
 	devl_unlock(dl);
 
Signature · 1 line
2.17.1
#12 lore raw

PATCH v10 12/14 pds_core: add the aux client API

In reply to Shannon Nelson

Add the client API operations for running adminq commands. The core registers the client with the FW, then the client has a context for requesting adminq services. We expect to add additional operations for other clients, including requesting additional private adminqs and IRQs, but don't have the need yet.

---
 drivers/net/ethernet/amd/pds_core/auxbus.c | 151 ++++++++++++++++++++-
 include/linux/pds/pds_auxbus.h             |   6 +
 include/linux/pds/pds_common.h             |   2 +
 3 files changed, 158 insertions(+), 1 deletion(-)
drivers/net/ethernet/amd/pds_core/auxbus.c +150 -1
index adee516b3f0c..0cf77a7eb94b 100644
@@ -6,6 +6,136 @@
 #include "core.h"
 #include <linux/pds/pds_auxbus.h>
 
+/**
+ * pds_client_register - Link the client to the firmware
+ * @pf_pdev:	ptr to the PF driver struct
+ * @devname:	name that includes service into, e.g. pds_core.vDPA
+ *
+ * Return: 0 on success, or
+ *         negative for error
+ */
+int pds_client_register(struct pci_dev *pf_pdev, char *devname)
+{
+	union pds_core_adminq_comp comp = {};
+	union pds_core_adminq_cmd cmd = {};
+	struct pdsc *pf;
+	int err;
+	u16 ci;
+
+	pf = pci_get_drvdata(pf_pdev);
+	if (pf->state)
+		return -ENXIO;
+
+	cmd.client_reg.opcode = PDS_AQ_CMD_CLIENT_REG;
+	strncpy(cmd.client_reg.devname, devname,
+		sizeof(cmd.client_reg.devname));
+
+	err = pdsc_adminq_post(pf, &cmd, &comp, false);
+	if (err) {
+		dev_info(pf->dev, "register dev_name %s with DSC failed, status %d: %pe\n",
+			 devname, comp.status, ERR_PTR(err));
+		return err;
+	}
+
+	ci = le16_to_cpu(comp.client_reg.client_id);
+	if (!ci) {
+		dev_err(pf->dev, "%s: device returned null client_id\n",
+			__func__);
+		return -EIO;
+	}
+
+	dev_dbg(pf->dev, "%s: device returned client_id %d for %s\n",
+		__func__, ci, devname);
+
+	return ci;
+}
+EXPORT_SYMBOL_GPL(pds_client_register);
+
+/**
+ * pds_client_unregister - Unlink the client from the firmware
+ * @pf_pdev:	ptr to the PF driver struct
+ * @client_id:	id returned from pds_client_register()
+ *
+ * Return: 0 on success, or
+ *         negative for error
+ */
+int pds_client_unregister(struct pci_dev *pf_pdev, u16 client_id)
+{
+	union pds_core_adminq_comp comp = {};
+	union pds_core_adminq_cmd cmd = {};
+	struct pdsc *pf;
+	int err;
+
+	pf = pci_get_drvdata(pf_pdev);
+	if (pf->state)
+		return -ENXIO;
+
+	cmd.client_unreg.opcode = PDS_AQ_CMD_CLIENT_UNREG;
+	cmd.client_unreg.client_id = cpu_to_le16(client_id);
+
+	err = pdsc_adminq_post(pf, &cmd, &comp, false);
+	if (err)
+		dev_info(pf->dev, "unregister client_id %d failed, status %d: %pe\n",
+			 client_id, comp.status, ERR_PTR(err));
+
+	return err;
+}
+EXPORT_SYMBOL_GPL(pds_client_unregister);
+
+/**
+ * pds_client_adminq_cmd - Process an adminq request for the client
+ * @padev:   ptr to the client device
+ * @req:     ptr to buffer with request
+ * @req_len: length of actual struct used for request
+ * @resp:    ptr to buffer where answer is to be copied
+ * @flags:   optional flags from pds_core_adminq_flags
+ *
+ * Return: 0 on success, or
+ *         negative for error
+ *
+ * Client sends pointers to request and response buffers
+ * Core copies request data into pds_core_client_request_cmd
+ * Core sets other fields as needed
+ * Core posts to AdminQ
+ * Core copies completion data into response buffer
+ */
+int pds_client_adminq_cmd(struct pds_auxiliary_dev *padev,
+			  union pds_core_adminq_cmd *req,
+			  size_t req_len,
+			  union pds_core_adminq_comp *resp,
+			  u64 flags)
+{
+	union pds_core_adminq_cmd cmd = {};
+	struct pci_dev *pf_pdev;
+	struct pdsc *pf;
+	size_t cp_len;
+	int err;
+
+	pf_pdev = pci_physfn(padev->vf_pdev);
+	pf = pci_get_drvdata(pf_pdev);
+
+	dev_dbg(pf->dev, "%s: %s opcode %d\n",
+		__func__, dev_name(&padev->aux_dev.dev), req->opcode);
+
+	if (pf->state)
+		return -ENXIO;
+
+	/* Wrap the client's request */
+	cmd.client_request.opcode = PDS_AQ_CMD_CLIENT_CMD;
+	cmd.client_request.client_id = cpu_to_le16(padev->client_id);
+	cp_len = min_t(size_t, req_len, sizeof(cmd.client_request.client_cmd));
+	memcpy(cmd.client_request.client_cmd, req, cp_len);
+
+	err = pdsc_adminq_post(pf, &cmd, resp,
+			       !!(flags & PDS_AQ_FLAG_FASTPOLL));
+	if (err && err != -EAGAIN)
+		dev_info(pf->dev, "client admin cmd failed: %pe\n",
+			 ERR_PTR(err));
+
+	return err;
+}
+EXPORT_SYMBOL_GPL(pds_client_adminq_cmd);
+
 static void pdsc_auxbus_dev_release(struct device *dev)
 {
 	struct pds_auxiliary_dev *padev =
@@ -16,6 +146,7 @@ static void pdsc_auxbus_dev_release(struct device *dev)
 
 static struct pds_auxiliary_dev *pdsc_auxbus_dev_register(struct pdsc *cf,
 							  struct pdsc *pf,
+							  u16 client_id,
 							  char *name)
 {
 	struct auxiliary_device *aux_dev;
@@ -27,6 +158,7 @@ static struct pds_auxiliary_dev *pdsc_auxbus_dev_register(struct pdsc *cf,
 		return ERR_PTR(-ENOMEM);
 
 	padev->vf_pdev = cf->pdev;
+	padev->client_id = client_id;
 
 	aux_dev = &padev->aux_dev;
 	aux_dev->name = name;
@@ -66,8 +198,10 @@ int pdsc_auxbus_dev_del(struct pdsc *cf, struct pdsc *pf)
 
 	padev = pf->vfs[cf->vf_id].padev;
 	if (padev) {
+		pds_client_unregister(pf->pdev, padev->client_id);
 		auxiliary_device_delete(&padev->aux_dev);
 		auxiliary_device_uninit(&padev->aux_dev);
+		padev->client_id = 0;
 	}
 	pf->vfs[cf->vf_id].padev = NULL;
 
@@ -79,7 +213,9 @@ int pdsc_auxbus_dev_add(struct pdsc *cf, struct pdsc *pf)
 {
 	struct pds_auxiliary_dev *padev;
 	enum pds_core_vif_types vt;
+	char devname[PDS_DEVNAME_LEN];
 	u16 vt_support;
+	int client_id;
 	int err = 0;
 
 	mutex_lock(&pf->config_lock);
@@ -101,9 +237,22 @@ int pdsc_auxbus_dev_add(struct pdsc *cf, struct pdsc *pf)
 	      pf->viftype_status[vt].enabled))
 		goto out_unlock;
 
-	padev = pdsc_auxbus_dev_register(cf, pf,
+	/* Need to register with FW and get the client_id before
+	 * creating the aux device so that the aux client can run
+	 * adminq commands as part its probe
+	 */
+	snprintf(devname, sizeof(devname), "%s.%s.%d",
+		 PDS_CORE_DRV_NAME, pf->viftype_status[vt].name, cf->uid);
+	client_id = pds_client_register(pf->pdev, devname);
+	if (client_id < 0) {
+		err = client_id;
+		goto out_unlock;
+	}
+
+	padev = pdsc_auxbus_dev_register(cf, pf, client_id,
 					 pf->viftype_status[vt].name);
 	if (IS_ERR(padev)) {
+		pds_client_unregister(pf->pdev, client_id);
 		err = PTR_ERR(padev);
 		goto out_unlock;
 	}
include/linux/pds/pds_auxbus.h +6 -0
index 493f75b1995e..214ef12302d0 100644
@@ -11,4 +11,10 @@ struct pds_auxiliary_dev {
 	struct pci_dev *vf_pdev;
 	u16 client_id;
 };
+
+int pds_client_adminq_cmd(struct pds_auxiliary_dev *padev,
+			  union pds_core_adminq_cmd *req,
+			  size_t req_len,
+			  union pds_core_adminq_comp *resp,
+			  u64 flags);
 #endif /* _PDSC_AUXBUS_H_ */
include/linux/pds/pds_common.h +2 -0
index 961b3d02c69f..4b37675fde3e 100644
@@ -61,4 +61,6 @@ enum pds_core_logical_qtype {
 };
 
 void *pdsc_get_pf_struct(struct pci_dev *vf_pdev);
+int pds_client_register(struct pci_dev *pf_pdev, char *devname);
+int pds_client_unregister(struct pci_dev *pf_pdev, u16 client_id);
 #endif /* _PDS_COMMON_H_ */
Signature · 1 line
2.17.1
#13 lore raw

PATCH v10 13/14 pds_core: publish events to the clients

In reply to Shannon Nelson

When the Core device gets an event from the device, or notices the device FW to be up or down, it needs to send those events on to the clients that have an event handler. Add the code to pass along the events to the clients.

The entry points pdsc_register_notify() and pdsc_unregister_notify() are EXPORTed for other drivers that want to listen for these events.

---
 drivers/net/ethernet/amd/pds_core/adminq.c |  2 ++
 drivers/net/ethernet/amd/pds_core/core.c   | 32 ++++++++++++++++++++++
 drivers/net/ethernet/amd/pds_core/core.h   |  3 ++
 include/linux/pds/pds_common.h             |  2 ++
 4 files changed, 39 insertions(+)
drivers/net/ethernet/amd/pds_core/adminq.c +2 -0
index fb2ba3f62480..045fe133f6ee 100644
@@ -29,11 +29,13 @@ static int pdsc_process_notifyq(struct pdsc_qcq *qcq)
 		case PDS_EVENT_LINK_CHANGE:
 			dev_info(pdsc->dev, "NotifyQ LINK_CHANGE ecode %d eid %lld\n",
 				 ecode, eid);
+			pdsc_notify(PDS_EVENT_LINK_CHANGE, comp);
 			break;
 
 		case PDS_EVENT_RESET:
 			dev_info(pdsc->dev, "NotifyQ RESET ecode %d eid %lld\n",
 				 ecode, eid);
+			pdsc_notify(PDS_EVENT_RESET, comp);
 			break;
 
 		case PDS_EVENT_XCVR:
drivers/net/ethernet/amd/pds_core/core.c +32 -0
index b2fca3b99f02..483a070d96fa 100644
@@ -6,6 +6,25 @@
 
 #include "core.h"
 
+static BLOCKING_NOTIFIER_HEAD(pds_notify_chain);
+
+int pdsc_register_notify(struct notifier_block *nb)
+{
+	return blocking_notifier_chain_register(&pds_notify_chain, nb);
+}
+EXPORT_SYMBOL_GPL(pdsc_register_notify);
+
+void pdsc_unregister_notify(struct notifier_block *nb)
+{
+	blocking_notifier_chain_unregister(&pds_notify_chain, nb);
+}
+EXPORT_SYMBOL_GPL(pdsc_unregister_notify);
+
+void pdsc_notify(unsigned long event, void *data)
+{
+	blocking_notifier_call_chain(&pds_notify_chain, event, data);
+}
+
 void pdsc_intr_free(struct pdsc *pdsc, int index)
 {
 	struct pdsc_intr_info *intr_info;
@@ -494,12 +513,19 @@ void pdsc_stop(struct pdsc *pdsc)
 
 static void pdsc_fw_down(struct pdsc *pdsc)
 {
+	union pds_core_notifyq_comp reset_event = {
+		.reset.ecode = cpu_to_le16(PDS_EVENT_RESET),
+		.reset.state = 0,
+	};
+
 	if (test_and_set_bit(PDSC_S_FW_DEAD, &pdsc->state)) {
 		dev_err(pdsc->dev, "%s: already happening\n", __func__);
 		return;
 	}
 
+	/* Notify clients of fw_down */
 	devlink_health_report(pdsc->fw_reporter, "FW down reported", pdsc);
+	pdsc_notify(PDS_EVENT_RESET, &reset_event);
 
 	pdsc_stop(pdsc);
 	pdsc_teardown(pdsc, PDSC_TEARDOWN_RECOVERY);
@@ -507,6 +533,10 @@ static void pdsc_fw_down(struct pdsc *pdsc)
 
 static void pdsc_fw_up(struct pdsc *pdsc)
 {
+	union pds_core_notifyq_comp reset_event = {
+		.reset.ecode = cpu_to_le16(PDS_EVENT_RESET),
+		.reset.state = 1,
+	};
 	int err;
 
 	if (!test_bit(PDSC_S_FW_DEAD, &pdsc->state)) {
@@ -522,9 +552,11 @@ static void pdsc_fw_up(struct pdsc *pdsc)
 	if (err)
 		goto err_out;
 
+	/* Notify clients of fw_up */
 	pdsc->fw_recoveries++;
 	devlink_health_reporter_state_update(pdsc->fw_reporter,
 					     DEVLINK_HEALTH_REPORTER_STATE_HEALTHY);
+	pdsc_notify(PDS_EVENT_RESET, &reset_event);
 
 	return;
 
drivers/net/ethernet/amd/pds_core/core.h +3 -0
index 9e01a9ee6868..e545fafc4819 100644
@@ -297,6 +297,9 @@ int pdsc_start(struct pdsc *pdsc);
 void pdsc_stop(struct pdsc *pdsc);
 void pdsc_health_thread(struct work_struct *work);
 
+int pdsc_register_notify(struct notifier_block *nb);
+void pdsc_unregister_notify(struct notifier_block *nb);
+void pdsc_notify(unsigned long event, void *data);
 int pdsc_auxbus_dev_add(struct pdsc *cf, struct pdsc *pf);
 int pdsc_auxbus_dev_del(struct pdsc *cf, struct pdsc *pf);
 
include/linux/pds/pds_common.h +2 -0
index 4b37675fde3e..060331486d50 100644
@@ -60,6 +60,8 @@ enum pds_core_logical_qtype {
 	PDS_CORE_QTYPE_MAX     = 16   /* don't change - used in struct size */
 };
 
+int pdsc_register_notify(struct notifier_block *nb);
+void pdsc_unregister_notify(struct notifier_block *nb);
 void *pdsc_get_pf_struct(struct pci_dev *vf_pdev);
 int pds_client_register(struct pci_dev *pf_pdev, char *devname);
 int pds_client_unregister(struct pci_dev *pf_pdev, u16 client_id);
Signature · 1 line
2.17.1
#14 lore raw

PATCH v10 14/14 pds_core: Kconfig and pds_core.rst

In reply to Shannon Nelson

Remaining documentation and Kconfig hook for building the driver.

---
 .../device_drivers/ethernet/amd/pds_core.rst     | 16 ++++++++++++++++
 MAINTAINERS                                      |  9 +++++++++
 drivers/net/ethernet/amd/Kconfig                 | 12 ++++++++++++
 drivers/net/ethernet/amd/Makefile                |  1 +
 4 files changed, 38 insertions(+)
Documentation/networking/device_drivers/ethernet/amd/pds_core.rst +16 -0
index b9f310de862e..9e8a16c44102 100644
@@ -114,6 +114,22 @@ The driver supports a devlink health reporter for FW status::
   # devlink health diagnose pci/0000:2b:00.0 reporter fw
    Status: healthy State: 1 Generation: 0 Recoveries: 0
 
+Enabling the driver
+===================
+
+The driver is enabled via the standard kernel configuration system,
+using the make command::
+
+  make oldconfig/menuconfig/etc.
+
+The driver is located in the menu structure at:
+
+  -> Device Drivers
+    -> Network device support (NETDEVICES [=y])
+      -> Ethernet driver support
+        -> AMD devices
+          -> AMD/Pensando Ethernet PDS_CORE Support
+
 Support
 =======
 
MAINTAINERS +9 -0
index b8b275e27cdb..2d9a95cffbb3 100644
@@ -1041,6 +1041,15 @@ F:	drivers/gpu/drm/amd/include/vi_structs.h
 F:	include/uapi/linux/kfd_ioctl.h
 F:	include/uapi/linux/kfd_sysfs.h
 
+AMD PDS CORE DRIVER
+M:	Shannon Nelson <[email protected]>
+M:	Brett Creeley <[email protected]>
+L:	[email protected]
+S:	Supported
+F:	Documentation/networking/device_drivers/ethernet/amd/pds_core.rst
+F:	drivers/net/ethernet/amd/pds_core/
+F:	include/linux/pds/
+
 AMD SPI DRIVER
 M:	Sanjay R Mehta <[email protected]>
 S:	Maintained
drivers/net/ethernet/amd/Kconfig +12 -0
index ab42f75b9413..235fcacef5c5 100644
@@ -186,4 +186,16 @@ config AMD_XGBE_HAVE_ECC
 	bool
 	default n
 
+config PDS_CORE
+	tristate "AMD/Pensando Data Systems Core Device Support"
+	depends on 64BIT && PCI
+	help
+	  This enables the support for the AMD/Pensando Core device family of
+	  adapters.  More specific information on this driver can be
+	  found in
+	  <file:Documentation/networking/device_drivers/ethernet/amd/pds_core.rst>.
+
+	  To compile this driver as a module, choose M here. The module
+	  will be called pds_core.
+
 endif # NET_VENDOR_AMD
drivers/net/ethernet/amd/Makefile +1 -0
index 42742afe9115..2dcfb84731e1 100644
@@ -17,3 +17,4 @@ obj-$(CONFIG_PCNET32) += pcnet32.o
 obj-$(CONFIG_SUN3LANCE) += sun3lance.o
 obj-$(CONFIG_SUNLANCE) += sunlance.o
 obj-$(CONFIG_AMD_XGBE) += xgbe/
+obj-$(CONFIG_PDS_CORE) += pds_core/
Signature · 1 line
2.17.1
#15 lore raw

Re: [PATCH v10 net-next 14/14] pds_core: Kconfig and pds_core.rst

kernel test robot noticed the following build warnings:

Hi Shannon,

kernel test robot noticed the following build warnings:

[auto build test WARNING on net-next/main]

url:    https://github.com/intel-lab-lkp/linux/commits/Shannon-Nelson/pds_core-initial-framework-for-pds_core-PF-driver/20230418-083612
patch link:    https://lore.kernel.org/r/20230418003228.28234-15-shannon.nelson%40amd.com
patch subject: [PATCH v10 net-next 14/14] pds_core: Kconfig and pds_core.rst
config: sparc-allyesconfig (https://download.01.org/0day-ci/archive/20230418/[email protected]/config)
compiler: sparc64-linux-gcc (GCC) 12.1.0

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/98284802ba918b756684dcf00cfa88bbab5cb498
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Shannon-Nelson/pds_core-initial-framework-for-pds_core-PF-driver/20230418-083612 git checkout 98284802ba918b756684dcf00cfa88bbab5cb498
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=sparc olddefconfig COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=sparc SHELL=/bin/bash drivers/net/

If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <[email protected]>
| Link: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All warnings (new ones prefixed by >>):

drivers/net/ethernet/amd/pds_core/auxbus.c: In function 'pds_client_register':

>> drivers/net/ethernet/amd/pds_core/auxbus.c:30:9: warning: 'strncpy' specified bound 32 equals destination size [-Wstringop-truncation]
      30 |         strncpy(cmd.client_reg.devname, devname,
         |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      31 |                 sizeof(cmd.client_reg.devname));
         |                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

vim +/strncpy +30 drivers/net/ethernet/amd/pds_core/auxbus.c

de85ef78c6b4f6 Shannon Nelson 2023-04-17 8
283e6c9307d7f4 Shannon Nelson 2023-04-17 9 /**

283e6c9307d7f4 Shannon Nelson 2023-04-17  10   * pds_client_register - Link the client to the firmware
283e6c9307d7f4 Shannon Nelson 2023-04-17  11   * @pf_pdev:	ptr to the PF driver struct
283e6c9307d7f4 Shannon Nelson 2023-04-17  12   * @devname:	name that includes service into, e.g. pds_core.vDPA
283e6c9307d7f4 Shannon Nelson 2023-04-17  13   *
283e6c9307d7f4 Shannon Nelson 2023-04-17  14   * Return: 0 on success, or
283e6c9307d7f4 Shannon Nelson 2023-04-17  15   *         negative for error
283e6c9307d7f4 Shannon Nelson 2023-04-17  16   */
283e6c9307d7f4 Shannon Nelson 2023-04-17  17  int pds_client_register(struct pci_dev *pf_pdev, char *devname)
283e6c9307d7f4 Shannon Nelson 2023-04-17  18  {
283e6c9307d7f4 Shannon Nelson 2023-04-17  19  	union pds_core_adminq_comp comp = {};
283e6c9307d7f4 Shannon Nelson 2023-04-17  20  	union pds_core_adminq_cmd cmd = {};
283e6c9307d7f4 Shannon Nelson 2023-04-17  21  	struct pdsc *pf;
283e6c9307d7f4 Shannon Nelson 2023-04-17  22  	int err;
283e6c9307d7f4 Shannon Nelson 2023-04-17  23  	u16 ci;
283e6c9307d7f4 Shannon Nelson 2023-04-17  24  
283e6c9307d7f4 Shannon Nelson 2023-04-17  25  	pf = pci_get_drvdata(pf_pdev);
283e6c9307d7f4 Shannon Nelson 2023-04-17  26  	if (pf->state)
283e6c9307d7f4 Shannon Nelson 2023-04-17  27  		return -ENXIO;
283e6c9307d7f4 Shannon Nelson 2023-04-17  28  
283e6c9307d7f4 Shannon Nelson 2023-04-17  29  	cmd.client_reg.opcode = PDS_AQ_CMD_CLIENT_REG;
283e6c9307d7f4 Shannon Nelson 2023-04-17 @30  	strncpy(cmd.client_reg.devname, devname,
283e6c9307d7f4 Shannon Nelson 2023-04-17  31  		sizeof(cmd.client_reg.devname));
283e6c9307d7f4 Shannon Nelson 2023-04-17  32  
283e6c9307d7f4 Shannon Nelson 2023-04-17  33  	err = pdsc_adminq_post(pf, &cmd, &comp, false);
283e6c9307d7f4 Shannon Nelson 2023-04-17  34  	if (err) {
283e6c9307d7f4 Shannon Nelson 2023-04-17  35  		dev_info(pf->dev, "register dev_name %s with DSC failed, status %d: %pe\n",
283e6c9307d7f4 Shannon Nelson 2023-04-17  36  			 devname, comp.status, ERR_PTR(err));
283e6c9307d7f4 Shannon Nelson 2023-04-17  37  		return err;
283e6c9307d7f4 Shannon Nelson 2023-04-17  38  	}
283e6c9307d7f4 Shannon Nelson 2023-04-17  39  
283e6c9307d7f4 Shannon Nelson 2023-04-17  40  	ci = le16_to_cpu(comp.client_reg.client_id);
283e6c9307d7f4 Shannon Nelson 2023-04-17  41  	if (!ci) {
283e6c9307d7f4 Shannon Nelson 2023-04-17  42  		dev_err(pf->dev, "%s: device returned null client_id\n",
283e6c9307d7f4 Shannon Nelson 2023-04-17  43  			__func__);
283e6c9307d7f4 Shannon Nelson 2023-04-17  44  		return -EIO;
283e6c9307d7f4 Shannon Nelson 2023-04-17  45  	}
283e6c9307d7f4 Shannon Nelson 2023-04-17  46  
283e6c9307d7f4 Shannon Nelson 2023-04-17  47  	dev_dbg(pf->dev, "%s: device returned client_id %d for %s\n",
283e6c9307d7f4 Shannon Nelson 2023-04-17  48  		__func__, ci, devname);
283e6c9307d7f4 Shannon Nelson 2023-04-17  49  
283e6c9307d7f4 Shannon Nelson 2023-04-17  50  	return ci;
283e6c9307d7f4 Shannon Nelson 2023-04-17  51  }
283e6c9307d7f4 Shannon Nelson 2023-04-17  52  EXPORT_SYMBOL_GPL(pds_client_register);
283e6c9307d7f4 Shannon Nelson 2023-04-17  53  
Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests
#16 lore raw

Re: [PATCH v10 net-next 14/14] pds_core: Kconfig and pds_core.rst

Quoted 27 lines · kernel test robot

Hi Shannon,

kernel test robot noticed the following build warnings:

[auto build test WARNING on net-next/main]

url:    https://github.com/intel-lab-lkp/linux/commits/Shannon-Nelson/pds_core-initial-framework-for-pds_core-PF-driver/20230418-083612
patch link:    https://lore.kernel.org/r/20230418003228.28234-15-shannon.nelson%40amd.com
patch subject: [PATCH v10 net-next 14/14] pds_core: Kconfig and pds_core.rst
config: sparc-allyesconfig (https://download.01.org/0day-ci/archive/20230418/[email protected]/config)
compiler: sparc64-linux-gcc (GCC) 12.1.0

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/98284802ba918b756684dcf00cfa88bbab5cb498
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Shannon-Nelson/pds_core-initial-framework-for-pds_core-PF-driver/20230418-083612 git checkout 98284802ba918b756684dcf00cfa88bbab5cb498
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=sparc olddefconfig COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=sparc SHELL=/bin/bash drivers/net/

If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <[email protected]>
| Link: https://lore.kernel.org/oe-kbuild-all/[email protected]/

Quoted 8 lines

All warnings (new ones prefixed by >>):

drivers/net/ethernet/amd/pds_core/auxbus.c: In function 'pds_client_register':

Quoted 1 line
Quoted 1 line

drivers/net/ethernet/amd/pds_core/auxbus.c:30:9: warning: 'strncpy' specified bound 32 equals destination size [-Wstringop-truncation]

       30 |         strncpy(cmd.client_reg.devname, devname,
          |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
       31 |                 sizeof(cmd.client_reg.devname));
          |                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Yes, this should be a strscpy(), not strncpy(). As much as I'd like to fix this one thing Right Now, I'll wait another day before I repost the full patchset for one little tweak.

sln

Quoted 54 lines

vim +/strncpy +30 drivers/net/ethernet/amd/pds_core/auxbus.c

de85ef78c6b4f6 Shannon Nelson 2023-04-17 8
283e6c9307d7f4 Shannon Nelson 2023-04-17 9 /**

283e6c9307d7f4 Shannon Nelson 2023-04-17  10   * pds_client_register - Link the client to the firmware
283e6c9307d7f4 Shannon Nelson 2023-04-17  11   * @pf_pdev:      ptr to the PF driver struct
283e6c9307d7f4 Shannon Nelson 2023-04-17  12   * @devname:      name that includes service into, e.g. pds_core.vDPA
283e6c9307d7f4 Shannon Nelson 2023-04-17  13   *
283e6c9307d7f4 Shannon Nelson 2023-04-17  14   * Return: 0 on success, or
283e6c9307d7f4 Shannon Nelson 2023-04-17  15   *         negative for error
283e6c9307d7f4 Shannon Nelson 2023-04-17  16   */
283e6c9307d7f4 Shannon Nelson 2023-04-17  17  int pds_client_register(struct pci_dev *pf_pdev, char *devname)
283e6c9307d7f4 Shannon Nelson 2023-04-17  18  {
283e6c9307d7f4 Shannon Nelson 2023-04-17  19    union pds_core_adminq_comp comp = {};
283e6c9307d7f4 Shannon Nelson 2023-04-17  20    union pds_core_adminq_cmd cmd = {};
283e6c9307d7f4 Shannon Nelson 2023-04-17  21    struct pdsc *pf;
283e6c9307d7f4 Shannon Nelson 2023-04-17  22    int err;
283e6c9307d7f4 Shannon Nelson 2023-04-17  23    u16 ci;
283e6c9307d7f4 Shannon Nelson 2023-04-17  24
283e6c9307d7f4 Shannon Nelson 2023-04-17  25    pf = pci_get_drvdata(pf_pdev);
283e6c9307d7f4 Shannon Nelson 2023-04-17  26    if (pf->state)
283e6c9307d7f4 Shannon Nelson 2023-04-17  27            return -ENXIO;
283e6c9307d7f4 Shannon Nelson 2023-04-17  28
283e6c9307d7f4 Shannon Nelson 2023-04-17  29    cmd.client_reg.opcode = PDS_AQ_CMD_CLIENT_REG;
283e6c9307d7f4 Shannon Nelson 2023-04-17 @30    strncpy(cmd.client_reg.devname, devname,
283e6c9307d7f4 Shannon Nelson 2023-04-17  31            sizeof(cmd.client_reg.devname));
283e6c9307d7f4 Shannon Nelson 2023-04-17  32
283e6c9307d7f4 Shannon Nelson 2023-04-17  33    err = pdsc_adminq_post(pf, &cmd, &comp, false);
283e6c9307d7f4 Shannon Nelson 2023-04-17  34    if (err) {
283e6c9307d7f4 Shannon Nelson 2023-04-17  35            dev_info(pf->dev, "register dev_name %s with DSC failed, status %d: %pe\n",
283e6c9307d7f4 Shannon Nelson 2023-04-17  36                     devname, comp.status, ERR_PTR(err));
283e6c9307d7f4 Shannon Nelson 2023-04-17  37            return err;
283e6c9307d7f4 Shannon Nelson 2023-04-17  38    }
283e6c9307d7f4 Shannon Nelson 2023-04-17  39
283e6c9307d7f4 Shannon Nelson 2023-04-17  40    ci = le16_to_cpu(comp.client_reg.client_id);
283e6c9307d7f4 Shannon Nelson 2023-04-17  41    if (!ci) {
283e6c9307d7f4 Shannon Nelson 2023-04-17  42            dev_err(pf->dev, "%s: device returned null client_id\n",
283e6c9307d7f4 Shannon Nelson 2023-04-17  43                    __func__);
283e6c9307d7f4 Shannon Nelson 2023-04-17  44            return -EIO;
283e6c9307d7f4 Shannon Nelson 2023-04-17  45    }
283e6c9307d7f4 Shannon Nelson 2023-04-17  46
283e6c9307d7f4 Shannon Nelson 2023-04-17  47    dev_dbg(pf->dev, "%s: device returned client_id %d for %s\n",
283e6c9307d7f4 Shannon Nelson 2023-04-17  48            __func__, ci, devname);
283e6c9307d7f4 Shannon Nelson 2023-04-17  49
283e6c9307d7f4 Shannon Nelson 2023-04-17  50    return ci;
283e6c9307d7f4 Shannon Nelson 2023-04-17  51  }
283e6c9307d7f4 Shannon Nelson 2023-04-17  52  EXPORT_SYMBOL_GPL(pds_client_register);
283e6c9307d7f4 Shannon Nelson 2023-04-17  53

--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests