New Microsemi PCI Switch Management Driver
Subsystems (4)
Logan Gunthorpe
maintainerv4 cover New Microsemi PCI Switch Management Driver
Changes since v3:
* Removed stdev_is_alive and moved pci deinit functions
into the device release function which only occurs after all cdev instances are closed. (per Bjorn)
* Reduced locking in read/write functions (per Bjorn)
* Use pci_alloc_irq_vectors to vastly improve ISR initialization (Bjorn)
* A few other minor nits and cleanup as noticed by Bjorn
Changes since v2:
* Collected reviewed and tested tags
* Very minor fix to the error path in the create function
Changes since v1:
* Rebased onto 4.10-rc6 (cleanly)
* Split the patch into a few more easily digestible patches (as suggested by Greg Kroah-Hartman)
* Folded switchtec.c into switchtec.h (per Greg)
* Fixed a bunch of 32bit build warnings caught by the kbuild test robot
* Fixed some issues in the documentation so it has a proper reStructredText format (as noted by Jonathan Corbet)
* Fixed padding and sizes in the IOCTL structures as noticed by Emil Velikov and used pahole to verify their consistency across 32 and 64 bit builds
* Reworked one of the IOCTL interfaces to be more future proof (per Emil).
Changes since RFC:
* Fixed incorrect use of the drive model as pointed out by Greg Kroah-Hartman
* Used devm functions as suggested by Keith Busch
* Added a handful of sysfs attributes to the switchtec class
* Added a handful of IOCTLs to the switchtec device
* A number of miscellaneous bug fixes
--
Hi,
This is a continuation of the RFC we posted lasted month [1] which proposes a management driver for Microsemi's Switchtec line of PCI switches. This hardware is still looking to be used in the Open Compute Platform
To make this entirely clear: the Switchtec products are compliant with the PCI specifications and are supported today with the standard in-kernel driver. However, these devices also expose a management endpoint on a separate PCI function address which can be used to perform some advanced operations. This is a driver for that function. See the patch for more information.
Since the RFC, we've made the changes requested by Greg Kroah-Hartman and Keith Busch, and we've also fleshed out a number of features. We've added a couple of IOCTLs and sysfs attributes which are documented in the patch. Significant work has also been done on the userspace tool which is available under a GPL license at [2]. We've also had testing done by some of the interested parties.
We hope to see this work included in either 4.11 or 4.12 assuming a smooth review process.
The patch is based off of the v4.10-rc6 release.
Thanks for your review,
Logan
[1] https://www.spinics.net/lists/linux-pci/msg56897.html
[2] https://github.com/sbates130272/switchtec-user
Logan Gunthorpe (4):
MicroSemi Switchtec management interface driver
switchtec: Add user interface documentation
switchtec: Add sysfs attributes to the Switchtec driver
switchtec: Add IOCTLs to the Switchtec driver
Documentation/ABI/testing/sysfs-class-switchtec | 96 ++ Documentation/ioctl/ioctl-number.txt | 1 + Documentation/switchtec.txt | 80 ++ MAINTAINERS | 11 + drivers/pci/Kconfig | 1 + drivers/pci/Makefile | 1 + drivers/pci/switch/Kconfig | 13 + drivers/pci/switch/Makefile | 1 + drivers/pci/switch/switchtec.c | 1503 +++++++++++++++++++++++ include/uapi/linux/switchtec_ioctl.h | 132 ++ 10 files changed, 1839 insertions(+) create mode 100644 Documentation/ABI/testing/sysfs-class-switchtec create mode 100644 Documentation/switchtec.txt create mode 100644 drivers/pci/switch/Kconfig create mode 100644 drivers/pci/switch/Makefile create mode 100644 drivers/pci/switch/switchtec.c create mode 100644 include/uapi/linux/switchtec_ioctl.h
--
2.1.4
Logan Gunthorpe
maintainerPATCH v4 1/4 MicroSemi Switchtec management interface driver
Microsemi's "Switchtec" line of PCI switch devices is already well supported by the kernel with standard PCI switch drivers. However, the Switchtec device advertises a special management endpoint with a separate PCI function address and class code. This endpoint enables some additional functionality which includes:
* Packet and Byte Counters
* Switch Firmware Upgrades
* Event and Error logs
* Querying port link status
* Custom user firmware commands
This patch introduces the switchtec kernel module which provides PCI driver that exposes a char device. The char device provides userspace access to this interface through read, write and (optionally) poll calls.
A userspace tool and library which utilizes this interface is available at [1]. This tool takes inspiration (and borrows some code) from nvme-cli [2]. The tool is largely complete at this time but additional features may be added in the future.
[1] https://github.com/sbates130272/switchtec-user
[2] https://github.com/linux-nvme/nvme-cli
--- MAINTAINERS | 8 + drivers/pci/Kconfig | 1 + drivers/pci/Makefile | 1 + drivers/pci/switch/Kconfig | 13 + drivers/pci/switch/Makefile | 1 + drivers/pci/switch/switchtec.c | 923 +++++++++++++++++++++++++++++++++++++++++ 6 files changed, 947 insertions(+) create mode 100644 drivers/pci/switch/Kconfig create mode 100644 drivers/pci/switch/Makefile create mode 100644 drivers/pci/switch/switchtec.c
MAINTAINERS +8 -0
@@ -9506,6 +9506,14 @@ S: Maintained F: Documentation/devicetree/bindings/pci/aardvark-pci.txt F: drivers/pci/host/pci-aardvark.c +PCI DRIVER FOR MICROSEMI SWITCHTEC +M: Kurt Schwemmer <kurt.schwemmer at microsemi.com> +M: Stephen Bates <stephen.bates at microsemi.com> +M: Logan Gunthorpe <logang at deltatee.com> +L: linux-pci at vger.kernel.org +S: Maintained +F: drivers/pci/switch/switchtec* + PCI DRIVER FOR NVIDIA TEGRA M: Thierry Reding <thierry.reding at gmail.com> L: linux-tegra at vger.kernel.org
drivers/pci/Kconfig +1 -0
@@ -133,3 +133,4 @@ config PCI_HYPERV source "drivers/pci/hotplug/Kconfig" source "drivers/pci/host/Kconfig" +source "drivers/pci/switch/Kconfig"
drivers/pci/Makefile +1 -0
@@ -68,3 +68,4 @@ ccflags-$(CONFIG_PCI_DEBUG) := -DDEBUG # PCI host controller drivers obj-y += host/ +obj-y += switch/
drivers/pci/switch/Kconfig new file +13 -0
@@ -0,0 +1,13 @@ +menu "PCI switch controller drivers" + depends on PCI + +config PCI_SW_SWITCHTEC + tristate "MicroSemi Switchtec PCIe Switch Management Driver" + help + Enables support for the management interface for the MicroSemi + Switchtec series of PCIe switches. Supports userspace access + to submit MRPC commands to the switch via /dev/switchtecX + devices. See <file:Documentation/switchtec.txt> for more + information. + +endmenu
drivers/pci/switch/Makefile new file +1 -0
@@ -0,0 +1 @@ +obj-$(CONFIG_PCI_SW_SWITCHTEC) += switchtec.o
drivers/pci/switch/switchtec.c new file +923 -0
923 lines not on this page · raw message
Signature · 1 line
2.1.4
Logan Gunthorpe
maintainerPATCH v4 2/4 switchtec: Add user interface documentation
In reply to Logan GunthorpeThis adds standard documentation for the sysfs switchtec attributes and a RST formatted text file which documents the char device interface. Jonathan Corbet has indicated he will move this to a new user-space developer documentation book once it's created.
--- Documentation/switchtec.txt | 53 +++++++++++++++++++++++++++++++++++++++++++++ MAINTAINERS | 1 + 2 files changed, 54 insertions(+) create mode 100644 Documentation/switchtec.txt
Documentation/switchtec.txt new file +53 -0
@@ -0,0 +1,53 @@ +======================== +Linux Switchtec Support +======================== + +Microsemi's "Switchtec" line of PCI switch devices is already +supported by the kernel with standard PCI switch drivers. However, the +Switchtec device advertises a special management endpoint which +enables some additional functionality. This includes: + +* Packet and Byte Counters +* Firmware Upgrades +* Event and Error logs +* Querying port link status +* Custom user firmware commands + +The switchtec kernel module implements this functionality. + + +Interface +========= + +The primary means of communicating with the Switchtec management firmware is +through the Memory-mapped Remote Procedure Call (MRPC) interface. +Commands are submitted to the interface with a 4-byte command +identifier and up to 1KB of command specific data. The firmware will +respond with a 4 bytes return code and up to 1KB of command specific +data. The interface only processes a single command at a time. + + +Userspace Interface +=================== + +The MRPC interface will be exposed to userspace through a simple char +device: /dev/switchtec#, one for each management endpoint in the system. + +The char device has the following semantics: + +* A write must consist of at least 4 bytes and no more than 1028 bytes. + The first four bytes will be interpreted as the command to run and + the remainder will be used as the input data. A write will send the + command to the firmware to begin processing. + +* Each write must be followed by exactly one read. Any double write will + produce an error and any read that doesn't follow a write will + produce an error. + +* A read will block until the firmware completes the command and return + the four bytes of status plus up to 1024 bytes of output data. (The + length will be specified by the size parameter of the read call -- + reading less than 4 bytes will produce an error. + +* The poll call will also be supported for userspace applications that + need to do other things while waiting for the command to complete.
MAINTAINERS +1 -0
@@ -9512,6 +9512,7 @@ M: Stephen Bates <stephen.bates at microsemi.com> M: Logan Gunthorpe <logang at deltatee.com> L: linux-pci at vger.kernel.org S: Maintained +F: Documentation/switchtec.txt F: drivers/pci/switch/switchtec* PCI DRIVER FOR NVIDIA TEGRA
Signature · 1 line
2.1.4
Logan Gunthorpe
maintainerPATCH v4 3/4 switchtec: Add sysfs attributes to the Switchtec driver
In reply to Logan GunthorpeThis patch adds a few read-only sysfs attributes which provide some device information that is exposed from the devices. Primarily component and device names and versions. These are documented in Documentation/ABI/testing/sysfs-class-switchtec.
--- Documentation/ABI/testing/sysfs-class-switchtec | 96 ++++++++++++++++++++ MAINTAINERS | 1 + drivers/pci/switch/switchtec.c | 113 ++++++++++++++++++++++++ 3 files changed, 210 insertions(+) create mode 100644 Documentation/ABI/testing/sysfs-class-switchtec
Documentation/ABI/testing/sysfs-class-switchtec new file +96 -0
@@ -0,0 +1,96 @@ +switchtec - Microsemi Switchtec PCI Switch Management Endpoint + +For details on this subsystem look at Documentation/switchtec.txt. + +What: /sys/class/switchtec +Date: 05-Jan-2017 +KernelVersion: v4.11 +Contact: Logan Gunthorpe <logang at deltatee.com> +Description: The switchtec class subsystem folder. + Each registered switchtec driver is represented by a switchtecX + subfolder (X being an integer >= 0). + + +What: /sys/class/switchtec/switchtec[0-9]+/component_id +Date: 05-Jan-2017 +KernelVersion: v4.11 +Contact: Logan Gunthorpe <logang at deltatee.com> +Description: Component identifier as stored in the hardware (eg. PM8543) + (read only) +Values: arbitrary string. + + +What: /sys/class/switchtec/switchtec[0-9]+/component_revision +Date: 05-Jan-2017 +KernelVersion: v4.11 +Contact: Logan Gunthorpe <logang at deltatee.com> +Description: Component revision stored in the hardware (read only) +Values: integer. + + +What: /sys/class/switchtec/switchtec[0-9]+/component_vendor +Date: 05-Jan-2017 +KernelVersion: v4.11 +Contact: Logan Gunthorpe <logang at deltatee.com> +Description: Component vendor as stored in the hardware (eg. MICROSEM) + (read only) +Values: arbitrary string. + + +What: /sys/class/switchtec/switchtec[0-9]+/device_version +Date: 05-Jan-2017 +KernelVersion: v4.11 +Contact: Logan Gunthorpe <logang at deltatee.com> +Description: Device version as stored in the hardware (read only) +Values: integer. + + +What: /sys/class/switchtec/switchtec[0-9]+/fw_version +Date: 05-Jan-2017 +KernelVersion: v4.11 +Contact: Logan Gunthorpe <logang at deltatee.com> +Description: Currently running firmware version (read only) +Values: integer (in hexadecimal). + + +What: /sys/class/switchtec/switchtec[0-9]+/partition +Date: 05-Jan-2017 +KernelVersion: v4.11 +Contact: Logan Gunthorpe <logang at deltatee.com> +Description: Partition number for this device in the switch (read only) +Values: integer. + + +What: /sys/class/switchtec/switchtec[0-9]+/partition_count +Date: 05-Jan-2017 +KernelVersion: v4.11 +Contact: Logan Gunthorpe <logang at deltatee.com> +Description: Total number of partitions in the switch (read only) +Values: integer. + + +What: /sys/class/switchtec/switchtec[0-9]+/product_id +Date: 05-Jan-2017 +KernelVersion: v4.11 +Contact: Logan Gunthorpe <logang at deltatee.com> +Description: Product identifier as stored in the hardware (eg. PSX 48XG3) + (read only) +Values: arbitrary string. + + +What: /sys/class/switchtec/switchtec[0-9]+/product_revision +Date: 05-Jan-2017 +KernelVersion: v4.11 +Contact: Logan Gunthorpe <logang at deltatee.com> +Description: Product revision stored in the hardware (eg. RevB) + (read only) +Values: arbitrary string. + + +What: /sys/class/switchtec/switchtec[0-9]+/product_vendor +Date: 05-Jan-2017 +KernelVersion: v4.11 +Contact: Logan Gunthorpe <logang at deltatee.com> +Description: Product vendor as stored in the hardware (eg. MICROSEM) + (read only) +Values: arbitrary string.
MAINTAINERS +1 -0
@@ -9513,6 +9513,7 @@ M: Logan Gunthorpe <logang at deltatee.com> L: linux-pci at vger.kernel.org S: Maintained F: Documentation/switchtec.txt +F: Documentation/ABI/testing/sysfs-class-switchtec F: drivers/pci/switch/switchtec* PCI DRIVER FOR NVIDIA TEGRA
drivers/pci/switch/switchtec.c +113 -0
125 lines not on this page · raw message
Signature · 1 line
2.1.4
Logan Gunthorpe
maintainerPATCH v4 4/4 switchtec: Add IOCTLs to the Switchtec driver
In reply to Logan GunthorpeThis patch introduces a couple of special IOCTLs which are provided to:
* Inform userspace of firmware partition locations
* Pass event counts and allow userspace to wait on events
* Translate between PFF numbers used by the switch to port numbers.
--- Documentation/ioctl/ioctl-number.txt | 1 + Documentation/switchtec.txt | 27 ++ MAINTAINERS | 1 + drivers/pci/switch/switchtec.c | 467 +++++++++++++++++++++++++++++++++++ include/uapi/linux/switchtec_ioctl.h | 132 ++++++++++ 5 files changed, 628 insertions(+) create mode 100644 include/uapi/linux/switchtec_ioctl.h
Documentation/ioctl/ioctl-number.txt +1 -0
@@ -191,6 +191,7 @@ Code Seq#(hex) Include File Comments 'W' 00-1F linux/watchdog.h conflict! 'W' 00-1F linux/wanrouter.h conflict! (pre 3.9) 'W' 00-3F sound/asound.h conflict! +'W' 40-5F drivers/pci/switch/switchtec.c 'X' all fs/xfs/xfs_fs.h conflict! and fs/xfs/linux-2.6/xfs_ioctl32.h and include/linux/falloc.h
Documentation/switchtec.txt +27 -0
@@ -51,3 +51,30 @@ The char device has the following semantics: * The poll call will also be supported for userspace applications that need to do other things while waiting for the command to complete. + +The following IOCTLs are also supported by the device: + +* SWITCHTEC_IOCTL_FLASH_INFO - Retrieve firmware length and number + of partitions in the device. + +* SWITCHTEC_IOCTL_FLASH_PART_INFO - Retrieve address and lengeth for + any specified partition in flash. + +* SWITCHTEC_IOCTL_EVENT_SUMMARY - Read a structure of bitmaps + indicating all uncleared events. + +* SWITCHTEC_IOCTL_EVENT_CTL - Get the current count, clear and set flags + for any event. This ioctl takes in a switchtec_ioctl_event_ctl struct + with the event_id, index and flags set (index being the partition or PFF + number for non-global events). It returns whether the event has + occurred, the number of times and any event specific data. The flags + can be used to clear the count or enable and disable actions to + happen when the event occurs. + By using the SWITCHTEC_IOCTL_EVENT_FLAG_EN_POLL flag, + you can set an event to trigger a poll command to return with + POLLPRI. In this way, userspace can wait for events to occur. + +* SWITCHTEC_IOCTL_PFF_TO_PORT and SWITCHTEC_IOCTL_PORT_TO_PFF convert + between PCI Function Framework number (used by the event system) + and Switchtec Logic Port ID and Partition number (which is more + user friendly).
MAINTAINERS +1 -0
@@ -9515,6 +9515,7 @@ S: Maintained F: Documentation/switchtec.txt F: Documentation/ABI/testing/sysfs-class-switchtec F: drivers/pci/switch/switchtec* +F: include/uapi/linux/switchtec_ioctl.h PCI DRIVER FOR NVIDIA TEGRA M: Thierry Reding <thierry.reding at gmail.com>
drivers/pci/switch/switchtec.c +467 -0
502 lines not on this page · raw message
include/uapi/linux/switchtec_ioctl.h new file +132 -0
132 lines not on this page · raw message
Signature · 1 line
2.1.4