Skip to contentSkip to conversation

PCIe Enclosure LED Management

PATCH In review Mariusz Tkaczyk 6 participants 10 messages Sep 4, 2024 – Sep 9, 2024

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v7 cover PCIe Enclosure LED Management

Patchset is named as PCIe Enclosure LED Management because it adds two features:
- Native PCIe Enclosure Management (NPEM)
- PCIe SSD Status LED Management (DSM)

Both are pattern oriented standards, they tell which "indication" should blink. It doesn't control physical LED or pattern visualization.

Overall, driver is simple but it was not simple to fit it into interfaces we have in kernel (We considered leds and enclosure interfaces). It reuses leds interface, this approach seems to be the best because:
- leds are actively maintained, no new interface added.
- leds do not require any extensions, enclosure needs to be adjusted first.

There are trade-offs:
- "brightness" is the name of sysfs file to control led. It is not natural to use brightness to set patterns, that is why multiple led devices are created (one per indication);
- Update of one led may affect other leds, led triggers may not work as expected.

Changes from v1:
- Renamed "pattern" to indication.
- DSM support added.
- Fixed nits reported by Bjorn.

Changes from v2:
- Introduce lazy loading to allow DELL _DSM quirks to work, reported by Stuart.
- leds class initcall moved up in Makefile, proposed by Dan.
- fix other nits reported by Dan and Iipo.

Changes from v3:
- Remove unnecessary packed attr.
- Fix doc issue reported by lkp.
- Fix read_poll_timeout() error handling reported by Iipo.
- Minor fixes reported by Christoph.

Changes from v4:
- Use 0 / 1 instead of LED_OFF/LED_ON, suggested by Marek.
- Documentation added, suggested by Bjorn.

Change from v5:
- Remove unnecessary _packed, reported by Christoph.
- Changed "led" to "LED" and other typos suggested by Randy.

Change from v6:
- Removed links, suggested by Bjorn.
- npem->active_inds_initialized:1 moved to DSM commit, suggested by Bjorn.
- Improve justification for active_inds_initialized, suggested by Bjorn.
- Chosen backed logging added, suggested by Bjorn.

Cc Arnd Bergmann <[email protected]>
Cc Bjorn Helgaas <[email protected]>
Cc Dan Williams <[email protected]>
Cc Greg Kroah-Hartman <[email protected]>
Cc Ilpo Jarvinen <[email protected]>
Cc Lukas Wunner <[email protected]>
Cc Lee Jones <[email protected]>
Cc Keith Busch <[email protected]>
Cc Marek Behun <[email protected]>
Cc Pavel Machek <[email protected]>
Cc Randy Dunlap <[email protected]>
Cc Andy Shevchenko <[email protected]>
Cc Stuart Hayes <[email protected]>

Mariusz Tkaczyk (3):
leds: Init leds class earlier
PCI/NPEM: Add Native PCIe Enclosure Management support PCI/NPEM: Add _DSM PCIe SSD status LED management

 Documentation/ABI/testing/sysfs-bus-pci |  72 +++
 drivers/Makefile                        |   4 +-
 drivers/pci/Kconfig                     |   9 +
 drivers/pci/Makefile                    |   1 +
 drivers/pci/npem.c                      | 597 ++++++++++++++++++++++++
 drivers/pci/pci.h                       |   8 +
 drivers/pci/probe.c                     |   2 +
 drivers/pci/remove.c                    |   2 +
 include/linux/pci.h                     |   3 +
 include/uapi/linux/pci_regs.h           |  35 ++
 10 files changed, 732 insertions(+), 1 deletion(-)
 create mode 100644 drivers/pci/npem.c
Signature · 1 line
2.35.3
#1 lore raw

PATCH v7 1/3 leds: Init leds class earlier

NPEM driver will require leds class, there is an init-order conflict. Make sure that LEDs initialization happens first and add comment.

---
 drivers/Makefile | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/Makefile +3 -1
index fe9ceb0d2288..45d1c3e630f7 100644
@@ -17,6 +17,9 @@ obj-$(CONFIG_PINCTRL)		+= pinctrl/
 obj-$(CONFIG_GPIOLIB)		+= gpio/
 obj-y				+= pwm/
 
+# LEDs must come before PCI, it is needed by NPEM driver
+obj-y				+= leds/
+
 obj-y				+= pci/
 
 obj-$(CONFIG_PARISC)		+= parisc/
@@ -130,7 +133,6 @@ obj-$(CONFIG_CPU_IDLE)		+= cpuidle/
 obj-y				+= mmc/
 obj-y				+= ufs/
 obj-$(CONFIG_MEMSTICK)		+= memstick/
-obj-y				+= leds/
 obj-$(CONFIG_INFINIBAND)	+= infiniband/
 obj-y				+= firmware/
 obj-$(CONFIG_CRYPTO)		+= crypto/
Signature · 1 line
2.35.3
maintainer
#2 lore raw

Re: [PATCH v7 1/3] leds: Init leds class earlier

Quoted 4 lines · Mariusz Tkaczyk

NPEM driver will require leds class, there is an init-order conflict. Make sure that LEDs initialization happens first and add comment.

Do you have a link to this discussion?

Quoted 12 lines
---
 drivers/Makefile | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/Makefile +0 -0
index fe9ceb0d2288..45d1c3e630f7 100644

drivers/Makefile obj-$(CONFIG_PINCTRL) += pinctrl/ · line 20

Quoted 4 lines
drivers/Makefile +1 -0
@@ -17,6 +17,9 @@ obj-$(CONFIG_PINCTRL)		+= pinctrl/
 obj-$(CONFIG_GPIOLIB)		+= gpio/
 obj-y				+= pwm/
 
+# LEDs must come before PCI, it is needed by NPEM driver

This seems very fragile.

Isn't there a better way to describe the dependency in Kconfig etc?

Quoted 16 lines
drivers/Makefile +2 -1
+obj-y				+= leds/
+
 obj-y				+= pci/
 
 obj-$(CONFIG_PARISC)		+= parisc/
@@ -130,7 +133,6 @@ obj-$(CONFIG_CPU_IDLE)		+= cpuidle/
 obj-y				+= mmc/
 obj-y				+= ufs/
 obj-$(CONFIG_MEMSTICK)		+= memstick/
-obj-y				+= leds/
 obj-$(CONFIG_INFINIBAND)	+= infiniband/
 obj-y				+= firmware/
 obj-$(CONFIG_CRYPTO)		+= crypto/
Signature · 1 line
2.35.3
Signature · 1 line
Lee Jones [李琼斯]
#3 lore raw

Re: [PATCH v7 1/3] leds: Init leds class earlier

Quoted 8 lines · On Mon, 9 Sep 2024 10:03:40 +0100 Lee Jones <[email protected]> wrote:
Quoted 4 lines · Mariusz Tkaczyk

NPEM driver will require leds class, there is an init-order conflict. Make sure that LEDs initialization happens first and add comment.

Do you have a link to this discussion?

https://lore.kernel.org/linux-pci/2024061429-handiness-waggle-d87a@gregkh/

I discussed this with Lukas and Dan offline and as a result I updated Makefile instead initcall change.

Quoted 16 lines
Quoted 16 lines
---
 drivers/Makefile | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/Makefile +0 -0
index fe9ceb0d2288..45d1c3e630f7 100644
@@ -17,6 +17,9 @@ obj-$(CONFIG_PINCTRL)		+= pinctrl/
 obj-$(CONFIG_GPIOLIB)		+= gpio/
 obj-y				+= pwm/
 
Quoted 5 lines

drivers/Makefile obj-$(CONFIG_PINCTRL) += pinctrl/ · line 20

Quoted 1 line
drivers/Makefile +1 -0
+# LEDs must come before PCI, it is needed by NPEM driver  

This seems very fragile.

Isn't there a better way to describe the dependency in Kconfig etc?

I don't know if it is better. What I know is (according to Greg) that maintainers are aware of possible ordering issues and not described dependencies so they are not likely to allow changing it.

Mariusz

#4 lore raw

Re: [PATCH v7 1/3] leds: Init leds class earlier

In reply to Lee Jones
Quoted 3 lines · Lee Jones
Quoted 2 lines · Mariusz Tkaczyk

NPEM driver will require leds class, there is an init-order conflict. Make sure that LEDs initialization happens first and add comment.

[...]

Quoted 23 lines
Quoted 13 lines
drivers/Makefile +3 -0
@@ -17,6 +17,9 @@ obj-$(CONFIG_PINCTRL)		+= pinctrl/
 obj-$(CONFIG_GPIOLIB)		+= gpio/
 obj-y				+= pwm/
 
+# LEDs must come before PCI, it is needed by NPEM driver
+obj-y				+= leds/
+
 obj-y				+= pci/
 
 obj-$(CONFIG_PARISC)		+= parisc/

drivers/Makefile obj-$(CONFIG_CPU_IDLE) += cpuidle/ · line 138

Quoted 7 lines
drivers/Makefile +0 -1
@@ -130,7 +133,6 @@ obj-$(CONFIG_CPU_IDLE)		+= cpuidle/
 obj-y				+= mmc/
 obj-y				+= ufs/
 obj-$(CONFIG_MEMSTICK)		+= memstick/
-obj-y				+= leds/
 obj-$(CONFIG_INFINIBAND)	+= infiniband/
 obj-y				+= firmware/
 obj-$(CONFIG_CRYPTO)		+= crypto/

This seems very fragile.

Quoted 1 line

Isn't there a better way to describe the dependency in Kconfig etc?

In v2 of this series, the correct init order was ensured by running leds_init() at "arch" initcall level, instead of "subsys":

https://lore.kernel.org/linux-pci/[email protected]/

If you prefer this alternative approach, then Bjorn or Krzysztof can probably replace the commit on the pci.git/npem branch.

AFAIK, all topic branches in pci.git are to be considered drafts that can be amended until the pull request is sent.

Thanks,

Lukas

#5 lore raw

PATCH v7 2/3 PCI/NPEM: Add Native PCIe Enclosure Management support

In reply to Mariusz Tkaczyk

Native PCIe Enclosure Management (NPEM, PCIe r6.1 sec 6.28) allows managing LED in storage enclosures. NPEM is indication oriented and it does not give direct access to LED. Although each of the indications *could* represent an individual LED, multiple indications could also be represented as a single, multi-color LED or a single LED blinking in a specific interval. The specification leaves that open.

Each enabled indication (capability register bit on) is represented as a ledclass_dev which can be controlled through sysfs. For every ledclass device only 2 brightness states are allowed: LED_ON (1) or LED_OFF (0). It is corresponding to NPEM control register (Indication bit on/off).

Ledclass devices appear in sysfs as child devices (subdirectory) of PCI device which has an NPEM Extended Capability and indication is enabled in NPEM capability register. For example, these are leds created for pcieport "10000:02:05.0" on my setup:

leds/

├── 10000:02:05.0:enclosure:fail
├── 10000:02:05.0:enclosure:locate
├── 10000:02:05.0:enclosure:ok
└── 10000:02:05.0:enclosure:rebuild

They can be also found in "/sys/class/leds" directory. Parent PCIe device bdf is used to guarantee uniqueness across leds subsystem.

To enable/disable fail indication "brightness" file can be edited:
echo 1 > ./leds/10000:02:05.0:enclosure:fail/brightness echo 0 > ./leds/10000:02:05.0:enclosure:fail/brightness

PCIe r6.1, sec 7.9.19.2 defines the possible indications.

Multiple indications for same parent PCIe device can conflict and hardware may update them when processing new request. To avoid issues, driver refresh all indications by reading back control register.

Driver is projected to be exclusive NPEM extended capability manager. It waits up to 1 second after imposing new request, it doesn't verify if controller is busy before write, assuming that mutex lock gives protection from concurrent updates.

Driver is not registered if _DSM LED management is available respectfully to PCI Firmware Spec r3.3 sec 4.7. In next patch DSM support is added, therefore info message is added to log backend chosen.

NPEM is a PCIe extended capability so it should be registered in pcie_init_capabilities() but it is not possible due to LED dependency. Parent pci_device must be added earlier for led_classdev_register() to be successful. NPEM does not require configuration on kernel side, it is safe to register LED devices later.

---
 Documentation/ABI/testing/sysfs-bus-pci |  63 ++++
 drivers/pci/Kconfig                     |   9 +
 drivers/pci/Makefile                    |   1 +
 drivers/pci/npem.c                      | 425 ++++++++++++++++++++++++
 drivers/pci/pci.h                       |   8 +
 drivers/pci/probe.c                     |   2 +
 drivers/pci/remove.c                    |   2 +
 include/linux/pci.h                     |   3 +
 include/uapi/linux/pci_regs.h           |  35 ++
 9 files changed, 548 insertions(+)
 create mode 100644 drivers/pci/npem.c
Documentation/ABI/testing/sysfs-bus-pci +63 -0
index ecf47559f495..9ddbb1364698 100644
@@ -500,3 +500,66 @@ Description:
 		console drivers from the device.  Raw users of pci-sysfs
 		resourceN attributes must be terminated prior to resizing.
 		Success of the resizing operation is not guaranteed.
+
+What:		/sys/bus/pci/devices/.../leds/*:enclosure:*/brightness
+What:		/sys/class/leds/*:enclosure:*/brightness
+Date:		August 2024
+KernelVersion:	6.12
+Description:
+		LED indications on PCIe storage enclosures which are controlled
+		through the NPEM interface (Native PCIe Enclosure Management,
+		PCIe r6.1 sec 6.28) are accessible as led class devices, both
+		below /sys/class/leds and below NPEM-capable PCI devices.
+
+		Although these led class devices could be manipulated manually,
+		in practice they are typically manipulated automatically by an
+		application such as ledmon(8).
+
+		The name of a led class device is as follows:
+		<bdf>:enclosure:<indication>
+		where:
+
+		- <bdf> is the domain, bus, device and function number
+		  (e.g. 10000:02:05.0)
+		- <indication> is a short description of the LED indication
+
+		Valid indications per PCIe r6.1 table 6-27 are:
+
+		- ok (drive is functioning normally)
+		- locate (drive is being identified by an admin)
+		- fail (drive is not functioning properly)
+		- rebuild (drive is part of an array that is rebuilding)
+		- pfa (drive is predicted to fail soon)
+		- hotspare (drive is marked to be used as a replacement)
+		- ica (drive is part of an array that is degraded)
+		- ifa (drive is part of an array that is failed)
+		- idt (drive is not the right type for the connector)
+		- disabled (drive is disabled, removal is safe)
+		- specific0 to specific7 (enclosure-specific indications)
+
+		Broadly, the indications fall into one of these categories:
+
+		- to signify drive state (ok, locate, fail, idt, disabled)
+		- to signify drive role or state in a software RAID array
+		  (rebuild, pfa, hotspare, ica, ifa)
+		- to signify any other role or state (specific0 to specific7)
+
+		Mandatory indications per PCIe r6.1 sec 7.9.19.2 comprise:
+		ok, locate, fail, rebuild. All others are optional.
+		A led class device is only visible if the corresponding
+		indication is supported by the device.
+
+		To manipulate the indications, write 0 (LED_OFF) or 1 (LED_ON)
+		to the "brightness" file. Note that manipulating an indication
+		may implicitly manipulate other indications at the vendor's
+		discretion. E.g. when the user lights up the "ok" indication,
+		the vendor may choose to automatically turn off the "fail"
+		indication. The current state of an indication can be
+		retrieved by reading its "brightness" file.
+
+		The PCIe Base Specification allows vendors leeway to choose
+		different colors or blinking patterns for the indications,
+		but they typically follow the IBPI standard. E.g. the "locate"
+		indication is usually presented as one or two LEDs blinking at
+		4 Hz frequency:
+		https://en.wikipedia.org/wiki/International_Blinking_Pattern_Interpretation
drivers/pci/Kconfig +9 -0
index aa4d1833f442..0d94e4a967d8 100644
@@ -143,6 +143,15 @@ config PCI_IOV
 
 	  If unsure, say N.
 
+config PCI_NPEM
+	bool "Native PCIe Enclosure Management"
+	depends on LEDS_CLASS=y
+	help
+	  Support for Native PCIe Enclosure Management. It allows managing LED
+	  indications in storage enclosures. Enclosure must support following
+	  indications: OK, Locate, Fail, Rebuild, other indications are
+	  optional.
+
 config PCI_PRI
 	bool "PCI PRI support"
 	select PCI_ATS
drivers/pci/Makefile +1 -0
index 8ddad57934a6..374c5c06d92f 100644
@@ -35,6 +35,7 @@ obj-$(CONFIG_XEN_PCIDEV_FRONTEND) += xen-pcifront.o
 obj-$(CONFIG_VGA_ARB)		+= vgaarb.o
 obj-$(CONFIG_PCI_DOE)		+= doe.o
 obj-$(CONFIG_PCI_DYNAMIC_OF_NODES) += of_property.o
+obj-$(CONFIG_PCI_NPEM)		+= npem.o
 
 # Endpoint library must be initialized before its users
 obj-$(CONFIG_PCI_ENDPOINT)	+= endpoint/
drivers/pci/npem.c new file +425 -0

425 lines not on this page · raw message

drivers/pci/pci.h +8 -0
index 79c8398f3938..554fd9dfe25c 100644
@@ -398,6 +398,14 @@ static inline void pci_doe_destroy(struct pci_dev *pdev) { }
 static inline void pci_doe_disconnected(struct pci_dev *pdev) { }
 #endif
 
+#ifdef CONFIG_PCI_NPEM
+void pci_npem_create(struct pci_dev *dev);
+void pci_npem_remove(struct pci_dev *dev);
+#else
+static inline void pci_npem_create(struct pci_dev *dev) { }
+static inline void pci_npem_remove(struct pci_dev *dev) { }
+#endif
+
 /**
  * pci_dev_set_io_state - Set the new error state if possible.
  *
drivers/pci/probe.c +2 -0
index b14b9876c030..17ee559c31a4 100644
@@ -2593,6 +2593,8 @@ void pci_device_add(struct pci_dev *dev, struct pci_bus *bus)
 	dev->match_driver = false;
 	ret = device_add(&dev->dev);
 	WARN_ON(ret < 0);
+
+	pci_npem_create(dev);
 }
 
 struct pci_dev *pci_scan_single_device(struct pci_bus *bus, int devfn)
drivers/pci/remove.c +2 -0
index 910387e5bdbf..da9629c3a688 100644
@@ -34,6 +34,8 @@ static void pci_destroy_dev(struct pci_dev *dev)
 	if (!dev->dev.kobj.parent)
 		return;
 
+	pci_npem_remove(dev);
+
 	device_del(&dev->dev);
 
 	down_write(&pci_bus_sem);
include/linux/pci.h +3 -0
index 4cf89a4b4cbc..c9db853e269f 100644
@@ -516,6 +516,9 @@ struct pci_dev {
 #endif
 #ifdef CONFIG_PCI_DOE
 	struct xarray	doe_mbs;	/* Data Object Exchange mailboxes */
+#endif
+#ifdef CONFIG_PCI_NPEM
+	struct npem	*npem;		/* Native PCIe Enclosure Management */
 #endif
 	u16		acs_cap;	/* ACS Capability offset */
 	phys_addr_t	rom;		/* Physical address if not from BAR */
include/uapi/linux/pci_regs.h +35 -0
index 94c00996e633..42f922c67139 100644
@@ -740,6 +740,7 @@
 #define PCI_EXT_CAP_ID_DVSEC	0x23	/* Designated Vendor-Specific */
 #define PCI_EXT_CAP_ID_DLF	0x25	/* Data Link Feature */
 #define PCI_EXT_CAP_ID_PL_16GT	0x26	/* Physical Layer 16.0 GT/s */
+#define PCI_EXT_CAP_ID_NPEM	0x29	/* Native PCIe Enclosure Management */
 #define PCI_EXT_CAP_ID_PL_32GT  0x2A    /* Physical Layer 32.0 GT/s */
 #define PCI_EXT_CAP_ID_DOE	0x2E	/* Data Object Exchange */
 #define PCI_EXT_CAP_ID_MAX	PCI_EXT_CAP_ID_DOE
@@ -1121,6 +1122,40 @@
 #define  PCI_PL_16GT_LE_CTRL_USP_TX_PRESET_MASK		0x000000F0
 #define  PCI_PL_16GT_LE_CTRL_USP_TX_PRESET_SHIFT	4
 
+/* Native PCIe Enclosure Management */
+#define PCI_NPEM_CAP     0x04 /* NPEM capability register */
+#define  PCI_NPEM_CAP_CAPABLE     0x00000001 /* NPEM Capable */
+
+#define PCI_NPEM_CTRL    0x08 /* NPEM control register */
+#define	 PCI_NPEM_CTRL_ENABLE     0x00000001 /* NPEM Enable */
+
+/*
+ * Native PCIe Enclosure Management indication bits and Reset command bit
+ * are corresponding for capability and control registers.
+ */
+#define  PCI_NPEM_CMD_RESET       0x00000002 /* NPEM Reset Command */
+#define  PCI_NPEM_IND_OK          0x00000004 /* NPEM indication OK */
+#define  PCI_NPEM_IND_LOCATE      0x00000008 /* NPEM indication Locate */
+#define  PCI_NPEM_IND_FAIL        0x00000010 /* NPEM indication Fail */
+#define  PCI_NPEM_IND_REBUILD     0x00000020 /* NPEM indication Rebuild */
+#define  PCI_NPEM_IND_PFA         0x00000040 /* NPEM indication Predicted Failure Analysis */
+#define  PCI_NPEM_IND_HOTSPARE    0x00000080 /* NPEM indication Hot Spare */
+#define  PCI_NPEM_IND_ICA         0x00000100 /* NPEM indication In Critical Array */
+#define  PCI_NPEM_IND_IFA         0x00000200 /* NPEM indication In Failed Array */
+#define  PCI_NPEM_IND_IDT         0x00000400 /* NPEM indication Invalid Device Type */
+#define  PCI_NPEM_IND_DISABLED    0x00000800 /* NPEM indication Disabled */
+#define  PCI_NPEM_IND_SPEC_0      0x01000000
+#define  PCI_NPEM_IND_SPEC_1      0x02000000
+#define  PCI_NPEM_IND_SPEC_2      0x04000000
+#define  PCI_NPEM_IND_SPEC_3      0x08000000
+#define  PCI_NPEM_IND_SPEC_4      0x10000000
+#define  PCI_NPEM_IND_SPEC_5      0x20000000
+#define  PCI_NPEM_IND_SPEC_6      0x40000000
+#define  PCI_NPEM_IND_SPEC_7      0x80000000
+
+#define PCI_NPEM_STATUS  0x0c /* NPEM status register */
+#define	 PCI_NPEM_STATUS_CC       0x00000001 /* NPEM Command completed */
+
 /* Data Object Exchange */
 #define PCI_DOE_CAP		0x04    /* DOE Capabilities Register */
 #define  PCI_DOE_CAP_INT_SUP			0x00000001  /* Interrupt Support */
Signature · 1 line
2.35.3
#6 lore raw

PATCH v7 3/3 PCI/NPEM: Add _DSM PCIe SSD status LED management

In reply to Mariusz Tkaczyk

Device Specific Method PCIe SSD Status LED Management (_DSM) defined in PCI Firmware Spec r3.3 sec 4.7 provides a way to manage LEDs via ACPI.

The design is similar to NPEM defined in PCIe Base Specification r6.1 sec 6.28:
- both standards are indication oriented,
- _DSM supported bits are corresponding to NPEM capability register bits
- _DSM control bits are corresponding to NPEM control register bits.

_DSM does not support enclosure specific indications and special NPEM commands NPEM_ENABLE and NPEM_RESET.

_DSM is implemented as a second op in NPEM driver. The standard used in background is logged with info priority. The interface is accessed same as NPEM.

According to spec, _DSM has higher priority and availability of _DSM in not limited to devices with NPEM support. It is followed in implementation.

DELL implementation of DSM is using acpi ipmi, which is not available imediatelly (in fact it tooks up to 10s for this interface
to be avialable). It can determine if DSM is supported (GET_SUPPORTED_STATES_DSM is working) but it cannot serve GET_STATE_DSM or SET_STATE_DSM commands in this time.
From userspace application perspective (primarly configured by systemd service) it is better to have not working but configured interface rather than have it available after few seconds.
For that reason, npem->active_indications cache is now loaded lazily i.e. any GET or SET request want cache to be updated if it is not done yet.

---
 Documentation/ABI/testing/sysfs-bus-pci |   9 ++
 drivers/pci/npem.c                      | 188 +++++++++++++++++++++++-
 2 files changed, 189 insertions(+), 8 deletions(-)
Documentation/ABI/testing/sysfs-bus-pci +9 -0
index 9ddbb1364698..7f63c7e97773 100644
@@ -563,3 +563,12 @@ Description:
 		indication is usually presented as one or two LEDs blinking at
 		4 Hz frequency:
 		https://en.wikipedia.org/wiki/International_Blinking_Pattern_Interpretation
+
+		PCI Firmware Specification r3.3 sec 4.7 defines a DSM interface
+		to facilitate shared access by operating system and platform
+		firmware to a device's NPEM registers. The kernel will use
+		this DSM interface where available, instead of accessing NPEM
+		registers directly. The DSM interface does not support the
+		enclosure-specific indications "specific0" to "specific7",
+		hence the corresponding led class devices are unavailable if
+		the DSM interface is used.
drivers/pci/npem.c +180 -8

251 lines not on this page · raw message

Signature · 1 line
2.35.3
#7 lore raw

Re: [PATCH v7 3/3] PCI/NPEM: Add _DSM PCIe SSD status LED management

Hello,

Very nice series!

[...]

drivers/pci/npem.c static bool npem_has_dsm(struct pci_dev *pdev) · line 393

Quoted 4 lines
drivers/pci/npem.c +4 -0
+static const struct npem_ops dsm_ops = {
+	.get_active_indications = dsm_get_active_indications,
+	.set_active_indications = dsm_set_active_indications,
+	.name =  "_DSM PCIe SSD Status LED Management",

A small nit: extra space.

Bjorn, we can mend this on the branch, if are OK with that.

Krzysztof

#8 lore raw
In reply to Mariusz Tkaczyk
Quoted 52 lines · Mariusz Tkaczyk

Patchset is named as PCIe Enclosure LED Management because it adds two features:
- Native PCIe Enclosure Management (NPEM)
- PCIe SSD Status LED Management (DSM)

Both are pattern oriented standards, they tell which "indication" should blink. It doesn't control physical LED or pattern visualization.

Overall, driver is simple but it was not simple to fit it into interfaces we have in kernel (We considered leds and enclosure interfaces). It reuses leds interface, this approach seems to be the best because:
- leds are actively maintained, no new interface added.
- leds do not require any extensions, enclosure needs to be adjusted first.

There are trade-offs:
- "brightness" is the name of sysfs file to control led. It is not natural to use brightness to set patterns, that is why multiple led devices are created (one per indication);
- Update of one led may affect other leds, led triggers may not work as expected.

Changes from v1:
- Renamed "pattern" to indication.
- DSM support added.
- Fixed nits reported by Bjorn.

Changes from v2:
- Introduce lazy loading to allow DELL _DSM quirks to work, reported by Stuart.
- leds class initcall moved up in Makefile, proposed by Dan.
- fix other nits reported by Dan and Iipo.

Changes from v3:
- Remove unnecessary packed attr.
- Fix doc issue reported by lkp.
- Fix read_poll_timeout() error handling reported by Iipo.
- Minor fixes reported by Christoph.

Changes from v4:
- Use 0 / 1 instead of LED_OFF/LED_ON, suggested by Marek.
- Documentation added, suggested by Bjorn.

Change from v5:
- Remove unnecessary _packed, reported by Christoph.
- Changed "led" to "LED" and other typos suggested by Randy.

Change from v6:
- Removed links, suggested by Bjorn.
- npem->active_inds_initialized:1 moved to DSM commit, suggested by Bjorn.
- Improve justification for active_inds_initialized, suggested by Bjorn.
- Chosen backed logging added, suggested by Bjorn.

Quoted 4 lines

Very nice. Applied to pci/npem for v6.12, thank you!

I noticed that b4 didn't pick up Stuart's Tested-by from the cover letter. I assume it covers the whole series, so I added it to each patch. Let me know if that's not what you intended.

Quoted 36 lines
Cc Arnd Bergmann <[email protected]>
Cc Bjorn Helgaas <[email protected]>
Cc Dan Williams <[email protected]>
Cc Greg Kroah-Hartman <[email protected]>
Cc Ilpo Jarvinen <[email protected]>
Cc Lukas Wunner <[email protected]>
Cc Lee Jones <[email protected]>
Cc Keith Busch <[email protected]>
Cc Marek Behun <[email protected]>
Cc Pavel Machek <[email protected]>
Cc Randy Dunlap <[email protected]>
Cc Andy Shevchenko <[email protected]>
Cc Stuart Hayes <[email protected]>

Mariusz Tkaczyk (3):
leds: Init leds class earlier
PCI/NPEM: Add Native PCIe Enclosure Management support PCI/NPEM: Add _DSM PCIe SSD status LED management

 Documentation/ABI/testing/sysfs-bus-pci |  72 +++
 drivers/Makefile                        |   4 +-
 drivers/pci/Kconfig                     |   9 +
 drivers/pci/Makefile                    |   1 +
 drivers/pci/npem.c                      | 597 ++++++++++++++++++++++++
 drivers/pci/pci.h                       |   8 +
 drivers/pci/probe.c                     |   2 +
 drivers/pci/remove.c                    |   2 +
 include/linux/pci.h                     |   3 +
 include/uapi/linux/pci_regs.h           |  35 ++
 10 files changed, 732 insertions(+), 1 deletion(-)
 create mode 100644 drivers/pci/npem.c
Signature · 1 line
2.35.3
#9 lore raw
Quoted 5 lines · Bjorn Helgaas

On Wed, Sep 04, 2024 at 12:48:45PM +0200, Mariusz Tkaczyk wrote:

I noticed that b4 didn't pick up Stuart's Tested-by from the cover letter. I assume it covers the whole series, so I added it to each patch. Let me know if that's not what you intended.

You forgot to add -t IIRC which does spread the tags against cover letter to all of the patches.

Signature · 2 lines
With Best Regards,
Andy Shevchenko