Skip to contentSkip to conversation

spi: Implement a atomic transfer as a last resort

PATCH New Heiko Stuebner 1 participant 7 messages Nov 19

Subsystems (2)
lore
Share & tools
lore

Heiko Stuebner

author maintainer
#0 lore raw

v4 cover spi: Implement a atomic transfer as a last resort

SPI devices at times are used as PMICs including a power-controller functionality that should turn off a system.

This happens in an atomic context very late during the shutdown, where no interrupts are available anymore and conflicts with the asynchronous nature of the SPI subsystem.

====== History ======
For the RK806 spi-connected PMIC this has already caused interesting artifacts.

A first change, tried to "fix" the issue by using the SYS_OFF_MODE_POWER_OFF_PREPARE step of poweroff. While this hides the issue, on some devices this now turns the pmic off too early before other important steps are done.

To mitigate that issue for "some" devices, a change [0] tried to use different poweroff phases for i2c and spi devices, where Lee rightfully said, that with this no proper solution would be done - and Lee was right, as that issue now is open since 2024.

The newest change is a simple revert back the old way [1] because of those issues caused on other devices, but that now breaks the SPI side again and also did not take into account that long history.

====== Now =======
The I2C framework has an atomic transfer callback for exactly this late stage transfers since
commit 63b96983a5dd ("i2c: core: introduce callbacks for atomic transfers") and during the discussion it was suggested implementing something like this for SPI too.

As this did not seem to have happened yet, I thought I'd try my luck.

Similar to I2C this is meant only for that latest stage so I stole the gating from there too.

I assume I did not take some SPI specifics into account, but I guess having some code to discuss is a starting point at least :-)

===== Testing =====
Adding the below diff allows the atomic transfer to also work as main transfer function for testing purposes. As SPI's ->transfer() call already lives inside a spinlock_irqsafe context, this allows to find any sleeping elements easily.

@@ -953,6 +953,9 @@

        /* Only register atomic transfer in host-mode */
        if (!target_mode) {
+               pm_runtime_resume_and_get(rs->dev);
+               ctlr->transfer = rockchip_spi_transfer_atomic;
+
                ctlr->transfer_atomic = rockchip_spi_transfer_atomic;
 
                ret = devm_register_sys_off_handler(&pdev->dev,

I've also changed the chip-select of my board to use the gpio path, and no sleeping parts were reported anymore after allowing both the spi_exec_delay and gpio chipselect to handle both variants.

===== New =====
changes in v4:
- drop the pm-runtime-get patch, before I dig myself deeper into an unrelated thing :-)

changes in v3:
- core: allow gpio chipselects to be used in atomic contexts
- core: split in-atomic check, a generic one for gpio-cs and delay handling, and specific one for limiting sync calls to power-off events
- rockchip: fail gracefully when not runtime resume (Sebastian Reichel)

changes in v2:
address review comments from Mark Brown
- core: preserve old behaviour for drivers not declaring an atomic transfer function
- core: don't use message optimization - safer, but still validate the message to get bits_per_word defaults etc
- core: make spi_delay_exec usable from atomic contexts
- rockchip: only register atomic transfer in host-mode
- rockchip: provide a wait_for_idle_atomic function, to
not require (non-working) jiffies for timeout
- rockchip: register additional restart handler to enable runtime pm before reboots too
- rockchip: drop runtime-pm from atomic transfer - would sleep anyway
- rockchip: don't prevent power_off, if runtime-resume fails
- rockchip: use core spi's spi_set_cs to also handle gpio-cs

[0] https://lore.kernel.org/lkml/[email protected]/
[1] https://lore.kernel.org/all/[email protected]/

Heiko Stuebner (6):
spi: Allow spi_toggle_csgpiod() to get called from atomic contexts
spi: Allow spi_delay_exec() to be used from atomic contexts
spi: Export spi_set_cs() function
spi: Add infrastructure for atomic sync transfers
spi: rockchip: Fix runtime resume error handling
spi: rockchip: Add an atomic transfer function

 drivers/spi/spi-rockchip.c | 125 ++++++++++++++++++++++++++++++++++++-
 drivers/spi/spi.c          |  90 +++++++++++++++++++++++---
 include/linux/spi/spi.h    |  13 ++++
 3 files changed, 217 insertions(+), 11 deletions(-)
Signature · 1 line
2.53.0
author maintainer
#1 lore raw

PATCH v4 1/6 spi: Allow spi_toggle_csgpiod() to get called from atomic contexts

From: Heiko Stuebner <[email protected]>

spi_toggle_csgpiod() by default allows for sleeping gpio controllers. In the upcoming atomic mode for late stage systems,

So determine which stage the system is at and select an appropriate gpio_set_value function to use.

If for whatever reason a sleeping gpio controller is combined with an atomic operation using a gpio-chipselect, this will cause a warning.

This also introduces a helper spi_in_atomic() to determine when that is the case. While the upcoming atomic transfer should really only be used during late system states, this helper just makes sure to distinguish between sleeping / non-sleeping.

---
 drivers/spi/spi.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
drivers/spi/spi.c +12 -2
index a6a9aaa47fde..5e05f2fef12e 100644
@@ -1244,8 +1244,18 @@ static inline bool spi_is_last_cs(struct spi_device *spi)
 	return last;
 }
 
+/* Check if sleeping is allowed */
+static inline bool spi_in_atomic(void)
+{
+	return IS_ENABLED(CONFIG_PREEMPT_COUNT) ? !preemptible() : irqs_disabled();
+}
+
 static void spi_toggle_csgpiod(struct spi_device *spi, u8 idx, bool enable, bool activate)
 {
+	/* Will complain if atomic transfer is combined with sleeping gpio */
+	int (*gpio_func)(struct gpio_desc *desc, int value) =
+		spi_in_atomic() ? gpiod_set_value : gpiod_set_value_cansleep;
+
 	/*
 	 * Historically ACPI has no means of the GPIO polarity and
 	 * thus the SPISerialBus() resource defines it on the per-chip
@@ -1257,10 +1267,10 @@ static void spi_toggle_csgpiod(struct spi_device *spi, u8 idx, bool enable, bool
 	 * into account.
 	 */
 	if (is_acpi_device_node(dev_fwnode(&spi->dev)))
-		gpiod_set_value_cansleep(spi_get_csgpiod(spi, idx), !enable);
+		gpio_func(spi_get_csgpiod(spi, idx), !enable);
 	else
 		/* Polarity handled by GPIO library */
-		gpiod_set_value_cansleep(spi_get_csgpiod(spi, idx), activate);
+		gpio_func(spi_get_csgpiod(spi, idx), activate);
 
 	if (activate)
 		spi_delay_exec(&spi->cs_setup, NULL);
Signature · 1 line
2.53.0
author maintainer
#2 lore raw

PATCH v4 2/6 spi: Allow spi_delay_exec() to be used from atomic contexts

In reply to Heiko Stuebner

From: Heiko Stuebner <[email protected]>

When using spi_delay_exec() only the fsleep in the subsequent _spi_transfer_delay_ns() can actually sleep.

As we might want to use spi_delay_exec() from atomic contexts in the future, move the might_sleep() up to the actual place that matters and fall back to the existing ndelay if called from an atomic context.

---
 drivers/spi/spi.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
drivers/spi/spi.c +2 -3
index 5e05f2fef12e..ec2074126910 100644
@@ -1730,11 +1730,12 @@ static void _spi_transfer_delay_ns(u32 ns)
 {
 	if (!ns)
 		return;
-	if (ns <= NSEC_PER_USEC) {
+	if (ns <= NSEC_PER_USEC || spi_in_atomic()) {
 		ndelay(ns);
 	} else {
 		u32 us = DIV_ROUND_UP(ns, NSEC_PER_USEC);
 
+		might_sleep();
 		fsleep(us);
 	}
 }
@@ -1782,8 +1783,6 @@ int spi_delay_exec(struct spi_delay *_delay, struct spi_transfer *xfer)
 {
 	int delay;
 
-	might_sleep();
-
 	if (!_delay)
 		return -EINVAL;
 
Signature · 1 line
2.53.0
author maintainer
#3 lore raw

PATCH v4 3/6 spi: Export spi_set_cs() function

In reply to Heiko Stuebner

From: Heiko Stuebner <[email protected]>

While spi_set_cs is currently only used when the SPI framework manages the core transfer function, it contains a lot of functionality that can be used for the upcoming atomic transfers, where a driver implementing that atomic transfer could profit from not having to duplicate the chip-select functionality.

The only sleeping part was the spi_delay_exec function doing the chip-select delays, which can now also run in atomic contexts.

So export the function.

---
 drivers/spi/spi.c       | 3 ++-
 include/linux/spi/spi.h | 2 ++
 2 files changed, 4 insertions(+), 1 deletion(-)
drivers/spi/spi.c +2 -1
index ec2074126910..0b0bc093e01f 100644
@@ -1278,7 +1278,7 @@ static void spi_toggle_csgpiod(struct spi_device *spi, u8 idx, bool enable, bool
 		spi_delay_exec(&spi->cs_inactive, NULL);
 }
 
-static void spi_set_cs(struct spi_device *spi, bool enable, bool force)
+void spi_set_cs(struct spi_device *spi, bool enable, bool force)
 {
 	bool activate = enable;
 	u8 idx;
@@ -1335,6 +1335,7 @@ static void spi_set_cs(struct spi_device *spi, bool enable, bool force)
 			spi_delay_exec(&spi->cs_inactive, NULL);
 	}
 }
+EXPORT_SYMBOL_GPL(spi_set_cs);
 
 #ifdef CONFIG_HAS_DMA
 static int spi_map_buf_attrs(struct spi_controller *ctlr, struct device *dev,
include/linux/spi/spi.h +2 -0
index fb4baa0d3398..0bf31776c54c 100644
@@ -349,6 +349,8 @@ static inline bool spi_is_csgpiod(struct spi_device *spi)
 	return false;
 }
 
+extern void spi_set_cs(struct spi_device *spi, bool enable, bool force);
+
 /**
  * struct spi_driver - Host side "protocol" driver
  * @id_table: List of SPI devices supported by this driver
Signature · 1 line
2.53.0
author maintainer
#4 lore raw

PATCH v4 4/6 spi: Add infrastructure for atomic sync transfers

In reply to Heiko Stuebner

From: Heiko Stuebner <[email protected]>

There are cases where SPI transfers are needed during times without interrupts like shutdown when a PMIC should turn off the system.

I2C already has atomic transfers for exactly that case, so introduce a similar concept for SPI. Similar to I2C, this is and should stay a corner-case, so is only allowed during the shutdown stages of the kernel, when interrupts are already turned off.

So add an optional transfer_atomic callback, drivers can implement to do immediate uninterrupted transfers and let spi_sync() determine if the state is late enough to use it.

For drivers without a transfer_atomic callback defined, the previous behaviour is kept for backwards compatibility and a helper spi_in_atomic_sync_mode() makes sure to only ever use this atomic mode during late system stages.

spi_async() does not require a sleeping context for itself, so while it could in theory also be included, its caller waiting for the completion can sleep, so would still cause the splat about it.

---
 drivers/spi/spi.c       | 68 ++++++++++++++++++++++++++++++++++++++---
 include/linux/spi/spi.h | 11 +++++++
 2 files changed, 75 insertions(+), 4 deletions(-)
drivers/spi/spi.c +64 -4
index 0b0bc093e01f..0569f30785e9 100644
@@ -5073,6 +5073,47 @@ static int __spi_sync(struct spi_device *spi, struct spi_message *message)
 	return status;
 }
 
+/*
+ * We only allow atomic transfers for very late communication, e.g. to access a
+ * PMIC when powering down. Atomic transfers are a corner case and not for
+ * generic use!
+ */
+static inline bool spi_in_atomic_sync_mode(void)
+{
+	return system_state > SYSTEM_RUNNING && spi_in_atomic();
+}
+
+static int __spi_sync_atomic(struct spi_device *spi, struct spi_message *message)
+{
+	struct spi_controller *ctlr = spi->controller;
+	unsigned long flags;
+	int status;
+
+	if (!ctlr->transfer_atomic) {
+		dev_err(&ctlr->dev, "No atomic transfer handler defined\n");
+		return -EOPNOTSUPP;
+	}
+
+	if (__spi_check_suspended(ctlr)) {
+		dev_warn_once(&spi->dev, "Attempted to sync while suspend\n");
+		return -ESHUTDOWN;
+	}
+
+	/* Not using spi_maybe_optimize_message, so just validate */
+	status = __spi_validate(spi, message);
+	if (status)
+		return status;
+
+	spin_lock_irqsave(&ctlr->bus_lock_spinlock, flags);
+	status = ctlr->transfer_atomic(spi, message);
+	spin_unlock_irqrestore(&ctlr->bus_lock_spinlock, flags);
+
+	if (status == 0)
+		status = message->status;
+
+	return status;
+}
+
 /**
  * spi_sync - blocking/synchronous SPI data transfers
  * @spi: device with which data will be exchanged
@@ -5083,6 +5124,10 @@ static int __spi_sync(struct spi_device *spi, struct spi_message *message)
  * is non-interruptible, and has no timeout.  Low-overhead controller
  * drivers may DMA directly into and out of the message buffers.
  *
+ * During the time when the system is being shutdown and a transfer_atomic
+ * function is set, this call may also be used from a context that does
+ * not allow sleeping, i.e. with disabled interrupts.
+ *
  * Note that the SPI device's chip select is active during the message,
  * and then is normally disabled between messages.  Drivers for some
  * frequently-used devices may want to minimize costs of selecting a chip,
@@ -5096,11 +5141,17 @@ static int __spi_sync(struct spi_device *spi, struct spi_message *message)
  */
 int spi_sync(struct spi_device *spi, struct spi_message *message)
 {
+	struct spi_controller *ctlr = spi->controller;
 	int ret;
 
-	mutex_lock(&spi->controller->bus_lock_mutex);
-	ret = __spi_sync(spi, message);
-	mutex_unlock(&spi->controller->bus_lock_mutex);
+	/* Keep old behaviour for clients that tolerate the warnings */
+	if (spi_in_atomic_sync_mode() && ctlr->transfer_atomic) {
+		ret = __spi_sync_atomic(spi, message);
+	} else {
+		mutex_lock(&spi->controller->bus_lock_mutex);
+		ret = __spi_sync(spi, message);
+		mutex_unlock(&spi->controller->bus_lock_mutex);
+	}
 
 	return ret;
 }
@@ -5116,6 +5167,10 @@ EXPORT_SYMBOL_GPL(spi_sync);
  * is non-interruptible, and has no timeout.  Low-overhead controller
  * drivers may DMA directly into and out of the message buffers.
  *
+ * During the time when the system is being shutdown and a transfer_atomic
+ * function is set, this call may also be used from a context that does
+ * not allow sleeping, i.e. with disabled interrupts.
+ *
  * This call should be used by drivers that require exclusive access to the
  * SPI bus. It has to be preceded by a spi_bus_lock call. The SPI bus must
  * be released by a spi_bus_unlock call when the exclusive access is over.
@@ -5124,7 +5179,12 @@ EXPORT_SYMBOL_GPL(spi_sync);
  */
 int spi_sync_locked(struct spi_device *spi, struct spi_message *message)
 {
-	return __spi_sync(spi, message);
+	struct spi_controller *ctlr = spi->controller;
+
+	if (spi_in_atomic_sync_mode() && ctlr->transfer_atomic)
+		return __spi_sync_atomic(spi, message);
+	else
+		return __spi_sync(spi, message);
 }
 EXPORT_SYMBOL_GPL(spi_sync_locked);
 
include/linux/spi/spi.h +11 -0
index 0bf31776c54c..f62b21af5d75 100644
@@ -461,6 +461,8 @@ extern struct spi_device *devm_spi_new_ancillary_device(struct spi_device *spi,
  * controller for configuring specific CS setup time, hold time and inactive
  * delay in terms of clock counts
  * @transfer: adds a message to the controller's transfer queue.
+ * @transfer_atomic: transfer a message immediately from a late-stage
+ *	synchronous transfer
  * @cleanup: frees controller-specific state
  * @can_dma: determine whether this controller supports DMA
  * @dma_map_dev: device which can be used for DMA mapping
@@ -721,6 +723,15 @@ struct spi_controller {
 	int			(*transfer)(struct spi_device *spi,
 						struct spi_message *mesg);
 
+	/*
+	 * Bidirectional bulk atomic transfers
+	 *
+	 * Special function that allows transfers in contexts where
+	 * no interrupts nor DMA are available.
+	 */
+	int			(*transfer_atomic)(struct spi_device *spi,
+						   struct spi_message *mesg);
+
 	/* Called on deregistration to free memory provided by spi_controller */
 	void			(*cleanup)(struct spi_device *spi);
 
Signature · 1 line
2.53.0
author maintainer
#5 lore raw

PATCH v4 5/6 spi: rockchip: Fix runtime resume error handling

In reply to Heiko Stuebner

From: Heiko Stuebner <[email protected]>

When enabling the apb-clock any error does get handled, but when an error happens enabling the spiclk itself, only the apb-clock is disabled but the function then returns successful.

Fix this by also reporting the error on enabling the second clock.

---
 drivers/spi/spi-rockchip.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/spi/spi-rockchip.c +3 -1
index f8d6cea2f3da..cc6bd7d8f877 100644
@@ -992,8 +992,10 @@ static int rockchip_spi_runtime_resume(struct device *dev)
 		return ret;
 
 	ret = clk_prepare_enable(rs->spiclk);
-	if (ret < 0)
+	if (ret < 0) {
 		clk_disable_unprepare(rs->apb_pclk);
+		return ret;
+	}
 
 	return 0;
 }
Signature · 1 line
2.53.0
author maintainer
#6 lore raw

PATCH v4 6/6 spi: rockchip: Add an atomic transfer function

In reply to Heiko Stuebner

From: Heiko Stuebner <[email protected]>

Rockchip SPI controllers are often used to handle SPI connected PMICs that also handle system power-off events, like the RK806.

When the kernel signals the power-off, all the interrupts are already disabled, so regular SPI with its mutexes and DMA transfers does not work anymore.

Implement an atomic transfer function that does SPI transfers polling the controller in a loop without any DMA or IRQ.

This can successfully handle the power-off command to RK806, but also read back the 2 chip-id registers just fine before that, as a test.

---
 drivers/spi/spi-rockchip.c | 121 +++++++++++++++++++++++++++++++++++++
 1 file changed, 121 insertions(+)
drivers/spi/spi-rockchip.c +121 -0
index cc6bd7d8f877..2cd2af3eb28f 100644
@@ -7,12 +7,14 @@
 #include <linux/clk.h>
 #include <linux/dmaengine.h>
 #include <linux/interrupt.h>
+#include <linux/iopoll.h>
 #include <linux/module.h>
 #include <linux/of.h>
 #include <linux/pinctrl/consumer.h>
 #include <linux/platform_device.h>
 #include <linux/spi/spi.h>
 #include <linux/pm_runtime.h>
+#include <linux/reboot.h>
 #include <linux/scatterlist.h>
 
 #define DRIVER_NAME "rockchip-spi"
@@ -218,6 +220,19 @@ static inline void wait_for_tx_idle(struct rockchip_spi *rs, bool target_mode)
 	dev_warn(rs->dev, "spi controller is in busy state!\n");
 }
 
+/* atomic transfers only in host-mode */
+static inline void wait_for_tx_idle_atomic(struct rockchip_spi *rs)
+{
+	u32 sr;
+	int ret;
+
+	ret = readl_relaxed_poll_timeout_atomic(rs->regs + ROCKCHIP_SPI_SR, sr,
+						!(sr & SR_BUSY),
+						10, 5 * USEC_PER_MSEC);
+	if (ret)
+		dev_warn(rs->dev, "spi controller still in busy state!\n");
+}
+
 static u32 get_fifo_len(struct rockchip_spi *rs)
 {
 	u32 ver;
@@ -722,6 +737,74 @@ static bool rockchip_spi_can_dma(struct spi_controller *ctlr,
 	return xfer->len / bytes_per_word >= rs->fifo_len;
 }
 
+static int rockchip_spi_transfer_atomic(struct spi_device *spi, struct spi_message *msg)
+{
+	struct spi_controller *ctlr = spi->controller;
+	struct rockchip_spi *rs = spi_controller_get_devdata(ctlr);
+	struct spi_transfer *xfer;
+	int ret = 0;
+
+	if (!pm_runtime_active(rs->dev)) {
+		dev_err(rs->dev, "Failed to do atomic SPI transfer due to runtime suspend\n");
+		return -EIO;
+	}
+
+	spi_set_cs(spi, true, false);
+
+	/* make sure all interrupts are masked and status cleared */
+	writel_relaxed(0, rs->regs + ROCKCHIP_SPI_IMR);
+	writel_relaxed(0xffffffff, rs->regs + ROCKCHIP_SPI_ICR);
+
+	list_for_each_entry(xfer, &msg->transfers, transfer_list) {
+		WARN_ON(readl_relaxed(rs->regs + ROCKCHIP_SPI_SSIENR) &&
+			(readl_relaxed(rs->regs + ROCKCHIP_SPI_SR) & SR_BUSY));
+
+		rs->n_bytes = xfer->bits_per_word <= 8 ? 1 : 2;
+
+		ret = rockchip_spi_config(rs, spi, xfer, false, false);
+		if (ret < 0)
+			goto out_disable;
+
+		writel_relaxed(0, rs->regs + ROCKCHIP_SPI_RXFTLR);
+
+		rs->tx = xfer->tx_buf;
+		rs->rx = xfer->rx_buf;
+		rs->tx_left = rs->tx ? xfer->len / rs->n_bytes : 0;
+		rs->rx_left = xfer->len / rs->n_bytes;
+
+		spi_enable_chip(rs, true);
+
+		while (rs->tx_left || rs->rx_left) {
+			u32 status = readl_relaxed(rs->regs + ROCKCHIP_SPI_SR);
+
+			if (rs->tx_left && !(status & SR_TF_FULL))
+				rockchip_spi_pio_writer(rs);
+
+			if (rs->rx_left && !(status & SR_RF_EMPTY))
+				rockchip_spi_pio_reader(rs);
+		}
+
+		wait_for_tx_idle_atomic(rs);
+
+		spi_enable_chip(rs, false);
+
+		if (xfer->cs_change && !list_is_last(&xfer->transfer_list, &msg->transfers)) {
+			spi_set_cs(spi, false, false);
+			spi_transfer_cs_change_delay_exec(msg, xfer);
+			spi_set_cs(spi, true, false);
+		}
+	}
+
+out_disable:
+	spi_set_cs(spi, false, false);
+	msg->status = ret;
+
+	if (msg->complete)
+		msg->complete(msg->context);
+
+	return ret;
+}
+
 static int rockchip_spi_setup(struct spi_device *spi)
 {
 	struct rockchip_spi *rs = spi_controller_get_devdata(spi->controller);
@@ -750,6 +833,19 @@ static int rockchip_spi_setup(struct spi_device *spi)
 	return 0;
 }
 
+static int rockchip_spi_power_off(struct sys_off_data *data)
+{
+	struct rockchip_spi *rs = data->cb_data;
+	int ret;
+
+	/* enable controller for possible calls from shutdown handlers */
+	ret = pm_runtime_resume_and_get(rs->dev);
+	if (ret)
+		dev_warn(rs->dev, "spi controller failed to runtime resume: %d\n", ret);
+
+	return 0;
+}
+
 static int rockchip_spi_probe(struct platform_device *pdev)
 {
 	struct device_node *np = pdev->dev.of_node;
@@ -858,6 +954,31 @@ static int rockchip_spi_probe(struct platform_device *pdev)
 	ctlr->max_transfer_size = rockchip_spi_max_transfer_size;
 	ctlr->handle_err = rockchip_spi_handle_err;
 
+	/* Only register atomic transfer in host-mode */
+	if (!target_mode) {
+		ctlr->transfer_atomic = rockchip_spi_transfer_atomic;
+
+		ret = devm_register_sys_off_handler(&pdev->dev,
+						    SYS_OFF_MODE_POWER_OFF_PREPARE,
+						    SYS_OFF_PRIO_DEFAULT,
+						    &rockchip_spi_power_off, rs);
+		if (ret) {
+			dev_err_probe(&pdev->dev, ret,
+				     "failed to register poweroff handler\n");
+			goto err_disable_pm_runtime;
+		}
+
+		ret = devm_register_sys_off_handler(&pdev->dev,
+						    SYS_OFF_MODE_RESTART_PREPARE,
+						    SYS_OFF_PRIO_DEFAULT,
+						    &rockchip_spi_power_off, rs);
+		if (ret) {
+			dev_err_probe(&pdev->dev, ret,
+				     "failed to register restart handler\n");
+			goto err_disable_pm_runtime;
+		}
+	}
+
 	ctlr->dma_tx = dma_request_chan(rs->dev, "tx");
 	if (IS_ERR(ctlr->dma_tx)) {
 		/* Check tx to see if we need to defer driver probing */
Signature · 1 line
2.53.0