Add enhance SPI DMA support for JHB100 SFC
v2 cover Add enhance SPI DMA support for JHB100 SFC
On the JHB100, there is only one channel available for DMA transfer. In flash access scenarios, it operates in half-duplex mode, so we can request this DMA channel for the direction that transfers a larger amount of data. Different transfer directions need to share a single DMA channel in a time-multiplexed manner.
This series has already been tested on the JHB100 EVB1.
Change since v1:
- Add a new patch 1.
- Keep DMA channels for the lifetime of the controller instead of requesting and releasing them for each transfer.
- Optimize the process of requesting/releasing DMA for every enhanced SPI transfer, so that the release/request DMA process is re-executed only when the direction of a new DMA transfer is different from that of the previous DMA transfer.
v1: https://lore.kernel.org/all/[email protected]/
Changhuang Liang (3):
spi: dw: Convert the Tx done delay from SCK cycles to time
spi: dw: Support DMA with a single channel
spi: dw: Add DMA support for enhanced memory operations
drivers/spi/spi-dw-core.c | 103 +++++++++----- drivers/spi/spi-dw-dma.c | 290 ++++++++++++++++++++++++++++++++++---- drivers/spi/spi-dw-mmio.c | 2 + drivers/spi/spi-dw.h | 8 ++ 4 files changed, 336 insertions(+), 67 deletions(-)
--
2.25.1
PATCH v2 1/3 spi: dw: Convert the Tx done delay from SCK cycles to time
Currently the delay in dw_spi_dma_wait_tx_done() is expressed in SCK cycles, which requires passing a valid struct spi_transfer pointer. This does not work for enhanced SPI memory operations, which are not described by a spi_transfer. Compute the delay in ns or us based on an explicit bus speed instead, so that the helper can also be used for those transfers.
--- drivers/spi/spi-dw-dma.c | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-)
drivers/spi/spi-dw-dma.c +14 -6
@@ -286,19 +286,27 @@ static inline bool dw_spi_dma_tx_busy(struct dw_spi *dws) (DW_SPI_SR_BUSY | DW_SPI_SR_TF_EMPT)) != DW_SPI_SR_TF_EMPT; } -static int dw_spi_dma_wait_tx_done (struct dw_spi *dws, -struct spi_transfer *xfer) +static int dw_spi_dma_wait_tx_done (struct dw_spi *dws,u32 speed_hz) {int retry =DW_SPI_WAIT_RETRIES ;struct spi_delay delay; +unsigned long ns, us;u32 nents; nents =dw_readl (dws,DW_SPI_TXFLR ); - delay.unit =SPI_DELAY_UNIT_SCK ; - delay.value = nents * dws->n_bytes *BITS_PER_BYTE ; + ns =DIV_ROUND_UP (NSEC_PER_SEC , speed_hz) * nents * + dws->n_bytes *BITS_PER_BYTE ; +if (ns <=NSEC_PER_USEC ) { + delay.unit =SPI_DELAY_UNIT_NSECS ; + delay.value = ns; + }else { + us =DIV_ROUND_UP (ns,NSEC_PER_USEC ); + delay.unit =SPI_DELAY_UNIT_USECS ; + delay.value =clamp_val (us,0 ,USHRT_MAX ); + }while (dw_spi_dma_tx_busy (dws) && retry--) -spi_delay_exec (&delay, xfer); +spi_delay_exec (&delay,NULL );if (retry <0 ) {dev_err (&dws->ctlr->dev,"Tx hanged up\n" );
@@ -659,7 +667,7 @@ static int dw_spi_dma_transfer(struct dw_spi *dws, struct spi_transfer *xfer)return ret;if (dws->ctlr->cur_msg->status == -EINPROGRESS ) { - ret =dw_spi_dma_wait_tx_done (dws, xfer); + ret =dw_spi_dma_wait_tx_done (dws, xfer->effective_speed_hz);if (ret)return ret; }
Signature · 1 line
2.25.1
PATCH v2 2/3 spi: dw: Support DMA with a single channel
In reply to Changhuang LiangSome platforms, such as StarFive JHB100, provide only one DMA channel for the DW SPI controller. dw_spi_dma_init_generic() requires both the Tx and Rx channels, so DMA is disabled entirely on such platforms.
Request both channels at probe time and decide what is usable from the return values. If the first (Rx) channel request fails, fall back to non-DMA transfers as before. If only the second (Tx) channel request fails, keep the Rx channel. Each successfully requested channel is counted in dws->dma_nr_chans. -EPROBE_DEFER on the second channel is still propagated.
Full-duplex standard SPI transfers need both channels, so only set ctlr->can_dma and SPI_CONTROLLER_MUST_TX when two channels are available. A single channel can still serve half-duplex transfers, which will be used by the enhanced SPI memory operations in a follow-up patch.
Make dw_spi_dma_caps_init() and dw_spi_dma_maxburst_init() cope with a missing channel, and set ctlr->dma_tx/dma_rx only after the capability check has passed so that they never point to a released channel.
--- drivers/spi/spi-dw-core.c | 3 ++ drivers/spi/spi-dw-dma.c | 88 ++++++++++++++++++++++++++++----------- drivers/spi/spi-dw.h | 1 + 3 files changed, 67 insertions(+), 25 deletions(-)
drivers/spi/spi-dw-core.c +3 -0
@@ -1360,6 +1360,9 @@ int dw_spi_add_controller(struct device *dev, struct dw_spi *dws) goto err_free_irq; }else if (ret) {dev_warn (dev,"DMA init failed\n" ); + }else if (dws->dma_nr_chans ==1 ) { +/* Full-duplex DMA transfers need both channels */ +dev_info (dev,"Single DMA channel, full-duplex DMA disabled\n" ); }else { ctlr->can_dma = dws->dma_ops->can_dma; ctlr->flags |=SPI_CONTROLLER_MUST_TX ;
drivers/spi/spi-dw-dma.c +63 -25
137 lines not on this page · raw message
drivers/spi/spi-dw.h +1 -0
@@ -227,6 +227,7 @@ struct dw_spi {u32 txburst;struct dma_chan *rxchan;u32 rxburst; +unsigned int dma_nr_chans;u32 dma_sg_burst;u32 dma_addr_widths;unsigned long dma_chan_busy;
Signature · 1 line
2.25.1
PATCH v2 3/3 spi: dw: Add DMA support for enhanced memory operations
In reply to Changhuang LiangImplement DMA support for enhanced SPI memory operations. These operations are half-duplex, so they work with either one or two DMA channels.
Add enhanced memory DMA callbacks (get_chan, setup, can_dma, transfer) to struct dw_spi_dma_ops and hook them into the generic DMA operations. They reuse the existing channel configuration, submission and wait helpers.
With two channels, both are requested at probe time and kept for the lifetime of the controller. With a single channel, as on JHB100, the channel is bound to one direction by its "tx"/"rx" DMA specifier. The channel currently held (dws->txchan or dws->rxchan) records which direction it serves. It is reused as long as the direction stays the same, and only released and requested for the other direction when the transfer direction changes.
In dw_spi_exec_enh_mem_op(), record the direction in dws->tx_dir and use the DMA path when a channel is available for that direction, can_dma_enh_mem() accepts the transfer, and the transfer exceeds the FIFO length; otherwise fall back to the interrupt path.
Also move the udelay(5) workaround into dw_spi_enh_write_cmd_addr() so it applies to both paths, and enable the generic DMA setup in dw_spi_jhb100_init().
--- drivers/spi/spi-dw-core.c | 100 +++++++++++++-------- drivers/spi/spi-dw-dma.c | 182 ++++++++++++++++++++++++++++++++++++++ drivers/spi/spi-dw-mmio.c | 2 + drivers/spi/spi-dw.h | 7 ++ 4 files changed, 255 insertions(+), 36 deletions(-)
drivers/spi/spi-dw-core.c +64 -36
@@ -937,6 +937,15 @@ static void dw_spi_enh_write_cmd_addr(struct dw_spi *dws, const struct spi_mem_odw_spi_set_cs (mem->spi, false); } + +/* +* FIXME: The exact reason for this delay is not fully understood, +* but empirical testing shows it significantly improves the stability +* of read/write operations. Without this delay, occasional transfer +* errors or timeouts may occur under certain conditions. +* Keeping it as a safeguard based on practical validation. +*/ +udelay (5 ); }static int dw_spi_exec_enh_mem_op (struct spi_mem *mem,const struct spi_mem_op *op)
@@ -948,6 +957,8 @@ static int dw_spi_exec_enh_mem_op(struct spi_mem *mem, const struct spi_mem_op *unsigned long long ms;int ret; + dws->dma_mapped = false; +switch (op->data.buswidth) {case 0 :case 1 :
@@ -971,10 +982,13 @@ static int dw_spi_exec_enh_mem_op(struct spi_mem *mem, const struct spi_mem_op * cfg.dfs =8 ; cfg.freq =clamp (op->max_freq,0U , dws->max_mem_freq); cfg.ndf = op->data.nbytes; -if (op->data.dir ==SPI_MEM_DATA_IN ) +if (op->data.dir ==SPI_MEM_DATA_IN ) { cfg.tmode =DW_SPI_CTRLR0_TMOD_RO ; -else + dws->tx_dir = false; + }else { cfg.tmode =DW_SPI_CTRLR0_TMOD_TO ; + dws->tx_dir = true; + }if (op->data.buswidth == op->addr.buswidth && op->data.buswidth == op->cmd.buswidth)
@@ -1010,47 +1024,61 @@ static int dw_spi_exec_enh_mem_op(struct spi_mem *mem, const struct spi_mem_op * } } -dw_spi_enh_write_cmd_addr (dws, op, mem); +if (dws->dma_nr_chans && dws->dma_ops->dma_enh_mem_get_chan && + op->data.nbytes > dws->fifo_len) { + ret = dws->dma_ops->dma_enh_mem_get_chan (ctlr->dev.parent, dws); +if (ret) { +/* fall back to the PIO/IRQ path */ +dev_dbg (&ctlr->dev,"DMA enh mem channel unavailable (%d)\n" , ret); + }else if (dws->dma_ops->can_dma_enh_mem (ctlr)) { + ret = dws->dma_ops->dma_enh_mem_setup (dws); +if (ret) +dev_err (&ctlr->dev,"DMA enh mem setup failed (%d)\n" , ret); +else + dws->dma_mapped = true; + } + } -/* -* FIXME: The exact reason for this delay is not fully understood, -* but empirical testing shows it significantly improves the stability -* of read/write operations. Without this delay, occasional transfer -* errors or timeouts may occur under certain conditions. -* Keeping it as a safeguard based on practical validation. -*/ -udelay (5 ); +if (dws->dma_mapped) { +dw_spi_enh_write_cmd_addr (dws, op, mem); -dw_spi_enh_irq_setup (dws); + ret = dws->dma_ops->dma_enh_mem_transfer (mem, op); -/* Use timeout calculation from spi_transfer_wait() */ - ms =8LL *MSEC_PER_SEC * (dws->rx_len ? dws->rx_len : dws->tx_len); -do_div (ms, dws->current_freq); +dw_spi_stop_mem_op (dws, mem->spi); + }else { +dw_spi_enh_write_cmd_addr (dws, op, mem); -/* -* Increase it twice and add 200 ms tolerance, use -* predefined maximum in case of overflow. -*/ - ms += ms +200 ; -if (ms >UINT_MAX ) - ms =UINT_MAX ; +dw_spi_enh_irq_setup (dws); - ms =wait_for_completion_timeout (&ctlr->xfer_completion, -msecs_to_jiffies (ms)); -if (ms ==0 ) { -dw_spi_mask_intr (dws,0xff ); -synchronize_irq (dws->irq); - dws->rx =NULL ; - dws->tx =NULL ; - dws->rx_len =0 ; - dws->tx_len =0 ; -dw_spi_stop_mem_op (dws, mem->spi); -return -EIO ; - } +/* Use timeout calculation from spi_transfer_wait() */ + ms =8LL *MSEC_PER_SEC * (dws->rx_len ? dws->rx_len : dws->tx_len); +do_div (ms, dws->current_freq); + +/* +* Increase it twice and add 200 ms tolerance, use +* predefined maximum in case of overflow. +*/ + ms += ms +200 ; +if (ms >UINT_MAX ) + ms =UINT_MAX ; + + ms =wait_for_completion_timeout (&ctlr->xfer_completion, +msecs_to_jiffies (ms)); +if (ms ==0 ) { +dw_spi_mask_intr (dws,0xff ); +synchronize_irq (dws->irq); + dws->rx =NULL ; + dws->tx =NULL ; + dws->rx_len =0 ; + dws->tx_len =0 ; +dw_spi_stop_mem_op (dws, mem->spi); +return -EIO ; + } - ret =dw_spi_wait_mem_op_done (dws); + ret =dw_spi_wait_mem_op_done (dws); -dw_spi_stop_mem_op (dws, mem->spi); +dw_spi_stop_mem_op (dws, mem->spi); + }return ret; }
drivers/spi/spi-dw-dma.c +182 -0
194 lines not on this page · raw message
drivers/spi/spi-dw-mmio.c +2 -0
@@ -363,6 +363,8 @@ static int dw_spi_jhb100_init(struct platform_device *pdev,dw_spi_jhb100_mask_intr (&dwsmmio->dws,0xff ); +dw_spi_dma_setup_generic (&dwsmmio->dws); +return 0 ; }
drivers/spi/spi-dw.h +7 -0
@@ -183,6 +183,12 @@ struct dw_spi_dma_ops { struct spi_transfer *xfer);int (*dma_transfer)(struct dw_spi *dws,struct spi_transfer *xfer);void (*dma_stop)(struct dw_spi *dws); + +/* enh mem dma ops */ +int (*dma_enh_mem_get_chan)(struct device *dev,struct dw_spi *dws); +int (*dma_enh_mem_setup)(struct dw_spi *dws); +bool (*can_dma_enh_mem)(struct spi_controller *ctlr); +int (*dma_enh_mem_transfer)(struct spi_mem *mem,const struct spi_mem_op *op); };struct dw_spi {
@@ -228,6 +234,7 @@ struct dw_spi {struct dma_chan *rxchan;u32 rxburst;unsigned int dma_nr_chans; +bool tx_dir;u32 dma_sg_burst;u32 dma_addr_widths;unsigned long dma_chan_busy;
Signature · 1 line
2.25.1
Mark Brown
maintainerRe: [PATCH v2 3/3] spi: dw: Add DMA support for enhanced memory operations
Quoted 8 lines · Changhuang Liang
Implement DMA support for enhanced SPI memory operations. These operations are half-duplex, so they work with either one or two DMA channels.
drivers/spi/spi-dw-dma.c +4
-0+static void dw_spi_enh_mem_dma_stop (struct dw_spi *dws) +{ +if (dws->tx_dir) { +if (test_bit (DW_SPI_TX_BUSY , &dws->dma_chan_busy)) {
drivers/spi/spi-dw-dma.c static void dw_spi_dma_stop(struct dw_spi *dws) · line 860
Quoted 10 lines
drivers/spi/spi-dw-dma.c +10
-0+dmaengine_terminate_sync (dws->txchan); +clear_bit (DW_SPI_TX_BUSY , &dws->dma_chan_busy); + } + }else { +if (test_bit (DW_SPI_RX_BUSY , &dws->dma_chan_busy)) { +dmaengine_terminate_sync (dws->rxchan); +clear_bit (DW_SPI_RX_BUSY , &dws->dma_chan_busy); + } + } +}
It would probably be safer to unconditionally stop the DMA and clear the busy bit, we're only doing this in error cases so we shouldn't rely on everything being in a joined up state. Equally, I see the same pattern in the existing code so if we're going to fix it we could fix both at once.
Mark Brown
maintainerQuoted 9 lines · Changhuang Liang
Add enhance SPI DMA support for JHB100 SFC
On the JHB100, there is only one channel available for DMA transfer. In flash access scenarios, it operates in half-duplex mode, so we can request this DMA channel for the direction that transfers a larger amount of data. Different transfer directions need to share a single DMA channel in a time-multiplexed manner.
[...]
Applied to
https://git.kernel.org/pub/scm/linux/kernel/git/broonie/spi.git for-7.4
Thanks!
[1/3] spi: dw: Convert the Tx done delay from SCK cycles to time
https://git.kernel.org/broonie/spi/c/5c99cb15169d [2/3] spi: dw: Support DMA with a single channel https://git.kernel.org/broonie/spi/c/929b82c85fc1 [3/3] spi: dw: Add DMA support for enhanced memory operations https://git.kernel.org/broonie/spi/c/8cc7be24bc5e
All being well this means that it will be integrated into the linux-next tree (usually sometime in the next 24 hours) and sent to Linus during the next merge window (or sooner if it is a bug fix), however if problems are discovered then the patch may be dropped or reverted.
You may get further e-mails resulting from automated or manual testing and review of the tree, please engage with people reporting problems and send followup patches addressing any issues that are reported if needed.
If any updates are required or you are submitting further changes they should be sent as incremental updates against current git, existing patches will not be replaced.
Please add any relevant lists and maintainers to the CCs when replying to this mail.
Thanks,
Mark