dmaengine: mv_xor: order descriptor writes before engine access

The descriptor pool is allocated with dma_alloc_wc(), so descriptor writes sit in the CPU write buffers and only reach the engine when the buffers are flushed. The engine can fetch a descriptor as soon as it is pointed at one, so those writes must be ordered against the MMIO registers and the descriptor links that hand ownership over.

mv_chan_set_next_descriptor() programs the next-descriptor register with writel() instead of writel_relaxed(); the barrier inside writel() drains prior descriptor writes before the MMIO store that points the engine at them.

The chain-append path in mv_xor_tx_submit() needs two more barriers. A dma_wmb() before mv_desc_set_next_desc() orders the stores that initialize sw_desc->hw_desc in the prep functions before the store that links the descriptor into the chain; otherwise a running engine could follow the new link and read a half-written descriptor. The existing mb() after it orders the link store before the busy-status and current-descriptor register reads, which the relaxed readl path would otherwise bypass while the link write is still in the write buffer, letting the CPU restart a channel the engine has already raced past the tail of.

---
v3: add back dma_wmb
v2: split off from main patch

 drivers/dma/mv_xor.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

index da8eea8789ae..c46ee95fe01a 100644
@@ -103,7 +103,11 @@ static u32 mv_chan_get_current_desc(struct mv_xor_chan *chan)
 static void mv_chan_set_next_descriptor(struct mv_xor_chan *chan,
 					u32 next_desc_addr)
 {
-	writel_relaxed(next_desc_addr, XOR_NEXT_DESC(chan));
+	/*
+	 * writel drains descriptor writes to DRAM before the engine
+	 * is pointed at them
+	 */
+	writel(next_desc_addr, XOR_NEXT_DESC(chan));
 }
 
 static void mv_chan_unmask_interrupts(struct mv_xor_chan *chan)

@@ -407,9 +411,18 @@ mv_xor_tx_submit(struct dma_async_tx_descriptor *tx)
 		dev_dbg(mv_chan_to_devp(mv_chan), "Append to last desc %pa\n",
 			&old_chain_tail->async_tx.phys);
 
+		/* commit the new descriptor before chaining it in */
+		dma_wmb();
+
 		/* fix up the hardware chain */
 		mv_desc_set_next_desc(old_chain_tail, sw_desc->async_tx.phys);
 
+		/*
+		 * make the new link visible to the engine before we read
+		 * the channel state, the device may fetch it at any point
+		 */
+		mb();
+
 		/* if the channel is not busy */
 		if (!mv_chan_is_busy(mv_chan)) {
 			u32 current_desc = mv_chan_get_current_desc(mv_chan);

2.55.0