Skip to contentSkip to conversation

async_tx: add support for asynchronous RAID6 recovery operations

PATCH In review Yuri Tikhonov 2 participants 5 messages Jan 13, 2009 – Jan 17, 2009

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH v3 3/11 async_tx: add support for asynchronous RAID6 recovery operations

This patch extends async_tx API with two operations for recovery operations on RAID6 array with two failed disks using new async_pq() operation. Patch introduces the following functions:

async_r6_dd_recov() recovers after double data disk failure

async_r6_dp_recov() recovers after D+P failure

---
 crypto/async_tx/Kconfig         |    5 +
 crypto/async_tx/Makefile        |    1 +
 crypto/async_tx/async_r6recov.c |  286 +++++++++++++++++++++++++++++++++++++++
 include/linux/async_tx.h        |   11 ++
 4 files changed, 303 insertions(+), 0 deletions(-)
 create mode 100644 crypto/async_tx/async_r6recov.c
crypto/async_tx/Kconfig +5 -0
index cb6d731..0b56224 100644
@@ -18,3 +18,8 @@ config ASYNC_PQ
 	tristate
 	select ASYNC_CORE
 
+config ASYNC_R6RECOV
+	tristate
+	select ASYNC_CORE
+	select ASYNC_PQ
+
crypto/async_tx/Makefile +1 -0
index 1b99265..0ed8f13 100644
@@ -3,3 +3,4 @@ obj-$(CONFIG_ASYNC_MEMCPY) += async_memcpy.o
 obj-$(CONFIG_ASYNC_MEMSET) += async_memset.o
 obj-$(CONFIG_ASYNC_XOR) += async_xor.o
 obj-$(CONFIG_ASYNC_PQ) += async_pq.o
+obj-$(CONFIG_ASYNC_R6RECOV) += async_r6recov.o
crypto/async_tx/async_r6recov.c new file +286 -0

286 lines not on this page · raw message

include/linux/async_tx.h +11 -0
index 2f92d87..7575f12 100644
@@ -173,5 +173,16 @@ async_syndrome_zero_sum(struct page **src_list, unsigned int offset,
 	enum async_tx_flags flags, struct dma_async_tx_descriptor *depend_tx,
 	dma_async_tx_callback callback, void *callback_param);
 
+struct dma_async_tx_descriptor *
+async_r6_dd_recov(int src_num, size_t bytes, int faila, int failb,
+	struct page **ptrs, enum async_tx_flags flags,
+	struct dma_async_tx_descriptor *depend_tx,
+	dma_async_tx_callback callback, void *callback_param);
+
+struct dma_async_tx_descriptor *
+async_r6_dp_recov(int src_num, size_t bytes, int faila, struct page **ptrs,
+	enum async_tx_flags flags, struct dma_async_tx_descriptor *depend_tx,
+	dma_async_tx_callback callback, void *callback_param);
+
 void async_tx_quiesce(struct dma_async_tx_descriptor **tx);
 #endif /* _ASYNC_TX_H_ */
Signature · 1 line
1.6.0.6
#1 lore raw
Quoted 4 lines · Yuri Tikhonov
crypto/async_tx/async_r6recov.c new file +4 -0
+       /* (2) Calculate Q+Qxy */
+       lptrs[0] = ptrs[failb];
+       lptrs[1] = ptrs[disks-1];
+       lptrs[2] = NULL;

crypto/async_tx/async_r6recov.c line 123

Quoted 10 lines
crypto/async_tx/async_r6recov.c new file +10 -0
+       tx = async_pq(lptrs, NULL, 0, 1, bytes, ASYNC_TX_DEP_ACK,
+                     tx, NULL, NULL);
+
+       /* (3) Calculate P+Pxy */
+       lptrs[0] = ptrs[faila];
+       lptrs[1] = ptrs[disks-2];
+       lptrs[2] = NULL;
+       tx = async_pq(lptrs, NULL, 0, 1, bytes, ASYNC_TX_DEP_ACK,
+                     tx, NULL, NULL);
+

These two calls convinced me that ASYNC_TX_PQ_ZERO_{P,Q} need to go. A 1-source async_pq operation does not make sense. These should be:

       /* (2) Calculate Q+Qxy */
       lptrs[0] = ptrs[disks-1];
       lptrs[1] = ptrs[failb];
       tx = async_xor(lptrs[0], lptrs, 0, 2, bytes,
                      ASYNC_TX_XOR_DROP_DST|ASYNC_TX_DEP_ACK, tx, NULL, NULL);

        /* (3) Calculate P+Pxy */
       lptrs[0] = ptrs[disks-2];
       lptrs[1] = ptrs[faila];
       tx = async_xor(lptrs[0], lptrs, 0, 2, bytes,
                      ASYNC_TX_XOR_DROP_DST|ASYNC_TX_DEP_ACK, tx, NULL, NULL);

Regards,
Dan

#2 lore raw
Quoted 10 lines · On Thursday, January 15, 2009 Dan Williams wrote:
Quoted 9 lines · Yuri Tikhonov
crypto/async_tx/async_r6recov.c new file +9 -0
+       /* (2) Calculate Q+Qxy */
+       lptrs[0] = ptrs[failb];
+       lptrs[1] = ptrs[disks-1];
+       lptrs[2] = NULL;
+       tx = async_pq(lptrs, NULL, 0, 1, bytes, ASYNC_TX_DEP_ACK,
+                     tx, NULL, NULL);
+
+       /* (3) Calculate P+Pxy */
+       lptrs[0] = ptrs[faila];
Quoted 8 lines

crypto/async_tx/async_r6recov.c line 123

Quoted 5 lines
crypto/async_tx/async_r6recov.c new file +5 -0
+       lptrs[1] = ptrs[disks-2];
+       lptrs[2] = NULL;
+       tx = async_pq(lptrs, NULL, 0, 1, bytes, ASYNC_TX_DEP_ACK,
+                     tx, NULL, NULL);
+

These two calls convinced me that ASYNC_TX_PQ_ZERO_{P,Q} need to go. A 1-source async_pq operation does not make sense.

Another source is hidden under not-set ASYNC_TX_PQ_ZERO_{P,Q} :)
Though, I agree, this looks rather misleading.

Quoted 13 lines

These should be:

       /* (2) Calculate Q+Qxy */
       lptrs[0] = ptrs[disks-1];
       lptrs[1] = ptrs[failb];
       tx = async_xor(lptrs[0], lptrs, 0, 2, bytes,
                      ASYNC_TX_XOR_DROP_DST|ASYNC_TX_DEP_ACK, tx, NULL, NULL);

        /* (3) Calculate P+Pxy */
       lptrs[0] = ptrs[disks-2];
       lptrs[1] = ptrs[faila];
       tx = async_xor(lptrs[0], lptrs, 0, 2, bytes,
                      ASYNC_TX_XOR_DROP_DST|ASYNC_TX_DEP_ACK, tx, NULL, NULL);

The reason why I preferred to use async_pq() instead of async_xor()
here is to maximize the chance that the whole D+D recovery operation will be handled in one ADMA device, i.e. without channels switch and the latency introduced because of that.

So, if we'll decide to stay with ASYNC_TX_PQ_ZERO_{P,Q}, then this
should be probably kept unchanged, but if we'll get rid of ASYNC_TX_PQ_ZERO_{P,Q}, then, obviously, we'll have to use async_xor()s here as you suggest.

Regards, Yuri

--
Yuri Tikhonov, Senior Software Engineer
Emcraft Systems, www.emcraft.com

#3 lore raw
Quoted 5 lines · Yuri Tikhonov

The reason why I preferred to use async_pq() instead of async_xor()
here is to maximize the chance that the whole D+D recovery operation will be handled in one ADMA device, i.e. without channels switch and the latency introduced because of that.

This should be a function of the async_tx_find_channel implementation.
The default version tries to keep a chain of operations on one
channel.

struct dma_chan *
__async_tx_find_channel(struct dma_async_tx_descriptor *depend_tx,
enum dma_transaction_type tx_type)

{
        /* see if we can keep the chain on one channel */
        if (depend_tx &&
            dma_has_cap(tx_type, depend_tx->chan->device->cap_mask))
                return depend_tx->chan;
        return dma_find_channel(tx_type);
}

--
Dan

#4 lore raw
Quoted 14 lines · On Friday, January 16, 2009 you wrote:
Quoted 5 lines · Yuri Tikhonov

The reason why I preferred to use async_pq() instead of async_xor()
here is to maximize the chance that the whole D+D recovery operation will be handled in one ADMA device, i.e. without channels switch and the latency introduced because of that.

This should be a function of the async_tx_find_channel implementation.
The default version tries to keep a chain of operations on one
channel.

struct dma_chan *
__async_tx_find_channel(struct dma_async_tx_descriptor *depend_tx,
enum dma_transaction_type tx_type)

Quoted 7 lines
{
        /* see if we can keep the chain on one channel */
        if (depend_tx &&
            dma_has_cap(tx_type, depend_tx->chan->device->cap_mask))
                return depend_tx->chan;
        return dma_find_channel(tx_type);
}

Right. Then I need to update my ADMA driver, and add support for
explicit DMA_XOR capability on channels which can process DMA_PQ. Thanks.

Regards, Yuri

--
Yuri Tikhonov, Senior Software Engineer
Emcraft Systems, www.emcraft.com