Skip to content

Hardware Accelerated MD RAID5: Introduction

Dan Williams linux-raid lkml 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest19 patches 29 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/19raid5: raid5_do_soft_block_opsdrivers/md/raid5.c include/linux/raid/raid5.h+415 -31 reply
2/19raid5: move write operations to a workqueuedrivers/md/Kconfig drivers/md/raid5.c include/linux/raid/raid5.h+190 -261 reply
3/19raid5: move check parity operations to a workqueuedrivers/md/raid5.c+42 -180 replies
4/19raid5: move compute block operations to a workqueuedrivers/md/raid5.c+107 -460 replies
5/19raid5: move read completion copies to a workqueuedrivers/md/raid5.c+38 -290 replies
6/19raid5: move the reconstruct write expansion operation to a workqueuedrivers/md/raid5.c+27 -90 replies
7/19raid5: remove compute_block and compute_parity5drivers/md/raid5.c+0 -1230 replies
8/19dmaengine: enable multiple clients and operationsdrivers/dma/dmaengine.c drivers/dma/ioatdma.c include/linux/dmaengine.h +2 more+443 -11513 replies
9/19dmaengine: reduce backend address permutationsdrivers/dma/dmaengine.c drivers/dma/ioatdma.c include/linux/dmaengine.h+249 -1451 reply
10/19dmaengine: expose per channel dma mapping characteristics to clientsdrivers/dma/dmaengine.c drivers/dma/ioatdma.c include/linux/dmaengine.h+73 -00 replies
11/19dmaengine: add memset as an asynchronous dma operationdrivers/dma/dmaengine.c drivers/dma/ioatdma.c include/linux/dmaengine.h+88 -01 reply
12/19dmaengine: dma_async_memcpy_err for DMA engines that do not support memcpydrivers/dma/dmaengine.c+13 -01 reply
13/19dmaengine: add support for dma xor zero sum operationsdrivers/dma/dmaengine.c drivers/dma/ioatdma.c include/linux/dmaengine.h+77 -00 replies
14/19dmaengine: add dma_sync_waitinclude/linux/dmaengine.h+12 -01 reply
15/19dmaengine: raid5 dma clientdrivers/dma/Kconfig drivers/dma/Makefile drivers/dma/raid5-dma.c +4 more+839 -71 reply
16/19dmaengine: Driver for the Intel IOP 32x, 33x, and 13xx RAID enginesdrivers/dma/Kconfig drivers/dma/Makefile drivers/dma/iop-adma.c +1 more+1624 -31 reply
17/19iop3xx: define IOP3XX_REG_ADDR[32|16|8] and clean up DMA/AAU defsinclude/asm-arm/arch-iop32x/entry-macro.S include/asm-arm/arch-iop32x/iop32x.h include/asm-arm/arch-iop33x/entry-macro.S +2 more+188 -2151 reply
18/19iop3xx: Give Linux control over PCI (ATU) initializationarch/arm/mach-iop32x/Kconfig arch/arm/mach-iop32x/ep80219.c arch/arm/mach-iop32x/iq31244.c +10 more+214 -121 reply
19/19iop3xx: IOP 32x and 33x support for the iop-adma driverarch/arm/mach-iop32x/iq80321.c arch/arm/mach-iop33x/iq80331.c arch/arm/mach-iop33x/iq80332.c +4 more+1201 -00 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [PATCH 00/19] Hardware Accelerated MD RAID5: Introduction

Neil,

The following patches implement hardware accelerated raid5 for the Intel Xscale® series of I/O Processors. The MD changes allow stripe operations to run outside the spin lock in a work queue. Hardware acceleration is achieved by using a dma-engine-aware work queue routine instead of the default software only routine.

Since the last release of the raid5 changes many bug fixes and other improvements have been made as a result of stress testing. See the per patch change logs for more information about what was fixed. This release is the first release of the full dma implementation.

The patches touch 3 areas, the md-raid5 driver, the generic dmaengine interface, and a platform device driver for IOPs. The raid5 changes follow your comments concerning making the acceleration implementation similar to how the stripe cache handles I/O requests. The dmaengine changes are the second release of this code. They expand the interface to handle more than memcpy operations, and add a generic raid5-dma client. The iop-adma driver supports dma memcpy, xor, xor zero sum, and memset across all IOP architectures (32x, 33x, and 13xx).

Concerning the context switching performance concerns raised at the previous release, I have observed the following. For the hardware accelerated case it appears that performance is always better with the work queue than without since it allows multiple stripes to be operated on simultaneously. I expect the same for an SMP platform, but so far my testing has been limited to IOPs. For a single-processor non-accelerated configuration I have not observed performance degradation with work queue support enabled, but in the Kconfig option help text I recommend disabling it (CONFIG_MD_RAID456_WORKQUEUE).

Please consider the patches for -mm.

-Dan

[PATCH 01/19] raid5: raid5_do_soft_block_ops
[PATCH 02/19] raid5: move write operations to a workqueue
[PATCH 03/19] raid5: move check parity operations to a workqueue
[PATCH 04/19] raid5: move compute block operations to a workqueue
[PATCH 05/19] raid5: move read completion copies to a workqueue
[PATCH 06/19] raid5: move the reconstruct write expansion operation to a workqueue [PATCH 07/19] raid5: remove compute_block and compute_parity5
[PATCH 08/19] dmaengine: enable multiple clients and operations
[PATCH 09/19] dmaengine: reduce backend address permutations
[PATCH 10/19] dmaengine: expose per channel dma mapping characteristics to clients [PATCH 11/19] dmaengine: add memset as an asynchronous dma operation [PATCH 12/19] dmaengine: dma_async_memcpy_err for DMA engines that do not support memcpy [PATCH 13/19] dmaengine: add support for dma xor zero sum operations [PATCH 14/19] dmaengine: add dma_sync_wait
[PATCH 15/19] dmaengine: raid5 dma client
[PATCH 16/19] dmaengine: Driver for the Intel IOP 32x, 33x, and 13xx RAID engines [PATCH 17/19] iop3xx: define IOP3XX_REG_ADDR[32|16|8] and clean up DMA/AAU defs [PATCH 18/19] iop3xx: Give Linux control over PCI (ATU) initialization [PATCH 19/19] iop3xx: IOP 32x and 33x support for the iop-adma driver

Note, the iop3xx patches apply against the iop3xx platform code re-factoring done by Lennert Buytenhek. His patches are reproduced, with permission, on the Xscale IOP SourceForge site.

Also available on SourceForge:

Linux Symposium Paper: MD RAID Acceleration Support for Asynchronous DMA/XOR Engines
http://prdownloads.sourceforge.net/xscaleiop/ols_paper_2006.pdf?download

Tar archive of the patch set
http://prdownloads.sourceforge.net/xscaleiop/md_raid_accel-2.6.18-rc6.tar.gz?download

[PATCH 01/19] http://prdownloads.sourceforge.net/xscaleiop/md-add-raid5-do-soft-block-ops.patch?download
[PATCH 02/19] http://prdownloads.sourceforge.net/xscaleiop/md-move-write-operations-to-a-workqueue.patch?download
[PATCH 03/19] http://prdownloads.sourceforge.net/xscaleiop/md-move-check-parity-operations-to-a-workqueue.patch?download
[PATCH 04/19] http://prdownloads.sourceforge.net/xscaleiop/md-move-compute-block-operations-to-a-workqueue.patch?download
[PATCH 05/19] http://prdownloads.sourceforge.net/xscaleiop/md-move-read-completion-copies-to-a-workqueue.patch?download
[PATCH 06/19] http://prdownloads.sourceforge.net/xscaleiop/md-move-expansion-operations-to-a-workqueue.patch?download
[PATCH 07/19] http://prdownloads.sourceforge.net/xscaleiop/md-remove-compute_block-and-compute_parity5.patch?download
[PATCH 08/19] http://prdownloads.sourceforge.net/xscaleiop/dmaengine-multiple-clients-and-multiple-operations.patch?download
[PATCH 09/19] http://prdownloads.sourceforge.net/xscaleiop/dmaengine-unite-backend-address-types.patch?download
[PATCH 10/19] http://prdownloads.sourceforge.net/xscaleiop/dmaengine-dma-async-map-page.patch?download
[PATCH 11/19] http://prdownloads.sourceforge.net/xscaleiop/dmaengine-dma-async-memset.patch?download
[PATCH 12/19] http://prdownloads.sourceforge.net/xscaleiop/dmaengine-dma-async-memcpy-err.patch?download
[PATCH 13/19] http://prdownloads.sourceforge.net/xscaleiop/dmaengine-dma-async-zero-sum.patch?download
[PATCH 14/19] http://prdownloads.sourceforge.net/xscaleiop/dmaengine-dma-sync-wait.patch?download
[PATCH 15/19] http://prdownloads.sourceforge.net/xscaleiop/md-raid5-dma-client.patch?download
[PATCH 16/19] http://prdownloads.sourceforge.net/xscaleiop/iop-adma-device-driver.patch?download
[PATCH 17/19] http://prdownloads.sourceforge.net/xscaleiop/iop3xx-register-macro-cleanup.patch?download
[PATCH 18/19] http://prdownloads.sourceforge.net/xscaleiop/iop3xx-pci-initialization.patch?download
[PATCH 19/19] http://prdownloads.sourceforge.net/xscaleiop/iop3xx-adma-support.patch?download

Optimal performance on IOPs is obtained with:
CONFIG_MD_RAID456_WORKQUEUE=y
CONFIG_MD_RAID5_HW_OFFLOAD=y
CONFIG_RAID5_DMA=y
CONFIG_INTEL_IOP_ADMA=y
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html