Poor write performance on software raid with 512k chunk size

Hello together,

i experienced severe performance problems when using a crypt mapping on a software raid 5 with 512k chunk size. Without a crypt mapping the write speed was ~100M/sec, with the crypt mapping it had only ~20M/sec. After is rebuild the raid on the same disks using 64k chunk size, the write performance with the crypt mapping is now ~90M/sec. I am using a self compiled 3.2.6 vanilla kernel on a debian squeeze with an AMD Phenom(tm) II X4 955 processor.

For now reducing the chunk size solved the problem for me but 512k chunk size was chosen as the default by mdadm so it would be nice to find the reason for this performance bottleneck.

Greets,
Dennis

As we've mentioned to others, try increasing
/sys/block/<devices>/queue/nr_requests
significantly (e.g. see if 50,000 has any impact).

Or experiment with these patches:
http://people.redhat.com/mpatocka/patches/kernel/dm-crypt-paralelizace/current/
(at your own risk, of course).

Alasdair

Thanks for the fast reply.

Am 26.02.2012 14:37, schrieb Alasdair G Kergon:

As I mentioned before, I chose a smaller chunk size for my RAID to solve this problem. As I am not interested in rebuilding a 6TB RAID again, I cannot verify if your suggestions increase write throughput with greater chunk size.

I would suggest that the device mapper should set default values that give maximum performance for the default values other kernel subsystems use. As the RAID subsystem seems to use 512k chunks by default, the device mapper should use settings by default that allow maximum performance for this chunk size.

Greets,
Dennis