dm-crypt and huge performance penalty
Hi,
I've just purchased three HDD drives and tried to setup mdadm RAID5 + dm-crypt.
What I've found is that dm-crypt reduces storage I/O performance very much while CPU is not bottleneck.
Firstly I've tried to setup raid5 and then dm-crypt on top of it. After discovering issue I've tried just raid5 and confirmed that mdadm itself works as expected.
And the last idea is to setup three dm-crypt partitions and then raid5 on top of independently encrypted drives.
So here are my results:
| mdadm | mdadm+dm-crypt | 3 x dm-crypt + mdadm | Seq. read | 168 Mb/s | 57 Mb/s | 119 Mb/s | Seq. write | 80 Mb/s | 36 Mb/s | 64.4 Mb/s |
Also here are some results using both dd and bonnie as benchmark:
http://pastebin.com/hwwkRRTd
I'm using 3 x WD1002FAEX-00Z3A0 HDD. SATA controller is just embedded to m/b:
00:1f.2 IDE interface: Intel Corporation 5 Series/3400 Series Chipset 4 port SATA IDE Controller (rev 06)
00:1f.5 IDE interface: Intel Corporation 5 Series/3400 Series Chipset 2 port SATA IDE Controller (rev 06)
dd tests were done using simple dd:
if=/dev/resulting-device of=/dev/zero bs=1m count=512
For Bonnie I've used 50GB reiserfs3 partition on top of resulting device.
Are these numbers expected? They looks really strange for me.
Signature · 1 line
WBR, Dmitry
Quoted 4 lines · Dmitry Nezhevenko
Firstly I've tried to setup raid5 and then dm-crypt on top of it. After discovering issue I've tried just raid5 and confirmed that mdadm itself works as expected.
Quoted 8 lines
And the last idea is to setup three dm-crypt partitions and then raid5 on top of independently encrypted drives.
So here are my results:
| mdadm | mdadm+dm-crypt | 3 x dm-crypt + mdadm | Seq. read | 168 Mb/s | 57 Mb/s | 119 Mb/s | Seq. write | 80 Mb/s | 36 Mb/s | 64.4 Mb/s |
dm-crypt will only use one core per device, so this is expected behavior (and I believe an issue that is being worked on). From these numbers it looks like you have a dual core or dual cpu system?
Signature · 1 line
Ryan Castellucci http://ryanc.org/
Quoted 10 lines · Ryan Castellucci
Quoted 9 lines · Dmitry Nezhevenko
Firstly I've tried to setup raid5 and then dm-crypt on top of it. After discovering issue I've tried just raid5 and confirmed that mdadm itself works as expected.
And the last idea is to setup three dm-crypt partitions and then raid5 on top of independently encrypted drives.
So here are my results:
Quoted 8 lines
Quoted 3 lines
| mdadm | mdadm+dm-crypt | 3 x dm-crypt + mdadm | Seq. read | 168 Mb/s | 57 Mb/s | 119 Mb/s | Seq. write | 80 Mb/s | 36 Mb/s | 64.4 Mb/s |dm-crypt will only use one core per device, so this is expected behavior (and I believe an issue that is being worked on). From these numbers it looks like you have a dual core or dual cpu system?
Hi, Yes, The machine is home NAS with inexpensive Intel Core i3 CPU that has two real cores and HT.
Signature · 1 line
WBR, Dmitry