Why is my LVM Mirror performance so bad?
Hi There. Hoping to get some clarity on some performance woes in a little testing environment that I've setup.
I am experimenting with LVM mirrors on a CentOS 6.5 cluster.
I have two VMs running on ESX5.5 that are sharing two RDM luns from an SRP target server running SCST.
Performance however seems to take a huge hit compared to a vanilla volume
[root@scst1 ~]# vgs
VG #PV #LV #SN Attr VSize VFree
hadata 2 5 0 wz--nc 999.99g 149.99g vg_scst1 1 2 0 wz--n- 39.51g 0
[root@scst1 ~]# lvs -a -o +devices
LV VG Attr LSize Pool Origin Data% Move Log Cpy%Sync Convert Devices mirror1 hadata mwi-a-m--- 100.00g 100.00 mirror1_mimage_0(0),mirror1_mimage_1(0) [mirror1_mimage_0] hadata iwi-aom--- 100.00g /dev/sdb1(0) [mirror1_mimage_1] hadata iwi-aom--- 100.00g /dev/sdc1(0) mirror2 hadata mwi-a-m--- 100.00g 100.00 mirror2_mimage_0(0),mirror2_mimage_1(0) [mirror2_mimage_0] hadata iwi-aom--- 100.00g /dev/sdb1(25600) [mirror2_mimage_1] hadata iwi-aom--- 100.00g /dev/sdc1(25600) mirror3 hadata mwi-a-m--- 100.00g 100.00 mirror3_mimage_0(0),mirror3_mimage_1(0) [mirror3_mimage_0] hadata iwi-aom--- 100.00g /dev/sdb1(51200) [mirror3_mimage_1] hadata iwi-aom--- 100.00g /dev/sdc1(51200) mirror4 hadata mwi-a-m--- 100.00g 100.00 mirror4_mimage_0(0),mirror4_mimage_1(0) [mirror4_mimage_0] hadata iwi-aom--- 100.00g /dev/sdb1(76800) [mirror4_mimage_1] hadata iwi-aom--- 100.00g /dev/sdc1(76800) test hadata -wi-a----- 50.00g /dev/sdb1(102400) lv_root vg_scst1 -wi-ao---- 31.65g /dev/sda2(0) lv_swap vg_scst1 -wi-ao---- 7.86g /dev/sda2(8102)
[root@scst1 ~]# dd if=/dev/sdb1 of=/dev/null bs=1M count=20000 20000+0 records in
20000+0 records out
20971520000 bytes (21 GB) copied, 41.0031 s, 511 MB/s
[root@scst1 ~]# dd if=/dev/hadata/test of=/dev/null bs=1M count=20000 20000+0 records in
20000+0 records out
20971520000 bytes (21 GB) copied, 44.344 s, 473 MB/s
[root@scst1 ~]# dd if=/dev/hadata/mirror1 of=/dev/null bs=1M count=20000 20000+0 records in
20000+0 records out
20971520000 bytes (21 GB) copied, 365.685 s, 57.3 MB/s
Is this just the life of a LVM mirror and the performance here is about as good as it gets?
Because it is LVM! :)
Quoted 75 lines · Errol Neal
Hi There. Hoping to get some clarity on some performance woes in a little testing environment that I've setup.
I am experimenting with LVM mirrors on a CentOS 6.5 cluster.
I have two VMs running on ESX5.5 that are sharing two RDM luns from an SRP target server running SCST.
Performance however seems to take a huge hit compared to a vanilla volume
[root@scst1 ~]# vgs
VG #PV #LV #SN Attr VSize VFreehadata 2 5 0 wz--nc 999.99g 149.99g vg_scst1 1 2 0 wz--n- 39.51g 0[root@scst1 ~]# lvs -a -o +devices
LV VG Attr LSize Pool Origin Data% Move
Log Cpy%Sync Convert Devices
mirror1 hadata mwi-a-m--- 100.00g100.00 mirror1_mimage_0(0),mirror1_mimage_1(0) [mirror1_mimage_0] hadata iwi-aom--- 100.00g /dev/sdb1(0) [mirror1_mimage_1] hadata iwi-aom--- 100.00g /dev/sdc1(0) mirror2 hadata mwi-a-m--- 100.00g 100.00 mirror2_mimage_0(0),mirror2_mimage_1(0) [mirror2_mimage_0] hadata iwi-aom--- 100.00g /dev/sdb1(25600) [mirror2_mimage_1] hadata iwi-aom--- 100.00g /dev/sdc1(25600) mirror3 hadata mwi-a-m--- 100.00g 100.00 mirror3_mimage_0(0),mirror3_mimage_1(0) [mirror3_mimage_0] hadata iwi-aom--- 100.00g /dev/sdb1(51200) [mirror3_mimage_1] hadata iwi-aom--- 100.00g /dev/sdc1(51200) mirror4 hadata mwi-a-m--- 100.00g 100.00 mirror4_mimage_0(0),mirror4_mimage_1(0) [mirror4_mimage_0] hadata iwi-aom--- 100.00g /dev/sdb1(76800) [mirror4_mimage_1] hadata iwi-aom--- 100.00g /dev/sdc1(76800) test hadata -wi-a----- 50.00g /dev/sdb1(102400) lv_root vg_scst1 -wi-ao---- 31.65g /dev/sda2(0) lv_swap vg_scst1 -wi-ao---- 7.86g /dev/sda2(8102)[root@scst1 ~]# dd if=/dev/sdb1 of=/dev/null bs=1M count=20000 20000+0 records in
20000+0 records out
20971520000 bytes (21 GB) copied, 41.0031 s, 511 MB/s[root@scst1 ~]# dd if=/dev/hadata/test of=/dev/null bs=1M count=20000 20000+0 records in
20000+0 records out
20971520000 bytes (21 GB) copied, 44.344 s, 473 MB/s[root@scst1 ~]# dd if=/dev/hadata/mirror1 of=/dev/null bs=1M count=20000 20000+0 records in
20000+0 records out
20971520000 bytes (21 GB) copied, 365.685 s, 57.3 MB/sIs this just the life of a LVM mirror and the performance here is about as good as it gets?
_______________________________________________
linux-lvm mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Dne 27.9.2014 v 22:35 Micky napsal(a):
Quoted 2 lines
Because it is LVM! :)
And that's what I call 'Very helpful answer'
Quoted 8 lines
On Sat, Sep 27, 2014 at 10:42 PM, Errol Neal <[email protected] <mailto:[email protected]>> wrote:
Hi There. Hoping to get some clarity on some performance woes in a little testing environment that I've setup.
I am experimenting with LVM mirrors on a CentOS 6.5 cluster.
Are you using 'exclusive' single node activation - or is your mirror active on multiple nodes with cmirrord ?
Quoted 6 lines
I have two VMs running on ESX5.5 that are sharing two RDM luns from an SRP target server running SCST.
Performance however seems to take a huge hit compared to a vanilla volume
Quoted 6 lines
[root@scst1 ~]# vgs
VG #PV #LV #SN Attr VSize VFree
hadata 2 5 0 wz--nc 999.99g 149.99g
vg_scst11 2 0 wz--n- 39.51g 0[root@scst1 ~]# lvs -a -o +devices
lvs -a -o+lv_active
If you don't need cluster-wide mirror activation - use the local version:
lvchange -aey vg/mirror
Note - new mdraid is supported only in 'exlusive' mode.
Zdenek
Quoted 3 lines · Errol Neal
Is this just the life of a LVM mirror and the performance here is about as good as it gets?
----
It's a bit confusing from what you have above, but *usually*, if you want performance, you set up 1 duplicate device (disk) to duplicate another of the same type. That way when the controller is writing to 1, it will be able to write to the other in the same location and expect *similar* speeds (speeds will vary more on desktop drives vs. enterprise, so overall RAID performance is lost).
From what it looks like you have multiple segments mirroring other (parallel?? not sure) segments... but as data is being written to one segment, it also may not have finished writing a parallel copy to another segment.
Don't setup multiple segments mirroring each other on 2 devices... have 1 be the mirror of the other and both be of similar type if you want near device speed.
i.e. you don't want 4 mirrors, but want to make all of sdb mirror all of sdc.
Quoted 42 lines · Errol Neal
Hi There. Hoping to get some clarity on some performance woes in a little testing environment that I've setup.
I am experimenting with LVM mirrors on a CentOS 6.5 cluster.
I have two VMs running on ESX5.5 that are sharing two RDM luns from an SRP target server running SCST.
Performance however seems to take a huge hit compared to a vanilla volume
[root@scst1 ~]# vgs
VG #PV #LV #SN Attr VSize VFreehadata 2 5 0 wz--nc 999.99g 149.99g vg_scst1 1 2 0 wz--n- 39.51g 0[root@scst1 ~]# lvs -a -o +devices
LV VG Attr LSize Pool Origin Data% Move Log Cpy%Sync Convert Devices mirror1 hadata mwi-a-m--- 100.00g 100.00 mirror1_mimage_0(0),mirror1_mimage_1(0) [mirror1_mimage_0] hadata iwi-aom--- 100.00g /dev/sdb1(0) [mirror1_mimage_1] hadata iwi-aom--- 100.00g /dev/sdc1(0) mirror2 hadata mwi-a-m--- 100.00g 100.00 mirror2_mimage_0(0),mirror2_mimage_1(0) [mirror2_mimage_0] hadata iwi-aom--- 100.00g /dev/sdb1(25600) [mirror2_mimage_1] hadata iwi-aom--- 100.00g /dev/sdc1(25600) mirror3 hadata mwi-a-m--- 100.00g 100.00 mirror3_mimage_0(0),mirror3_mimage_1(0) [mirror3_mimage_0] hadata iwi-aom--- 100.00g /dev/sdb1(51200) [mirror3_mimage_1] hadata iwi-aom--- 100.00g /dev/sdc1(51200) mirror4 hadata mwi-a-m--- 100.00g 100.00 mirror4_mimage_0(0),mirror4_mimage_1(0) [mirror4_mimage_0] hadata iwi-aom--- 100.00g /dev/sdb1(76800) [mirror4_mimage_1] hadata iwi-aom--- 100.00g /dev/sdc1(76800) test hadata -wi-a----- 50.00g /dev/sdb1(102400) lv_root vg_scst1 -wi-ao---- 31.65g /dev/sda2(0) lv_swap vg_scst1 -wi-ao---- 7.86g /dev/sda2(8102)[root@scst1 ~]# dd if=/dev/sdb1 of=/dev/null bs=1M count=20000 20000+0 records in
20000+0 records out
20971520000 bytes (21 GB) copied, 41.0031 s, 511 MB/s[root@scst1 ~]# dd if=/dev/hadata/test of=/dev/null bs=1M count=20000 20000+0 records in
20000+0 records out
20971520000 bytes (21 GB) copied, 44.344 s, 473 MB/s
Quoted 7 lines
[root@scst1 ~]# dd if=/dev/hadata/mirror1 of=/dev/null bs=1M count=20000 20000+0 records in
20000+0 records out
20971520000 bytes (21 GB) copied, 365.685 s, 57.3 MB/sIs this just the life of a LVM mirror and the performance here is about as good as it gets?
Is the performance also degraded in this way when using single machine (i.e. not cluster)?
brassow