reiserfs3/ext4/btrfs RAID read performance
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 13 lines · On Sep 20, 11:50 am, [email protected] wrote:
On Sun, Sep 20, 2009 at 3:47 AM, Daniel J Blueman
Quoted 10 lines · <[email protected]> wrote:
Quoted 8 lines · On Sep 19, 7:20 pm, [email protected] wrote:
RAID details:
md8 : active raid10 sda7[0] sdd7[3] sdc7[2] sdb7[1]
62925824 blocks 256K chunks 2 far-copies [4/4] [UUUU]Ext4:
mkfs.ext4 -E stride=64,stripe-width=128/dev/md8
mount -t ext4 -o noatime,auto_da_alloc,commit=600/dev/md8/mnt/md8
Here, stripe-width should be 4* stride, not that it'll make much difference.
Quoted 44 lines
Quoted 30 lines
Quoted 17 lines
Reiser3:
mount -t reiserfs/dev/md8/mnt/md8
mount -t reiserfs -o noatime,notail/dev/md8/dev/md8Ext4 results:
intial create total runs 10 avg 172.76 MB/s (user 0.43s sys 0.60s) create total runs 14 avg 36.49 MB/s (user 0.42s sys 0.59s) patch total runs 15 avg 15.16 MB/s (user 0.24s sys 0.49s) compile total runs 14 avg 64.07 MB/s (user 0.10s sys 0.59s) clean total runs 10 avg 393.43 MB/s (user 0.02s sys 0.06s) read tree total runs 11 avg 20.47 MB/s (user 0.53s sys 0.74s) read compiled tree total runs 4 avg 32.94 MB/s (user 0.61s sys 1.17s) delete tree total runs 10 avg 2.51 seconds (user 0.24s sys 0.42s) delete compiled tree total runs 4 avg 2.63 seconds (user 0.28s sys 0.50s) stat tree total runs 11 avg 1.99 seconds (user 0.23s sys 0.18s) stat compiled tree total runs 7 avg 2.11 seconds (user 0.27s sys 0.21s)Quoted 13 lines
Reiser3 results:
intial create total runs 10 avg 82.74 MB/s (user 0.45s sys 1.13s) create total runs 14 avg 28.54 MB/s (user 0.45s sys 1.19s) patch total runs 15 avg 10.91 MB/s (user 0.24s sys 0.86s) compile total runs 14 avg 47.49 MB/s (user 0.10s sys 1.27s) clean total runs 10 avg 270.21 MB/s (user 0.02s sys 0.15s) read tree total runs 11 avg 26.33 MB/s (user 0.54s sys 0.81s) read compiled tree total runs 4 avg 41.82 MB/s (user 0.62s sys 1.36s) delete tree total runs 10 avg 3.38 seconds (user 0.24s sys 0.72s) delete compiled tree total runs 4 avg 4.14 seconds (user 0.27s sys 0.88s) stat tree total runs 11 avg 2.09 seconds (user 0.22s sys 0.18s) stat compiled tree total runs 7 avg 2.27 seconds (user 0.25s sys 0.21s)Quoted 2 lines
It would be interesting to also compare against BTRFS if on 2.6.30 or newer, if you can.
BTRFS 2.6.31
mkfs.btrfs -d raid10 -m raid10
/dev/sda7/dev/sdb7/dev/sdc7/dev/sdd7mount -t btrfs -o noatime/dev/sda7/mnt/md8intial create total runs 10 avg 158.85 MB/s (user 0.45s sys 0.93s) create total runs 14 avg 32.67 MB/s (user 0.44s sys 0.90s) patch total runs 15 avg 8.91 MB/s (user 0.22s sys 0.84s) compile total runs 14 avg 61.02 MB/s (user 0.09s sys 0.50s) clean total runs 10 avg 245.12 MB/s (user 0.02s sys 0.18s)
Quoted 7 lines
read tree total runs 11 avg 14.03 MB/s (user 0.48s sys 0.87s) read compiled tree total runs 4 avg 29.14 MB/s (user 0.54s sys 1.37s)
delete tree total runs 10 avg 9.77 seconds (user 0.28s sys 1.37s) delete compiled tree total runs 4 avg 11.91 seconds (user 0.31s sys 1.60s) stat tree total runs 11 avg 4.36 seconds (user 0.25s sys 0.33s) stat compiled tree total runs 7 avg 5.29 seconds (user 0.29s sys 0.37s)
Not ext4 specific, but I was finding 64KB chunk-size RAID 10 (layout f2 if using MD) and increasing readahead of /dev/mdX to drives*256 to give the best performance on a 4-drive SATA array. Consider using aligned partitions (ext4 has internal alignment, I don't think BTRFS does) and at the outside of identical disks if using non-SSDs.
Thanks,
Daniel
--
Daniel J Blueman