Missing capacity in a RAID 5 array?
I've created a software RAID 5 array consisting of 8 160 GB drives. One of the drives had prior data I wanted to keep, so I first created a 7-drive array, copied the data from the original drive to the array, then added the 8th drive using raidreconf. Worked like a charm (took about 27 hours, though!)
The OS now reports that the total capacity on /dev/md0 is about 920 GB:
/dev/md0 923029884 172872252 750157632 19% /mnt/store
Now, even if we take into account that the drives are 160 decimal GB, which equates to 149 binary GB, my array should still have a total capacity somewhere around 7 x 149 = 1043 GB.
Am I really losing 120 GB to reserved space for RAID and/or filesystem accounting? I feel like I'm missing something obvious here. Can I get my full terabyte somehow?
Here's my /etc/raidtab:
raiddev /dev/md0
raid-level 5
nr-raid-disks 8
nr-spare-disks 0
persistent-superblock 1
parity-algorithm left-symmetric
chunk-size 32
device /dev/hdc
raid-disk 0
device /dev/hdd
raid-disk 1
device /dev/hde
raid-disk 2
device /dev/hdg
raid-disk 3
device /dev/hdh
raid-disk 4
device /dev/hdi
raid-disk 5
device /dev/hdk
raid-disk 6
device /dev/hdf
raid-disk 7
And here's my /proc/mdstat:
Personalities : [raid5]
md0 : active raid5 hdf[7] hdk[6] hdi[5] hdh[4] hdg[3] hde[2] hdd[1] hdc[0]
1094035712 blocks level 5, 32k chunk, algorithm 2 [8/8] [UUUUUUUU]
unused devices: <none>
Thanks for the help.
Jeff Morrow
Quoted 9 lines · Jeff Morrow
I've created a software RAID 5 array consisting of 8 160 GB drives. One of the drives had prior data I wanted to keep, so I first created a 7-drive array, copied the data from the original drive to the array, then added the 8th drive using raidreconf. Worked like a charm (took about 27 hours, though!)
The OS now reports that the total capacity on
/dev/md0is about 920 GB:
/dev/md0923029884 172872252 750157632 19%/mnt/storeNow, even if we take into account that the drives are 160 decimal GB, which equates to 149 binary GB, my array should still have a total capacity somewhere around 7 x 149 = 1043 GB.
Am I really losing 120 GB to reserved space for RAID and/or filesystem accounting? I feel like I'm missing something obvious here. Can I get my full terabyte somehow?
What filesystem are you using and how much space does it reserve for Root?
Plus, how much space does the filesystem metadata take up? Perhaps reduce the number of inodes to reclaim some space (Can't do that once there is data on there though)
On my 2.1TB I told mke2fs not to reserve any space for root as I was going to lose about 105MB.
Regards,
Brad
Quoted 2 lines · Brad Campbell
On my 2.1TB I told mke2fs not to reserve any space for root as I was going to lose about 105MB [GB?].
As you fill up this filesystem, you're going to see decreased performance, as the reserved space isn't just "insurance" for root processes, it's also essential for managing fragmentation. This doesn't apply, as much, if you are just adding new files, without ever modifying or deleting the old ones, or if you are working with a static file set altogether.
Quoted 9 lines · Jeff Morrow
I've created a software RAID 5 array consisting of 8 160 GB drives. One of the drives had prior data I wanted to keep, so I first created a 7-drive array, copied the data from the original drive to the array, then added the 8th drive using raidreconf. Worked like a charm (took about 27 hours, though!)
The OS now reports that the total capacity on
/dev/md0is about 920 GB:
/dev/md0923029884 172872252 750157632 19%/mnt/store
Did you grow the filesystem after you grew the raid device? If you used ext2s you should look at resize2fs.
Signature · 1 line
Mikael Abrahamsson email: [email protected]
Quoted 1 line
Did you grow the filesystem after you grew the raid device? If you used ext2s you should look at resize2fs.
Yep, that did it. Many thanks for the quick responses!
Jeff
Quoted 10 lines · Jeff Morrow
I've created a software RAID 5 array consisting of 8 160 GB drives. One of the drives had prior data I wanted to keep, so I first created a 7-drive array, copied the data from the original drive to the array, then added the 8th drive using raidreconf. Worked like a charm (took about 27 hours, though!)
The OS now reports that the total capacity on
/dev/md0is about 920 GB:
/dev/md0923029884 172872252 750157632 19%/mnt/store
Quoted 7 lines
Now, even if we take into account that the drives are 160 decimal GB, which equates to 149 binary GB, my array should still have a total capacity somewhere around 7 x 149 = 1043 GB.
Am I really losing 120 GB to reserved space for RAID and/or filesystem accounting? I feel like I'm missing something obvious here. Can I get my full terabyte somehow?
So you created a RAID5 array out of 7 disk. The capacity of a RAID5 array of equally sized disk can be calculated like this:total_size = (n -1) * single_disk_size
which calculates to a total size of 894MB - 960MB (depending on how you calculate a MegaByte).
The extra disk that you added is most certainly now seen as a spare disk which adds now capacity to the array, just a instant replacement in case of failures.
What you should have done:
Create a degraded RAID5 array out of 8 drives with the 8th one marked as missing/failed. Then copy all the data from the 8th drive to the array. After this hot-add the 8th drive to degraded array and let it rebuild.
Regards,
Juri
Signature · 1 line
Juri Haberland <[email protected]>