Skip to contentSkip to conversation

Looking for advice on creating XFS on a large (to me) RAID

DISCUSSION Krzysztof Adamski 2 participants 3 messages Nov 3, 2011 VFS

lore
Share & tools
lore
#0 lore raw

Hi All,

I'm about to create a XFS file system on a MD RAID6 made up of 12 3T drives.

This is the command line that I was planning to use:

# mkfs.xfs  -l lazy-count=1 -s size=4096 -N /dev/md7
         =                       sectsz=4096  attr=2, projid32bit=0
data     =                       bsize=4096   blocks=7320776704,imaxpct=5
         =                       sunit=128    swidth=1280 blks
naming   =version 2              bsize=4096   ascii-ci=0
log      =internal log           bsize=4096   blocks=521728, version=2
         =                       sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

This is the MD examine for one of the drives:

Nitrogen:~# mdadm -E /dev/sda2
/dev/sda2:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x1
     Array UUID : ee8cb70e:2f40f34f:79d46ade:4b12c78b
  Creation Time : Tue Nov  1 17:03:38 2011
     Raid Level : raid6
   Raid Devices : 12

Avail Dev Size : 5856624640 (2792.66 GiB 2998.59 GB)

     Array Size : 58566236160 (27926.56 GiB 29985.91 GB)

Used Dev Size : 5856623616 (2792.66 GiB 2998.59 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
State : clean
Device UUID : 72fa45ca:775f244d:15b030fe:f569fe8c

Internal Bitmap : 8 sectors from superblock
Update Time : Thu Nov 3 11:07:39 2011
Checksum : 4c72bfd4 - correct
Events : 11029

Layout : left-symmetric
Chunk Size : 512K

Device Role : Active device 0
Array State : AAAAAAAAAAAA ('A' == active, '.' == missing)

# mkfs.xfs -V
mkfs.xfs version 3.1.4

System is Debian Squeeze with self compiled 3.0.8 kernel (64bit)

Should I change any default parameters to mkfs.xfs

If I didn't provide any important info let me know.

Thanks in advance,
K

#1 lore raw
Quoted 10 lines · Krzysztof Adamski

Hi All,

I'm about to create a XFS file system on a MD RAID6 made up of 12 3T drives.

This is the command line that I was planning to use:

# mkfs.xfs

-l lazy-count=1

This is the default in any semi-recent xfsprogs version.

Quoted 1 line

-s size=4096

using a sector size larger than the actual disk sector size might risk data loss, as the log code expects sector sized writes to be atomic.

If you have a real 4k logical sector disk it should be picked up automatically with the kernel topology support. We don't quite do the right thing for 512 byte logical / 4k physical disks, in which case you might need this line.

Quoted 1 line

-N

That won't actually create you a filesystem :)

Quoted 1 line

= sunit=128 swidth=1280 blks

In general you're better off with a 32k stripe unit / chunk size, than the extremly large default of 512k.

Quoted 1 line

Should I change any default parameters to mkfs.xfs

Usually not.

#2 lore raw
Quoted 10 lines · Christoph Hellwig
Quoted 8 lines · Krzysztof Adamski

Hi All,

I'm about to create a XFS file system on a MD RAID6 made up of 12 3T drives.

This is the command line that I was planning to use:

-s size=4096

Quoted 7 lines

using a sector size larger than the actual disk sector size might risk data loss, as the log code expects sector sized writes to be atomic.

If you have a real 4k logical sector disk it should be picked up automatically with the kernel topology support. We don't quite do the right thing for 512 byte logical / 4k physical disks, in which case you might need this line.

The drives I'm using are Hitachi H3IK30003272SW (0S03208) 3TB, I belive they are the 512 byte logical / 4k physical disks.

Quoted 4 lines
Quoted 1 line

= sunit=128 swidth=1280 blks

In general you're better off with a 32k stripe unit / chunk size, than the extremly large default of 512k.

I'll compare a simple dd benchmark of the raw md device with 32k and 512k and report back. The resync takes over 23 hours :-(

Thanks for the info.
K