Change sector size on existing partition

Hi,

I'm new to XFS, I have RAID-10 array with 4 disk, when I check with xfs_info, the information print like this.

$ xfs_info /var/lib/mysql
meta-data=/dev/mapper/vg_catalystdb01-lv_database isize=256
agcount=16, agsize=1600000 blks
         =                       sectsz=4096  attr=2, projid32bit=0
data     =                       bsize=4096   blocks=25600000, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0
log      =internal               bsize=4096   blocks=12500, version=2
         =                       sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

Is it possible to change sectsz value to 512 without re-format it? Or any suggestion? I have issue with current sector size, my TokuDB engines[1] can't start because of this.

[1] https://groups.google.com/forum/#!topic/tokudb-user/kvQFJLCmKwo

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

Quoted 16 lines · Dewangga Bachrul Alam

Hi,

I'm new to XFS, I have RAID-10 array with 4 disk, when I check with xfs_info, the information print like this.

$ xfs_info /var/lib/mysql
meta-data=/dev/mapper/vg_catalystdb01-lv_database isize=256
agcount=16, agsize=1600000 blks
         =                       sectsz=4096  attr=2, projid32bit=0
data     =                       bsize=4096   blocks=25600000, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0
log      =internal               bsize=4096   blocks=12500, version=2
         =                       sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

Quoted 4 lines

Is it possible to change sectsz value to 512 without re-format it? Or any suggestion? I have issue with current sector size, my TokuDB engines[1] can't start because of this.

The only way to set things like sector size, block size, etc. is to reformat. I believe the default sector size is dependent on the physical device. You might want to report the following from your array device and perhaps from some or all of the member devices:

blockdev --getss --getpbsz --getiomin --getioopt <device>

Brian

Quoted 6 lines

[1] https://groups.google.com/forum/#!topic/tokudb-user/kvQFJLCmKwo

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

Hi,

The device reported like this :

$ blockdev --getss --getpbsz --getiomin --getioopt /dev/dm-3
512
4096
4096
0

Then, the sector size should be 512. Don't know why it can be 4096. :( I will try to backup them and reformat. Any suggestion for formating on raid-10 array? The device is 4 x 1TB drives.

From skimming through the link posted earlier, it sounds like you have

Quoted 29 lines · Dewangga Bachrul Alam

Quoted 16 lines

Hi,

I'm new to XFS, I have RAID-10 array with 4 disk, when I check with xfs_info, the information print like this.

$ xfs_info /var/lib/mysql
meta-data=/dev/mapper/vg_catalystdb01-lv_database isize=256
agcount=16, agsize=1600000 blks
         =                       sectsz=4096  attr=2, projid32bit=0
data     =                       bsize=4096   blocks=25600000, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0
log      =internal               bsize=4096   blocks=12500, version=2
         =                       sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

Quoted 4 lines

Is it possible to change sectsz value to 512 without re-format it? Or any suggestion? I have issue with current sector size, my TokuDB engines[1] can't start because of this.

The only way to set things like sector size, block size, etc. is to reformat. I believe the default sector size is dependent on the physical device. You might want to report the following from your array device and perhaps from some or all of the member devices:

blockdev --getss --getpbsz --getiomin --getioopt <device>

Quoted 8 lines

Brian

Quoted 6 lines

[1] https://groups.google.com/forum/#!topic/tokudb-user/kvQFJLCmKwo

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

Quoted 10 lines · Dewangga Bachrul Alam

Hi,

The device reported like this :

$ blockdev --getss --getpbsz --getiomin --getioopt /dev/dm-3
512
4096
4096
0

What is dm-3? Is that a logical volume based on a volume group on top of your physical array? It might be good to get the equivalent data for the array device (md?) and at least one of the physical devices (sd?).

Quoted 4 lines

Then, the sector size should be 512. Don't know why it can be 4096. :( I will try to backup them and reformat. Any suggestion for formating on raid-10 array? The device is 4 x 1TB drives.

According to the above you have 4k physical sectors and 512b logical sectors. IIUC, this means mkfs will use the physical size by default, but you can specify and the device can handle down to 512 sectors.

From skimming through the link posted earlier, it sounds like you have an application that has a hardcoded dependency on 512b direct I/O requirements (e.g., buffer alignment) rather than being configurable..? Can you disable direct I/O and verify whether that works? Regardless, it might be wise to test out this 512b sector configuration (perhaps with a single or spare drive?) and verify this fixes the problem you're trying to solve before reconfiguring everything.

Brian

Quoted 38 lines · Brian Foster

Quoted 37 lines · Dewangga Bachrul Alam

Quoted 16 lines

Hi,

I'm new to XFS, I have RAID-10 array with 4 disk, when I check with xfs_info, the information print like this.

$ xfs_info /var/lib/mysql
meta-data=/dev/mapper/vg_catalystdb01-lv_database isize=256
agcount=16, agsize=1600000 blks
         =                       sectsz=4096  attr=2, projid32bit=0
data     =                       bsize=4096   blocks=25600000, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0
log      =internal               bsize=4096   blocks=12500, version=2
         =                       sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

Quoted 4 lines

Is it possible to change sectsz value to 512 without re-format it? Or any suggestion? I have issue with current sector size, my TokuDB engines[1] can't start because of this.

The only way to set things like sector size, block size, etc. is to reformat. I believe the default sector size is dependent on the physical device. You might want to report the following from your array device and perhaps from some or all of the member devices:

blockdev --getss --getpbsz --getiomin --getioopt <device>

Brian

Quoted 6 lines

[1] https://groups.google.com/forum/#!topic/tokudb-user/kvQFJLCmKwo

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

Hi,

I'm new to XFS, I have RAID-10 array with 4 disk, when I check with xfs_info, the information print like this.

$ xfs_info /var/lib/mysql
meta-data=/dev/mapper/vg_catalystdb01-lv_database isize=256
agcount=16, agsize=1600000 blks
         =                       sectsz=4096  attr=2, projid32bit=0
data     =                       bsize=4096   blocks=25600000, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0
log      =internal               bsize=4096   blocks=12500, version=2
         =                       sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

Is it possible to change sectsz value to 512 without re-format it? Or any suggestion? I have issue with current sector size, my TokuDB engines[1] can't start because of this.

[1] https://groups.google.com/forum/#!topic/tokudb-user/kvQFJLCmKwo

Quoted 16 lines · Dewangga Bachrul Alam

Hi,

I'm new to XFS, I have RAID-10 array with 4 disk, when I check with xfs_info, the information print like this.

$ xfs_info /var/lib/mysql
meta-data=/dev/mapper/vg_catalystdb01-lv_database isize=256
agcount=16, agsize=1600000 blks
         =                       sectsz=4096  attr=2, projid32bit=0
data     =                       bsize=4096   blocks=25600000, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0
log      =internal               bsize=4096   blocks=12500, version=2
         =                       sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

Quoted 5 lines

Is it possible to change sectsz value to 512 without re-format it? Or any suggestion? I have issue with current sector size, my TokuDB engines[1] can't start because of this.

[1] https://groups.google.com/forum/#!topic/tokudb-user/kvQFJLCmKwo

You almost certainly need this commit to resolve the issue.

You didn't say here what kernel you were using, but from your other post it looks like CentOS 6.4. Are you using stock xfsprogs from centos6.4 as well? I don't remember if it chooses 4k sectors for 4k physical block devices or not; unless you have a custom xfsprogs version or overrode sector size at mkfs time, I guess it must.

commit 7c71ee78031c248dca13fc94dea9a4cc217db6cf

Author: Eric Sandeen <[email protected]>
Date:   Tue Jan 21 16:46:23 2014 -0600

xfs: allow logical-sector sized O_DIRECT

Some time ago, mkfs.xfs started picking the storage physical sector size as the default filesystem "sector size" in order to avoid RMW costs incurred by doing IOs at logical sector size alignments.

However, this means that for a filesystem made with i.e. a 4k sector size on an "advanced format" 4k/512 disk, 512-byte direct IOs are no longer allowed. This means that XFS has essentially turned this AF drive into a hard 4K device, from the filesystem on up.

XFS's mkfs-specified "sector size" is really just controlling the minimum size & alignment of filesystem metadata.

There is no real need to tightly couple XFS's minimal metadata size to the minimum allowed direct IO size; XFS can continue doing metadata in optimal sizes, but still allow smaller DIOs for apps which issue them, for whatever reason.

This patch adds a new field to the xfs_buftarg, so that we now track 2 sizes:

1) The metadata sector size, which is the minimum unit and alignment of IO which will be performed by metadata operations.
2) The device logical sector size

The first is used internally by the file system for metadata alignment and IOs.
The second is used for the minimum allowed direct IO alignment.

This has passed xfstests on filesystems made with 4k sectors, including when run under the patch I sent to ignore XFS_IOC_DIOINFO, and issue 512 DIOs anyway. I also directly tested end of block behavior on preallocated, sparse, and existing files when we do a 512 IO into a 4k file on a 4k-sector filesystem, to be sure there were no unexpected behaviors.

Signed-off-by Eric Sandeen <[email protected]>

Hi,

I'm sorry, didnt fill any information here, but here is my nodes details.

$ uname -a
Linux catalyst-db01.jkt3d.xxx 2.6.32-504.3.3.el6.x86_64 #1 SMP Wed Dec
17 01:55:02 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

$ cat /etc/redhat-release
CentOS release 6.6 (Final)

The xfs and partition table build from anaconda from first install, instalation came from CentOS 6.6. But it's weird, only this node has 4k sector size, the others is 512.

catalyst-db01$ yum history info 1 | grep xfsprogs | fpaste Uploading (0.2KiB)...
http://ur1.ca/jihyu -> http://paste.fedoraproject.org/173606/27434142

Quoted 11 lines · Dewangga Bachrul Alam

Hi,

I'm sorry, didnt fill any information here, but here is my nodes details.

$ uname -a
Linux catalyst-db01.jkt3d.xxx 2.6.32-504.3.3.el6.x86_64 #1 SMP Wed Dec
17 01:55:02 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

$ cat /etc/redhat-release
CentOS release 6.6 (Final)

Quoted 7 lines

The xfs and partition table build from anaconda from first install, instalation came from CentOS 6.6. But it's weird, only this node has 4k sector size, the others is 512.

catalyst-db01$ yum history info 1 | grep xfsprogs | fpaste Uploading (0.2KiB)...
http://ur1.ca/jihyu -> http://paste.fedoraproject.org/173606/27434142

so xfsprogs v3.1.1

This went into v3.1.8:

commit 287d168b550857ce40e04b5f618d7eb91b87022f

Author: Eric Sandeen <[email protected]>
Date:   Thu Mar 1 22:46:35 2012 -0600

mkfs.xfs: properly handle physical sector size

This splits the fs_topology structure "sectorsize" into logical & physical, and gets both via blkid_get_topology().

This primarily allows us to default to using the physical sectorsize for mkfs's "sector size" value, the fundamental size of any IOs the filesystem will perform.

We reduce mkfs.xfs's "sector size" to logical if a block size < physical sector size is specified. This is suboptimal, but permissable.

For block size < sector size, differentiate the error message based on whether the sector size was manually specified, or deduced.

Signed-off-by Eric Sandeen <[email protected]>

Reviewed-by Dave Chinner <[email protected]>

but was backported to the RHEL6 xfsprogs:

* Tue Sep 25 2012 Eric Sandeen <[email protected]> 3.1.1-8
- mkfs.xfs: better handle misaligned 4k devices (#836433)
- mkfs.xfs: default to physical sectorsize (#836433)

So, not *exactly* a bug, because the assumption that 512-byte DIO will always work is not a good one, but the commit I mentioned in my first email will let 512-byte DIOs work again.

I'd tell you to file a bug with your RHEL support people, but Centos ... ;) We probably should get that kernel commit into RHEL6 if possible. I'm kind of surprised we haven't seen other reports.

But, if you ever wind up with hard 4k/4k drives, your database still won't work. On any filesystem. :)

If you don't mind following up with this informtation in the other forum, that might help others.

-Eric

Hi,

Thanks for your kind, but I don't know how to reproduce the errors, it happens randomly. I try to reproduce on VM but nothing helps, I can't create raid array on VM. :)

But, I'm quite sure there is miscalculation or something else that make 4k sector size on RAID-10 array.