What is a recommended XFS sector size for hybrid (512e) advanced format hard drives?
Hi.
Looking around I saw some references that it's preferable to use sector size (sectsz) of 4 KB when creating XFS partitions on hybrid (512e) advanced format hard drives and also some mentions of making that a default in mkfs.xfs. However I noticed that my current mkfs.xfs (3.2.1) used with current Debian testing (Linux 3.16.0) still uses 512 B sectors by default.
Can you please clarify what after all is the recommended sector size for such drives and why isn't it a default in mkfs.xfs (since supposedly defaults are generally recommended optimal settings unless you have some special use case).
Can you please also add this topic to the XFS FAQ (http://www.xfs.org/index.php/XFS_FAQ[1]), since I couldn't find any information about it there.
Thanks!
Hillel Lubman.
--------
[1] http://www.xfs.org/index.php/XFS_FAQ
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
On 01/04/2015 06:56 PM, Hillel Lubman wrote:
...
Quoted 6 lines
Looking around I saw some references that it's preferable to use sector size (sectsz) of 4 KB when creating XFS partitions on hybrid (512e) advanced format hard drives and also some mentions of making that a default in mkfs.xfs. However I noticed that my current mkfs.xfs (3.2.1) used with current Debian testing (Linux 3.16.0) still uses 512 B sectors by default.
...
Quoted 4 lines
Can you please clarify what after all is the recommended sector size for such drives and why isn't it a default in mkfs.xfs (since supposedly defaults are generally recommended optimal settings unless you have some special use case).
...
XFS sectsz is unimportant with these drives. What matters is that any partitions you create start and end on 4KB boundaries. This will prevent adjacent hardware sector RMW internal to the drive. XFS writes in 4KB filesystem blocks on Linux. As long as the fsblocks are aligned to the 4KB hardware sectors there's nothing more you can do to avoid performance penalties with these drives.
Stan
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 10 lines · On Sunday, January 04, 2015 21:52:51 Stan Hoeppner wrote:
On 01/04/2015 06:56 PM, Hillel Lubman wrote:
...Quoted 1 line
when creating XFS partitions on hybrid (512e)
...
Quoted 2 lines
Can you please clarify what after all is the recommended sector size for such drives
...
XFS sectsz is unimportant with these drives. What matters is that any partitions you create start and end on 4KB boundaries.
Thanks. So there shouldn't be any negative impact either if sectz is set to 4 KB? In one particular case I experience some perceived slowness on 2 TB WD drive (and I created XFS partition on that drive using sectz as 4 KB). I didn't test however how it compares to default settings on the same drive which would set it to 512 B.
Regards,
Hillel.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 18 lines · Hillel Lubman
Quoted 15 lines · On Sunday, January 04, 2015 21:52:51 Stan Hoeppner wrote:
On 01/04/2015 06:56 PM, Hillel Lubman wrote:
...
Quoted 1 line
when creating XFS partitions on hybrid (512e)
...
Quoted 3 lines
Can you please clarify what after all is the recommended sector size for
such drives
...
Quoted 8 lines
Quoted 3 lines
XFS sectsz is unimportant with these drives. What matters is that any
partitions you create start and end on 4KB boundaries.
Thanks. So there shouldn't be any negative impact either if sectz is set to 4 KB?
Again, as long as your partitions are 4KB aligned.
Quoted 3 lines
In one particular case I experience some perceived slowness on 2 TB WD drive (and I created XFS partition on that drive using sectz as 4 KB).
Likely due to a misaligned partition. In this case many XFS IOs are going to cause RMW in two adjacent hardware (4KB) disk sectors as each XFS block overlaps two sectors. This may significantly hamper drive performance.
Quoted 2 lines
I didn't test however how it compares to default settings on the same drive which would set it to 512 B.
As long as your underlying partition is 4KB aligned the only advantage you'll likely see with 4B sectsz is a little faster log IO. So for non metadata heavy workloads you won't see any difference between 512B and 4KB sectsz.
Stan
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 10 lines · On Monday, January 05, 2015 21:21:25 Stan Hoeppner wrote:
Quoted 3 lines
I didn't test however how it compares to default settings on the same drive which would set it to 512 B.
As long as your underlying partition is 4KB aligned the only advantage you'll likely see with 4B sectsz is a little faster log IO. So for non metadata heavy workloads you won't see any difference between 512B and 4KB sectsz.
Stan
It's actually created by gparted with 1 MB alignment, so it's implicitly 4 KB aligned as well. I guess something else is causing the slowness or it's just subjective. Raw I/O performance test looks normal with hdparm.
Here is what gdisk reports (third partition is the XFS one):
Found valid GPT with protective MBR; using GPT. Disk /dev/sda: 3907029168 sectors, 1.8 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): ...
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 3907029134 Partitions will be aligned on 2048-sector boundaries Total free space is 4205 sectors (2.1 MiB)
Number Start (sector) End (sector) Size Code Name 1 2048 534527 260.0 MiB EF00 2 534528 1763327 600.0 MiB 0700 3 1763328 3904931839 1.8 TiB 0700 4 3904931840 3907026943 1023.0 MiB 8200
Regards,
Hillel.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 13 lines · Stan Hoeppner
On 01/04/2015 06:56 PM, Hillel Lubman wrote:
...Quoted 6 lines
Looking around I saw some references that it's preferable to use sector size (sectsz) of 4 KB when creating XFS partitions on hybrid (512e) advanced format hard drives and also some mentions of making that a default in mkfs.xfs. However I noticed that my current mkfs.xfs (3.2.1) used with current Debian testing (Linux 3.16.0) still uses 512 B sectors by default.
...
Quoted 4 lines
Can you please clarify what after all is the recommended sector size for such drives and why isn't it a default in mkfs.xfs (since supposedly defaults are generally recommended optimal settings unless you have some special use case).
Quoted 8 lines
...
XFS sectsz is unimportant with these drives. What matters is that any partitions you create start and end on 4KB boundaries. This will prevent adjacent hardware sector RMW internal to the drive. XFS writes in 4KB filesystem blocks on Linux. As long as the fsblocks are aligned to the 4KB hardware sectors there's nothing more you can do to avoid performance penalties with these drives.
Not in my experience. My first (and lying about it) AF HDD had horrible write performance until i re'mkfs.xfs'ed it with 4k sector-size. And yes the aligned was 4k.
But that was many years ago and i don't know if newer XFS (IIRC that was before delay-log) would behave better. But as all of my storage after the first AF HDD was either an AF HDD or a SSD i have been formating everything with 4k sector-size for years.
Signature · 6 lines
Matthias _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Quoted 7 lines · Hillel Lubman
Hi.
Can you please clarify what after all is the recommended sector size for such drives and why isn't it a default in mkfs.xfs (since supposedly defaults are generally recommended optimal settings unless you have some special use case).
It is indeed the default.
[root@sandeen ~]# blockdev --getss --getpbsz /dev/sde 512
4096
[root@sandeen ~]# mkfs.xfs /dev/sde
meta-data=/dev/sde isize=256 agcount=1, agsize=4097 blks
= sectsz=4096 attr=2, projid32bit=0
data = bsize=4096 blocks=4097, imaxpct=25
= sunit=1 swidth=8 blks
naming =version 2 bsize=4096 ascii-ci=0
log =internal log bsize=4096 blocks=1424, version=2
= sectsz=4096 sunit=1 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0however, some drives lie about these sizes, and then mkfs.xfs can't know. Try the blockdev command above to see.
-Eric
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 2 lines · Eric Sandeen
however, some drives lie about these sizes, and then mkfs.xfs can't know. Try the blockdev command above to see.
blockdev and parted seem to get this wrong for a device for which smartctl and hdparm get correct
smartctl -a reports:
Sector Sizes: 512 bytes logical, 4096 bytes physical
hdparm -I reports:
Logical Sector size: 512 bytes Physical Sector size: 4096 bytes
parted reports:
Sector size (logical/physical): 512B/512B
# blockdev --getss /dev/sdc 512 # blockdev --getpbsz /dev/sdc 512
And thus xfs_info on this device says sectsz=512 since I formatted with defaults.
Signature · 6 lines
Chris Murphy _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Quoted 6 lines · Chris Murphy
Quoted 2 lines · Eric Sandeen
however, some drives lie about these sizes, and then mkfs.xfs can't know. Try the blockdev command above to see.
blockdev and parted seem to get this wrong for a device for which smartctl and hdparm get correct
I don't think they get it wrong, they are just reporting what the drive says over that interface.
I ... don't actually know where smartctl/hdparm get the values, vs. the values exported to blockdev. Is the drive reporting different values over different query interfaces? Hrm.
mkfs.xfs uses the same interface as used by blockdev.
-Eric
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 10 lines · Eric Sandeen
Quoted 6 lines · Chris Murphy
Quoted 2 lines · Eric Sandeen
however, some drives lie about these sizes, and then mkfs.xfs can't know. Try the blockdev command above to see.
blockdev and parted seem to get this wrong for a device for which smartctl and hdparm get correct
I don't think they get it wrong, they are just reporting what the drive says over that interface.
The problem may be that the drive isn't being queried, but rather the bridge chipset involved. All drives are AF drives in USB 3 enclosures. But somehow hdparm and smartctl are getting actual drive info from the drive through this interface; while the kernel appears to be fooled by bridge.
I don't immediately have a way to directly connect any to SATA. I will bet dollars to donuts this is what's going on though, because I know one of these drives when it was connected via SATA was reported by parted as 512B/4096B but currently it isn't.
Signature · 6 lines
Chris Murphy _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
What's sectsz= used for? Historically this would be sectsz=512, bsize=4096 before AF hard drives appeared. So is sectsz used for optimizing sub blocksize changes? e.g. if only 50 bytes needs changing, the fs doesn't need to read modify and write the entire 4096 block, just change the affected sector?
If that's true, then sectsz=512 for a 4096 physical sector drive would cause a lot of in-drive RMW, depending on the workdlow. On older AF drives this can be a huge penalty hit.
Chris Murphy
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 5 lines · Chris Murphy
What's sectsz= used for? Historically this would be sectsz=512, bsize=4096 before AF hard drives appeared. So is sectsz used for optimizing sub blocksize changes? e.g. if only 50 bytes needs changing, the fs doesn't need to read modify and write the entire 4096 block, just change the affected sector?
Nope, filesystem data blocks are always fully written, but the sector size is i.e. the minimum _log_ IO size.
+/*
+ * The xfs_buftarg contains 2 notions of "sector size" -
+ *
+ * 1) The metadata sector size, which is the minimum unit and
+ * alignment of IO which will be performed by metadata operations.
+ * 2) The device logical sector size
+ *
+ * The first is specified at mkfs time, and is stored on-disk in the
+ * superblock's sb_sectsize.
+ *
+ * The latter is derived from the underlying device, and controls direct IO
+ * alignment constraints.
-Eric
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 9 lines · Eric Sandeen
Quoted 5 lines · Chris Murphy
What's sectsz= used for? Historically this would be sectsz=512, bsize=4096 before AF hard drives appeared. So is sectsz used for optimizing sub blocksize changes? e.g. if only 50 bytes needs changing, the fs doesn't need to read modify and write the entire 4096 block, just change the affected sector?
Nope, filesystem data blocks are always fully written, but the sector size is i.e. the minimum _log_ IO size.
So maybe the people reporting performance problems with sectsz=512 on AF drives compared to manually specifying 4096 bytes, have a high metadata workload with older AF drives and are getting lot of in-drive RMW...
Signature · 6 lines
Chris Murphy _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Quoted 11 lines · On Tuesday, January 06, 2015 11:02:09 Eric Sandeen wrote:
Quoted 7 lines · Hillel Lubman
Hi.
Can you please clarify what after all is the recommended sector size for such drives and why isn't it a default in mkfs.xfs (since supposedly defaults are generally recommended optimal settings unless you have some special use case).
It is indeed the default.
Quoted 8 lines
[root@sandeen ~]# blockdev --getss --getpbsz
/dev/sde512
4096
...however, some drives lie about these sizes, and then mkfs.xfs can't know. Try the blockdev command above to see.
I see. Indeed, on the drive where I get sectsz as 512 by default, blockdev --getss --getpbsz reports:
512
512
However hdparm -I reports:
Logical Sector size: 512 bytes Physical Sector size: 4096 bytes
So is it still worthwhile making sectsz 4 KB explicitly instead of using mkfs.xfs default in such case? And on a side note, since there is a more reliable way to figure that info out, may be mkfs.xfs should rather use that?
Regards,
Hillel.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 4 lines · Hillel Lubman
I see. Indeed, on the drive where I get sectsz as 512 by default, blockdev --getss --getpbsz reports:
512
512
what kind of drive is this; is it usb, SATA, or ?
Quoted 11 lines
However hdparm -I reports:
Logical Sector size: 512 bytes
Physical Sector size: 4096 bytes
So is it still worthwhile making sectsz 4 KB explicitly instead of using mkfs.xfs default in such case? And on a side note, since there is a more reliable way to figure that info out, may be mkfs.xfs should rather use that?
a) probably can't hurt
b) probably doesn't matter ;)
c) not sure why there are 2 different answers. Knowing what kind of drive it is would be good; mkfs.xfs just uses whatever blkid says.
-Eric
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 4 lines · On Tuesday, January 06, 2015 23:30:26 Eric Sandeen wrote:
Quoted 2 lines · Hillel Lubman
I see. Indeed, on the drive where I get sectsz as 512 by default, blockdev --getss --getpbsz reports:
Quoted 8 lines
Quoted 2 lines
512
512what kind of drive is this; is it usb, SATA, or ?
c) not sure why there are 2 different answers. Knowing what kind of drive it is would be good; mkfs.xfs just uses whatever blkid says.
One is a recent portable WD USB3 drive (elements) and another is a regular 3.5" internal AF WD black in an enclosure (also attached through USB3). I didn't recently test the case of a drive attached through regular SATA.
Regards,
Hillel.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 3 lines · Hillel Lubman
One is a recent portable WD USB3 drive (elements) and another is a regular 3.5" internal AF WD black in an enclosure (also attached through USB3). I didn't recently test the case of a drive attached through regular SATA.
A common theme emerges, so far 6 for 6 drives behaving this way are in USB enclosures. At least one vendor ships an external drive (drive inside an enclosure) that causes parted to report 4096B/4096B logical/physical sectors. The drive inside is actually 512e so it's 512B/4096B, and the result of this is that partitioning/formatting it inside the enclosure means it's completely unreadable outside the enclosure and vice versa. So cute and adorable.
Signature · 6 lines
Chris Murphy _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs