update BDI {io,ra}_pages values based on the RT device limits v2
Subsystems (2)
Hi all,
we ran into a bit of a funny case where adding an SSD to store metadata to a (zoned) XFS file system reduced the performance vs using a HDD for both data and metadata. It turns out this is due to readahead code looking at the BDI limits, including the io_pages value that can't be inspected or tuned from userspace.
Changes since v1:
- restore the old value at unmount, unless the BDI values changed since mount
PATCH 1/1 xfs: update BDI {io,ra}_pages values based on the RT device limits
When using XFS with a main device on an SSD that stores metadata and a RT device to store data on a HDD, we fail to take the I/O sizes for the RT device into accounting, leading to up to 5% slower read performance when using an SSD for metadata vs storing data and metadata on the HDD.
Fix this up by taking the RT settings into account at mount an restoring the old settings at unmount time, unless the BDI settings have changed from those set by XFS.
--- fs/xfs/xfs_mount.h | 7 ++++++ fs/xfs/xfs_super.c | 53 +++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 59 insertions(+), 1 deletion(-)
fs/xfs/xfs_mount.h +7 -0
@@ -349,6 +349,13 @@ typedef struct xfs_mount {/* Index of uuid record in the uuid xarray. */ unsigned int m_uuid_table_index; + +/* +* Old io_pages/ra_pages valued in the main bdev BDI, and our initial +* calculated values. +*/ +unsigned long m_old_io_pages, m_initial_io_pages; +unsigned long m_old_ra_pages, m_initial_ra_pages; } xfs_mount_t;#define M_IGEO (mp) (&(mp)->m_ino_geo)
fs/xfs/xfs_super.c +52 -1
72 lines not on this page · raw message
Signature · 1 line
2.53.0
Re: [PATCH] xfs: update BDI {io,ra}_pages values based on the RT device limits
Quoted 11 lines · Christoph Hellwig
When using XFS with a main device on an SSD that stores metadata and a RT device to store data on a HDD, we fail to take the I/O sizes for the RT device into accounting, leading to up to 5% slower read performance when using an SSD for metadata vs storing data and metadata on the HDD.
Fix this up by taking the RT settings into account at mount an restoring the old settings at unmount time, unless the BDI settings have changed from those set by XFS.
Reported-by Filip Blagojevic <[email protected]>Signed-off-by Christoph Hellwig <[email protected]>
[...]
Quoted 9 lines
+/*
+ * When using a RT device some or all data I/O is using the RT device, but
+ * the BDI is inherited from the main data device. When the underlying block
+ * device for the RT device has larger I/O sizes, the BDI settings might be
+ * incorrect, which is especially bad if the main device is a SSD and the
+ * RT device is a HDD, as theio_optfixup inblk_apply_bdi_limitsis missing
+ * for this case.
+ *
+ * Update the BDI values to the max of the data and RT device to cover our
s/data/main ?
I am still confused about the proper name for the device holding metadata :)
Other than this, this looks OK to me.
Signature · 2 lines
Damien Le Moal Western Digital Research
Re: [PATCH] xfs: update BDI {io,ra}_pages values based on the RT device limits
Quoted 6 lines · Damien Le Moal
Quoted 1 line
+ * Update the BDI values to the max of the data and RT device to cover our
s/data/main ?
I am still confused about the proper name for the device holding metadata :)Other than this, this looks OK to me.
Historically the main device is called the data device, which is also reflected in all kinds of field naming. But yes, it is confusing especially for zoned xfs setups. Maybe I should do a little global naming sweep..
Carlos Maiolino
maintainerRe: [PATCH] xfs: update BDI {io,ra}_pages values based on the RT device limits
Quoted 6 lines · Christoph Hellwig
Quoted 5 lines · Damien Le Moal
Quoted 1 line
+ * Update the BDI values to the max of the data and RT device to cover our
s/data/main ?
I am still confused about the proper name for the device holding metadata :)
Quoted 8 lines
Quoted 1 line
Other than this, this looks OK to me.
Historically the main device is called the data device, which is also reflected in all kinds of field naming. But yes, it is confusing especially for zoned xfs setups. Maybe I should do a little global naming sweep..
+1, please keep /data/ here, this makes much more sense in xfs lingo... Perhaps we should start naming them to something like ndata (for normal data) and zdata for zoned data, but that's talk for another thread.
Re: [PATCH] xfs: update BDI {io,ra}_pages values based on the RT device limits
Quoted 16 lines · Carlos Maiolino
Quoted 6 lines · Christoph Hellwig
Quoted 5 lines · Damien Le Moal
Quoted 1 line
+ * Update the BDI values to the max of the data and RT device to cover our
s/data/main ?
I am still confused about the proper name for the device holding metadata :)Quoted 8 lines
Quoted 1 line
Other than this, this looks OK to me.
Historically the main device is called the data device, which is also reflected in all kinds of field naming. But yes, it is confusing especially for zoned xfs setups. Maybe I should do a little global naming sweep..
Quoted 3 lines
+1, please keep /data/ here, this makes much more sense in xfs lingo... Perhaps we should start naming them to something like ndata (for normal data) and zdata for zoned data, but that's talk for another thread.
I've been trying to be consistent at calling them the 'datadev' and 'rtdev' (no spaces) so it's more obvious that I'm using a jargon. But 'rt' makes no sense these days and I never did come up with a good retcon for it though.
"RelaTive"? Because the rtdev is "related" to the datadev by metadata?
:D
--D
Re: [PATCH] xfs: update BDI {io,ra}_pages values based on the RT device limits
In reply to Christoph HellwigQuoted 11 lines · Christoph Hellwig
When using XFS with a main device on an SSD that stores metadata and a RT device to store data on a HDD, we fail to take the I/O sizes for the RT device into accounting, leading to up to 5% slower read performance when using an SSD for metadata vs storing data and metadata on the HDD.
Fix this up by taking the RT settings into account at mount an restoring the old settings at unmount time, unless the BDI settings have changed from those set by XFS.
Reported-by Filip Blagojevic <[email protected]>Signed-off-by Christoph Hellwig <[email protected]>
Quoted 106 lines
--- fs/xfs/xfs_mount.h | 7 ++++++ fs/xfs/xfs_super.c | 53 +++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 59 insertions(+), 1 deletion(-)fs/xfs/xfs_mount.h +7
-0@@ -349,6 +349,13 @@ typedef struct xfs_mount {/* Index of uuid record in the uuid xarray. */ unsigned int m_uuid_table_index; + +/* +* Old io_pages/ra_pages valued in the main bdev BDI, and our initial +* calculated values. +*/ +unsigned long m_old_io_pages, m_initial_io_pages; +unsigned long m_old_ra_pages, m_initial_ra_pages; } xfs_mount_t;#define M_IGEO (mp) (&(mp)->m_ino_geo)fs/xfs/xfs_super.c +52
-1@@ -548,6 +548,52 @@ xfs_open_devices(return error; } +/* +* When using a RT device some or all data I/O is using the RT device, but +* the BDI is inherited from the main data device. When the underlying block +* device for the RT device has larger I/O sizes, the BDI settings might be +* incorrect, which is especially bad if the main device is a SSD and the +* RT device is a HDD, as the io_opt fixup in blk_apply_bdi_limits is missing +* for this case. +* +* Update the BDI values to the max of the data and RT device to cover our +* bases. +*/ +static void +xfs_update_bdi_rahead ( +struct xfs_mount *mp) +{ +struct backing_dev_info *rt_bdi = + mp->m_rtdev_targp->bt_bdev->bd_disk->bdi; +struct backing_dev_info *sb_bdi = mp->m_super->s_bdi; + + mp->m_old_io_pages = sb_bdi->io_pages; + mp->m_old_ra_pages = sb_bdi->ra_pages; + + sb_bdi->io_pages = mp->m_initial_io_pages = +max (sb_bdi->io_pages, rt_bdi->io_pages); + sb_bdi->ra_pages = mp->m_initial_ra_pages = +max (sb_bdi->ra_pages, rt_bdi->ra_pages); +} + +static void +xfs_restore_bdi_rahead ( +struct xfs_mount *mp) +{ +struct backing_dev_info *sb_bdi = mp->m_super->s_bdi; + +if (sb_bdi->io_pages == mp->m_initial_io_pages) + sb_bdi->io_pages = mp->m_old_io_pages; +else +xfs_info (mp,"io_pages changed from %lu to %lu, not restoring." , + mp->m_initial_io_pages, sb_bdi->io_pages); +if (sb_bdi->ra_pages == mp->m_initial_ra_pages) + sb_bdi->ra_pages = mp->m_old_ra_pages; +else +xfs_info (mp,"ra_pages changed from %lu to %lu, not restoring." , + mp->m_initial_ra_pages, sb_bdi->ra_pages); +} +/* * Setup xfs_mount buffer target pointers based on superblock */ @@ -585,6 +631,7 @@ xfs_setup_devices( mp->m_sb.sb_sectsize, mp->m_sb.sb_rblocks);if (error)return error; +xfs_update_bdi_rahead (mp); }return 0 ;@@ -2283,8 +2330,12 @@ static voidxfs_kill_sb ( struct super_block *sb) { +struct xfs_mount *mp =XFS_M (sb); + +if (mp->m_rtdev_targp && mp->m_rtdev_targp != mp->m_ddev_targp) +xfs_restore_bdi_rahead (mp);kill_block_super (sb); -xfs_mount_free (XFS_M (sb)); +xfs_mount_free (mp); }static struct file_system_type xfs_fs_type = {Signature · 1 line
2.53.0
Re: [PATCH] xfs: update BDI {io,ra}_pages values based on the RT device limits
In reply to Christoph HellwigQuoted 10 lines · Christoph Hellwig
When using XFS with a main device on an SSD that stores metadata and a RT device to store data on a HDD, we fail to take the I/O sizes for the RT device into accounting, leading to up to 5% slower read performance when using an SSD for metadata vs storing data and metadata on the HDD.
Fix this up by taking the RT settings into account at mount an restoring the old settings at unmount time, unless the BDI settings have changed from those set by XFS.
[...]
Applied to for-next, thanks!
[1/1] xfs: update BDI {io,ra}_pages values based on the RT device limits
commit: 39de8c9f3e4bc059bb0bc85a39affcf52664c1ed
Best regards,
Signature · 1 line
Carlos Maiolino <[email protected]>