Skip to contentSkip to conversation

xfs: allow sunit mount option to repair bad primary sb stripe values

PATCH Superseded Dave Chinner 2 participants 4 messages Mar 12, 2024 – Mar 13, 2024 XFS filesystem

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 xfs: allow sunit mount option to repair bad primary sb stripe values

From: Dave Chinner <[email protected]>

If a filesystem has a busted stripe alignment configuration on disk (e.g. because broken RAID firmware told mkfs that swidth was smaller than sunit), then the filesystem will refuse to mount due to the stripe validation failing. This failure is triggering during distro upgrades from old kernels lacking this check to newer kernels with this check, and currently the only way to fix it is with offline xfs_db surgery.

This runtime validity checking occurs when we read the superblock for the first time and causes the mount to fail immediately. This prevents the rewrite of stripe unit/width via
mount options that occurs later in the mount process. Hence there is no way to recover this situation without resorting to offline xfs_db rewrite of the values.

However, we parse the mount options long before we read the superblock, and we know if the mount has been asked to re-write the stripe alignment configuration when we are reading the superblock and verifying it for the first time. Hence we can conditionally ignore stripe verification failures if the mount options specified will correct the issue.

We validate that the new stripe unit/width are valid before we overwrite the superblock values, so we can ignore the invalid config at verification and fail the mount later if the new values are not valid. This, at least, gives users the chance of correcting the issue after a kernel upgrade without having to resort to xfs-db hacks.

---
 fs/xfs/libxfs/xfs_sb.c | 40 +++++++++++++++++++++++++++++++---------
 fs/xfs/libxfs/xfs_sb.h |  3 ++-
 2 files changed, 33 insertions(+), 10 deletions(-)
fs/xfs/libxfs/xfs_sb.c +31 -9
index d991eec05436..f51b1efa2cae 100644
@@ -530,7 +530,8 @@ xfs_validate_sb_common(
 	}
 
 	if (!xfs_validate_stripe_geometry(mp, XFS_FSB_TO_B(mp, sbp->sb_unit),
-			XFS_FSB_TO_B(mp, sbp->sb_width), 0, false))
+			XFS_FSB_TO_B(mp, sbp->sb_width), 0,
+			xfs_buf_daddr(bp) == XFS_SB_DADDR, false))
 		return -EFSCORRUPTED;
 
 	/*
@@ -1323,8 +1324,10 @@ xfs_sb_get_secondary(
 }
 
 /*
- * sunit, swidth, sectorsize(optional with 0) should be all in bytes,
- * so users won't be confused by values in error messages.
+ * sunit, swidth, sectorsize(optional with 0) should be all in bytes, so users
+ * won't be confused by values in error messages. This returns false if a value
+ * is invalid and it is not the primary superblock that going to be corrected
+ * later in the mount process.
  */
 bool
 xfs_validate_stripe_geometry(
@@ -1332,20 +1335,21 @@ xfs_validate_stripe_geometry(
 	__s64			sunit,
 	__s64			swidth,
 	int			sectorsize,
+	bool			primary_sb,
 	bool			silent)
 {
 	if (swidth > INT_MAX) {
 		if (!silent)
 			xfs_notice(mp,
 "stripe width (%lld) is too large", swidth);
-		return false;
+		goto check_override;
 	}
 
 	if (sunit > swidth) {
 		if (!silent)
 			xfs_notice(mp,
 "stripe unit (%lld) is larger than the stripe width (%lld)", sunit, swidth);
-		return false;
+		goto check_override;
 	}
 
 	if (sectorsize && (int)sunit % sectorsize) {
@@ -1353,21 +1357,21 @@ xfs_validate_stripe_geometry(
 			xfs_notice(mp,
 "stripe unit (%lld) must be a multiple of the sector size (%d)",
 				   sunit, sectorsize);
-		return false;
+		goto check_override;
 	}
 
 	if (sunit && !swidth) {
 		if (!silent)
 			xfs_notice(mp,
 "invalid stripe unit (%lld) and stripe width of 0", sunit);
-		return false;
+		goto check_override;
 	}
 
 	if (!sunit && swidth) {
 		if (!silent)
 			xfs_notice(mp,
 "invalid stripe width (%lld) and stripe unit of 0", swidth);
-		return false;
+		goto check_override;
 	}
 
 	if (sunit && (int)swidth % (int)sunit) {
@@ -1375,9 +1379,27 @@ xfs_validate_stripe_geometry(
 			xfs_notice(mp,
 "stripe width (%lld) must be a multiple of the stripe unit (%lld)",
 				   swidth, sunit);
-		return false;
+		goto check_override;
 	}
 	return true;
+
+check_override:
+	if (!primary_sb)
+		return false;
+	/*
+	 * During mount, mp->m_dalign will not be set unless the sunit mount
+	 * option was set. If it was set, ignore the bad stripe alignment values
+	 * and allow the validation and overwrite later in the mount process to
+	 * attempt to overwrite the bad stripe alignment values with the values
+	 * supplied by mount options.
+	 */
+	if (!mp->m_dalign)
+		return false;
+	if (!silent)
+		xfs_notice(mp,
+"Will try to correct with specified mount options sunit (%d) and swidth (%d)",
+			BBTOB(mp->m_dalign), BBTOB(mp->m_swidth));
+	return true;
 }
 
 /*
fs/xfs/libxfs/xfs_sb.h +2 -1
index 67a40069724c..58798b9c70ba 100644
@@ -35,7 +35,8 @@ extern int	xfs_sb_get_secondary(struct xfs_mount *mp,
 				struct xfs_buf **bpp);
 
 extern bool	xfs_validate_stripe_geometry(struct xfs_mount *mp,
-		__s64 sunit, __s64 swidth, int sectorsize, bool silent);
+		__s64 sunit, __s64 swidth, int sectorsize, bool primary_sb,
+		bool silent);
 
 uint8_t xfs_compute_rextslog(xfs_rtbxlen_t rtextents);
 
Signature · 1 line
2.43.0
#1 lore raw
Quoted 52 lines · Dave Chinner

From: Dave Chinner <[email protected]>

If a filesystem has a busted stripe alignment configuration on disk (e.g. because broken RAID firmware told mkfs that swidth was smaller than sunit), then the filesystem will refuse to mount due to the stripe validation failing. This failure is triggering during distro upgrades from old kernels lacking this check to newer kernels with this check, and currently the only way to fix it is with offline xfs_db surgery.

This runtime validity checking occurs when we read the superblock for the first time and causes the mount to fail immediately. This prevents the rewrite of stripe unit/width via
mount options that occurs later in the mount process. Hence there is no way to recover this situation without resorting to offline xfs_db rewrite of the values.

However, we parse the mount options long before we read the superblock, and we know if the mount has been asked to re-write the stripe alignment configuration when we are reading the superblock and verifying it for the first time. Hence we can conditionally ignore stripe verification failures if the mount options specified will correct the issue.

We validate that the new stripe unit/width are valid before we overwrite the superblock values, so we can ignore the invalid config at verification and fail the mount later if the new values are not valid. This, at least, gives users the chance of correcting the issue after a kernel upgrade without having to resort to xfs-db hacks.

---
 fs/xfs/libxfs/xfs_sb.c | 40 +++++++++++++++++++++++++++++++---------
 fs/xfs/libxfs/xfs_sb.h |  3 ++-
 2 files changed, 33 insertions(+), 10 deletions(-)
fs/xfs/libxfs/xfs_sb.c +2 -1
index d991eec05436..f51b1efa2cae 100644
@@ -530,7 +530,8 @@ xfs_validate_sb_common(
 	}
 
 	if (!xfs_validate_stripe_geometry(mp, XFS_FSB_TO_B(mp, sbp->sb_unit),
-			XFS_FSB_TO_B(mp, sbp->sb_width), 0, false))
+			XFS_FSB_TO_B(mp, sbp->sb_width), 0,
+			xfs_buf_daddr(bp) == XFS_SB_DADDR, false))
 		return -EFSCORRUPTED;
 
 	/*

fs/xfs/libxfs/xfs_sb.c xfs_sb_get_secondary( · line 1330

Quoted 9 lines
fs/xfs/libxfs/xfs_sb.c +4 -2
@@ -1323,8 +1324,10 @@ xfs_sb_get_secondary(
 }
 
 /*
- * sunit, swidth, sectorsize(optional with 0) should be all in bytes,
- * so users won't be confused by values in error messages.
+ * sunit, swidth, sectorsize(optional with 0) should be all in bytes, so users
+ * won't be confused by values in error messages. This returns false if a value
+ * is invalid and it is not the primary superblock that going to be corrected
+ * later in the mount process.

Hmm, I found this last sentence a little confusing. How about:

"This function returns false if the stripe geometry is invalid and no attempt will be made to correct it later in the mount process."

Quoted 60 lines
fs/xfs/libxfs/xfs_sb.c +7 -6
  */
 bool
 xfs_validate_stripe_geometry(
@@ -1332,20 +1335,21 @@ xfs_validate_stripe_geometry(
 	__s64			sunit,
 	__s64			swidth,
 	int			sectorsize,
+	bool			primary_sb,
 	bool			silent)
 {
 	if (swidth > INT_MAX) {
 		if (!silent)
 			xfs_notice(mp,
 "stripe width (%lld) is too large", swidth);
-		return false;
+		goto check_override;
 	}
 
 	if (sunit > swidth) {
 		if (!silent)
 			xfs_notice(mp,
 "stripe unit (%lld) is larger than the stripe width (%lld)", sunit, swidth);
-		return false;
+		goto check_override;
 	}
 
 	if (sectorsize && (int)sunit % sectorsize) {
@@ -1353,21 +1357,21 @@ xfs_validate_stripe_geometry(
 			xfs_notice(mp,
 "stripe unit (%lld) must be a multiple of the sector size (%d)",
 				   sunit, sectorsize);
-		return false;
+		goto check_override;
 	}
 
 	if (sunit && !swidth) {
 		if (!silent)
 			xfs_notice(mp,
 "invalid stripe unit (%lld) and stripe width of 0", sunit);
-		return false;
+		goto check_override;
 	}
 
 	if (!sunit && swidth) {
 		if (!silent)
 			xfs_notice(mp,
 "invalid stripe width (%lld) and stripe unit of 0", swidth);
-		return false;
+		goto check_override;
 	}
 
 	if (sunit && (int)swidth % (int)sunit) {
@@ -1375,9 +1379,27 @@ xfs_validate_stripe_geometry(
 			xfs_notice(mp,
 "stripe width (%lld) must be a multiple of the stripe unit (%lld)",
 				   swidth, sunit);
-		return false;
+		goto check_override;
 	}
 	return true;

fs/xfs/libxfs/xfs_sb.c xfs_validate_stripe_geometry( · line 1394

Quoted 10 lines
fs/xfs/libxfs/xfs_sb.c +10 -0
+
+check_override:
+	if (!primary_sb)
+		return false;
+	/*
+	 * During mount, mp->m_dalign will not be set unless the sunit mount
+	 * option was set. If it was set, ignore the bad stripe alignment values
+	 * and allow the validation and overwrite later in the mount process to
+	 * attempt to overwrite the bad stripe alignment values with the values
+	 * supplied by mount options.

What catches the case of if m_dalign/m_swidth also being garbage values? Is it xfs_check_new_dalign? Should that fail the mount if the replacement values are also garbage?

Quoted 16 lines
fs/xfs/libxfs/xfs_sb.c +8 -0
+	 */
+	if (!mp->m_dalign)
+		return false;
+	if (!silent)
+		xfs_notice(mp,
+"Will try to correct with specified mount options sunit (%d) and swidth (%d)",
+			BBTOB(mp->m_dalign), BBTOB(mp->m_swidth));
+	return true;
 }
 
 /*
fs/xfs/libxfs/xfs_sb.h +0 -0
index 67a40069724c..58798b9c70ba 100644

fs/xfs/libxfs/xfs_sb.h extern int xfs_sb_get_secondary(struct xfs_mount *mp, · line 37

Quoted 3 lines
fs/xfs/libxfs/xfs_sb.h +0 -0
@@ -35,7 +35,8 @@ extern int	xfs_sb_get_secondary(struct xfs_mount *mp,
 				struct xfs_buf **bpp);
 
 extern bool	xfs_validate_stripe_geometry(struct xfs_mount *mp,

This declaration might as well lose the extern here too.

fs/xfs/libxfs/xfs_sb.h extern int xfs_sb_get_secondary(struct xfs_mount *mp, · line 39

Quoted 3 lines
fs/xfs/libxfs/xfs_sb.h +2 -1
-		__s64 sunit, __s64 swidth, int sectorsize, bool silent);
+		__s64 sunit, __s64 swidth, int sectorsize, bool primary_sb,
+		bool silent);

What should value for @primary_sb should mkfs pass into xfs_validate_stripe_geometry from calc_stripe_factors?

--D

Quoted 7 lines
fs/xfs/libxfs/xfs_sb.h +0 -0
 uint8_t xfs_compute_rextslog(xfs_rtbxlen_t rtextents);
 
Signature · 1 line
2.43.0
#2 lore raw
Quoted 65 lines · Darrick J. Wong
Quoted 52 lines · Dave Chinner

From: Dave Chinner <[email protected]>

If a filesystem has a busted stripe alignment configuration on disk (e.g. because broken RAID firmware told mkfs that swidth was smaller than sunit), then the filesystem will refuse to mount due to the stripe validation failing. This failure is triggering during distro upgrades from old kernels lacking this check to newer kernels with this check, and currently the only way to fix it is with offline xfs_db surgery.

This runtime validity checking occurs when we read the superblock for the first time and causes the mount to fail immediately. This prevents the rewrite of stripe unit/width via
mount options that occurs later in the mount process. Hence there is no way to recover this situation without resorting to offline xfs_db rewrite of the values.

However, we parse the mount options long before we read the superblock, and we know if the mount has been asked to re-write the stripe alignment configuration when we are reading the superblock and verifying it for the first time. Hence we can conditionally ignore stripe verification failures if the mount options specified will correct the issue.

We validate that the new stripe unit/width are valid before we overwrite the superblock values, so we can ignore the invalid config at verification and fail the mount later if the new values are not valid. This, at least, gives users the chance of correcting the issue after a kernel upgrade without having to resort to xfs-db hacks.

---
 fs/xfs/libxfs/xfs_sb.c | 40 +++++++++++++++++++++++++++++++---------
 fs/xfs/libxfs/xfs_sb.h |  3 ++-
 2 files changed, 33 insertions(+), 10 deletions(-)
fs/xfs/libxfs/xfs_sb.c +2 -1
index d991eec05436..f51b1efa2cae 100644
@@ -530,7 +530,8 @@ xfs_validate_sb_common(
 	}
 
 	if (!xfs_validate_stripe_geometry(mp, XFS_FSB_TO_B(mp, sbp->sb_unit),
-			XFS_FSB_TO_B(mp, sbp->sb_width), 0, false))
+			XFS_FSB_TO_B(mp, sbp->sb_width), 0,
+			xfs_buf_daddr(bp) == XFS_SB_DADDR, false))
 		return -EFSCORRUPTED;
 
 	/*

fs/xfs/libxfs/xfs_sb.c xfs_sb_get_secondary( · line 1330

Quoted 9 lines
fs/xfs/libxfs/xfs_sb.c +4 -2
@@ -1323,8 +1324,10 @@ xfs_sb_get_secondary(
 }
 
 /*
- * sunit, swidth, sectorsize(optional with 0) should be all in bytes,
- * so users won't be confused by values in error messages.
+ * sunit, swidth, sectorsize(optional with 0) should be all in bytes, so users
+ * won't be confused by values in error messages. This returns false if a value
+ * is invalid and it is not the primary superblock that going to be corrected
+ * later in the mount process.

Hmm, I found this last sentence a little confusing. How about:

Quoted 2 lines

"This function returns false if the stripe geometry is invalid and no attempt will be made to correct it later in the mount process."

Sure.

.....

Quoted 14 lines
Quoted 14 lines
fs/xfs/libxfs/xfs_sb.c +7 -1
@@ -1375,9 +1379,27 @@ xfs_validate_stripe_geometry(
 			xfs_notice(mp,
 "stripe width (%lld) must be a multiple of the stripe unit (%lld)",
 				   swidth, sunit);
-		return false;
+		goto check_override;
 	}
 	return true;
+
+check_override:
+	if (!primary_sb)
+		return false;
+	/*
+	 * During mount, mp->m_dalign will not be set unless the sunit mount
Quoted 8 lines

fs/xfs/libxfs/xfs_sb.c xfs_validate_stripe_geometry( · line 1394

Quoted 4 lines
fs/xfs/libxfs/xfs_sb.c +4 -0
+	 * option was set. If it was set, ignore the bad stripe alignment values
+	 * and allow the validation and overwrite later in the mount process to
+	 * attempt to overwrite the bad stripe alignment values with the values
+	 * supplied by mount options.

What catches the case of if m_dalign/m_swidth also being garbage values? Is it xfs_check_new_dalign? Should that fail the mount if the replacement values are also garbage?

xfs_validate_new_dalign().

It returns -EINVAL is the new striped alignment cannot be used (i.e. not aligned to block or ag sizes) and that causes the mount to fail.

Quoted 11 lines

fs/xfs/libxfs/xfs_sb.h extern int xfs_sb_get_secondary(struct xfs_mount *mp, · line 37

Quoted 8 lines
fs/xfs/libxfs/xfs_sb.h +0 -0
index 67a40069724c..58798b9c70ba 100644
@@ -35,7 +35,8 @@ extern int	xfs_sb_get_secondary(struct xfs_mount *mp,
 				struct xfs_buf **bpp);
 
 extern bool	xfs_validate_stripe_geometry(struct xfs_mount *mp,

This declaration might as well lose the extern here too.

Quoted 6 lines

fs/xfs/libxfs/xfs_sb.h extern int xfs_sb_get_secondary(struct xfs_mount *mp, · line 39

Quoted 3 lines
fs/xfs/libxfs/xfs_sb.h +2 -1
-		__s64 sunit, __s64 swidth, int sectorsize, bool silent);
+		__s64 sunit, __s64 swidth, int sectorsize, bool primary_sb,
+		bool silent);

What should value for @primary_sb should mkfs pass into xfs_validate_stripe_geometry from calc_stripe_factors?

Userspace problem, really. i.e. mkfs is already abusing this function by passing a NULL xfs_mount and so will crash if the function tries to dereference mp. Hence it can pass false for "primary_sb" so it doesn't run any of the kernel side primary sb recovery code that dereferences mp because it doesn't need to (can't!) run it.

-Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#3 lore raw
Quoted 95 lines · Dave Chinner
Quoted 65 lines · Darrick J. Wong
Quoted 52 lines · Dave Chinner

From: Dave Chinner <[email protected]>

If a filesystem has a busted stripe alignment configuration on disk (e.g. because broken RAID firmware told mkfs that swidth was smaller than sunit), then the filesystem will refuse to mount due to the stripe validation failing. This failure is triggering during distro upgrades from old kernels lacking this check to newer kernels with this check, and currently the only way to fix it is with offline xfs_db surgery.

This runtime validity checking occurs when we read the superblock for the first time and causes the mount to fail immediately. This prevents the rewrite of stripe unit/width via
mount options that occurs later in the mount process. Hence there is no way to recover this situation without resorting to offline xfs_db rewrite of the values.

However, we parse the mount options long before we read the superblock, and we know if the mount has been asked to re-write the stripe alignment configuration when we are reading the superblock and verifying it for the first time. Hence we can conditionally ignore stripe verification failures if the mount options specified will correct the issue.

We validate that the new stripe unit/width are valid before we overwrite the superblock values, so we can ignore the invalid config at verification and fail the mount later if the new values are not valid. This, at least, gives users the chance of correcting the issue after a kernel upgrade without having to resort to xfs-db hacks.

---
 fs/xfs/libxfs/xfs_sb.c | 40 +++++++++++++++++++++++++++++++---------
 fs/xfs/libxfs/xfs_sb.h |  3 ++-
 2 files changed, 33 insertions(+), 10 deletions(-)
fs/xfs/libxfs/xfs_sb.c +2 -1
index d991eec05436..f51b1efa2cae 100644
@@ -530,7 +530,8 @@ xfs_validate_sb_common(
 	}
 
 	if (!xfs_validate_stripe_geometry(mp, XFS_FSB_TO_B(mp, sbp->sb_unit),
-			XFS_FSB_TO_B(mp, sbp->sb_width), 0, false))
+			XFS_FSB_TO_B(mp, sbp->sb_width), 0,
+			xfs_buf_daddr(bp) == XFS_SB_DADDR, false))
 		return -EFSCORRUPTED;
 
 	/*

fs/xfs/libxfs/xfs_sb.c xfs_sb_get_secondary( · line 1330

Quoted 9 lines
fs/xfs/libxfs/xfs_sb.c +4 -2
@@ -1323,8 +1324,10 @@ xfs_sb_get_secondary(
 }
 
 /*
- * sunit, swidth, sectorsize(optional with 0) should be all in bytes,
- * so users won't be confused by values in error messages.
+ * sunit, swidth, sectorsize(optional with 0) should be all in bytes, so users
+ * won't be confused by values in error messages. This returns false if a value
+ * is invalid and it is not the primary superblock that going to be corrected
+ * later in the mount process.

Hmm, I found this last sentence a little confusing. How about:

Quoted 2 lines

"This function returns false if the stripe geometry is invalid and no attempt will be made to correct it later in the mount process."

Sure.

.....

Quoted 14 lines
Quoted 14 lines
fs/xfs/libxfs/xfs_sb.c +7 -1
@@ -1375,9 +1379,27 @@ xfs_validate_stripe_geometry(
 			xfs_notice(mp,
 "stripe width (%lld) must be a multiple of the stripe unit (%lld)",
 				   swidth, sunit);
-		return false;
+		goto check_override;
 	}
 	return true;
+
+check_override:
+	if (!primary_sb)
+		return false;
+	/*
+	 * During mount, mp->m_dalign will not be set unless the sunit mount
Quoted 8 lines

fs/xfs/libxfs/xfs_sb.c xfs_validate_stripe_geometry( · line 1394

Quoted 4 lines
fs/xfs/libxfs/xfs_sb.c +4 -0
+	 * option was set. If it was set, ignore the bad stripe alignment values
+	 * and allow the validation and overwrite later in the mount process to
+	 * attempt to overwrite the bad stripe alignment values with the values
+	 * supplied by mount options.

What catches the case of if m_dalign/m_swidth also being garbage values? Is it xfs_check_new_dalign? Should that fail the mount if the replacement values are also garbage?

Quoted 4 lines

xfs_validate_new_dalign().

It returns -EINVAL is the new striped alignment cannot be used (i.e. not aligned to block or ag sizes) and that causes the mount to fail.

Ok good.

Quoted 18 lines
Quoted 11 lines

fs/xfs/libxfs/xfs_sb.h extern int xfs_sb_get_secondary(struct xfs_mount *mp, · line 37

Quoted 8 lines
fs/xfs/libxfs/xfs_sb.h +0 -0
index 67a40069724c..58798b9c70ba 100644
@@ -35,7 +35,8 @@ extern int	xfs_sb_get_secondary(struct xfs_mount *mp,
 				struct xfs_buf **bpp);
 
 extern bool	xfs_validate_stripe_geometry(struct xfs_mount *mp,

This declaration might as well lose the extern here too.

Quoted 6 lines

fs/xfs/libxfs/xfs_sb.h extern int xfs_sb_get_secondary(struct xfs_mount *mp, · line 39

Quoted 3 lines
fs/xfs/libxfs/xfs_sb.h +2 -1
-		__s64 sunit, __s64 swidth, int sectorsize, bool silent);
+		__s64 sunit, __s64 swidth, int sectorsize, bool primary_sb,
+		bool silent);

What should value for @primary_sb should mkfs pass into xfs_validate_stripe_geometry from calc_stripe_factors?

Quoted 6 lines

Userspace problem, really. i.e. mkfs is already abusing this function by passing a NULL xfs_mount and so will crash if the function tries to dereference mp. Hence it can pass false for "primary_sb" so it doesn't run any of the kernel side primary sb recovery code that dereferences mp because it doesn't need to (can't!) run it.

At least it's mkfs so there's nothing /to/ recover ;)

I've vaguely wondered if this function ought to drop the @mp argument and instead pass back an error enum, and then the caller can decide if it wants to emit a message or do something else. That would be a worthwhile cleanup for the lurking logic bomb in mkfs; maybe I'll try to code something up later.

--D

Quoted 5 lines

-Dave.

Signature · 2 lines
Dave Chinner
[email protected]