Skip to contentSkip to conversation

xfs_repair: zero out unused parts of superblocks

PATCH In review Eric Sandeen 3 participants 5 messages Aug 15, 2013 – Aug 27, 2013

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 xfs_repair: zero out unused parts of superblocks

Prior to:
1375cb65 xfs: growfs: don't read garbage for new secondary superblocks

we ran the risk of allowing garbage in secondary superblocks beyond the in-use sb fields. With kernels 3.10 and beyond, the verifiers will kick these out as invalid, but xfs_repair does not detect or repair this condition.

There is superblock stale-data zeroing code, but it is under a narrow conditional - the bug addressed in the above commit did not meet that conditional. So change this to check unconditionally.

Further, the checking code was looking at the in-memory superblock buffer, which was zeroed prior to population, and would therefore never possibly show any stale data beyond the last up-rev superblock field.

So instead, check the disk buffer for this garbage condition.

If we detect garbage, we must zero out both the in-memory sb and the disk buffer; the former may contain unused data in up-rev sb fields which will be written back out; the latter may contain garbage beyond all fields, which won't be updated when we translate the in-memory sb back to disk.

The V4 superblock case was zeroing out the sb_bad_features2 field; we also fix that to leave that field alone.

Lastly, use offsetof() instead of the tortured (__psint_t) casts & pointer math.

---

Michael - this will need slight tweaking to apply against older xfsprogs.

Also:

With more of Dave's changes, I think we can swap out:

size = offsetof(xfs_sb_t, sb_lsn)
+ sizeof(sb->sb_lsn);
for

		size = xfs_sb_info[XFS_SBS_LSN + 1].offset;

but this version is a bit easier to backport, and works in the current git tree...

repair/agheader.c +54 -51
index b0f38ba..53e47b6 100644
@@ -256,60 +256,63 @@ secondary_sb_wack(xfs_mount_t *mp, xfs_buf_t *sbuf, xfs_sb_t *sb,
 	rval = do_bzero = 0;
 
 	/*
-	 * mkfs's that stamped a feature bit besides the ones in the mask
-	 * (e.g. were pre-6.5 beta) could leave garbage in the secondary
-	 * superblock sectors.  Anything stamping the shared fs bit or better
-	 * into the secondaries is ok and should generate clean secondary
-	 * superblock sectors.  so only run the zero check on the
-	 * potentially garbaged secondaries.
+	 * Check for garbage beyond the last valid field.
+	 * Use field addresses instead so this code will still
+	 * work against older filesystems when the superblock
+	 * gets rev'ed again with new fields appended.
+	 *
+	 * size is the size of data which is valid for this sb.
 	 */
-	if (pre_65_beta ||
-	    (sb->sb_versionnum & XR_GOOD_SECSB_VNMASK) == 0 ||
-	    sb->sb_versionnum < XFS_SB_VERSION_4)  {
-		/*
-		 * Check for garbage beyond the last field.
-		 * Use field addresses instead so this code will still
-		 * work against older filesystems when the superblock
-		 * gets rev'ed again with new fields appended.
-		 */
-		if (xfs_sb_version_hasmorebits(sb))
-			size = (__psint_t)&sb->sb_features2
-				+ sizeof(sb->sb_features2) - (__psint_t)sb;
-		else if (xfs_sb_version_haslogv2(sb))
-			size = (__psint_t)&sb->sb_logsunit
-				+ sizeof(sb->sb_logsunit) - (__psint_t)sb;
-		else if (xfs_sb_version_hassector(sb))
-			size = (__psint_t)&sb->sb_logsectsize
-				+ sizeof(sb->sb_logsectsize) - (__psint_t)sb;
-		else if (xfs_sb_version_hasdirv2(sb))
-			size = (__psint_t)&sb->sb_dirblklog
-				+ sizeof(sb->sb_dirblklog) - (__psint_t)sb;
-		else
-			size = (__psint_t)&sb->sb_width
-				+ sizeof(sb->sb_width) - (__psint_t)sb;
-		for (ip = (char *)((__psint_t)sb + size);
-		     ip < (char *)((__psint_t)sb + mp->m_sb.sb_sectsize);
-		     ip++)  {
-			if (*ip)  {
-				do_bzero = 1;
-				break;
-			}
-		}
-
-		if (do_bzero)  {
-			rval |= XR_AG_SB_SEC;
-			if (!no_modify)  {
-				do_warn(
-		_("zeroing unused portion of %s superblock (AG #%u)\n"),
-					!i ? _("primary") : _("secondary"), i);
-				memset((void *)((__psint_t)sb + size), 0,
-					mp->m_sb.sb_sectsize - size);
-			} else
-				do_warn(
-		_("would zero unused portion of %s superblock (AG #%u)\n"),
-					!i ? _("primary") : _("secondary"), i);
+	if (xfs_sb_version_hascrc(sb))
+		size = offsetof(xfs_sb_t, sb_lsn)
+			+ sizeof(sb->sb_lsn);
+	else if (xfs_sb_version_hasmorebits(sb))
+		size = offsetof(xfs_sb_t, sb_bad_features2)
+			+ sizeof(sb->sb_bad_features2);
+	else if (xfs_sb_version_haslogv2(sb))
+		size = offsetof(xfs_sb_t, sb_logsunit)
+			+ sizeof(sb->sb_logsunit);
+	else if (xfs_sb_version_hassector(sb))
+		size = offsetof(xfs_sb_t, sb_logsectsize)
+			+ sizeof(sb->sb_logsectsize);
+	else if (xfs_sb_version_hasdirv2(sb))
+		size = offsetof(xfs_sb_t, sb_dirblklog)
+			+ sizeof(sb->sb_dirblklog);
+	else
+		size = offsetof(xfs_sb_t, sb_width)
+			+ sizeof(sb->sb_width);
+
+	/* Check the buffer we read from disk for garbage outside size */
+	for (ip = XFS_BUF_PTR(sbuf) + size;
+	     ip < XFS_BUF_PTR(sbuf) + mp->m_sb.sb_sectsize;
+	     ip++)  {
+		if (*ip)  {
+			do_bzero = 1;
+			break;
 		}
 	}
+	if (do_bzero)  {
+		rval |= XR_AG_SB_SEC;
+		if (!no_modify)  {
+			do_warn(
+	_("zeroing unused portion of %s superblock (AG #%u)\n"),
+				!i ? _("primary") : _("secondary"), i);
+			/*
+			 * zero both the in-memory sb and the disk buffer,
+			 * because the former was read from disk and
+			 * may contain newer version fields that shouldn't
+			 * be set, and the latter is never updated past
+			 * the last field - just zap them both.
+			 */
+			memset((void *)((__psint_t)sb + size), 0,
+				mp->m_sb.sb_sectsize - size);
+			memset(XFS_BUF_PTR(sbuf) + size, 0,
+				mp->m_sb.sb_sectsize - size);
+		} else
+			do_warn(
+	_("would zero unused portion of %s superblock (AG #%u)\n"),
+				!i ? _("primary") : _("secondary"), i);
+	}
 
 	/*
 	 * now look for the fields we can manipulate directly.

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#1 lore raw
Quoted 42 lines · Eric Sandeen

Prior to:
1375cb65 xfs: growfs: don't read garbage for new secondary superblocks

we ran the risk of allowing garbage in secondary superblocks beyond the in-use sb fields. With kernels 3.10 and beyond, the verifiers will kick these out as invalid, but xfs_repair does not detect or repair this condition.

There is superblock stale-data zeroing code, but it is under a narrow conditional - the bug addressed in the above commit did not meet that conditional. So change this to check unconditionally.

Further, the checking code was looking at the in-memory superblock buffer, which was zeroed prior to population, and would therefore never possibly show any stale data beyond the last up-rev superblock field.

So instead, check the disk buffer for this garbage condition.

If we detect garbage, we must zero out both the in-memory sb and the disk buffer; the former may contain unused data in up-rev sb fields which will be written back out; the latter may contain garbage beyond all fields, which won't be updated when we translate the in-memory sb back to disk.

The V4 superblock case was zeroing out the sb_bad_features2 field; we also fix that to leave that field alone.

Lastly, use offsetof() instead of the tortured (__psint_t) casts & pointer math.

---

Michael - this will need slight tweaking to apply against older xfsprogs.

Also:

With more of Dave's changes, I think we can swap out:

Quoted 7 lines

size = offsetof(xfs_sb_t, sb_lsn)
+ sizeof(sb->sb_lsn);
for

		size = xfs_sb_info[XFS_SBS_LSN + 1].offset;

but this version is a bit easier to backport, and works in the current git tree...

I tested with current git tree and it worked as expected for me! Afterwards I was able to run xfs_growfs (from 3.1.11) w/o any problem.

Please excuse me - forgot to set the english locale before starting. Hope you can guess what it should be in english :-).

Phase 1 - Superblock finden und überprüfen...
Schreiben verändert primären Superblock
Phase 2 - ein internes Protokoll benutzen
- Null-Protokoll...
- freier Speicher und Inode-Karten des Dateisystems werden
gescannt...
ungenutzten Anteil des »primär«-Superblocks nullen (AG #0) primäre/sekundärer Superblock-1-Konflikt - AG-Superblock-Geometrie-Info hat einen Konflikt mit der
Dateisystem-Geometrie
falscher sb für ag 1 wird zurückgesetzt
primäre/sekundärer Superblock-2-Konflikt - AG-Superblock-Geometrie-Info hat einen Konflikt mit der
Dateisystem-Geometrie
falscher sb für ag 2 wird zurückgesetzt
primäre/sekundärer Superblock-3-Konflikt - AG-Superblock-Geometrie-Info hat einen Konflikt mit der
Dateisystem-Geometrie
falscher sb für ag 3 wird zurückgesetzt
primäre/sekundärer Superblock-4-Konflikt - AG-Superblock-Geometrie-Info hat einen Konflikt mit der
Dateisystem-Geometrie
falscher sb für ag 4 wird zurückgesetzt
primäre/sekundärer Superblock-5-Konflikt - AG-Superblock-Geometrie-Info hat einen Konflikt mit der
Dateisystem-Geometrie
falscher sb für ag 5 wird zurückgesetzt
primäre/sekundärer Superblock-6-Konflikt - AG-Superblock-Geometrie-Info hat einen Konflikt mit der
Dateisystem-Geometrie
falscher sb für ag 6 wird zurückgesetzt
primäre/sekundärer Superblock-7-Konflikt - AG-Superblock-Geometrie-Info hat einen Konflikt mit der
Dateisystem-Geometrie
falscher sb für ag 7 wird zurückgesetzt
primäre/sekundärer Superblock-8-Konflikt - AG-Superblock-Geometrie-Info hat einen Konflikt mit der
Dateisystem-Geometrie
falscher sb für ag 8 wird zurückgesetzt
primäre/sekundärer Superblock-9-Konflikt - AG-Superblock-Geometrie-Info hat einen Konflikt mit der
Dateisystem-Geometrie
falscher sb für ag 9 wird zurückgesetzt
primäre/sekundärer Superblock-10-Konflikt - AG-Superblock-Geometrie-Info hat einen Konflikt mit der
Dateisystem-Geometrie
falscher sb für ag 10 wird zurückgesetzt
primäre/sekundärer Superblock-11-Konflikt - AG-Superblock-Geometrie-Info hat einen Konflikt mit der
Dateisystem-Geometrie
falscher sb für ag 11 wird zurückgesetzt
primäre/sekundärer Superblock-12-Konflikt - AG-Superblock-Geometrie-Info hat einen Konflikt mit der
Dateisystem-Geometrie
falscher sb für ag 12 wird zurückgesetzt
primäre/sekundärer Superblock-13-Konflikt - AG-Superblock-Geometrie-Info hat einen Konflikt mit der
Dateisystem-Geometrie
falscher sb für ag 13 wird zurückgesetzt
primäre/sekundärer Superblock-14-Konflikt - AG-Superblock-Geometrie-Info hat einen Konflikt mit der
Dateisystem-Geometrie
falscher sb für ag 14 wird zurückgesetzt
primäre/sekundärer Superblock-15-Konflikt - AG-Superblock-Geometrie-Info hat einen Konflikt mit der
Dateisystem-Geometrie
falscher sb für ag 15 wird zurückgesetzt
xfs_sb_read_verify: XFS_CORRUPTION_ERROR
ungenutzten Anteil des »sekundär«-Superblocks nullen (AG #16)
xfs_sb_read_verify: XFS_CORRUPTION_ERROR
ungenutzten Anteil des »sekundär«-Superblocks nullen (AG #17)
xfs_sb_read_verify: XFS_CORRUPTION_ERROR
ungenutzten Anteil des »sekundär«-Superblocks nullen (AG #18)
xfs_sb_read_verify: XFS_CORRUPTION_ERROR
ungenutzten Anteil des »sekundär«-Superblocks nullen (AG #19)
xfs_sb_read_verify: XFS_CORRUPTION_ERROR
ungenutzten Anteil des »sekundär«-Superblocks nullen (AG #20)
xfs_sb_read_verify: XFS_CORRUPTION_ERROR
ungenutzten Anteil des »sekundär«-Superblocks nullen (AG #21)
xfs_sb_read_verify: XFS_CORRUPTION_ERROR
ungenutzten Anteil des »sekundär«-Superblocks nullen (AG #22)
xfs_sb_read_verify: XFS_CORRUPTION_ERROR
ungenutzten Anteil des »sekundär«-Superblocks nullen (AG #23)
xfs_sb_read_verify: XFS_CORRUPTION_ERROR
ungenutzten Anteil des »sekundär«-Superblocks nullen (AG #24)
xfs_sb_read_verify: XFS_CORRUPTION_ERROR
ungenutzten Anteil des »sekundär«-Superblocks nullen (AG #25)
xfs_sb_read_verify: XFS_CORRUPTION_ERROR
ungenutzten Anteil des »sekundär«-Superblocks nullen (AG #26)
xfs_sb_read_verify: XFS_CORRUPTION_ERROR
ungenutzten Anteil des »sekundär«-Superblocks nullen (AG #27)
xfs_sb_read_verify: XFS_CORRUPTION_ERROR
ungenutzten Anteil des »sekundär«-Superblocks nullen (AG #28)
xfs_sb_read_verify: XFS_CORRUPTION_ERROR
ungenutzten Anteil des »sekundär«-Superblocks nullen (AG #29) invalid start block 4096000 in record 1 of bno btree block 41/1 invalid start block 4096000 in record 1 of cnt btree block 41/2 agf_freeblks 3604481, gezählt 1 in ag 41
agf_longest 3604480, gezählt 1 in ag 41
sb_icount 0, counted 6528
sb_ifree 0, counted 669
sb_fdblocks 0, counted 80515
- Wurzel-Inode-Stück gefunden
Phase 3 - für jedes AG...
- agi unverknüpfte Listen werden gescannt und bereinigt...
- bekannte Inodes werden behandelt und Inode-Entdeckung wird
durchgeführt...
- agno = 0
- agno = 1
- agno = 2
- agno = 3
- agno = 4
- agno = 5
- agno = 6
- agno = 7
- agno = 8
- agno = 9
- agno = 10
- agno = 11
- agno = 12
- agno = 13
- agno = 14
- agno = 15
- agno = 16
- agno = 17
- agno = 18
- agno = 19
- agno = 20
- agno = 21
- agno = 22
- agno = 23
- agno = 24
- agno = 25
- agno = 26
- agno = 27
- agno = 28
- agno = 29
- agno = 30
- agno = 31
- agno = 32
- agno = 33
- agno = 34
- agno = 35
- agno = 36
- agno = 37
- agno = 38
- agno = 39
- agno = 40
- agno = 41
- neu entdeckte Inodes werden behandelt...
Phase 4 - auf doppelte Blöcke überprüfen...
- Liste mit doppeltem Ausmaß wird eingerichtet...
- es wird geprüft ob Inodes Blocks doppelt beanspruchen...
- agno = 0
- agno = 2
- agno = 1
- agno = 4
- agno = 3
- agno = 6
- agno = 7
- agno = 5
- agno = 8
- agno = 9
- agno = 10
- agno = 11
- agno = 12
- agno = 13
- agno = 14
- agno = 15
- agno = 16
- agno = 17
- agno = 18
- agno = 19
- agno = 20
- agno = 21
- agno = 22
- agno = 23
- agno = 24
- agno = 25
- agno = 26
- agno = 27
- agno = 29
- agno = 30
- agno = 28
- agno = 31
- agno = 33
- agno = 32
- agno = 34
- agno = 35
- agno = 36
- agno = 38
- agno = 37
- agno = 40
- agno = 39
- agno = 41
Phase 5 - AG-Köpfe und Bäume werden erneut gebildet...
- Superblock wird zurückgesetzt...
Phase 6 - Inode-Verbindbarkeit wird geprüft...
- Inhalte der Echtzeit-Bitmaps und Zusammenfassungs-Inodes
werden zurückgesetzt
- Dateisystem wird durchquert ...
- durchqueren beendet ...
- nicht verbundene Inodes werden nach lost+found verschoben ...
Phase 7 - Verweisanzahl wird geprüft und berichtigt... erledigt

Thanks,
kind regards,
Michael

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#2 lore raw
Quoted 48 lines · Michael Maier
Quoted 47 lines · Eric Sandeen

Prior to:
1375cb65 xfs: growfs: don't read garbage for new secondary superblocks

we ran the risk of allowing garbage in secondary superblocks beyond the in-use sb fields. With kernels 3.10 and beyond, the verifiers will kick these out as invalid, but xfs_repair does not detect or repair this condition.

There is superblock stale-data zeroing code, but it is under a narrow conditional - the bug addressed in the above commit did not meet that conditional. So change this to check unconditionally.

Further, the checking code was looking at the in-memory superblock buffer, which was zeroed prior to population, and would therefore never possibly show any stale data beyond the last up-rev superblock field.

So instead, check the disk buffer for this garbage condition.

If we detect garbage, we must zero out both the in-memory sb and the disk buffer; the former may contain unused data in up-rev sb fields which will be written back out; the latter may contain garbage beyond all fields, which won't be updated when we translate the in-memory sb back to disk.

The V4 superblock case was zeroing out the sb_bad_features2 field; we also fix that to leave that field alone.

Lastly, use offsetof() instead of the tortured (__psint_t) casts & pointer math.

---

Michael - this will need slight tweaking to apply against older xfsprogs.

Also:

With more of Dave's changes, I think we can swap out:

size = offsetof(xfs_sb_t, sb_lsn)
+ sizeof(sb->sb_lsn);
for

		size = xfs_sb_info[XFS_SBS_LSN + 1].offset;
Quoted 9 lines
Quoted 2 lines

but this version is a bit easier to backport, and works in the current git tree...

I tested with current git tree and it worked as expected for me! Afterwards I was able to run xfs_growfs (from 3.1.11) w/o any problem.

Please excuse me - forgot to set the english locale before starting. Hope you can guess what it should be in english :-).

Kein Problem, ich kann etwas Deutsch zu lesen. :)

Glad it worked for you!

-Eric

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

Reviewed-by
#3 lore raw
Quoted 52 lines · Eric Sandeen
Quoted 51 lines · Michael Maier
Quoted 49 lines · Eric Sandeen

Prior to:
1375cb65 xfs: growfs: don't read garbage for new secondary superblocks

we ran the risk of allowing garbage in secondary superblocks beyond the in-use sb fields. With kernels 3.10 and beyond, the verifiers will kick these out as invalid, but xfs_repair does not detect or repair this condition.

There is superblock stale-data zeroing code, but it is under a narrow conditional - the bug addressed in the above commit did not meet that conditional. So change this to check unconditionally.

Further, the checking code was looking at the in-memory superblock buffer, which was zeroed prior to population, and would therefore never possibly show any stale data beyond the last up-rev superblock field.

So instead, check the disk buffer for this garbage condition.

If we detect garbage, we must zero out both the in-memory sb and the disk buffer; the former may contain unused data in up-rev sb fields which will be written back out; the latter may contain garbage beyond all fields, which won't be updated when we translate the in-memory sb back to disk.

The V4 superblock case was zeroing out the sb_bad_features2 field; we also fix that to leave that field alone.

Lastly, use offsetof() instead of the tortured (__psint_t) casts & pointer math.

---

Michael - this will need slight tweaking to apply against older xfsprogs.

Also:

With more of Dave's changes, I think we can swap out:

size = offsetof(xfs_sb_t, sb_lsn)
+ sizeof(sb->sb_lsn);
for

		size = xfs_sb_info[XFS_SBS_LSN + 1].offset;

but this version is a bit easier to backport, and works in the current git tree...

Quoted 8 lines
Quoted 6 lines

I tested with current git tree and it worked as expected for me! Afterwards I was able to run xfs_growfs (from 3.1.11) w/o any problem.

Please excuse me - forgot to set the english locale before starting. Hope you can guess what it should be in english :-).

Kein Problem, ich kann etwas Deutsch zu lesen. :)

Ich auch :)

Code looks good.

Quoted 10 lines

Glad it worked for you!

-Eric

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs