xfs: clean up directory data block header padding
v3 cover xfs: clean up directory data block header padding
The pad field in xfs_dir3_data_hdr exists for 64-bit alignment and should always be zero. Old kernels zeroed only the xfs_dir3_blk_hdr portion of the header during initialization and initialized the remaining header contents separately.
This series takes a broader approach as suggested by Dave Chinner:
zero the entire header region unconditionally, self-heal stale non-zero padding during writes, and report remaining non-zero padding in scrub.
Patch 1 zeroes the entire directory data block header region in xfs_dir3_data_init instead of just the xfs_dir3_blk_hdr. Patch 2 zeroes the pad field in xfs_dir3_data_write_verify so that existing directories self-heal when modified.
Patch 3 checks the pad field in scrub and adds xchk_fblock_set_preen.
Changes in v3:
- Zero the entire header region (geo->data_entry_offset bytes) unconditionally instead of changing the memset type to xfs_dir3_data_hdr.
- Remove the now-unnecessary bestfree manual zeroing loop.
- Zero pad in xfs_dir3_data_write_verify for runtime self-healing
of old metadata.
- Drop unnecessary cpu_to_be32() on comparison to zero.
- Split into a 3-patch series for easier review.
- Link to v2: https://lore.kernel.org/all/[email protected]/#t
Changes in v2:
- Fix xfs_dir3_data_init to zero the full xfs_dir3_data_hdr instead of just xfs_dir3_blk_hdr so that the pad field is covered by the memset.
- Use xchk_fblock_set_preen instead of xchk_fblock_set_corrupt since
old kernels may have written non-zero padding without issues.
- Add xchk_fblock_set_preen helper function.
- Link to v1: https://lore.kernel.org/all/[email protected]/#t
Yuto Ohnuki (3):
xfs: zero entire directory data block header region at init
xfs: zero directory data block padding on write verification
xfs: check directory data block header padding in scrub
fs/xfs/libxfs/xfs_dir2_data.c | 18 +++++++++++------- fs/xfs/scrub/common.c | 11 +++++++++++ fs/xfs/scrub/common.h | 2 ++ fs/xfs/scrub/dir.c | 7 ++++++- 4 files changed, 30 insertions(+), 8 deletions(-)
Signature · 8 lines
2.50.1 Amazon Web Services EMEA SARL, 38 avenue John F. Kennedy, L-1855 Luxembourg, R.C.S. Luxembourg B186284 Amazon Web Services EMEA SARL, Irish Branch, One Burlington Plaza, Burlington Road, Dublin 4, Ireland, branch registration number 908705
PATCH v3 1/3 xfs: zero entire directory data block header region at init
xfs_dir3_data_init currently zeroes only the xfs_dir3_blk_hdr portion of the directory data block header, then manually initializes the bestfree entries in a loop. This leaves the pad field in xfs_dir3_data_hdr uninitialized and requires explicit zeroing of each bestfree slot.
Zero the entire header region (geo->data_entry_offset bytes) unconditionally before setting individual fields. This covers all current and future header fields, all padding (implicit and explicit), and the bestfree array, so the manual zeroing loop for bestfree can be removed.
--- fs/xfs/libxfs/xfs_dir2_data.c | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-)
fs/xfs/libxfs/xfs_dir2_data.c +5 -7
@@ -728,7 +728,6 @@ xfs_dir3_data_init(struct xfs_dir2_data_unused *dup;struct xfs_dir2_data_free *bf;int error; -int i;/* * Get the buffer set up for the block.
@@ -741,13 +740,16 @@ xfs_dir3_data_init(xfs_trans_buf_set_type (tp, bp,XFS_BLFT_DIR_DATA_BUF );/* -* Initialize the header. +* Initialize the whole directory header region to zero +* so that all padding, bestfree entries, and any +* future header fields are clean. */ hdr = bp->b_addr; +memset (hdr,0 , geo->data_entry_offset); +if (xfs_has_crc (mp)) {struct xfs_dir3_blk_hdr *hdr3 = bp->b_addr; -memset (hdr3,0 ,sizeof (*hdr3)); hdr3->magic =cpu_to_be32 (XFS_DIR3_DATA_MAGIC ); hdr3->blkno =cpu_to_be64 (xfs_buf_daddr (bp)); hdr3->owner =cpu_to_be64 (args->owner);
@@ -759,10 +761,6 @@ xfs_dir3_data_init( bf =xfs_dir2_data_bestfree_p (mp, hdr); bf[0 ].offset =cpu_to_be16 (geo->data_entry_offset); bf[0 ].length =cpu_to_be16 (geo->blksize - geo->data_entry_offset); -for (i =1 ; i <XFS_DIR2_DATA_FD_COUNT ; i++) { - bf[i].length =0 ; - bf[i].offset =0 ; - }/* * Set up an unused entry for the block's body.
Signature · 8 lines
2.50.1 Amazon Web Services EMEA SARL, 38 avenue John F. Kennedy, L-1855 Luxembourg, R.C.S. Luxembourg B186284 Amazon Web Services EMEA SARL, Irish Branch, One Burlington Plaza, Burlington Road, Dublin 4, Ireland, branch registration number 908705
Re: [PATCH v3 1/3] xfs: zero entire directory data block header region at init
Quoted 5 lines · Yuto Ohnuki
xfs_dir3_data_initcurrently zeroes only thexfs_dir3_blk_hdrportion of the directory data block header, then manually initializes the bestfree entries in a loop. This leaves the pad field inxfs_dir3_data_hdruninitialized and requires explicit zeroing of each bestfree slot.
Quoted 8 lines
Zero the entire header region (
geo->data_entry_offsetbytes) unconditionally before setting individual fields. This covers all current and future header fields, all padding (implicit and explicit), and the bestfree array, so the manual zeroing loop for bestfree can be removed.Suggested-by Dave Chinner <[email protected]>Signed-off-by Yuto Ohnuki <[email protected]>
Seems reasonable to me to zero the entire head including the bestfree areas.
--D
Quoted 59 lines
--- fs/xfs/libxfs/xfs_dir2_data.c | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-)fs/xfs/libxfs/xfs_dir2_data.c +5
-7@@ -728,7 +728,6 @@ xfs_dir3_data_init(struct xfs_dir2_data_unused *dup;struct xfs_dir2_data_free *bf;int error; -int i;/* * Get the buffer set up for the block. @@ -741,13 +740,16 @@ xfs_dir3_data_init(xfs_trans_buf_set_type (tp, bp,XFS_BLFT_DIR_DATA_BUF );/* -* Initialize the header. +* Initialize the whole directory header region to zero +* so that all padding, bestfree entries, and any +* future header fields are clean. */ hdr = bp->b_addr; +memset (hdr,0 , geo->data_entry_offset); +if (xfs_has_crc (mp)) {struct xfs_dir3_blk_hdr *hdr3 = bp->b_addr; -memset (hdr3,0 ,sizeof (*hdr3)); hdr3->magic =cpu_to_be32 (XFS_DIR3_DATA_MAGIC ); hdr3->blkno =cpu_to_be64 (xfs_buf_daddr (bp)); hdr3->owner =cpu_to_be64 (args->owner);@@ -759,10 +761,6 @@ xfs_dir3_data_init( bf =xfs_dir2_data_bestfree_p (mp, hdr); bf[0 ].offset =cpu_to_be16 (geo->data_entry_offset); bf[0 ].length =cpu_to_be16 (geo->blksize - geo->data_entry_offset); -for (i =1 ; i <XFS_DIR2_DATA_FD_COUNT ; i++) { - bf[i].length =0 ; - bf[i].offset =0 ; - }/* * Set up an unused entry for the block's body. Signature · 8 lines
2.50.1 Amazon Web Services EMEA SARL, 38 avenue John F. Kennedy, L-1855 Luxembourg, R.C.S. Luxembourg B186284 Amazon Web Services EMEA SARL, Irish Branch, One Burlington Plaza, Burlington Road, Dublin 4, Ireland, branch registration number 908705
PATCH v3 2/3 xfs: zero directory data block padding on write verification
In reply to Yuto OhnukiOld kernels did not zero the pad field in xfs_dir3_data_hdr when initializing directory data blocks, so existing filesystems may have non-zero padding on disk.
Zero the pad field in xfs_dir3_data_write_verify alongside the existing LSN and checksum updates. The pad field is pure alignment padding with no runtime meaning, so zeroing it during write verification is safe and has no additional I/O cost. This lets filesystems gradually self-heal stale non-zero padding as directories are modified, without requiring an explicit repair pass.
--- fs/xfs/libxfs/xfs_dir2_data.c | 6 ++++++ 1 file changed, 6 insertions(+)
fs/xfs/libxfs/xfs_dir2_data.c +6 -0
@@ -382,6 +382,7 @@ xfs_dir3_data_write_verify(struct xfs_mount *mp = bp->b_mount;struct xfs_buf_log_item *bip = bp->b_log_item;struct xfs_dir3_blk_hdr *hdr3 = bp->b_addr; +struct xfs_dir3_data_hdr *datahdr3 = bp->b_addr;xfs_failaddr_t fa; fa =xfs_dir3_data_verify (bp);
@@ -396,6 +397,11 @@ xfs_dir3_data_write_verify(if (bip) hdr3->lsn =cpu_to_be64 (bip->bli_item.li_lsn); +/* +* Zero padding that may be stale from old kernels. +*/ + datahdr3->pad =0 ; +xfs_buf_update_cksum (bp,XFS_DIR3_DATA_CRC_OFF ); }
Signature · 8 lines
2.50.1 Amazon Web Services EMEA SARL, 38 avenue John F. Kennedy, L-1855 Luxembourg, R.C.S. Luxembourg B186284 Amazon Web Services EMEA SARL, Irish Branch, One Burlington Plaza, Burlington Road, Dublin 4, Ireland, branch registration number 908705
Re: [PATCH v3 2/3] xfs: zero directory data block padding on write verification
Quoted 30 lines · Yuto Ohnuki
Old kernels did not zero the pad field in
xfs_dir3_data_hdrwhen initializing directory data blocks, so existing filesystems may have non-zero padding on disk.Zero the pad field in
xfs_dir3_data_write_verifyalongside the existing LSN and checksum updates. The pad field is pure alignment padding with no runtime meaning, so zeroing it during write verification is safe and has no additional I/O cost. This lets filesystems gradually self-heal stale non-zero padding as directories are modified, without requiring an explicit repair pass.Suggested-by Dave Chinner <[email protected]>Signed-off-by Yuto Ohnuki <[email protected]>--- fs/xfs/libxfs/xfs_dir2_data.c | 6 ++++++ 1 file changed, 6 insertions(+)fs/xfs/libxfs/xfs_dir2_data.c +1
-0@@ -382,6 +382,7 @@ xfs_dir3_data_write_verify(struct xfs_mount *mp = bp->b_mount;struct xfs_buf_log_item *bip = bp->b_log_item;struct xfs_dir3_blk_hdr *hdr3 = bp->b_addr; +struct xfs_dir3_data_hdr *datahdr3 = bp->b_addr;xfs_failaddr_t fa; fa =xfs_dir3_data_verify (bp);
fs/xfs/libxfs/xfs_dir2_data.c xfs_dir3_data_write_verify( · line 403
Quoted 7 lines
fs/xfs/libxfs/xfs_dir2_data.c +4
-0@@ -396,6 +397,11 @@ xfs_dir3_data_write_verify(if (bip) hdr3->lsn =cpu_to_be64 (bip->bli_item.li_lsn); +/* +* Zero padding that may be stale from old kernels. +*/ + datahdr3->pad =0 ;
This needs to be gated on the directory block actually having a dir3 data header (aka xfs_has_crc()).
--D
Quoted 17 lines
fs/xfs/libxfs/xfs_dir2_data.c +1
-0+xfs_buf_update_cksum (bp,XFS_DIR3_DATA_CRC_OFF ); }Signature · 8 lines
2.50.1 Amazon Web Services EMEA SARL, 38 avenue John F. Kennedy, L-1855 Luxembourg, R.C.S. Luxembourg B186284 Amazon Web Services EMEA SARL, Irish Branch, One Burlington Plaza, Burlington Road, Dublin 4, Ireland, branch registration number 908705
Re: [PATCH v3 2/3] xfs: zero directory data block padding on write verification
Quoted 34 lines · Darrick J. Wong
Quoted 33 lines · Yuto Ohnuki
Old kernels did not zero the pad field in
xfs_dir3_data_hdrwhen initializing directory data blocks, so existing filesystems may have non-zero padding on disk.Zero the pad field in
xfs_dir3_data_write_verifyalongside the existing LSN and checksum updates. The pad field is pure alignment padding with no runtime meaning, so zeroing it during write verification is safe and has no additional I/O cost. This lets filesystems gradually self-heal stale non-zero padding as directories are modified, without requiring an explicit repair pass.Suggested-by Dave Chinner <[email protected]>Signed-off-by Yuto Ohnuki <[email protected]>--- fs/xfs/libxfs/xfs_dir2_data.c | 6 ++++++ 1 file changed, 6 insertions(+)fs/xfs/libxfs/xfs_dir2_data.c +1
-0@@ -382,6 +382,7 @@ xfs_dir3_data_write_verify(struct xfs_mount *mp = bp->b_mount;struct xfs_buf_log_item *bip = bp->b_log_item;struct xfs_dir3_blk_hdr *hdr3 = bp->b_addr; +struct xfs_dir3_data_hdr *datahdr3 = bp->b_addr;xfs_failaddr_t fa; fa =xfs_dir3_data_verify (bp);@@ -396,6 +397,11 @@ xfs_dir3_data_write_verify(if (bip) hdr3->lsn =cpu_to_be64 (bip->bli_item.li_lsn);
Quoted 7 lines
fs/xfs/libxfs/xfs_dir2_data.c xfs_dir3_data_write_verify( · line 403
Quoted 4 lines
fs/xfs/libxfs/xfs_dir2_data.c +4
-0+/* +* Zero padding that may be stale from old kernels. +*/ + datahdr3->pad =0 ;This needs to be gated on the directory block actually having a dir3 data header (aka
xfs_has_crc()).
It already is gated correctly - this check is done 5 lines above:
if (!xfs_has_crc(mp)) return;
-Dave.
Signature · 2 lines
Dave Chinner [email protected]
Darrick J. Wong
Reviewed-byRe: [PATCH v3 2/3] xfs: zero directory data block padding on write verification
Quoted 43 lines · Dave Chinner
Quoted 34 lines · Darrick J. Wong
Quoted 33 lines · Yuto Ohnuki
Old kernels did not zero the pad field in
xfs_dir3_data_hdrwhen initializing directory data blocks, so existing filesystems may have non-zero padding on disk.Zero the pad field in
xfs_dir3_data_write_verifyalongside the existing LSN and checksum updates. The pad field is pure alignment padding with no runtime meaning, so zeroing it during write verification is safe and has no additional I/O cost. This lets filesystems gradually self-heal stale non-zero padding as directories are modified, without requiring an explicit repair pass.Suggested-by Dave Chinner <[email protected]>Signed-off-by Yuto Ohnuki <[email protected]>--- fs/xfs/libxfs/xfs_dir2_data.c | 6 ++++++ 1 file changed, 6 insertions(+)fs/xfs/libxfs/xfs_dir2_data.c +1
-0@@ -382,6 +382,7 @@ xfs_dir3_data_write_verify(struct xfs_mount *mp = bp->b_mount;struct xfs_buf_log_item *bip = bp->b_log_item;struct xfs_dir3_blk_hdr *hdr3 = bp->b_addr; +struct xfs_dir3_data_hdr *datahdr3 = bp->b_addr;xfs_failaddr_t fa; fa =xfs_dir3_data_verify (bp);@@ -396,6 +397,11 @@ xfs_dir3_data_write_verify(if (bip) hdr3->lsn =cpu_to_be64 (bip->bli_item.li_lsn);Quoted 7 lines
fs/xfs/libxfs/xfs_dir2_data.c xfs_dir3_data_write_verify( · line 403
Quoted 4 lines
fs/xfs/libxfs/xfs_dir2_data.c +4
-0+/* +* Zero padding that may be stale from old kernels. +*/ + datahdr3->pad =0 ;This needs to be gated on the directory block actually having a dir3 data header (aka
xfs_has_crc()).
Quoted 4 lines
It already is gated correctly - this check is done 5 lines above:
if (!xfs_has_crc(mp)) return;
So it is. My bad;
--D
Quoted 7 lines
-Dave.
Signature · 2 lines
Dave Chinner [email protected]
PATCH v3 3/3 xfs: check directory data block header padding in scrub
In reply to Yuto OhnukiAdd the missing scrub check for the pad field in directory data block headers. Old kernels may have written non-zero padding without issue, and the write path now self-heals stale padding on modification. Flag non-zero padding as an optimization opportunity (preen) rather than corruption.
Add xchk_fblock_set_preen helper for reporting file fork block issues that could be optimized. The trace event xchk_fblock_preen already exists.
--- fs/xfs/scrub/common.c | 11 +++++++++++ fs/xfs/scrub/common.h | 2 ++ fs/xfs/scrub/dir.c | 7 ++++++- 3 files changed, 19 insertions(+), 1 deletion(-)
fs/xfs/scrub/common.c +11 -0
@@ -251,6 +251,17 @@ xchk_ino_set_preen(trace_xchk_ino_preen (sc, ino, __return_address); } +/* Record a block indexed by a file fork that could be optimized. */ +void +xchk_fblock_set_preen ( +struct xfs_scrub *sc, +int whichfork, +xfs_fileoff_t offset) +{ + sc->sm->sm_flags |=XFS_SCRUB_OFLAG_PREEN ; +trace_xchk_fblock_preen (sc, whichfork, offset, __return_address); +} +/* Record something being wrong with the filesystem primary superblock. */ void xchk_set_corrupt(
fs/xfs/scrub/common.h +2 -0
@@ -25,6 +25,8 @@ bool xchk_fblock_xref_process_error(struct xfs_scrub *sc,void xchk_block_set_preen (struct xfs_scrub *sc,struct xfs_buf *bp);void xchk_ino_set_preen (struct xfs_scrub *sc,xfs_ino_t ino); +void xchk_fblock_set_preen (struct xfs_scrub *sc, +int whichfork,xfs_fileoff_t offset);void xchk_set_corrupt (struct xfs_scrub *sc); void xchk_block_set_corrupt(struct xfs_scrub *sc,
fs/xfs/scrub/dir.c +6 -1
@@ -492,7 +492,12 @@ xchk_directory_data_bestfree( goto out;xchk_buffer_recheck (sc, bp); -/* XXX: Check xfs_dir3_data_hdr.pad is zero once we start setting it. */ +if (xfs_has_crc (sc->mp)) { +struct xfs_dir3_data_hdr *hdr3 = bp->b_addr; + +if (hdr3->pad) +xchk_fblock_set_preen (sc,XFS_DATA_FORK , lblk); + }if (sc->sm->sm_flags &XFS_SCRUB_OFLAG_CORRUPT ) goto out_buf;
Signature · 8 lines
2.50.1 Amazon Web Services EMEA SARL, 38 avenue John F. Kennedy, L-1855 Luxembourg, R.C.S. Luxembourg B186284 Amazon Web Services EMEA SARL, Irish Branch, One Burlington Plaza, Burlington Road, Dublin 4, Ireland, branch registration number 908705
Re: [PATCH v3 3/3] xfs: check directory data block header padding in scrub
Quoted 57 lines · Yuto Ohnuki
Add the missing scrub check for the pad field in directory data block headers. Old kernels may have written non-zero padding without issue, and the write path now self-heals stale padding on modification. Flag non-zero padding as an optimization opportunity (preen) rather than corruption.
Add
xchk_fblock_set_preenhelper for reporting file fork block issues that could be optimized. The trace eventxchk_fblock_preenalready exists.Signed-off-by Yuto Ohnuki <[email protected]>--- fs/xfs/scrub/common.c | 11 +++++++++++ fs/xfs/scrub/common.h | 2 ++ fs/xfs/scrub/dir.c | 7 ++++++- 3 files changed, 19 insertions(+), 1 deletion(-)fs/xfs/scrub/common.c +11
-0@@ -251,6 +251,17 @@ xchk_ino_set_preen(trace_xchk_ino_preen (sc, ino, __return_address); } +/* Record a block indexed by a file fork that could be optimized. */ +void +xchk_fblock_set_preen ( +struct xfs_scrub *sc, +int whichfork, +xfs_fileoff_t offset) +{ + sc->sm->sm_flags |=XFS_SCRUB_OFLAG_PREEN ; +trace_xchk_fblock_preen (sc, whichfork, offset, __return_address); +} +/* Record something being wrong with the filesystem primary superblock. */ void xchk_set_corrupt(fs/xfs/scrub/common.h +2
-0@@ -25,6 +25,8 @@ bool xchk_fblock_xref_process_error(struct xfs_scrub *sc,void xchk_block_set_preen (struct xfs_scrub *sc,struct xfs_buf *bp);void xchk_ino_set_preen (struct xfs_scrub *sc,xfs_ino_t ino); +void xchk_fblock_set_preen (struct xfs_scrub *sc, +int whichfork,xfs_fileoff_t offset);void xchk_set_corrupt (struct xfs_scrub *sc); void xchk_block_set_corrupt(struct xfs_scrub *sc,fs/xfs/scrub/dir.c +0
-0
fs/xfs/scrub/dir.c xchk_directory_data_bestfree( · line 499
Quoted 9 lines
fs/xfs/scrub/dir.c +5
-1@@ -492,7 +492,12 @@ xchk_directory_data_bestfree( goto out;xchk_buffer_recheck (sc, bp); -/* XXX: Check xfs_dir3_data_hdr.pad is zero once we start setting it. */ +if (xfs_has_crc (sc->mp)) { +struct xfs_dir3_data_hdr *hdr3 = bp->b_addr; + +if (hdr3->pad) +xchk_fblock_set_preen (sc,XFS_DATA_FORK , lblk);
Looks good! Thanks for patching this hole :)
--D
Quoted 17 lines
fs/xfs/scrub/dir.c +1
-0+ }if (sc->sm->sm_flags &XFS_SCRUB_OFLAG_CORRUPT ) goto out_buf;Signature · 8 lines
2.50.1 Amazon Web Services EMEA SARL, 38 avenue John F. Kennedy, L-1855 Luxembourg, R.C.S. Luxembourg B186284 Amazon Web Services EMEA SARL, Irish Branch, One Burlington Plaza, Burlington Road, Dublin 4, Ireland, branch registration number 908705
Quoted 10 lines · Yuto Ohnuki
The pad field in
xfs_dir3_data_hdrexists for 64-bit alignment and should always be zero. Old kernels zeroed only thexfs_dir3_blk_hdrportion of the header during initialization and initialized the remaining header contents separately.This series takes a broader approach as suggested by Dave Chinner:
zero the entire header region unconditionally, self-heal stale non-zero padding during writes, and report remaining non-zero padding in scrub.[...]
Applied to for-next, thanks!
[1/3] xfs: zero entire directory data block header region at init
commit: 00dd8d7ec5253c6273023a0fd6dc08683e0bdfef
[2/3] xfs: zero directory data block padding on write verification
commit: 8fbb1877dfa5e26bda1baf8cc6abd3f805098486
[3/3] xfs: check directory data block header padding in scrub
commit: 939919ccddfcc379bb82b1f90f732d9a5cb32cc8Best regards,
Signature · 1 line
Carlos Maiolino <[email protected]>