xfs_db: Scan entire file system when using 'frag'
PATCH 1/1 xfs_db: Scan entire file system when using 'frag'
From: Jorge Guerra <[email protected]>
While running the 'frag' command of 'xfs_db' we noticed that the tool is not scanning all the files in the file system. We noticed this when we modified the tool to print the inodes of all the files scanned. For example:
$ find /mnt/xfsdisk -type f | wc -l
1782674
$ xfs_db -r -c frag /dev/sdXX | grep MB | awk '{print $5}' | paste -s -d+ | bc
656818Upon inspecting the code we noticed that the scanfunc_ino function stops processing a given inode block once it encounters a free leaf. However, in practice we see that inodes are necessarily always layed out contiguously on the leaf node. This resulted in the 'frag' command skipping some valid inodes.
In this change we modify the scanfunc_ino function to skip freed inodes. With the change in place we ran the same experiment again and noticed a more accurate file count:
$ find /mnt/d0 -type f | wc -l
1810442
$ xfs_db -r -c frag /dev/sdXX | grep MB | awk '{print $5}' | paste -s -d+ | bc
1810442--- db/frag.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
db/frag.c +1 -1
@@ -507,7 +507,7 @@ scanfunc_ino(for (j =0 ; j < inodes_per_buf; j++) {if (XFS_INOBT_IS_FREE_DISK (&rp[i], ioff + j)) - goto next_buf; +continue ; dip = (xfs_dinode_t *)((char *)iocur_top->data + ((off + j) << mp->m_sb.sb_inodelog));process_inode (agf, agino + ioff + j, dip);
Signature · 1 line
2.13.5
Quoted 22 lines · Jorge Guerra
From: Jorge Guerra <[email protected]>
While running the 'frag' command of '
xfs_db' we noticed that the tool is not scanning all the files in the file system. We noticed this when we modified the tool to print the inodes of all the files scanned. For example:$ find /mnt/xfsdisk -type f | wc -l 1782674 $ xfs_db -r -c frag /dev/sdXX | grep MB | awk '{print $5}' | paste -s -d+ | bc 656818Upon inspecting the code we noticed that the
scanfunc_inofunction stops processing a given inode block once it encounters a free leaf. However, in practice we see that inodes are necessarily always layed out contiguously on the leaf node. This resulted in the 'frag' command skipping some valid inodes.In this change we modify the
scanfunc_inofunction to skip freed inodes. With the change in place we ran the same experiment again and noticed a more accurate file count:
Quoted 6 lines
$ find /mnt/d0 -type f | wc -l 1810442 $ xfs_db -r -c frag /dev/sdXX | grep MB | awk '{print $5}' | paste -s -d+ | bc 1810442Signed-off-by Jorge Guerra <[email protected]>
This looks right, but I'll warn you that xfs_db's frag command is largely useless in the first place. ;)
Also, I think:
Thanks!
Quoted 18 lines
--- db/frag.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)db/frag.c +1
-1@@ -507,7 +507,7 @@ scanfunc_ino(for (j =0 ; j < inodes_per_buf; j++) {if (XFS_INOBT_IS_FREE_DISK (&rp[i], ioff + j)) - goto next_buf; +continue ; dip = (xfs_dinode_t *)((char *)iocur_top->data + ((off + j) << mp->m_sb.sb_inodelog));process_inode (agf, agino + ioff + j, dip);
Darrick J. Wong
Reviewed-byQuoted 28 lines · Eric Sandeen
Quoted 27 lines · Jorge Guerra
From: Jorge Guerra <[email protected]>
While running the 'frag' command of '
xfs_db' we noticed that the tool is not scanning all the files in the file system. We noticed this when we modified the tool to print the inodes of all the files scanned. For example:$ find /mnt/xfsdisk -type f | wc -l 1782674 $ xfs_db -r -c frag /dev/sdXX | grep MB | awk '{print $5}' | paste -s -d+ | bc 656818Upon inspecting the code we noticed that the
scanfunc_inofunction stops processing a given inode block once it encounters a free leaf. However, in practice we see that inodes are necessarily always layed out contiguously on the leaf node. This resulted in the 'frag' command skipping some valid inodes.In this change we modify the
scanfunc_inofunction to skip freed inodes. With the change in place we ran the same experiment again and noticed a more accurate file count:$ find /mnt/d0 -type f | wc -l 1810442 $ xfs_db -r -c frag /dev/sdXX | grep MB | awk '{print $5}' | paste -s -d+ | bc 1810442
Quoted 4 lines
Quoted 1 line
Signed-off-by Jorge Guerra <[email protected]>This looks right, but I'll warn you that xfs_db's frag command is largely useless in the first place. ;)
I was looking through the manpages for the part where it says frag is useless but couldn't find it. Oh well. :(
--D
Quoted 27 lines
Also, I think:
Fixes 2a5eb70c ("xfs_db: teach the frag command about sparse inode chunks")Reviewed-by Eric Sandeen <[email protected]>Thanks!
Quoted 18 lines
--- db/frag.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)db/frag.c +1
-1@@ -507,7 +507,7 @@ scanfunc_ino(for (j =0 ; j < inodes_per_buf; j++) {if (XFS_INOBT_IS_FREE_DISK (&rp[i], ioff + j)) - goto next_buf; +continue ; dip = (xfs_dinode_t *)((char *)iocur_top->data + ((off + j) << mp->m_sb.sb_inodelog));process_inode (agf, agino + ioff + j, dip);
Lol!
Thanks Darrick!
Quoted 69 lines · Darrick J. Wong
Quoted 28 lines · Eric Sandeen
Quoted 27 lines · Jorge Guerra
From: Jorge Guerra <[email protected]>
While running the 'frag' command of '
xfs_db' we noticed that the tool is not scanning all the files in the file system. We noticed this when we modified the tool to print the inodes of all the files scanned. For example:$ find /mnt/xfsdisk -type f | wc -l 1782674 $ xfs_db -r -c frag /dev/sdXX | grep MB | awk '{print $5}' | paste -s -d+ | bc 656818Upon inspecting the code we noticed that the
scanfunc_inofunction stops processing a given inode block once it encounters a free leaf. However, in practice we see that inodes are necessarily always layed out contiguously on the leaf node. This resulted in the 'frag' command skipping some valid inodes.In this change we modify the
scanfunc_inofunction to skip freed inodes. With the change in place we ran the same experiment again and noticed a more accurate file count:$ find /mnt/d0 -type f | wc -l 1810442 $ xfs_db -r -c frag /dev/sdXX | grep MB | awk '{print $5}' | paste -s -d+ | bc 1810442Quoted 4 lines
Quoted 1 line
Signed-off-by Jorge Guerra <[email protected]>This looks right, but I'll warn you that xfs_db's frag command is largely useless in the first place. ;)
I was looking through the manpages for the part where it says frag is useless but couldn't find it. Oh well. :(
Reviewed-by Darrick J. Wong <[email protected]>--D
Quoted 27 lines
Also, I think:
Fixes 2a5eb70c ("xfs_db: teach the frag command about sparse inode chunks")Reviewed-by Eric Sandeen <[email protected]>Thanks!
Quoted 18 lines
--- db/frag.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)db/frag.c +1
-1@@ -507,7 +507,7 @@ scanfunc_ino(for (j =0 ; j < inodes_per_buf; j++) {if (XFS_INOBT_IS_FREE_DISK (&rp[i], ioff + j)) - goto next_buf; +continue ; dip = (xfs_dinode_t *)((char *)iocur_top->data + ((off + j) << mp->m_sb.sb_inodelog));process_inode (agf, agino + ioff + j, dip);
Signature · 1 line
Jorge E Guerra D
Thanks Eric,
I'll update the title.
We are using the frag command to quickly scan the file system and obtain info such as file size distribution and overheads. I'll send that change out for review soon :)
Hopefully that will make the frag command great again! :D
Quoted 60 lines · Eric Sandeen
Quoted 22 lines · Jorge Guerra
From: Jorge Guerra <[email protected]>
While running the 'frag' command of '
xfs_db' we noticed that the tool is not scanning all the files in the file system. We noticed this when we modified the tool to print the inodes of all the files scanned. For example:$ find /mnt/xfsdisk -type f | wc -l 1782674 $ xfs_db -r -c frag /dev/sdXX | grep MB | awk '{print $5}' | paste -s -d+ | bc 656818Upon inspecting the code we noticed that the
scanfunc_inofunction stops processing a given inode block once it encounters a free leaf. However, in practice we see that inodes are necessarily always layed out contiguously on the leaf node. This resulted in the 'frag' command skipping some valid inodes.In this change we modify the
scanfunc_inofunction to skip freed inodes. With the change in place we ran the same experiment again and noticed a more accurate file count:Quoted 6 lines
$ find /mnt/d0 -type f | wc -l 1810442 $ xfs_db -r -c frag /dev/sdXX | grep MB | awk '{print $5}' | paste -s -d+ | bc 1810442Signed-off-by Jorge Guerra <[email protected]>This looks right, but I'll warn you that xfs_db's frag command is largely useless in the first place. ;)
Also, I think:
Fixes 2a5eb70c ("xfs_db: teach the frag command about sparse inode chunks")Reviewed-by Eric Sandeen <[email protected]>Thanks!
Quoted 18 lines
--- db/frag.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)db/frag.c +1
-1@@ -507,7 +507,7 @@ scanfunc_ino(for (j =0 ; j < inodes_per_buf; j++) {if (XFS_INOBT_IS_FREE_DISK (&rp[i], ioff + j)) - goto next_buf; +continue ; dip = (xfs_dinode_t *)((char *)iocur_top->data + ((off + j) << mp->m_sb.sb_inodelog));process_inode (agf, agino + ioff + j, dip);
Signature · 1 line
Jorge E Guerra D
Quoted 3 lines · Jorge Guerra
Thanks Eric,
I'll update the title.
No need, I can add the fixes tag when I queue it up.
Quoted 5 lines
We are using the frag command to quickly scan the file system and obtain info such as file size distribution and overheads. I'll send that change out for review soon :)
Hopefully that will make the frag command great again! :D
;) ok, if it becomes less meaningless, you may want to remove the printf which claims that it is meaningless. :)
-Eric
Quoted 64 lines · Eric Sandeen
Quoted 60 lines · Jorge Guerra
Quoted 22 lines
From: Jorge Guerra <[email protected]>
While running the 'frag' command of '
xfs_db' we noticed that the tool is not scanning all the files in the file system. We noticed this when we modified the tool to print the inodes of all the files scanned. For example:$ find /mnt/xfsdisk -type f | wc -l 1782674 $ xfs_db -r -c frag /dev/sdXX | grep MB | awk '{print $5}' | paste -s -d+ | bc 656818Upon inspecting the code we noticed that the
scanfunc_inofunction stops processing a given inode block once it encounters a free leaf. However, in practice we see that inodes are necessarily always layed out contiguously on the leaf node. This resulted in the 'frag' command skipping some valid inodes.In this change we modify the
scanfunc_inofunction to skip freed inodes. With the change in place we ran the same experiment again and noticed a more accurate file count:Quoted 6 lines
$ find /mnt/d0 -type f | wc -l 1810442 $ xfs_db -r -c frag /dev/sdXX | grep MB | awk '{print $5}' | paste -s -d+ | bc 1810442Signed-off-by Jorge Guerra <[email protected]>This looks right, but I'll warn you that xfs_db's frag command is largely useless in the first place. ;)
Also, I think:
Fixes 2a5eb70c ("xfs_db: teach the frag command about sparse inode chunks")Reviewed-by Eric Sandeen <[email protected]>Thanks!
Quoted 18 lines
--- db/frag.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)db/frag.c +1
-1@@ -507,7 +507,7 @@ scanfunc_ino(for (j =0 ; j < inodes_per_buf; j++) {if (XFS_INOBT_IS_FREE_DISK (&rp[i], ioff + j)) - goto next_buf; +continue ; dip = (xfs_dinode_t *)((char *)iocur_top->data + ((off + j) << mp->m_sb.sb_inodelog));process_inode (agf, agino + ioff + j, dip);