btrfs-progs: map-logical: handle corrupted fs better
PATCH 1/1 btrfs-progs: map-logical: handle corrupted fs better
Currently if running btrfs-map-logical on a filesystem with corrupted extent tree, it will fail due to open_ctree() error.
But the truth is, btrfs-map-logical only requires chunk tree to do logical bytenr mapping.
Make btrfs-map-logical more robust by:
- Loosen the open_ctree() requirement
Now it doesn't require an extent tree to work.
- Don't return error for map_one_extent()
Function map_one_extent() is too lookup extent tree to ensure there is at least one extent for the range we're looking for.
But since now we don't require extent tree at all, there is no hard requirement for that function.
Thus here we change it to return void, and only do the check when possible.
Now btrfs-map-logical can work on a filesystem with corrupted extent tree.
--- btrfs-map-logical.c | 50 +++++++++++---------------------------------- 1 file changed, 12 insertions(+), 38 deletions(-)
btrfs-map-logical.c +12 -38
@@ -38,8 +38,8 @@* */ static FILE *info_file; -static int map_one_extent (struct btrfs_fs_info *fs_info, -u64 *logical_ret,u64 *len_ret,int search_forward) +static void map_one_extent (struct btrfs_fs_info *fs_info, +u64 *logical_ret,u64 *len_ret,int search_forward) {struct btrfs_path *path;struct btrfs_key key;
@@ -52,7 +52,7 @@ static int map_one_extent(struct btrfs_fs_info *fs_info, path =btrfs_alloc_path ();if (!path) -return -ENOMEM ; +return ; key.objectid = logical; key.type =0 ;
@@ -94,7 +94,11 @@ out:if (len_ret) *len_ret = len; } -return ret; +/* +* Ignore any error for extent item lookup, it can be corrupted +* extent tree or whatever. In that case, just ignore the +* extent item lookup and reset @ret to 0. +*/ } static int __print_mapping_info(struct btrfs_fs_info *fs_info, u64 logical,
@@ -261,7 +265,8 @@ int main(int argc, char **argv)radix_tree_init ();cache_tree_init (&root_cache); - root =open_ctree (dev,0 ,0 ); + root =open_ctree (dev,0 ,OPEN_CTREE_PARTIAL | +OPEN_CTREE_NO_BLOCK_GROUPS );if (!root) {fprintf (stderr,"Open ctree failed\n" );free (output_file);
@@ -293,34 +298,7 @@ int main(int argc, char **argv) cur_len = bytes;/* First find the nearest extent */ - ret =map_one_extent (root->fs_info, &cur_logical, &cur_len,0 ); -if (ret <0 ) { - errno = -ret; -fprintf (stderr,"Failed to find extent at [%llu,%llu): %m\n" , - cur_logical, cur_logical + cur_len); - goto out_close_fd; - } -/* -* Normally, search backward should be OK, but for special case like -* given logical is quite small where no extents are before it, -* we need to search forward. -*/ -if (ret >0 ) { - ret =map_one_extent (root->fs_info, &cur_logical, &cur_len,1 ); -if (ret <0 ) { - errno = -ret; -fprintf (stderr, -"Failed to find extent at [%llu,%llu): %m\n" , - cur_logical, cur_logical + cur_len); - goto out_close_fd; - } -if (ret >0 ) { -fprintf (stderr, -"Failed to find any extent at [%llu,%llu)\n" , - cur_logical, cur_logical + cur_len); - goto out_close_fd; - } - } +map_one_extent (root->fs_info, &cur_logical, &cur_len,0 );while (cur_logical + cur_len >= logical && cur_logical < logical + bytes) {
@@ -328,11 +306,7 @@ int main(int argc, char **argv)u64 real_len; found =1 ; - ret =map_one_extent (root->fs_info, &cur_logical, &cur_len,1 ); -if (ret <0 ) - goto out_close_fd; -if (ret >0 ) -break ; +map_one_extent (root->fs_info, &cur_logical, &cur_len,1 );/* check again if there is overlap. */ if (cur_logical + cur_len < logical || cur_logical >= logical + bytes)
Signature · 1 line
2.32.0
Quoted 98 lines · Qu Wenruo
Currently if running btrfs-map-logical on a filesystem with corrupted extent tree, it will fail due to
open_ctree()error.But the truth is, btrfs-map-logical only requires chunk tree to do logical bytenr mapping.
Make btrfs-map-logical more robust by:
- Loosen the
open_ctree()requirement
Now it doesn't require an extent tree to work.- Don't return error for
map_one_extent()
Functionmap_one_extent()is too lookup extent tree to ensure there is at least one extent for the range we're looking for.But since now we don't require extent tree at all, there is no hard requirement for that function.
Thus here we change it to return void, and only do the check when possible.Now btrfs-map-logical can work on a filesystem with corrupted extent tree.
Signed-off-by Qu Wenruo <[email protected]>--- btrfs-map-logical.c | 50 +++++++++++---------------------------------- 1 file changed, 12 insertions(+), 38 deletions(-)btrfs-map-logical.c +10
-24@@ -38,8 +38,8 @@* */ static FILE *info_file; -static int map_one_extent (struct btrfs_fs_info *fs_info, -u64 *logical_ret,u64 *len_ret,int search_forward) +static void map_one_extent (struct btrfs_fs_info *fs_info, +u64 *logical_ret,u64 *len_ret,int search_forward) {struct btrfs_path *path;struct btrfs_key key;@@ -52,7 +52,7 @@ static int map_one_extent(struct btrfs_fs_info *fs_info, path =btrfs_alloc_path ();if (!path) -return -ENOMEM ; +return ; key.objectid = logical; key.type =0 ;@@ -94,7 +94,11 @@ out:if (len_ret) *len_ret = len; } -return ret; +/* +* Ignore any error for extent item lookup, it can be corrupted +* extent tree or whatever. In that case, just ignore the +* extent item lookup and reset @ret to 0. +*/ } static int __print_mapping_info(struct btrfs_fs_info *fs_info, u64 logical,@@ -261,7 +265,8 @@ int main(int argc, char **argv)radix_tree_init ();cache_tree_init (&root_cache); - root =open_ctree (dev,0 ,0 ); + root =open_ctree (dev,0 ,OPEN_CTREE_PARTIAL | +OPEN_CTREE_NO_BLOCK_GROUPS );if (!root) {fprintf (stderr,"Open ctree failed\n" );free (output_file);@@ -293,34 +298,7 @@ int main(int argc, char **argv) cur_len = bytes;/* First find the nearest extent */ - ret =map_one_extent (root->fs_info, &cur_logical, &cur_len,0 ); -if (ret <0 ) { - errno = -ret; -fprintf (stderr,"Failed to find extent at [%llu,%llu): %m\n" , - cur_logical, cur_logical + cur_len); - goto out_close_fd; - } -/* -* Normally, search backward should be OK, but for special case like -* given logical is quite small where no extents are before it, -* we need to search forward. -*/ -if (ret >0 ) { - ret =map_one_extent (root->fs_info, &cur_logical, &cur_len,1 ); -if (ret <0 ) { - errno = -ret; -fprintf (stderr, -"Failed to find extent at [%llu,%llu): %m\n" , - cur_logical, cur_logical + cur_len);
btrfs-map-logical.c int main(int argc, char **argv) · line 301
Quoted 10 lines
btrfs-map-logical.c +1
-9- goto out_close_fd; - } -if (ret >0 ) { -fprintf (stderr, -"Failed to find any extent at [%llu,%llu)\n" , - cur_logical, cur_logical + cur_len); - goto out_close_fd; - } - } +map_one_extent (root->fs_info, &cur_logical, &cur_len,0 );
You essentially make map_one_extent fail silently in this case how can the call to it be reliable at all? Shouldn't it be removed altogether? alternatively, the function can be made to return an error, yet it should be up to the caller to choose to ignore it. Also if the tree is corrupted what pervents for the btrfs_search_slot in map_one_extent to return 0 which will trigger a BUG_ON ?
Furthermore with map_one_extent present the semantics of the program is that it prints the logical mapping of the real extent rather then the passed in bytes. Because the user is allowed to pass an offset for which there isn't a real extent. So if we want to retain this your change is a no-go. OTOH if we want to have btrfs_map_logical to serve as a simple calculation aid i.e you pass in some logical byte, irrespective whether it contains a real extent or not, and have the program return what the physical mapping is then map_one_extent becomes redundant altogether.
<snip>
Quoted 113 lines · Nikolay Borisov
Quoted 98 lines · Qu Wenruo
Currently if running btrfs-map-logical on a filesystem with corrupted extent tree, it will fail due to
open_ctree()error.But the truth is, btrfs-map-logical only requires chunk tree to do logical bytenr mapping.
Make btrfs-map-logical more robust by:
- Loosen the
open_ctree()requirement
Now it doesn't require an extent tree to work.- Don't return error for
map_one_extent()
Functionmap_one_extent()is too lookup extent tree to ensure there is at least one extent for the range we're looking for.But since now we don't require extent tree at all, there is no hard requirement for that function.
Thus here we change it to return void, and only do the check when possible.Now btrfs-map-logical can work on a filesystem with corrupted extent tree.
Signed-off-by Qu Wenruo <[email protected]>--- btrfs-map-logical.c | 50 +++++++++++---------------------------------- 1 file changed, 12 insertions(+), 38 deletions(-)btrfs-map-logical.c +10
-24@@ -38,8 +38,8 @@* */ static FILE *info_file; -static int map_one_extent (struct btrfs_fs_info *fs_info, -u64 *logical_ret,u64 *len_ret,int search_forward) +static void map_one_extent (struct btrfs_fs_info *fs_info, +u64 *logical_ret,u64 *len_ret,int search_forward) {struct btrfs_path *path;struct btrfs_key key;@@ -52,7 +52,7 @@ static int map_one_extent(struct btrfs_fs_info *fs_info, path =btrfs_alloc_path ();if (!path) -return -ENOMEM ; +return ; key.objectid = logical; key.type =0 ;@@ -94,7 +94,11 @@ out:if (len_ret) *len_ret = len; } -return ret; +/* +* Ignore any error for extent item lookup, it can be corrupted +* extent tree or whatever. In that case, just ignore the +* extent item lookup and reset @ret to 0. +*/ } static int __print_mapping_info(struct btrfs_fs_info *fs_info, u64 logical,@@ -261,7 +265,8 @@ int main(int argc, char **argv)radix_tree_init ();cache_tree_init (&root_cache); - root =open_ctree (dev,0 ,0 ); + root =open_ctree (dev,0 ,OPEN_CTREE_PARTIAL | +OPEN_CTREE_NO_BLOCK_GROUPS );if (!root) {fprintf (stderr,"Open ctree failed\n" );free (output_file);@@ -293,34 +298,7 @@ int main(int argc, char **argv) cur_len = bytes;/* First find the nearest extent */ - ret =map_one_extent (root->fs_info, &cur_logical, &cur_len,0 ); -if (ret <0 ) { - errno = -ret; -fprintf (stderr,"Failed to find extent at [%llu,%llu): %m\n" , - cur_logical, cur_logical + cur_len); - goto out_close_fd; - } -/* -* Normally, search backward should be OK, but for special case like -* given logical is quite small where no extents are before it, -* we need to search forward. -*/ -if (ret >0 ) { - ret =map_one_extent (root->fs_info, &cur_logical, &cur_len,1 ); -if (ret <0 ) { - errno = -ret; -fprintf (stderr, -"Failed to find extent at [%llu,%llu): %m\n" , - cur_logical, cur_logical + cur_len);btrfs-map-logical.c int main(int argc, char **argv) · line 301
Quoted 10 lines
btrfs-map-logical.c +1
-9- goto out_close_fd; - } -if (ret >0 ) { -fprintf (stderr, -"Failed to find any extent at [%llu,%llu)\n" , - cur_logical, cur_logical + cur_len); - goto out_close_fd; - } - } +map_one_extent (root->fs_info, &cur_logical, &cur_len,0 );
Quoted 6 lines
You essentially make
map_one_extentfail silently in this case how can the call to it be reliable at all? Shouldn't it be removed altogether? alternatively, the function can be made to return an error, yet it should be up to the caller to choose to ignore it. Also if the tree is corrupted what pervents for thebtrfs_search_slotinmap_one_extentto return 0 which will trigger aBUG_ON?
For corrupted extent tree root, open_ctree() in btrfs-progs will populate the root node with an empty leaf.
That's why the BUG_ON() will never trigger, as in such case, btrfs_search_slot() will return 1 other than -EIO.
Quoted 7 lines
Furthermore with
map_one_extentpresent the semantics of the program is that it prints the logical mapping of the real extent rather then the passed in bytes. Because the user is allowed to pass an offset for which there isn't a real extent. So if we want to retain this your change is a no-go.
The change just makes the extent item lookup an optional operation.
If by somehow we failed to lookup the extent item, we just continue with the values passed-in, no longer to verify whether there is an extent.
This is especially important for fs with corrupted extent tree.
Quoted 4 lines
OTOH if we want to have
btrfs_map_logicalto serve as a simple calculation aid i.e you pass in some logical byte, irrespective whether it contains a real extent or not, and have the program return what the physical mapping is thenmap_one_extentbecomes redundant altogether.
Yeah, I was also thinking about that, but not sure if we should completely remove map_one_extent().
Thus I took the middle land by rendering it optional.
But if that's the way to go, I can't be more happier as that greatly simplify the workflow of btrfs-map-logical.
Thanks,
Qu
Quoted 3 lines
<snip>
Quoted 32 lines · Qu Wenruo
Quoted 29 lines · Nikolay Borisov
Quoted 26 lines · Qu Wenruo
Currently if running btrfs-map-logical on a filesystem with corrupted extent tree, it will fail due to
open_ctree()error.But the truth is, btrfs-map-logical only requires chunk tree to do logical bytenr mapping.
Make btrfs-map-logical more robust by:
- Loosen the
open_ctree()requirement
Now it doesn't require an extent tree to work.- Don't return error for
map_one_extent()
Functionmap_one_extent()is too lookup extent tree to ensure there is
at least one extent for the range we're looking for.But since now we don't require extent tree at all, there is no hard requirement for that function.
Thus here we change it to return void, and only do the check when possible.Now btrfs-map-logical can work on a filesystem with corrupted extent tree.
Signed-off-by Qu Wenruo <[email protected]>---
<snip>
Quoted 22 lines
Quoted 7 lines
Furthermore with
map_one_extentpresent the semantics of the program is that it prints the logical mapping of the real extent rather then the passed in bytes. Because the user is allowed to pass an offset for which there isn't a real extent. So if we want to retain this your change is a no-go.The change just makes the extent item lookup an optional operation.
If by somehow we failed to lookup the extent item, we just continue with the values passed-in, no longer to verify whether there is an extent.
This is especially important for fs with corrupted extent tree.
Quoted 4 lines
OTOH if we want to have
btrfs_map_logicalto serve as a simple calculation aid i.e you pass in some logical byte, irrespective whether it contains a real extent or not, and have the program return what the physical mapping is thenmap_one_extentbecomes redundant altogether.
Quoted 4 lines
Yeah, I was also thinking about that, but not sure if we should completely remove
map_one_extent().Thus I took the middle land by rendering it optional.
IMO the middle ground would be to add a command line switch i.e force which would ignore map_one_extent. In such exceptional cases a user with a problem could be instructed to run the command with an '-f' switch for example. The rest of the time the program would preserve its old logic which guarantees returning mapping for real extents.
Quoted 11 lines
But if that's the way to go, I can't be more happier as that greatly simplify the workflow of btrfs-map-logical.
Thanks,
QuQuoted 3 lines
<snip>
Quoted 63 lines · Nikolay Borisov
Quoted 32 lines · Qu Wenruo
Quoted 29 lines · Nikolay Borisov
Quoted 26 lines · Qu Wenruo
Currently if running btrfs-map-logical on a filesystem with corrupted extent tree, it will fail due to
open_ctree()error.But the truth is, btrfs-map-logical only requires chunk tree to do logical bytenr mapping.
Make btrfs-map-logical more robust by:
- Loosen the
open_ctree()requirement
Now it doesn't require an extent tree to work.- Don't return error for
map_one_extent()
Functionmap_one_extent()is too lookup extent tree to ensure
there is
at least one extent for the range we're looking for.But since now we don't require extent tree at all, there is no hard requirement for that function.
Thus here we change it to return void, and only do the check when possible.Now btrfs-map-logical can work on a filesystem with corrupted extent tree.
Signed-off-by Qu Wenruo <[email protected]>---
<snip>
Quoted 25 lines
Quoted 7 lines
Furthermore with
map_one_extentpresent the semantics of the program is that it prints the logical mapping of the real extent rather then the passed in bytes. Because the user is allowed to pass an offset for which there isn't a real extent. So if we want to retain this your change is a no-go.The change just makes the extent item lookup an optional operation.
If by somehow we failed to lookup the extent item, we just continue with the values passed-in, no longer to verify whether there is an extent.
This is especially important for fs with corrupted extent tree.
Quoted 4 lines
OTOH if we want to have
btrfs_map_logicalto serve as a simple calculation aid i.e you pass in some logical byte, irrespective whether it contains a real extent or not, and have the program return what the physical mapping is thenmap_one_extentbecomes redundant altogether.Yeah, I was also thinking about that, but not sure if we should completely remove
map_one_extent().
Quoted 7 lines
Quoted 1 line
Thus I took the middle land by rendering it optional.
IMO the middle ground would be to add a command line switch i.e force which would ignore
map_one_extent. In such exceptional cases a user with a problem could be instructed to run the command with an '-f' switch for example. The rest of the time the program would preserve its old logic which guarantees returning mapping for real extents.
But now I'm wondering, do we really need that extent check?
I mean, yes rejecting the lookup for non-existing range may be helpful, but it's not perfect.
For example, if we are looking up a unwritten preallocated range, we can pass the extent check, but still only gets garbage from the disk.
So why not just let users do whatever they want?
(Well, I should ask the same questions to the older me from 5 years ago)
Thanks,
Qu
Quoted 12 lines
Quoted 12 lines
But if that's the way to go, I can't be more happier as that greatly simplify the workflow of btrfs-map-logical.
Thanks,
QuQuoted 3 lines
<snip>
Quoted 77 lines · Qu Wenruo
Quoted 66 lines · Nikolay Borisov
Quoted 33 lines · Qu Wenruo
Quoted 30 lines · Nikolay Borisov
Quoted 27 lines · Qu Wenruo
Currently if running btrfs-map-logical on a filesystem with corrupted extent tree, it will fail due to
open_ctree()error.But the truth is, btrfs-map-logical only requires chunk tree to do logical bytenr mapping.
Make btrfs-map-logical more robust by:
- Loosen the
open_ctree()requirement
Now it doesn't require an extent tree to work.- Don't return error for
map_one_extent()
Functionmap_one_extent()is too lookup extent tree to ensure there is
at least one extent for the range we're looking for.But since now we don't require extent tree at all, there is no hard
requirement for that function.
Thus here we change it to return void, and only do the check when possible.Now btrfs-map-logical can work on a filesystem with corrupted extent tree.
Signed-off-by Qu Wenruo <[email protected]>---
<snip>
Quoted 27 lines
Quoted 9 lines
Furthermore with
map_one_extentpresent the semantics of the program is that it prints the logical mapping of the real extent rather then the passed in bytes. Because the user is allowed to pass an offset for which
there isn't a real extent. So if we want to retain this your change is a
no-go.The change just makes the extent item lookup an optional operation.
If by somehow we failed to lookup the extent item, we just continue with the values passed-in, no longer to verify whether there is an extent.
This is especially important for fs with corrupted extent tree.
Quoted 4 lines
OTOH if we want to have
btrfs_map_logicalto serve as a simple calculation aid i.e you pass in some logical byte, irrespective whether it contains a real extent or not, and have the program return what the physical mapping is thenmap_one_extentbecomes redundant altogether.Yeah, I was also thinking about that, but not sure if we should completely remove
map_one_extent().Quoted 7 lines
Quoted 1 line
Thus I took the middle land by rendering it optional.
IMO the middle ground would be to add a command line switch i.e force which would ignore
map_one_extent. In such exceptional cases a user with a problem could be instructed to run the command with an '-f' switch for example. The rest of the time the program would preserve its old logic which guarantees returning mapping for real extents.
Quoted 7 lines
But now I'm wondering, do we really need that extent check?
I mean, yes rejecting the lookup for non-existing range may be helpful, but it's not perfect.
For example, if we are looking up a unwritten preallocated range, we can pass the extent check, but still only gets garbage from the disk.
Well yes and no. We will get a garbage if we try to read directly from the disk by using the resulting offset. OTOH we are getting valid data for an extent that exists - preallocated extents are real extents at the end of the day.
Again, it all boils down to what semantics we'd like to offer. But the more I think about it, the more I'm inclined to say keep it simple, and do a conversion from logical->physical in which case map_one_extent can be removed as well.
Quoted 20 lines
So why not just let users do whatever they want?
(Well, I should ask the same questions to the older me from 5 years ago)
Thanks,
QuQuoted 12 lines
Quoted 12 lines
But if that's the way to go, I can't be more happier as that greatly simplify the workflow of btrfs-map-logical.
Thanks,
QuQuoted 3 lines
<snip>
Quoted 93 lines · Nikolay Borisov
Quoted 77 lines · Qu Wenruo
Quoted 66 lines · Nikolay Borisov
Quoted 33 lines · Qu Wenruo
Quoted 30 lines · Nikolay Borisov
Quoted 27 lines · Qu Wenruo
Currently if running btrfs-map-logical on a filesystem with corrupted extent tree, it will fail due to
open_ctree()error.But the truth is, btrfs-map-logical only requires chunk tree to do logical bytenr mapping.
Make btrfs-map-logical more robust by:
- Loosen the
open_ctree()requirement
Now it doesn't require an extent tree to work.- Don't return error for
map_one_extent()
Functionmap_one_extent()is too lookup extent tree to ensure
there is
at least one extent for the range we're looking for.But since now we don't require extent tree at all, there is no
hard
requirement for that function.
Thus here we change it to return void, and only do the check when possible.Now btrfs-map-logical can work on a filesystem with corrupted extent tree.
Signed-off-by Qu Wenruo <[email protected]>---
<snip>
Quoted 27 lines
Quoted 9 lines
Furthermore with
map_one_extentpresent the semantics of the program is that it prints the logical mapping of the real extent rather then the passed in bytes. Because the user is allowed to pass an offset for which
there isn't a real extent. So if we want to retain this your change is a
no-go.The change just makes the extent item lookup an optional operation.
If by somehow we failed to lookup the extent item, we just continue with the values passed-in, no longer to verify whether there is an extent.
This is especially important for fs with corrupted extent tree.
Quoted 4 lines
OTOH if we want to have
btrfs_map_logicalto serve as a simple calculation aid i.e you pass in some logical byte, irrespective whether it contains a real extent or not, and have the program return what the physical mapping is thenmap_one_extentbecomes redundant altogether.Yeah, I was also thinking about that, but not sure if we should completely remove
map_one_extent().Quoted 7 lines
Quoted 1 line
Thus I took the middle land by rendering it optional.
IMO the middle ground would be to add a command line switch i.e force which would ignore
map_one_extent. In such exceptional cases a user with a problem could be instructed to run the command with an '-f' switch for example. The rest of the time the program would preserve its old logic which guarantees returning mapping for real extents.Quoted 7 lines
But now I'm wondering, do we really need that extent check?
I mean, yes rejecting the lookup for non-existing range may be helpful, but it's not perfect.
For example, if we are looking up a unwritten preallocated range, we can pass the extent check, but still only gets garbage from the disk.
Well yes and no. We will get a garbage if we try to read directly from the disk by using the resulting offset. OTOH we are getting valid data for an extent that exists - preallocated extents are real extents at the end of the day.
Quoted 4 lines
Again, it all boils down to what semantics we'd like to offer. But the more I think about it, the more I'm inclined to say keep it simple, and do a conversion from
logical->physicalin which casemap_one_extentcan be removed as well.
Yep, let's try to make it simpler by making it just a logical -> physical mapping.
And remove invalid cases like unaligned start/len.
Thanks,
Qu
Quoted 22 lines
Quoted 21 lines
So why not just let users do whatever they want?
(Well, I should ask the same questions to the older me from 5 years ago)
Thanks,
QuQuoted 12 lines
Quoted 12 lines
But if that's the way to go, I can't be more happier as that greatly simplify the workflow of btrfs-map-logical.
Thanks,
QuQuoted 3 lines
<snip>