btrfs-progs: cmds: Add subcommand that dumps file extents
Subsystems (2)
PATCH v2 1/1 btrfs-progs: cmds: Add subcommand that dumps file extents
This patch adds an subcommand in inspect-internal. It dumps file extents of the file that user provided. It helps to show the internal information about file extents comprise the file.
---
v2:
- Prints type and compression
- Use the terms from file_extents_item like disk_bytenr not like physical"
--- Makefile | 2 +- cmds/commands.h | 2 +- cmds/inspect-dump-file-extents.c | 165 +++++++++++++++++++++++++++++++ cmds/inspect.c | 1 + 4 files changed, 168 insertions(+), 2 deletions(-) create mode 100644 cmds/inspect-dump-file-extents.c
Makefile +1 -1
@@ -156,7 +156,7 @@ cmds_objects = cmds/subvolume.o cmds/filesystem.o cmds/device.o cmds/scrub.o \ cmds/restore.o cmds/rescue.o cmds/rescue-chunk-recover.o \ cmds/rescue-super-recover.o \ cmds/property.o cmds/filesystem-usage.o cmds/inspect-dump-tree.o \ - cmds/inspect-dump-super.o cmds/inspect-tree-stats.o cmds/filesystem-du.o \ + cmds/inspect-dump-super.o cmds/inspect-tree-stats.o cmds/inspect-dump-file-extents.o cmds/filesystem-du.o \ mkfs/common.o check/mode-common.o check/mode-lowmem.o libbtrfs_objects = common/send-stream.o common/send-utils.o kernel-lib/rbtree.o btrfs-list.o \ kernel-lib/radix-tree.o common/extent-cache.o kernel-shared/extent_io.o \
cmds/commands.h +1 -1
@@ -154,5 +154,5 @@ DECLARE_COMMAND(select_super);DECLARE_COMMAND (dump_super);DECLARE_COMMAND (debug_tree);DECLARE_COMMAND (rescue); - +DECLARE_COMMAND (inspect_dump_file_extents);#endif
cmds/inspect-dump-file-extents.c new file +165 -0
165 lines not on this page · raw message
cmds/inspect.c +1 -0
@@ -696,6 +696,7 @@ static const struct cmd_group inspect_cmd_group = { &cmd_struct_inspect_dump_tree, &cmd_struct_inspect_dump_super, &cmd_struct_inspect_tree_stats, + &cmd_struct_inspect_dump_file_extents, NULL } };
Signature · 1 line
2.25.1
Quoted 3 lines · Sidong Yang
This patch adds an subcommand in inspect-internal. It dumps file extents of the file that user provided. It helps to show the internal information about file extents comprise the file.
Despite the comments inlined below for the technical details, I'm not determined if we really want the tool.
On one hand, fiemap doesn't provide all detailed btrfs specific info, and it's common to utilize the tree search ioctl to do the work, just like "compsize".
But on the other hand, I'm not sure if it provides enough info compared to things like "btrfs ins dump-tree".
For now I don't have any objection nor preference.
Thus it's again David's call on this.
Quoted 32 lines
Signed-off-by Sidong Yang <[email protected]>---
v2:
- Prints type and compression
- Use the terms fromfile_extents_itemlikedisk_bytenrnot like physical"--- Makefile | 2 +- cmds/commands.h | 2 +- cmds/inspect-dump-file-extents.c | 165 +++++++++++++++++++++++++++++++ cmds/inspect.c | 1 + 4 files changed, 168 insertions(+), 2 deletions(-)create mode 100644
cmds/inspect-dump-file-extents.cMakefile +1
-1@@ -156,7 +156,7 @@ cmds_objects = cmds/subvolume.o cmds/filesystem.o cmds/device.o cmds/scrub.o \ cmds/restore.o cmds/rescue.o cmds/rescue-chunk-recover.o \ cmds/rescue-super-recover.o \ cmds/property.o cmds/filesystem-usage.o cmds/inspect-dump-tree.o \ - cmds/inspect-dump-super.o cmds/inspect-tree-stats.o cmds/filesystem-du.o \ + cmds/inspect-dump-super.o cmds/inspect-tree-stats.o cmds/inspect-dump-file-extents.o cmds/filesystem-du.o \ mkfs/common.o check/mode-common.o check/mode-lowmem.o libbtrfs_objects = common/send-stream.o common/send-utils.o kernel-lib/rbtree.o btrfs-list.o \ kernel-lib/radix-tree.o common/extent-cache.o kernel-shared/extent_io.o \cmds/commands.h +0
-0
cmds/commands.h DECLARE_COMMAND(select_super); · line 155
Quoted 2 lines
cmds/commands.h +0
-0@@ -154,5 +154,5 @@ DECLARE_COMMAND(select_super);DECLARE_COMMAND (dump_super);DECLARE_COMMAND (debug_tree);
Off-topic here.
Those "dump_super" and "debug_tree" makes me wonder, do we need to cleanup them?
I mean, we have inspect_dump_super for "btrfs ins dump-super", but what's "dump_super" here for? And what's the "debug_tree" here for?
cmds/commands.h DECLARE_COMMAND(select_super); · line 157
Quoted 3 lines
cmds/commands.h +1
-1DECLARE_COMMAND (rescue); - +DECLARE_COMMAND (inspect_dump_file_extents);
I would be better to put this line where the other "inpsect" subcommands are.
Quoted 47 lines
cmds/commands.h +0
-0#endif cmds/inspect-dump-file-extents.c new file +40
-0@@ -0,0 +1,165 @@ +/* +* This program is free software; you can redistribute it and/or +* modify it under the terms of the GNU General Public +* License v2 as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public +* License along with this program; if not, write to the +* Free Software Foundation, Inc., 59 Temple Place - Suite 330, +* Boston, MA 021110-1307, USA. +*/ + +#include <unistd.h> +#include <stdio.h> +#include <fcntl.h> + +#include <sys/ioctl.h> + +#include "common/utils.h" +#include "cmds/commands.h" + +static const char *const cmd_inspect_dump_file_extents_usage[] = { +"btrfs inspect-internal dump-extent path" , +"Dump file extent in a textual form" , +NULL +}; + +static void compress_type_to_str (u8 compress_type,char *ret) +{ +switch (compress_type) { +case BTRFS_COMPRESS_NONE : +strcpy (ret,"none" ); +break ; +case BTRFS_COMPRESS_ZLIB : +strcpy (ret,"zlib" ); +break ;
cmds/inspect-dump-file-extents.c line 50
Quoted 10 lines
cmds/inspect-dump-file-extents.c new file +10
-0+case BTRFS_COMPRESS_LZO : +strcpy (ret,"lzo" ); +break ; +case BTRFS_COMPRESS_ZSTD : +strcpy (ret,"zstd" ); +break ; +default : +sprintf (ret,"UNKNOWN.%d" , compress_type); + } +}
It would be better to just export the function with the same name in "kernel-shared/print-tree.c" so we don't have duplicated code.
cmds/inspect-dump-file-extents.c line 60
Quoted 10 lines
cmds/inspect-dump-file-extents.c new file +10
-0+ +static const char *file_extent_type_to_str (u8 type) +{ +switch (type) { +case BTRFS_FILE_EXTENT_INLINE :return "inline" ; +case BTRFS_FILE_EXTENT_PREALLOC :return "prealloc" ; +case BTRFS_FILE_EXTENT_REG :return "regular" ; +default :return "unknown" ; + } +}
The same here.
Quoted 45 lines
cmds/inspect-dump-file-extents.c new file +45
-0+ +static int cmd_inspect_dump_file_extents (const struct cmd_struct *cmd, +int argc,char **argv) +{ +int fd; +struct stat statbuf; +struct btrfs_ioctl_ino_lookup_args lookup; +struct btrfs_ioctl_search_args args; +struct btrfs_ioctl_search_key *sk = &args.key; +struct btrfs_file_extent_item *extent_item; +struct btrfs_ioctl_search_header *header; +u64 pos; +u64 buf_off; +u64 len; +u64 begin; +u64 disk_bytenr; +u64 disk_num_bytes; +u64 offset; +int ret; +int i; +char compress_str[16 ]; + + fd =open (argv[optind],O_RDONLY ); +if (fd <0 ) { +error ("cannot open %s: %m" , argv[optind]); + ret =1 ; + goto out; + } + +if (fstat (fd, &statbuf) <0 ) { +error ("failed to fstat %s: %m" , argv[optind]); + ret =1 ; + goto out; + } + + lookup.treeid =0 ; + lookup.objectid =BTRFS_FIRST_FREE_OBJECTID ; + +if (ioctl (fd,BTRFS_IOC_INO_LOOKUP , &lookup) <0 ) { +error ("failed to lookup inode %s: %m" , argv[optind]); + ret =1 ; + goto out; + } + + pos =0 ;
cmds/inspect-dump-file-extents.c line 115
Quoted 10 lines
cmds/inspect-dump-file-extents.c new file +10
-0+ + sk->tree_id = lookup.treeid; + sk->min_objectid = statbuf.st_ino; + sk->max_objectid = statbuf.st_ino; + + sk->max_offset =UINT64_MAX ; + sk->min_transid =0 ; + sk->max_transid =UINT64_MAX ; + sk->min_type = sk->max_type =BTRFS_EXTENT_DATA_KEY ; + sk->nr_items =4096 ;
You may want to do the tree search ioctl in a loop, as it's pretty common for super large or heavily fragmented inode to have way more items than one ioctl can return.
Quoted 22 lines
cmds/inspect-dump-file-extents.c new file +22
-0+ +while (statbuf.st_size > pos) { + sk->min_offset = pos; +if (ioctl (fd,BTRFS_IOC_TREE_SEARCH , &args)) { +error ("failed to search tree ioctl %s: %m" , argv[optind]); + ret =1 ; + goto out; + } + + buf_off =0 ; +for (i=0 ; i<sk->nr_items; ++i) { + header = (struct btrfs_ioctl_search_header *)(args.buf + buf_off); + +if (btrfs_search_header_type (header) ==BTRFS_EXTENT_DATA_KEY ) { + extent_item = (struct btrfs_file_extent_item *)(header +1 ); + begin =btrfs_search_header_offset (header); + +printf ("type = %s, begin = %llu, " , +file_extent_type_to_str (extent_item->type), begin); +switch (extent_item->type) { +case BTRFS_FILE_EXTENT_INLINE : + len =le64_to_cpu (extent_item->ram_bytes);
cmds/inspect-dump-file-extents.c line 147
Quoted 10 lines
cmds/inspect-dump-file-extents.c new file +10
-0+printf ("end = %llu\n" , begin + len); +break ; +case BTRFS_FILE_EXTENT_REG : +case BTRFS_FILE_EXTENT_PREALLOC : + len =le64_to_cpu (extent_item->num_bytes); + disk_bytenr =le64_to_cpu (extent_item->disk_bytenr); + disk_num_bytes =le64_to_cpu (extent_item->disk_num_bytes); + offset =le64_to_cpu (extent_item->offset); +compress_type_to_str (extent_item->compression, compress_str); + printf("end = %llu, disk_bytenr = %llu, disk_num_bytes = %llu,"
For "end" we normally mean inclusive end.
E.g, for @start = 1M, @len = 4K, then the @end should be 1M + 4K - 1.
Thus it would be better to just output the length, not the end.
(I know this sounds a little nitpicking, but trust me, when you have seen too many bugs caused by such offset-by-one behavior, you will be as sensitive as me on this)
Thanks,
Qu
Quoted 30 lines
cmds/inspect-dump-file-extents.c new file +17
-0+" offset = %llu, compression = %s\n" , + begin + len, disk_bytenr, disk_num_bytes, offset, compress_str); > + + break; + } + + } + buf_off +=sizeof (*header) +btrfs_search_header_len (header); + pos += len; + } + + } + ret =0 ; +out: +close (fd); +return ret; +} +DEFINE_SIMPLE_COMMAND (inspect_dump_file_extents,"dump-file-extents" );cmds/inspect.c +1
-0@@ -696,6 +696,7 @@ static const struct cmd_group inspect_cmd_group = { &cmd_struct_inspect_dump_tree, &cmd_struct_inspect_dump_super, &cmd_struct_inspect_tree_stats, + &cmd_struct_inspect_dump_file_extents, NULL } };
Quoted 58 lines · Qu Wenruo
Quoted 3 lines · Sidong Yang
This patch adds an subcommand in inspect-internal. It dumps file extents of the file that user provided. It helps to show the internal information about file extents comprise the file.
Despite the comments inlined below for the technical details, I'm not determined if we really want the tool.
On one hand, fiemap doesn't provide all detailed btrfs specific info, and it's common to utilize the tree search ioctl to do the work, just like "compsize".
But on the other hand, I'm not sure if it provides enough info compared to things like "btrfs ins dump-tree".
For now I don't have any objection nor preference.
Thus it's again David's call on this.
Quoted 32 lines
Signed-off-by Sidong Yang <[email protected]>---
v2:
- Prints type and compression
- Use the terms fromfile_extents_itemlikedisk_bytenrnot like physical"--- Makefile | 2 +- cmds/commands.h | 2 +- cmds/inspect-dump-file-extents.c | 165 +++++++++++++++++++++++++++++++ cmds/inspect.c | 1 + 4 files changed, 168 insertions(+), 2 deletions(-)create mode 100644
cmds/inspect-dump-file-extents.cMakefile +1
-1@@ -156,7 +156,7 @@ cmds_objects = cmds/subvolume.o cmds/filesystem.o cmds/device.o cmds/scrub.o \ cmds/restore.o cmds/rescue.o cmds/rescue-chunk-recover.o \ cmds/rescue-super-recover.o \ cmds/property.o cmds/filesystem-usage.o cmds/inspect-dump-tree.o \ - cmds/inspect-dump-super.o cmds/inspect-tree-stats.o cmds/filesystem-du.o \ + cmds/inspect-dump-super.o cmds/inspect-tree-stats.o cmds/inspect-dump-file-extents.o cmds/filesystem-du.o \ mkfs/common.o check/mode-common.o check/mode-lowmem.o libbtrfs_objects = common/send-stream.o common/send-utils.o kernel-lib/rbtree.o btrfs-list.o \ kernel-lib/radix-tree.o common/extent-cache.o kernel-shared/extent_io.o \cmds/commands.h +0
-0cmds/commands.h DECLARE_COMMAND(select_super); · line 155
Quoted 2 lines
cmds/commands.h +0
-0@@ -154,5 +154,5 @@ DECLARE_COMMAND(select_super);DECLARE_COMMAND (dump_super);DECLARE_COMMAND (debug_tree);Off-topic here.
Quoted 6 lines
Those "
dump_super" and "debug_tree" makes me wonder, do we need to cleanup them?I mean, we have
inspect_dump_superfor "btrfs ins dump-super", but what's "dump_super" here for?
And what's the "debug_tree" here for?
there is no command dump_super and debug_tree. And these aren't need to compile.
Quoted 7 lines
cmds/commands.h DECLARE_COMMAND(select_super); · line 157
Quoted 4 lines
cmds/commands.h +1
-1DECLARE_COMMAND (rescue); - +DECLARE_COMMAND (inspect_dump_file_extents);I would be better to put this line where the other "inpsect" subcommands are.
Okay, I'll do it.
Quoted 53 lines
Quoted 53 lines
cmds/commands.h +0
-0#endif cmds/inspect-dump-file-extents.c new file +45
-0@@ -0,0 +1,165 @@ +/* +* This program is free software; you can redistribute it and/or +* modify it under the terms of the GNU General Public +* License v2 as published by the Free Software Foundation. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public +* License along with this program; if not, write to the +* Free Software Foundation, Inc., 59 Temple Place - Suite 330, +* Boston, MA 021110-1307, USA. +*/ + +#include <unistd.h> +#include <stdio.h> +#include <fcntl.h> + +#include <sys/ioctl.h> + +#include "common/utils.h" +#include "cmds/commands.h" + +static const char *const cmd_inspect_dump_file_extents_usage[] = { +"btrfs inspect-internal dump-extent path" , +"Dump file extent in a textual form" , +NULL +}; + +static void compress_type_to_str (u8 compress_type,char *ret) +{ +switch (compress_type) { +case BTRFS_COMPRESS_NONE : +strcpy (ret,"none" ); +break ; +case BTRFS_COMPRESS_ZLIB : +strcpy (ret,"zlib" ); +break ; +case BTRFS_COMPRESS_LZO : +strcpy (ret,"lzo" ); +break ; +case BTRFS_COMPRESS_ZSTD : +strcpy (ret,"zstd" );
Quoted 8 lines
cmds/inspect-dump-file-extents.c line 50
Quoted 5 lines
cmds/inspect-dump-file-extents.c new file +5
-0+break ; +default : +sprintf (ret,"UNKNOWN.%d" , compress_type); + } +}It would be better to just export the function with the same name in "
kernel-shared/print-tree.c" so we don't have duplicated code.
Yes, I would be better.
Quoted 13 lines
cmds/inspect-dump-file-extents.c line 60
Quoted 11 lines
cmds/inspect-dump-file-extents.c new file +10
-0+ +static const char *file_extent_type_to_str (u8 type) +{ +switch (type) { +case BTRFS_FILE_EXTENT_INLINE :return "inline" ; +case BTRFS_FILE_EXTENT_PREALLOC :return "prealloc" ; +case BTRFS_FILE_EXTENT_REG :return "regular" ; +default :return "unknown" ; + } +}The same here.
Okay.
Quoted 52 lines
Quoted 52 lines
cmds/inspect-dump-file-extents.c new file +51
-0+ +static int cmd_inspect_dump_file_extents (const struct cmd_struct *cmd, +int argc,char **argv) +{ +int fd; +struct stat statbuf; +struct btrfs_ioctl_ino_lookup_args lookup; +struct btrfs_ioctl_search_args args; +struct btrfs_ioctl_search_key *sk = &args.key; +struct btrfs_file_extent_item *extent_item; +struct btrfs_ioctl_search_header *header; +u64 pos; +u64 buf_off; +u64 len; +u64 begin; +u64 disk_bytenr; +u64 disk_num_bytes; +u64 offset; +int ret; +int i; +char compress_str[16 ]; + + fd =open (argv[optind],O_RDONLY ); +if (fd <0 ) { +error ("cannot open %s: %m" , argv[optind]); + ret =1 ; + goto out; + } + +if (fstat (fd, &statbuf) <0 ) { +error ("failed to fstat %s: %m" , argv[optind]); + ret =1 ; + goto out; + } + + lookup.treeid =0 ; + lookup.objectid =BTRFS_FIRST_FREE_OBJECTID ; + +if (ioctl (fd,BTRFS_IOC_INO_LOOKUP , &lookup) <0 ) { +error ("failed to lookup inode %s: %m" , argv[optind]); + ret =1 ; + goto out; + } + + pos =0 ; + + sk->tree_id = lookup.treeid; + sk->min_objectid = statbuf.st_ino; + sk->max_objectid = statbuf.st_ino; + + sk->max_offset =UINT64_MAX ;
Quoted 8 lines
cmds/inspect-dump-file-extents.c line 115
Quoted 4 lines
cmds/inspect-dump-file-extents.c new file +4
-0+ sk->min_transid =0 ; + sk->max_transid =UINT64_MAX ; + sk->min_type = sk->max_type =BTRFS_EXTENT_DATA_KEY ; + sk->nr_items =4096 ;You may want to do the tree search ioctl in a loop, as it's pretty common for super large or heavily fragmented inode to have way more items than one ioctl can return.
I don't think much about this. I wonder if it's proper way to search tree. is there any better way than this code?
Quoted 37 lines
Quoted 23 lines
cmds/inspect-dump-file-extents.c new file +22
-0+ +while (statbuf.st_size > pos) { + sk->min_offset = pos; +if (ioctl (fd,BTRFS_IOC_TREE_SEARCH , &args)) { +error ("failed to search tree ioctl %s: %m" , argv[optind]); + ret =1 ; + goto out; + } + + buf_off =0 ; +for (i=0 ; i<sk->nr_items; ++i) { + header = (struct btrfs_ioctl_search_header *)(args.buf + buf_off); + +if (btrfs_search_header_type (header) ==BTRFS_EXTENT_DATA_KEY ) { + extent_item = (struct btrfs_file_extent_item *)(header +1 ); + begin =btrfs_search_header_offset (header); + +printf ("type = %s, begin = %llu, " , +file_extent_type_to_str (extent_item->type), begin); +switch (extent_item->type) { +case BTRFS_FILE_EXTENT_INLINE : + len =le64_to_cpu (extent_item->ram_bytes);cmds/inspect-dump-file-extents.c line 147
Quoted 10 lines
cmds/inspect-dump-file-extents.c new file +10
-0+printf ("end = %llu\n" , begin + len); +break ; +case BTRFS_FILE_EXTENT_REG : +case BTRFS_FILE_EXTENT_PREALLOC : + len =le64_to_cpu (extent_item->num_bytes); + disk_bytenr =le64_to_cpu (extent_item->disk_bytenr); + disk_num_bytes =le64_to_cpu (extent_item->disk_num_bytes); + offset =le64_to_cpu (extent_item->offset); +compress_type_to_str (extent_item->compression, compress_str); + printf("end = %llu, disk_bytenr = %llu, disk_num_bytes = %llu,"For "end" we normally mean inclusive end.
E.g, for @start = 1M, @len = 4K, then the @end should be 1M + 4K - 1.
Quoted 1 line
Thus it would be better to just output the length, not the end.
Okay, Printing output as start and len would be more explicit.
Thanks,
Sidong
Quoted 39 lines
(I know this sounds a little nitpicking, but trust me, when you have seen too many bugs caused by such offset-by-one behavior, you will be as sensitive as me on this)
Thanks,
QuQuoted 30 lines
cmds/inspect-dump-file-extents.c new file +17
-0+" offset = %llu, compression = %s\n" , + begin + len, disk_bytenr, disk_num_bytes, offset, compress_str); > + + break; + } + + } + buf_off +=sizeof (*header) +btrfs_search_header_len (header); + pos += len; + } + + } + ret =0 ; +out: +close (fd); +return ret; +} +DEFINE_SIMPLE_COMMAND (inspect_dump_file_extents,"dump-file-extents" );cmds/inspect.c +1
-0@@ -696,6 +696,7 @@ static const struct cmd_group inspect_cmd_group = { &cmd_struct_inspect_dump_tree, &cmd_struct_inspect_dump_super, &cmd_struct_inspect_tree_stats, + &cmd_struct_inspect_dump_file_extents, NULL } };
On 2021/7/12 下午2:40, Sidong Yang wrote:
[...]
Quoted 10 lines
Quoted 9 lines
cmds/inspect-dump-file-extents.c line 115
Quoted 5 lines
cmds/inspect-dump-file-extents.c new file +5
-0+ sk->max_offset =UINT64_MAX ; + sk->min_transid =0 ; + sk->max_transid =UINT64_MAX ; + sk->min_type = sk->max_type =BTRFS_EXTENT_DATA_KEY ; + sk->nr_items =4096 ;You may want to do the tree search ioctl in a loop, as it's pretty common for super large or heavily fragmented inode to have way more items than one ioctl can return.
Quoted 2 lines
I don't think much about this. I wonder if it's proper way to search tree. is there any better way than this code?
Here is one example:
https://github.com/kilobyte/compsize/blob/master/compsize.c#L179
It goes @again label to restart from last offset + 1.
Thanks,
Qu
Quoted 14 lines · Qu Wenruo
On 2021/7/12 下午2:40, Sidong Yang wrote:
[...]Quoted 10 lines
Quoted 9 lines
cmds/inspect-dump-file-extents.c line 115
Quoted 5 lines
cmds/inspect-dump-file-extents.c new file +5
-0+ sk->max_offset =UINT64_MAX ; + sk->min_transid =0 ; + sk->max_transid =UINT64_MAX ; + sk->min_type = sk->max_type =BTRFS_EXTENT_DATA_KEY ; + sk->nr_items =4096 ;You may want to do the tree search ioctl in a loop, as it's pretty common for super large or heavily fragmented inode to have way more items than one ioctl can return.
Quoted 8 lines
Quoted 2 lines
I don't think much about this. I wonder if it's proper way to search tree. is there any better way than this code?
Here is one example:
https://github.com/kilobyte/compsize/blob/master/compsize.c#L179
It goes @again label to restart from last offset + 1.
Thanks, I read the code you provided. It seems that compsize is also doing to what this patch do.
The 'do_file' function checks if nr_items is larger than 512. I wonder that the number '512' is such an magic number. I'll check it by my own effort.
Thanks,
Sidong
Quoted 4 lines
Thanks,
Qu
Quoted 19 lines · Sidong Yang
cmds/inspect-dump-file-extents.c new file +19
-0+static void compress_type_to_str (u8 compress_type,char *ret) +{ +switch (compress_type) { +case BTRFS_COMPRESS_NONE : +strcpy (ret,"none" ); +break ; +case BTRFS_COMPRESS_ZLIB : +strcpy (ret,"zlib" ); +break ; +case BTRFS_COMPRESS_LZO : +strcpy (ret,"lzo" ); +break ; +case BTRFS_COMPRESS_ZSTD : +strcpy (ret,"zstd" ); +break ; +default : +sprintf (ret,"UNKNOWN.%d" , compress_type); + } +}
[....]
Quoted 1 line
cmds/inspect-dump-file-extents.c new file +1
-0+char compress_str[16 ];
[...]
cmds/inspect-dump-file-extents.c line 146
Quoted 1 line
cmds/inspect-dump-file-extents.c new file +1
-0+compress_type_to_str (extent_item->compression, compress_str);
While this looks safe at a first glance, can we change this to:
#define COMPRESS_STR_LEN 5
[...]
switch (compress_type) {
case BTRFS_COMPRESS_NONE:
strncpy(ret, "none", COMPRESS_STR_LEN);
[...]
char compress_str[COMPRESS_STR_LEN];
One day someone will factor out 'compress_type_to_str()' and make it public, read user supplied input and then it's a disaster waiting to happen.
Thanks,
Johannes
On Mon, Jul 12, 2021 at 06:52:28AM +0000, Johannes Thumshirn wrote:
Hi, Thanks for review!
Quoted 36 lines · Sidong Yang
Quoted 19 lines
cmds/inspect-dump-file-extents.c new file +19
-0+static void compress_type_to_str (u8 compress_type,char *ret) +{ +switch (compress_type) { +case BTRFS_COMPRESS_NONE : +strcpy (ret,"none" ); +break ; +case BTRFS_COMPRESS_ZLIB : +strcpy (ret,"zlib" ); +break ; +case BTRFS_COMPRESS_LZO : +strcpy (ret,"lzo" ); +break ; +case BTRFS_COMPRESS_ZSTD : +strcpy (ret,"zstd" ); +break ; +default : +sprintf (ret,"UNKNOWN.%d" , compress_type); + } +}[....]
Quoted 1 line
cmds/inspect-dump-file-extents.c new file +1
-0+char compress_str[16 ];[...]
cmds/inspect-dump-file-extents.c line 146
Quoted 1 line
cmds/inspect-dump-file-extents.c new file +1
-0+compress_type_to_str (extent_item->compression, compress_str);While this looks safe at a first glance, can we change this to:
#define
COMPRESS_STR_LEN5[...]
switch (
compress_type) {
caseBTRFS_COMPRESS_NONE:
Quoted 8 lines
strncpy(ret, "none", COMPRESS_STR_LEN);[...]
char compress_str[COMPRESS_STR_LEN];
One day someone will factor out '
compress_type_to_str()' and make it public, read user supplied input and then it's a disaster waiting to happen.
This can be happen when @ret is too short? If it so, I think it also has problem when @ret is shorter than COMPRESS_STR_LEN. I wonder that I understood this problem you said.
Thanks,
Sidong
Quoted 3 lines
Thanks,
Johannes
Quoted 44 lines · Sidong Yang
On Mon, Jul 12, 2021 at 06:52:28AM +0000, Johannes Thumshirn wrote:
Hi, Thanks for review!
Quoted 40 lines · Sidong Yang
Quoted 19 lines
cmds/inspect-dump-file-extents.c new file +19
-0+static void compress_type_to_str (u8 compress_type,char *ret) +{ +switch (compress_type) { +case BTRFS_COMPRESS_NONE : +strcpy (ret,"none" ); +break ; +case BTRFS_COMPRESS_ZLIB : +strcpy (ret,"zlib" ); +break ; +case BTRFS_COMPRESS_LZO : +strcpy (ret,"lzo" ); +break ; +case BTRFS_COMPRESS_ZSTD : +strcpy (ret,"zstd" ); +break ; +default : +sprintf (ret,"UNKNOWN.%d" , compress_type); + } +}[....]
Quoted 1 line
cmds/inspect-dump-file-extents.c new file +1
-0+char compress_str[16 ];[...]
cmds/inspect-dump-file-extents.c line 146
Quoted 1 line
cmds/inspect-dump-file-extents.c new file +1
-0+compress_type_to_str (extent_item->compression, compress_str);While this looks safe at a first glance, can we change this to:
#define
COMPRESS_STR_LEN5[...]
switch (
compress_type) {
caseBTRFS_COMPRESS_NONE:strncpy(ret, "none", COMPRESS_STR_LEN);[...]
Quoted 8 lines
Quoted 4 lines
char compress_str[COMPRESS_STR_LEN];
One day someone will factor out '
compress_type_to_str()' and make it public, read user supplied input and then it's a disaster waiting to happen.This can be happen when @ret is too short? If it so, I think it also has problem when @ret is shorter than
COMPRESS_STR_LEN. I wonder that I understood this problem you said.
I guess Johannes means that, if we support more and more new compression algos, it can go beyond your original 16 bytes length for the compression algo name.
While using strncpy, it will never go beyond COMPRESS_STR_LEN forever. (Although it will truncating the output, but that would be easier to spot the problem and then enlarge the macro)
Thanks,
Qu
Quoted 7 lines
Thanks,
SidongQuoted 3 lines
Thanks,
Johannes
Quoted 56 lines · Qu Wenruo
Quoted 44 lines · Sidong Yang
On Mon, Jul 12, 2021 at 06:52:28AM +0000, Johannes Thumshirn wrote:
Hi, Thanks for review!
Quoted 40 lines · Sidong Yang
Quoted 19 lines
cmds/inspect-dump-file-extents.c new file +19
-0+static void compress_type_to_str (u8 compress_type,char *ret) +{ +switch (compress_type) { +case BTRFS_COMPRESS_NONE : +strcpy (ret,"none" ); +break ; +case BTRFS_COMPRESS_ZLIB : +strcpy (ret,"zlib" ); +break ; +case BTRFS_COMPRESS_LZO : +strcpy (ret,"lzo" ); +break ; +case BTRFS_COMPRESS_ZSTD : +strcpy (ret,"zstd" ); +break ; +default : +sprintf (ret,"UNKNOWN.%d" , compress_type); + } +}[....]
Quoted 1 line
cmds/inspect-dump-file-extents.c new file +1
-0+char compress_str[16 ];[...]
cmds/inspect-dump-file-extents.c line 146
Quoted 1 line
cmds/inspect-dump-file-extents.c new file +1
-0+compress_type_to_str (extent_item->compression, compress_str);While this looks safe at a first glance, can we change this to:
#define
COMPRESS_STR_LEN5[...]
switch (
compress_type) {
caseBTRFS_COMPRESS_NONE:strncpy(ret, "none", COMPRESS_STR_LEN);[...]
Quoted 8 lines
Quoted 4 lines
char compress_str[COMPRESS_STR_LEN];
One day someone will factor out '
compress_type_to_str()' and make it public, read user supplied input and then it's a disaster waiting to happen.This can be happen when @ret is too short? If it so, I think it also has problem when @ret is shorter than
COMPRESS_STR_LEN. I wonder that I understood this problem you said.
Quoted 7 lines
I guess Johannes means that, if we support more and more new compression algos, it can go beyond your original 16 bytes length for the compression algo name.
While using strncpy, it will never go beyond
COMPRESS_STR_LENforever. (Although it will truncating the output, but that would be easier to spot the problem and then enlarge the macro)
Okay, I understood. I'll do this in next version.
Thanks,
Sidong
Quoted 11 lines
Thanks,
QuQuoted 7 lines
Thanks,
SidongQuoted 3 lines
Thanks,
Johannes