btrfs-progs: qgroups usability
Subsystems (3)
cover btrfs-progs: qgroups usability
From: Jeff Mahoney <[email protected]>
Hi all -
The following series addresses some usability issues with the qgroups UI.
1) Adds -W option so we can wait on a rescan completing without starting one.
2) Adds qgroup information to 'btrfs subvolume show'
3) Adds a -P option to show pathnames for first-level qgroups (or member of nested qgroups with -v)
4) Allows exporting the qgroup table in JSON format for use by external programs/scripts.
-Jeff
Jeff Mahoney (8):
btrfs-progs: quota: Add -W option to rescan to wait without starting
rescan
btrfs-progs: qgroups: fix misleading index check
btrfs-progs: constify pathnames passed as arguments
btrfs-progs: qgroups: add pathname to show output
btrfs-progs: qgroups: introduce and use info and limit structures
btrfs-progs: qgroups: introduce btrfs_qgroup_query
btrfs-progs: subvolume: add quota info to btrfs sub show
btrfs-progs: qgroups: export qgroups usage information as JSON
Documentation/btrfs-qgroup.asciidoc | 8 + Documentation/btrfs-quota.asciidoc | 10 +- Makefile.inc.in | 4 +- chunk-recover.c | 4 +- cmds-device.c | 2 +- cmds-fi-usage.c | 6 +- cmds-qgroup.c | 49 +++- cmds-quota.c | 21 +- cmds-rescue.c | 4 +- cmds-subvolume.c | 46 ++++ configure.ac | 6 + kerncompat.h | 1 + qgroup.c | 526 ++++++++++++++++++++++++++++++------ qgroup.h | 22 +- send-utils.c | 4 +- utils.c | 22 +- utils.h | 2 + 17 files changed, 621 insertions(+), 116 deletions(-)
Signature · 1 line
2.15.1
PATCH 1/8 btrfs-progs: quota: Add -W option to rescan to wait without starting rescan
From: Jeff Mahoney <[email protected]>
This patch adds a new -W option to wait for a rescan without starting a new operation. This is useful for things like xfstests where we want do to do a "btrfs quota enable" and not continue until the subsequent rescan has finished.
In addition to documenting the new option in the man page, I've cleaned up the rescan entry to document the -w option a bit better.
--- Documentation/btrfs-quota.asciidoc | 10 +++++++--- cmds-quota.c | 21 +++++++++++++++------ 2 files changed, 22 insertions(+), 9 deletions(-)
Documentation/btrfs-quota.asciidoc +7 -3
@@ -238,15 +238,19 @@ Disable subvolume quota support for a filesystem. *enable* <path>:: Enable subvolume quota support for a filesystem. -*rescan* [-s] <path>:: +*rescan* [-s|-w|-W] <path>:: Trash all qgroup numbers and scan the metadata again with the current config. + `Options` + -s:::: -show status of a running rescan operation. +Show status of a running rescan operation. + -w:::: -wait for rescan operation to finish(can be already in progress). +Start rescan operation and wait until it has finished before exiting. If a rescan is already running, wait until it finishes and then exit without starting a new one. + +-W:::: +Wait for rescan operation to finish and then exit. If a rescan is not already running, exit silently. EXIT STATUS -----------
cmds-quota.c +15 -6
@@ -120,14 +120,20 @@ static int cmd_quota_rescan(int argc, char **argv)int wait_for_completion =0 ;while (1 ) { -int c =getopt (argc, argv,"sw" ); +int c =getopt (argc, argv,"swW" );if (c <0 )break ;switch (c) {case 's' : ioctlnum =BTRFS_IOC_QUOTA_RESCAN_STATUS ;break ; +case 'W' : + ioctlnum =0 ; + wait_for_completion =1 ; +break ;case 'w' : +/* Reset it in case the user did both -W and -w */ + ioctlnum =BTRFS_IOC_QUOTA_RESCAN ; wait_for_completion =1 ;break ;default :
@@ -135,8 +141,9 @@ static int cmd_quota_rescan(int argc, char **argv) } } -if (ioctlnum !=BTRFS_IOC_QUOTA_RESCAN && wait_for_completion) { -error ("switch -w cannot be used with -s" ); +if (ioctlnum ==BTRFS_IOC_QUOTA_RESCAN_STATUS && wait_for_completion) { +error ("switch -%c cannot be used with -s" , + ioctlnum ?'w' :'W' );return 1 ; }
@@ -150,8 +157,10 @@ static int cmd_quota_rescan(int argc, char **argv)if (fd <0 )return 1 ; - ret =ioctl (fd, ioctlnum, &args); - e = errno; +if (ioctlnum) { + ret =ioctl (fd, ioctlnum, &args); + e = errno; + }if (ioctlnum ==BTRFS_IOC_QUOTA_RESCAN_STATUS ) {close_file_or_dir (fd, dirstream);
@@ -167,7 +176,7 @@ static int cmd_quota_rescan(int argc, char **argv)return 0 ; } -if (ret ==0 ) { +if (ioctlnum ==BTRFS_IOC_QUOTA_RESCAN && ret ==0 ) {printf ("quota rescan started\n" );fflush (stdout); }else if (ret <0 && (!wait_for_completion || e !=EINPROGRESS )) {
Signature · 1 line
2.15.1
PATCH 2/8 btrfs-progs: qgroups: fix misleading index check
In reply to [email protected]From: Jeff Mahoney <[email protected]>
In print_single_qgroup_table we check the loop index against BTRFS_QGROUP_CHILD, but what we really mean is "last column." Since we have an enum value to indicate the last value, use that instead of assuming that BTRFS_QGROUP_CHILD is always last.
--- qgroup.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
qgroup.c +1 -1
@@ -267,7 +267,7 @@ static void print_single_qgroup_table(struct btrfs_qgroup *qgroup)continue ;print_qgroup_column (qgroup, i); -if (i !=BTRFS_QGROUP_CHILD ) +if (i !=BTRFS_QGROUP_ALL -1 )printf (" " ); }printf ("\n" );
Signature · 1 line
2.15.1
PATCH 3/8 btrfs-progs: constify pathnames passed as arguments
In reply to [email protected]From: Jeff Mahoney <[email protected]>
It's unlikely we're going to modify a pathname argument, so codify that and use const.
--- chunk-recover.c | 4 ++-- cmds-device.c | 2 +- cmds-fi-usage.c | 6 +++--- cmds-rescue.c | 4 ++-- send-utils.c | 4 ++-- 5 files changed, 10 insertions(+), 10 deletions(-)
chunk-recover.c +2 -2
@@ -1492,7 +1492,7 @@ out:return ERR_PTR (ret); } -static int recover_prepare (struct recover_control *rc,char *path) +static int recover_prepare (struct recover_control *rc,const char *path) {int ret;int fd;
@@ -2296,7 +2296,7 @@ static void validate_rebuild_chunks(struct recover_control *rc)/* * Return 0 when successful, < 0 on error and > 0 if aborted by user */ -int btrfs_recover_chunk_tree (char *path,int verbose,int yes) +int btrfs_recover_chunk_tree (const char *path,int verbose,int yes) {int ret =0 ;struct btrfs_root *root =NULL ;
cmds-device.c +1 -1
@@ -526,7 +526,7 @@ static const char * const cmd_device_usage_usage[] = { NULL }; -static int _cmd_device_usage (int fd,char *path,unsigned unit_mode) +static int _cmd_device_usage (int fd,const char *path,unsigned unit_mode) {int i;int ret =0 ;
cmds-fi-usage.c +3 -3
@@ -227,7 +227,7 @@ static int cmp_btrfs_ioctl_space_info(const void *a, const void *b)/* * This function load all the information about the space usage */ -static struct btrfs_ioctl_space_args *load_space_info (int fd,char *path) +static struct btrfs_ioctl_space_args *load_space_info (int fd,const char *path) {struct btrfs_ioctl_space_args *sargs =NULL , *sargs_orig =NULL ;int ret, count;
@@ -305,7 +305,7 @@ static void get_raid56_used(struct chunk_info *chunks, int chunkcount,#define MIN_UNALOCATED_THRESH SZ_16Mstatic int print_filesystem_usage_overall (int fd,struct chunk_info *chunkinfo,int chunkcount,struct device_info *devinfo,int devcount, -char *path,unsigned unit_mode) +const char *path,unsigned unit_mode) {struct btrfs_ioctl_space_args *sargs =NULL ;int i;
@@ -931,7 +931,7 @@ static void _cmd_filesystem_usage_linear(unsigned unit_mode,static int print_filesystem_usage_by_chunk (int fd,struct chunk_info *chunkinfo,int chunkcount,struct device_info *devinfo,int devcount, -char *path,unsigned unit_mode,int tabular) +const char *path,unsigned unit_mode,int tabular) {struct btrfs_ioctl_space_args *sargs;int ret =0 ;
cmds-rescue.c +2 -2
@@ -32,8 +32,8 @@ static const char * const rescue_cmd_group_usage[] = {NULL }; -int btrfs_recover_chunk_tree (char *path,int verbose,int yes); -int btrfs_recover_superblocks (char *path,int verbose,int yes); +int btrfs_recover_chunk_tree (const char *path,int verbose,int yes); +int btrfs_recover_superblocks (const char *path,int verbose,int yes);static const char *const cmd_rescue_chunk_recover_usage[] = {"btrfs rescue chunk-recover [options] <device>" ,
send-utils.c +2 -2
@@ -28,8 +28,8 @@#include "ioctl.h" #include "btrfs-list.h" -static int btrfs_subvolid_resolve_sub (int fd,char *path,size_t *path_len, -u64 subvol_id); +static int btrfs_subvolid_resolve_sub (int fd,char *path, +size_t *path_len,u64 subvol_id);static int btrfs_get_root_id_by_sub_path (int mnt_fd,const char *sub_path,u64 *root_id)
Signature · 1 line
2.15.1
PATCH 4/8 btrfs-progs: qgroups: add pathname to show output
In reply to [email protected]From: Jeff Mahoney <[email protected]>
The btrfs qgroup show command currently only exports qgroup IDs, forcing the user to resolve which subvolume each corresponds to.
This patch adds pathname resolution to qgroup show so that when the -P option is used, the last column contains the pathname of the root of the subvolume it describes. In the case of nested qgroups, it will show the number of member qgroups or the paths of the members if the -v option is used.
Pathname can also be used as a sort parameter.
--- Documentation/btrfs-qgroup.asciidoc | 4 + cmds-qgroup.c | 17 ++++- kerncompat.h | 1 + qgroup.c | 142 ++++++++++++++++++++++++++++++++---- qgroup.h | 4 +- utils.c | 22 ++++-- utils.h | 2 + 7 files changed, 166 insertions(+), 26 deletions(-)
Documentation/btrfs-qgroup.asciidoc +4 -0
@@ -97,10 +97,14 @@ print child qgroup id. print limit of referenced size of qgroup. -e:::: print limit of exclusive size of qgroup. +-P:::: +print pathname to the root of the subvolume managed by qgroup. For nested qgroups, the number of members will be printed unless -v is specified. -F:::: list all qgroups which impact the given path(include ancestral qgroups) -f:::: list all qgroups which impact the given path(exclude ancestral qgroups) +-v:::: +Be more verbose. Print pathnames of member qgroups when nested. --raw:::: raw numbers in bytes, without the 'B' suffix. --human-readable::::
cmds-qgroup.c +14 -3
@@ -280,8 +280,10 @@ static const char * const cmd_qgroup_show_usage[] = {" (including ancestral qgroups)" ,"-f list all qgroups which impact the given path" ," (excluding ancestral qgroups)" , +"-P print first-level qgroups using pathname" , +"-v verbose, prints all nested subvolumes" ,HELPINFO_UNITS_LONG , -"--sort=qgroupid,rfer,excl,max_rfer,max_excl" , +"--sort=qgroupid,rfer,excl,max_rfer,max_excl,pathname" ," list qgroups sorted by specified items" ," you can use '+' or '-' in front of each item." , " (+:ascending, -:descending, ascending default)",
@@ -299,6 +301,7 @@ static int cmd_qgroup_show(int argc, char **argv)int filter_flag =0 ;unsigned unit_mode;int sync =0 ; +bool verbose = false;struct btrfs_qgroup_comparer_set *comparer_set;struct btrfs_qgroup_filter_set *filter_set;
@@ -316,10 +319,11 @@ static int cmd_qgroup_show(int argc, char **argv)static const struct option long_options[] = { {"sort" , required_argument,NULL ,GETOPT_VAL_SORT }, {"sync" , no_argument,NULL ,GETOPT_VAL_SYNC }, + {"verbose" , no_argument,NULL ,'v' }, {NULL ,0 ,NULL ,0 } }; - c =getopt_long (argc, argv,"pcreFf" , long_options,NULL ); + c =getopt_long (argc, argv,"pPcreFfv" , long_options,NULL );if (c <0 )break ;switch (c) {
@@ -327,6 +331,10 @@ static int cmd_qgroup_show(int argc, char **argv)btrfs_qgroup_setup_print_column (BTRFS_QGROUP_PARENT );break ; +case 'P' : +btrfs_qgroup_setup_print_column ( +BTRFS_QGROUP_PATHNAME ); +break ;case 'c' :btrfs_qgroup_setup_print_column (BTRFS_QGROUP_CHILD );
@@ -354,6 +362,9 @@ static int cmd_qgroup_show(int argc, char **argv)case GETOPT_VAL_SYNC : sync =1 ;break ; +case 'v' : + verbose = true; +break ;default :usage (cmd_qgroup_show_usage); }
@@ -394,7 +405,7 @@ static int cmd_qgroup_show(int argc, char **argv)BTRFS_QGROUP_FILTER_PARENT , qgroupid); } - ret =btrfs_show_qgroups (fd, filter_set, comparer_set); + ret =btrfs_show_qgroups (fd, filter_set, comparer_set, verbose);close_file_or_dir (fd, dirstream);free (filter_set);free (comparer_set);
kerncompat.h +1 -0
@@ -29,6 +29,7 @@#include <stddef.h> #include <linux/types.h> #include <stdint.h> +#include <stdbool.h> #include <features.h>
qgroup.c +127 -15
270 lines not on this page · raw message
qgroup.h +3 -1
@@ -59,11 +59,13 @@ enum btrfs_qgroup_column_enum {BTRFS_QGROUP_MAX_EXCL ,BTRFS_QGROUP_PARENT ,BTRFS_QGROUP_CHILD , +BTRFS_QGROUP_PATHNAME , BTRFS_QGROUP_ALL, };enum btrfs_qgroup_comp_enum {BTRFS_QGROUP_COMP_QGROUPID , +BTRFS_QGROUP_COMP_PATHNAME ,BTRFS_QGROUP_COMP_RFER ,BTRFS_QGROUP_COMP_EXCL , BTRFS_QGROUP_COMP_MAX_RFER,
@@ -80,7 +82,7 @@ enum btrfs_qgroup_filter_enum {int btrfs_qgroup_parse_sort_string (const char *opt_arg,struct btrfs_qgroup_comparer_set **comps);int btrfs_show_qgroups (int fd,struct btrfs_qgroup_filter_set *, -struct btrfs_qgroup_comparer_set *); +struct btrfs_qgroup_comparer_set *,bool verbose);void btrfs_qgroup_setup_print_column (enum btrfs_qgroup_column_enum column);void btrfs_qgroup_setup_units (unsigned unit_mode);struct btrfs_qgroup_filter_set *btrfs_qgroup_alloc_filter_set (void );
utils.c +15 -7
@@ -2556,15 +2556,9 @@ out:return ret; } -int get_subvol_info_by_rootid (const char *mnt,struct root_info *get_ri,u64 r_id) +int get_subvol_info_by_rootid_fd (int fd,struct root_info *get_ri,u64 r_id) { -int fd;int ret; -DIR *dirstream =NULL ; - - fd =btrfs_open_dir (mnt, &dirstream,1 ); -if (fd <0 ) -return -EINVAL ;memset (get_ri,0 ,sizeof (*get_ri)); get_ri->root_id = r_id;
@@ -2574,6 +2568,20 @@ int get_subvol_info_by_rootid(const char *mnt, struct root_info *get_ri, u64 r_ielse ret =btrfs_get_subvol (fd, get_ri); +return ret; +} + +int get_subvol_info_by_rootid (const char *mnt,struct root_info *get_ri,u64 r_id) +{ +int fd; +int ret; +DIR *dirstream =NULL ; + + fd =btrfs_open_dir (mnt, &dirstream,1 ); +if (fd <0 ) +return -EINVAL ; + + ret =get_subvol_info_by_rootid_fd (fd, get_ri, r_id);if (ret)error ("can't find rootid '%llu' on '%s': %d" , r_id, mnt, ret);
utils.h +2 -0
@@ -154,6 +154,8 @@ int test_isdir(const char *path);const char *subvol_strip_mountpoint (const char *mnt,const char *full_path);int get_subvol_info (const char *fullpath,struct root_info *get_ri); +int get_subvol_info_by_rootid_fd (int fd,struct root_info *get_ri, +u64 rootid_arg);int get_subvol_info_by_rootid (const char *mnt,struct root_info *get_ri,u64 rootid_arg); int get_subvol_info_by_uuid(const char *mnt, struct root_info *get_ri,
Signature · 1 line
2.15.1
PATCH 5/8 btrfs-progs: qgroups: export qgroups usage information as JSON
In reply to [email protected]From: Jeff Mahoney <[email protected]>
One of the common requests I receive is for 'df' like facilities for subvolume usage. Really, the request is for monitoring tools to be able to understand when subvolumes may be approaching quota in the same manner traditional file systems approach ENOSPC.
This patch allows us to export the qgroups data in a machine-readable format so that monitoring tools can parse it easily.
There are two modes since JSON can technically handle 64-bit numbers but JavaScript proper cannot. show -j enables JSON mode using 64-bit integers directly. --json-compat presents 64-bit numbers as an array of two 32-bit numbers (high followed by low).
--- Documentation/btrfs-qgroup.asciidoc | 4 + Makefile.inc.in | 4 +- cmds-qgroup.c | 36 +++++- configure.ac | 6 + qgroup.c | 211 ++++++++++++++++++++++++++++++++++++ qgroup.h | 3 + 6 files changed, 258 insertions(+), 6 deletions(-)
Documentation/btrfs-qgroup.asciidoc +4 -0
@@ -105,6 +105,10 @@ list all qgroups which impact the given path(include ancestral qgroups) list all qgroups which impact the given path(exclude ancestral qgroups) -v:::: Be more verbose. Print pathnames of member qgroups when nested. +-j:::: +If enabled, export qgroup usage information in JSON format. This implies --raw. +--json-compat:::: +By default, JSON output contains full 64-bit integers, which may be incompatible with some JSON parsers. This option exports those values as an array of 32-bit numbers in [high, low] format. --raw:::: raw numbers in bytes, without the 'B' suffix. --human-readable::::
Makefile.inc.in +2 -2
@@ -18,9 +18,9 @@ BTRFSRESTORE_ZSTD = @BTRFSRESTORE_ZSTD@ SUBST_CFLAGS = @CFLAGS@ SUBST_LDFLAGS = @LDFLAGS@ -LIBS_BASE = @UUID_LIBS@ @BLKID_LIBS@ -L. -pthread +LIBS_BASE = @UUID_LIBS@ @BLKID_LIBS@ @JSON_LIBS@ -L. -pthread LIBS_COMP = @ZLIB_LIBS@ @LZO2_LIBS@ @ZSTD_LIBS@ -STATIC_LIBS_BASE = @UUID_LIBS_STATIC@ @BLKID_LIBS_STATIC@ -L. -pthread +STATIC_LIBS_BASE = @UUID_LIBS_STATIC@ @BLKID_LIBS_STATIC@ @JSON_LIBS_STATIC@ -L. -pthread STATIC_LIBS_COMP = @ZLIB_LIBS_STATIC@ @LZO2_LIBS_STATIC@ @ZSTD_LIBS_STATIC@ prefix ?= @prefix@
cmds-qgroup.c +32 -4
@@ -282,6 +282,10 @@ static const char * const cmd_qgroup_show_usage[] = {" (excluding ancestral qgroups)" ,"-P print first-level qgroups using pathname" ,"-v verbose, prints all nested subvolumes" , +#ifdef HAVE_JSON +"-j export in JSON format" , +"--json-compat export in JSON compatibility mode" , +#endifHELPINFO_UNITS_LONG ,"--sort=qgroupid,rfer,excl,max_rfer,max_excl,pathname" , " list qgroups sorted by specified items",
@@ -302,6 +306,8 @@ static int cmd_qgroup_show(int argc, char **argv)unsigned unit_mode;int sync =0 ;bool verbose = false; +bool export_json = false; +bool compat_json = false;struct btrfs_qgroup_comparer_set *comparer_set;struct btrfs_qgroup_filter_set *filter_set;
@@ -314,16 +320,26 @@ static int cmd_qgroup_show(int argc, char **argv)int c;enum {GETOPT_VAL_SORT =256 , -GETOPT_VAL_SYNC +GETOPT_VAL_SYNC , +GETOPT_VAL_JSCOMPAT , };static const struct option long_options[] = { {"sort" , required_argument,NULL ,GETOPT_VAL_SORT }, {"sync" , no_argument,NULL ,GETOPT_VAL_SYNC }, {"verbose" , no_argument,NULL ,'v' }, +#ifdef HAVE_JSON + {"json-compat" , no_argument,NULL ,GETOPT_VAL_JSCOMPAT }, +#endif {NULL ,0 ,NULL ,0 } }; - - c =getopt_long (argc, argv,"pPcreFfv" , long_options,NULL ); +const char getopt_chars[] = { +'p' ,'P' ,'c' ,'r' ,'e' ,'F' ,'f' ,'v' , +#ifdef HAVE_JSON +'j' , +#endif +'\0' }; + + c =getopt_long (argc, argv, getopt_chars, long_options,NULL );if (c <0 )break ;switch (c) {
@@ -353,6 +369,14 @@ static int cmd_qgroup_show(int argc, char **argv)case 'f' : filter_flag |=0x2 ;break ; +#ifdef HAVE_JSON +case GETOPT_VAL_JSCOMPAT : + compat_json = true; +case 'j' : + unit_mode =UNITS_RAW ; + export_json = true; +break ; +#endif case GETOPT_VAL_SORT : ret =btrfs_qgroup_parse_sort_string (optarg, &comparer_set);
@@ -405,7 +429,11 @@ static int cmd_qgroup_show(int argc, char **argv)BTRFS_QGROUP_FILTER_PARENT , qgroupid); } - ret =btrfs_show_qgroups (fd, filter_set, comparer_set, verbose); +if (export_json) + ret =btrfs_export_qgroups_json (fd, filter_set, comparer_set, + compat_json); +else + ret =btrfs_show_qgroups (fd, filter_set, comparer_set, verbose);close_file_or_dir (fd, dirstream);free (filter_set);free (comparer_set);
configure.ac +6 -0
@@ -197,6 +197,12 @@ PKG_STATIC(UUID_LIBS_STATIC, [uuid]) PKG_CHECK_MODULES(ZLIB, [zlib]) PKG_STATIC(ZLIB_LIBS_STATIC, [zlib]) +PKG_CHECK_MODULES(JSON, [json-c], [ + AC_DEFINE(HAVE_JSON, [1], [Have JSON]), + PKG_STATIC(JSON_LIBS_STATIC, [json-c], [ + AC_DEFINE(HAVE_JSON_STATIC, [1], [Have JSON static])], [true]) + ], [true]) + AC_ARG_ENABLE([zstd], AS_HELP_STRING([--disable-zstd], [build without zstd support]), [], [enable_zstd=yes]
qgroup.c +211 -0
229 lines not on this page · raw message
qgroup.h +3 -0
@@ -83,6 +83,9 @@ int btrfs_qgroup_parse_sort_string(const char *opt_arg, struct btrfs_qgroup_comparer_set **comps);int btrfs_show_qgroups (int fd,struct btrfs_qgroup_filter_set *,struct btrfs_qgroup_comparer_set *,bool verbose); +int btrfs_export_qgroups_json (int fd,struct btrfs_qgroup_filter_set *, +struct btrfs_qgroup_comparer_set *, +bool compat);void btrfs_qgroup_setup_print_column (enum btrfs_qgroup_column_enum column);void btrfs_qgroup_setup_units (unsigned unit_mode);struct btrfs_qgroup_filter_set *btrfs_qgroup_alloc_filter_set (void );
Signature · 1 line
2.15.1
PATCH 5/8 btrfs-progs: qgroups: introduce and use info and limit structures
In reply to [email protected]From: Jeff Mahoney <[email protected]>
We use structures to pass the info and limit from the kernel as items but store the individual values separately in btrfs_qgroup. We already have a btrfs_qgroup_limit structure that's used for setting the limit.
This patch introduces a btrfs_qgroup_info structure and uses that and btrfs_qgroup_limit in btrfs_qgroup.
--- qgroup.c | 73 +++++++++++++++++++++++++++++++--------------------------------- qgroup.h | 8 +++++++ 2 files changed, 43 insertions(+), 38 deletions(-)
qgroup.c +35 -38
@@ -46,20 +46,12 @@ struct btrfs_qgroup {/* * info_item */ -u64 generation; -u64 rfer;/*referenced*/ -u64 rfer_cmpr;/*referenced compressed*/ -u64 excl;/*exclusive*/ -u64 excl_cmpr;/*exclusive compressed*/ +struct btrfs_qgroup_info info;/* *limit_item */ -u64 flags;/*which limits are set*/ -u64 max_rfer; -u64 max_excl; -u64 rsv_rfer; -u64 rsv_excl; +struct btrfs_qgroup_limit limit;/*qgroups this group is member of*/ struct list_head qgroups;
@@ -272,24 +264,24 @@ static void print_qgroup_column(struct btrfs_qgroup *qgroup,print_qgroup_column_add_blank (BTRFS_QGROUP_QGROUPID , len);break ;case BTRFS_QGROUP_RFER : - len =printf ("%*s" , max_len,pretty_size_mode (qgroup->rfer, unit_mode)); + len =printf ("%*s" , max_len,pretty_size_mode (qgroup->info.referenced, unit_mode));break ;case BTRFS_QGROUP_EXCL : - len =printf ("%*s" , max_len,pretty_size_mode (qgroup->excl, unit_mode)); + len =printf ("%*s" , max_len,pretty_size_mode (qgroup->info.exclusive, unit_mode));break ;case BTRFS_QGROUP_PARENT : len =print_parent_column (qgroup);print_qgroup_column_add_blank (BTRFS_QGROUP_PARENT , len);break ;case BTRFS_QGROUP_MAX_RFER : -if (qgroup->flags &BTRFS_QGROUP_LIMIT_MAX_RFER ) - len =printf ("%*s" , max_len,pretty_size_mode (qgroup->max_rfer, unit_mode)); +if (qgroup->limit.flags &BTRFS_QGROUP_LIMIT_MAX_RFER ) + len =printf ("%*s" , max_len,pretty_size_mode (qgroup->limit.max_referenced, unit_mode));else len =printf ("%*s" , max_len,"none" );break ;case BTRFS_QGROUP_MAX_EXCL : -if (qgroup->flags &BTRFS_QGROUP_LIMIT_MAX_EXCL ) - len =printf ("%*s" , max_len,pretty_size_mode (qgroup->max_excl, unit_mode)); +if (qgroup->limit.flags &BTRFS_QGROUP_LIMIT_MAX_EXCL ) + len =printf ("%*s" , max_len,pretty_size_mode (qgroup->limit.max_exclusive, unit_mode));else len =printf ("%*s" , max_len,"none" );break ;
@@ -432,9 +424,9 @@ static int comp_entry_with_rfer(struct btrfs_qgroup *entry1, {int ret; -if (entry1->rfer > entry2->rfer) +if (entry1->info.referenced > entry2->info.referenced) ret =1 ; -else if (entry1->rfer < entry2->rfer) +else if (entry1->info.referenced < entry2->info.referenced) ret =-1 ;else ret =0 ;
@@ -448,9 +440,9 @@ static int comp_entry_with_excl(struct btrfs_qgroup *entry1, {int ret; -if (entry1->excl > entry2->excl) +if (entry1->info.exclusive > entry2->info.exclusive) ret =1 ; -else if (entry1->excl < entry2->excl) +else if (entry1->info.exclusive < entry2->info.exclusive) ret =-1 ;else ret =0 ;
@@ -464,9 +456,9 @@ static int comp_entry_with_max_rfer(struct btrfs_qgroup *entry1, {int ret; -if (entry1->max_rfer > entry2->max_rfer) +if (entry1->limit.max_referenced > entry2->limit.max_referenced) ret =1 ; -else if (entry1->max_rfer < entry2->max_rfer) +else if (entry1->limit.max_referenced < entry2->limit.max_referenced) ret =-1 ;else ret =0 ;
@@ -480,9 +472,9 @@ static int comp_entry_with_max_excl(struct btrfs_qgroup *entry1, {int ret; -if (entry1->max_excl > entry2->max_excl) +if (entry1->limit.max_exclusive > entry2->limit.max_exclusive) ret =1 ; -else if (entry1->max_excl < entry2->max_excl) +else if (entry1->limit.max_exclusive < entry2->limit.max_exclusive) ret =-1 ;else ret =0 ;
@@ -739,11 +731,13 @@ static int update_qgroup_info(int fd, struct qgroup_lookup *qgroup_lookup,if (IS_ERR_OR_NULL (bq))return PTR_ERR (bq); - bq->generation =btrfs_stack_qgroup_info_generation (info); - bq->rfer =btrfs_stack_qgroup_info_referenced (info); - bq->rfer_cmpr =btrfs_stack_qgroup_info_referenced_compressed (info); - bq->excl =btrfs_stack_qgroup_info_exclusive (info); - bq->excl_cmpr =btrfs_stack_qgroup_info_exclusive_compressed (info); + bq->info.generation =btrfs_stack_qgroup_info_generation (info); + bq->info.referenced =btrfs_stack_qgroup_info_referenced (info); + bq->info.referenced_compressed = +btrfs_stack_qgroup_info_referenced_compressed (info); + bq->info.exclusive =btrfs_stack_qgroup_info_exclusive (info); + bq->info.exclusive_compressed = +btrfs_stack_qgroup_info_exclusive_compressed (info);return 0 ; }
@@ -758,11 +752,14 @@ static int update_qgroup_limit(int fd, struct qgroup_lookup *qgroup_lookup,if (IS_ERR_OR_NULL (bq))return PTR_ERR (bq); - bq->flags =btrfs_stack_qgroup_limit_flags (limit); - bq->max_rfer =btrfs_stack_qgroup_limit_max_referenced (limit); - bq->max_excl =btrfs_stack_qgroup_limit_max_exclusive (limit); - bq->rsv_rfer =btrfs_stack_qgroup_limit_rsv_referenced (limit); - bq->rsv_excl =btrfs_stack_qgroup_limit_rsv_exclusive (limit); + bq->limit.flags =btrfs_stack_qgroup_limit_flags (limit); + bq->limit.max_referenced = +btrfs_stack_qgroup_limit_max_referenced (limit); + bq->limit.max_exclusive = +btrfs_stack_qgroup_limit_max_exclusive (limit); + bq->limit.rsv_referenced = +btrfs_stack_qgroup_limit_rsv_referenced (limit); + bq->limit.rsv_exclusive =btrfs_stack_qgroup_limit_rsv_exclusive (limit);return 0 ; }
@@ -1053,22 +1050,22 @@ static void __update_columns_max_len(struct btrfs_qgroup *bq, btrfs_qgroup_columns[column].max_len = len;break ;case BTRFS_QGROUP_RFER : - len =strlen (pretty_size_mode (bq->rfer, unit_mode)); + len =strlen (pretty_size_mode (bq->info.referenced, unit_mode));if (btrfs_qgroup_columns[column].max_len < len) btrfs_qgroup_columns[column].max_len = len;break ;case BTRFS_QGROUP_EXCL : - len =strlen (pretty_size_mode (bq->excl, unit_mode)); + len =strlen (pretty_size_mode (bq->info.exclusive, unit_mode));if (btrfs_qgroup_columns[column].max_len < len) btrfs_qgroup_columns[column].max_len = len;break ;case BTRFS_QGROUP_MAX_RFER : - len =strlen (pretty_size_mode (bq->max_rfer, unit_mode)); + len =strlen (pretty_size_mode (bq->limit.max_referenced, unit_mode));if (btrfs_qgroup_columns[column].max_len < len) btrfs_qgroup_columns[column].max_len = len;break ;case BTRFS_QGROUP_MAX_EXCL : - len =strlen (pretty_size_mode (bq->max_excl, unit_mode)); + len =strlen (pretty_size_mode (bq->limit.max_exclusive, unit_mode));if (btrfs_qgroup_columns[column].max_len < len) btrfs_qgroup_columns[column].max_len = len;break ;
qgroup.h +8 -0
@@ -79,6 +79,14 @@ enum btrfs_qgroup_filter_enum { BTRFS_QGROUP_FILTER_MAX, }; +struct btrfs_qgroup_info { +u64 generation; +u64 referenced; +u64 referenced_compressed; +u64 exclusive; +u64 exclusive_compressed; +}; +int btrfs_qgroup_parse_sort_string (const char *opt_arg,struct btrfs_qgroup_comparer_set **comps); int btrfs_show_qgroups(int fd, struct btrfs_qgroup_filter_set *,
Signature · 1 line
2.15.1
PATCH 6/8 btrfs-progs: qgroups: introduce and use info and limit structures
In reply to [email protected]From: Jeff Mahoney <[email protected]>
We use structures to pass the info and limit from the kernel as items but store the individual values separately in btrfs_qgroup. We already have a btrfs_qgroup_limit structure that's used for setting the limit.
This patch introduces a btrfs_qgroup_info structure and uses that and btrfs_qgroup_limit in btrfs_qgroup.
--- qgroup.c | 83 +++++++++++++++++++++++++++++++--------------------------------- qgroup.h | 8 +++++++ 2 files changed, 48 insertions(+), 43 deletions(-)
qgroup.c +40 -43
@@ -50,20 +50,12 @@ struct btrfs_qgroup {/* * info_item */ -u64 generation; -u64 rfer;/*referenced*/ -u64 rfer_cmpr;/*referenced compressed*/ -u64 excl;/*exclusive*/ -u64 excl_cmpr;/*exclusive compressed*/ +struct btrfs_qgroup_info info;/* *limit_item */ -u64 flags;/*which limits are set*/ -u64 max_rfer; -u64 max_excl; -u64 rsv_rfer; -u64 rsv_excl; +struct btrfs_qgroup_limit limit;/*qgroups this group is member of*/ struct list_head qgroups;
@@ -276,24 +268,24 @@ static void print_qgroup_column(struct btrfs_qgroup *qgroup,print_qgroup_column_add_blank (BTRFS_QGROUP_QGROUPID , len);break ;case BTRFS_QGROUP_RFER : - len =printf ("%*s" , max_len,pretty_size_mode (qgroup->rfer, unit_mode)); + len =printf ("%*s" , max_len,pretty_size_mode (qgroup->info.referenced, unit_mode));break ;case BTRFS_QGROUP_EXCL : - len =printf ("%*s" , max_len,pretty_size_mode (qgroup->excl, unit_mode)); + len =printf ("%*s" , max_len,pretty_size_mode (qgroup->info.exclusive, unit_mode));break ;case BTRFS_QGROUP_PARENT : len =print_parent_column (qgroup);print_qgroup_column_add_blank (BTRFS_QGROUP_PARENT , len);break ;case BTRFS_QGROUP_MAX_RFER : -if (qgroup->flags &BTRFS_QGROUP_LIMIT_MAX_RFER ) - len =printf ("%*s" , max_len,pretty_size_mode (qgroup->max_rfer, unit_mode)); +if (qgroup->limit.flags &BTRFS_QGROUP_LIMIT_MAX_RFER ) + len =printf ("%*s" , max_len,pretty_size_mode (qgroup->limit.max_referenced, unit_mode));else len =printf ("%*s" , max_len,"none" );break ;case BTRFS_QGROUP_MAX_EXCL : -if (qgroup->flags &BTRFS_QGROUP_LIMIT_MAX_EXCL ) - len =printf ("%*s" , max_len,pretty_size_mode (qgroup->max_excl, unit_mode)); +if (qgroup->limit.flags &BTRFS_QGROUP_LIMIT_MAX_EXCL ) + len =printf ("%*s" , max_len,pretty_size_mode (qgroup->limit.max_exclusive, unit_mode));else len =printf ("%*s" , max_len,"none" );break ;
@@ -436,9 +428,9 @@ static int comp_entry_with_rfer(struct btrfs_qgroup *entry1, {int ret; -if (entry1->rfer > entry2->rfer) +if (entry1->info.referenced > entry2->info.referenced) ret =1 ; -else if (entry1->rfer < entry2->rfer) +else if (entry1->info.referenced < entry2->info.referenced) ret =-1 ;else ret =0 ;
@@ -452,9 +444,9 @@ static int comp_entry_with_excl(struct btrfs_qgroup *entry1, {int ret; -if (entry1->excl > entry2->excl) +if (entry1->info.exclusive > entry2->info.exclusive) ret =1 ; -else if (entry1->excl < entry2->excl) +else if (entry1->info.exclusive < entry2->info.exclusive) ret =-1 ;else ret =0 ;
@@ -468,9 +460,9 @@ static int comp_entry_with_max_rfer(struct btrfs_qgroup *entry1, {int ret; -if (entry1->max_rfer > entry2->max_rfer) +if (entry1->limit.max_referenced > entry2->limit.max_referenced) ret =1 ; -else if (entry1->max_rfer < entry2->max_rfer) +else if (entry1->limit.max_referenced < entry2->limit.max_referenced) ret =-1 ;else ret =0 ;
@@ -484,9 +476,9 @@ static int comp_entry_with_max_excl(struct btrfs_qgroup *entry1, {int ret; -if (entry1->max_excl > entry2->max_excl) +if (entry1->limit.max_exclusive > entry2->limit.max_exclusive) ret =1 ; -else if (entry1->max_excl < entry2->max_excl) +else if (entry1->limit.max_exclusive < entry2->limit.max_exclusive) ret =-1 ;else ret =0 ;
@@ -743,11 +735,13 @@ static int update_qgroup_info(int fd, struct qgroup_lookup *qgroup_lookup,if (IS_ERR_OR_NULL (bq))return PTR_ERR (bq); - bq->generation =btrfs_stack_qgroup_info_generation (info); - bq->rfer =btrfs_stack_qgroup_info_referenced (info); - bq->rfer_cmpr =btrfs_stack_qgroup_info_referenced_compressed (info); - bq->excl =btrfs_stack_qgroup_info_exclusive (info); - bq->excl_cmpr =btrfs_stack_qgroup_info_exclusive_compressed (info); + bq->info.generation =btrfs_stack_qgroup_info_generation (info); + bq->info.referenced =btrfs_stack_qgroup_info_referenced (info); + bq->info.referenced_compressed = +btrfs_stack_qgroup_info_referenced_compressed (info); + bq->info.exclusive =btrfs_stack_qgroup_info_exclusive (info); + bq->info.exclusive_compressed = +btrfs_stack_qgroup_info_exclusive_compressed (info);return 0 ; }
@@ -762,11 +756,14 @@ static int update_qgroup_limit(int fd, struct qgroup_lookup *qgroup_lookup,if (IS_ERR_OR_NULL (bq))return PTR_ERR (bq); - bq->flags =btrfs_stack_qgroup_limit_flags (limit); - bq->max_rfer =btrfs_stack_qgroup_limit_max_referenced (limit); - bq->max_excl =btrfs_stack_qgroup_limit_max_exclusive (limit); - bq->rsv_rfer =btrfs_stack_qgroup_limit_rsv_referenced (limit); - bq->rsv_excl =btrfs_stack_qgroup_limit_rsv_exclusive (limit); + bq->limit.flags =btrfs_stack_qgroup_limit_flags (limit); + bq->limit.max_referenced = +btrfs_stack_qgroup_limit_max_referenced (limit); + bq->limit.max_exclusive = +btrfs_stack_qgroup_limit_max_exclusive (limit); + bq->limit.rsv_referenced = +btrfs_stack_qgroup_limit_rsv_referenced (limit); + bq->limit.rsv_exclusive =btrfs_stack_qgroup_limit_rsv_exclusive (limit);return 0 ; }
@@ -1057,22 +1054,22 @@ static void __update_columns_max_len(struct btrfs_qgroup *bq, btrfs_qgroup_columns[column].max_len = len;break ;case BTRFS_QGROUP_RFER : - len =strlen (pretty_size_mode (bq->rfer, unit_mode)); + len =strlen (pretty_size_mode (bq->info.referenced, unit_mode));if (btrfs_qgroup_columns[column].max_len < len) btrfs_qgroup_columns[column].max_len = len;break ;case BTRFS_QGROUP_EXCL : - len =strlen (pretty_size_mode (bq->excl, unit_mode)); + len =strlen (pretty_size_mode (bq->info.exclusive, unit_mode));if (btrfs_qgroup_columns[column].max_len < len) btrfs_qgroup_columns[column].max_len = len;break ;case BTRFS_QGROUP_MAX_RFER : - len =strlen (pretty_size_mode (bq->max_rfer, unit_mode)); + len =strlen (pretty_size_mode (bq->limit.max_referenced, unit_mode));if (btrfs_qgroup_columns[column].max_len < len) btrfs_qgroup_columns[column].max_len = len;break ;case BTRFS_QGROUP_MAX_EXCL : - len =strlen (pretty_size_mode (bq->max_excl, unit_mode)); + len =strlen (pretty_size_mode (bq->limit.max_exclusive, unit_mode));if (btrfs_qgroup_columns[column].max_len < len) btrfs_qgroup_columns[column].max_len = len;break ;
@@ -1367,27 +1364,27 @@ static bool export_one_qgroup(json_object *container, goto failure;json_object_object_add (obj,"qgroupid_raw" , tmp); - tmp =export_one_u64 (qgroup->generation, compat); + tmp =export_one_u64 (qgroup->info.generation, compat);if (!tmp) goto failure;json_object_object_add (obj,"generation" , tmp); - tmp =export_one_u64 (qgroup->rfer, compat); + tmp =export_one_u64 (qgroup->info.referenced, compat);if (!tmp) goto failure;json_object_object_add (obj,"referenced_bytes" , tmp); - tmp =export_one_u64 (qgroup->excl, compat); + tmp =export_one_u64 (qgroup->info.exclusive, compat);if (!tmp) goto failure;json_object_object_add (obj,"exclusive_bytes" , tmp); - tmp =export_one_u64 (qgroup->max_rfer, compat); + tmp =export_one_u64 (qgroup->limit.max_referenced, compat);if (!tmp) goto failure;json_object_object_add (obj,"referenced_limit_bytes" , tmp); - tmp =export_one_u64 (qgroup->max_excl, compat); + tmp =export_one_u64 (qgroup->limit.max_exclusive, compat);if (!tmp) goto failure;json_object_object_add (obj,"exclusive_limit_bytes" , tmp);
qgroup.h +8 -0
@@ -79,6 +79,14 @@ enum btrfs_qgroup_filter_enum { BTRFS_QGROUP_FILTER_MAX, }; +struct btrfs_qgroup_info { +u64 generation; +u64 referenced; +u64 referenced_compressed; +u64 exclusive; +u64 exclusive_compressed; +}; +int btrfs_qgroup_parse_sort_string (const char *opt_arg,struct btrfs_qgroup_comparer_set **comps); int btrfs_show_qgroups(int fd, struct btrfs_qgroup_filter_set *,
Signature · 1 line
2.15.1
PATCH 6/8 btrfs-progs: qgroups: introduce btrfs_qgroup_query
In reply to [email protected]From: Jeff Mahoney <[email protected]>
The only mechanism we have in the progs for searching qgroups is to load all of them and filter the results. This works for qgroup show but to add quota information to 'btrfs subvoluem show' it's pretty wasteful.
This patch splits out setting up the search and performing the search so we can search for a single qgroupid more easily.
--- qgroup.c | 98 +++++++++++++++++++++++++++++++++++++++++++++------------------- qgroup.h | 7 +++++ 2 files changed, 77 insertions(+), 28 deletions(-)
qgroup.c +70 -28
@@ -1146,11 +1146,11 @@ static inline void print_status_flag_warning(u64 flags)warning ("qgroup data inconsistent, rescan recommended" ); } -static int __qgroups_search (int fd,struct qgroup_lookup *qgroup_lookup) +static int __qgroups_search (int fd,struct btrfs_ioctl_search_args *args, +struct qgroup_lookup *qgroup_lookup) {int ret; -struct btrfs_ioctl_search_args args; -struct btrfs_ioctl_search_key *sk = &args.key; +struct btrfs_ioctl_search_key *sk = &args->key;struct btrfs_ioctl_search_header *sh;unsigned long off =0 ;unsigned int i;
@@ -1161,30 +1161,12 @@ static int __qgroups_search(int fd, struct qgroup_lookup *qgroup_lookup)u64 qgroupid;u64 qgroupid1; -memset (&args,0 ,sizeof (args)); - - sk->tree_id =BTRFS_QUOTA_TREE_OBJECTID ; - sk->max_type =BTRFS_QGROUP_RELATION_KEY ; - sk->min_type =BTRFS_QGROUP_STATUS_KEY ; - sk->max_objectid = (u64)-1 ; - sk->max_offset = (u64)-1 ; - sk->max_transid = (u64)-1 ; - sk->nr_items =4096 ; -qgroup_lookup_init (qgroup_lookup);while (1 ) { - ret =ioctl (fd,BTRFS_IOC_TREE_SEARCH , &args); + ret =ioctl (fd,BTRFS_IOC_TREE_SEARCH , args);if (ret <0 ) { -if (errno ==ENOENT ) { -error ("can't list qgroups: quotas not enabled" ); - ret = -ENOTTY ; - }else { -error ("can't list qgroups: %s" , -strerror (errno)); - ret = -errno; - } - + ret = -errno;break ; }
@@ -1198,14 +1180,14 @@ static int __qgroups_search(int fd, struct qgroup_lookup *qgroup_lookup)* read the root_ref item it contains */ for (i =0 ; i < sk->nr_items; i++) { - sh = (struct btrfs_ioctl_search_header *)(args.buf + + sh = (struct btrfs_ioctl_search_header *)(args->buf + off); off +=sizeof (*sh);switch (btrfs_search_header_type (sh)) {case BTRFS_QGROUP_STATUS_KEY : si = (struct btrfs_qgroup_status_item *) - (args.buf + off); + (args->buf + off); flags =btrfs_stack_qgroup_status_flags (si);print_status_flag_warning (flags);
@@ -1213,7 +1195,7 @@ static int __qgroups_search(int fd, struct qgroup_lookup *qgroup_lookup)case BTRFS_QGROUP_INFO_KEY : qgroupid =btrfs_search_header_offset (sh); info = (struct btrfs_qgroup_info_item *) - (args.buf + off); + (args->buf + off); ret =update_qgroup_info (fd, qgroup_lookup, qgroupid, info);
@@ -1221,7 +1203,7 @@ static int __qgroups_search(int fd, struct qgroup_lookup *qgroup_lookup)case BTRFS_QGROUP_LIMIT_KEY : qgroupid =btrfs_search_header_offset (sh); limit = (struct btrfs_qgroup_limit_item *) - (args.buf + off); + (args->buf + off); ret =update_qgroup_limit (fd, qgroup_lookup, qgroupid, limit);
@@ -1267,6 +1249,66 @@ static int __qgroups_search(int fd, struct qgroup_lookup *qgroup_lookup)return ret; } +static int qgroups_search_all (int fd,struct qgroup_lookup *qgroup_lookup) +{ +struct btrfs_ioctl_search_args args = { + .key = { + .tree_id =BTRFS_QUOTA_TREE_OBJECTID , + .max_type =BTRFS_QGROUP_RELATION_KEY , + .min_type =BTRFS_QGROUP_STATUS_KEY , + .max_objectid = (u64)-1 , + .max_offset = (u64)-1 , + .max_transid = (u64)-1 , + .nr_items =4096 , + }, + }; +int ret; + + ret =__qgroups_search (fd, &args, qgroup_lookup); +if (ret == -ENOTTY ) +error ("can't list qgroups: quotas not enabled" ); +else if (ret <0 ) +error ("can't list qgroups: %s" ,strerror (-ret)); +return ret; +} + +int btrfs_qgroup_query (int fd,u64 qgroupid,struct btrfs_qgroup_stats *stats) +{ +struct btrfs_ioctl_search_args args = { + .key = { + .tree_id =BTRFS_QUOTA_TREE_OBJECTID , + .min_type =BTRFS_QGROUP_INFO_KEY , + .max_type =BTRFS_QGROUP_LIMIT_KEY , + .max_objectid =0 , + .max_offset = qgroupid, + .max_transid = (u64)-1 , + .nr_items =4096 ,/* should be 2, i think */ + }, + }; +struct qgroup_lookup qgroup_lookup; +struct btrfs_qgroup *qgroup; +struct rb_node *n; +int ret; + + ret =__qgroups_search (fd, &args, &qgroup_lookup); +if (ret <0 ) +return ret; + + ret = -ENODATA ; + n =rb_first (&qgroup_lookup.root); +if (n) { + qgroup = rb_entry(n, struct btrfs_qgroup, rb_node); + stats->qgroupid = qgroup->qgroupid; + stats->info = qgroup->info; + stats->limit = qgroup->limit; + + ret =0 ; + } + +__free_all_qgroups (&qgroup_lookup); +return ret; +} +static void print_all_qgroups (struct qgroup_lookup *qgroup_lookup,bool verbose) {
@@ -1293,7 +1335,7 @@ int btrfs_show_qgroups(int fd,struct qgroup_lookup sort_tree;int ret; - ret =__qgroups_search (fd, &qgroup_lookup); + ret =qgroups_search_all (fd, &qgroup_lookup);if (ret)return ret; __filter_and_sort_qgroups(&qgroup_lookup, &sort_tree,
qgroup.h +7 -0
@@ -87,6 +87,12 @@ struct btrfs_qgroup_info {u64 exclusive_compressed; }; +struct btrfs_qgroup_stats { +u64 qgroupid; +struct btrfs_qgroup_info info; +struct btrfs_qgroup_limit limit; +}; +int btrfs_qgroup_parse_sort_string (const char *opt_arg,struct btrfs_qgroup_comparer_set **comps); int btrfs_show_qgroups(int fd, struct btrfs_qgroup_filter_set *,
@@ -105,4 +111,5 @@ int qgroup_inherit_add_group(struct btrfs_qgroup_inherit **inherit, char *arg);int qgroup_inherit_add_copy (struct btrfs_qgroup_inherit **inherit,char *arg,int type); +int btrfs_qgroup_query (int fd,u64 qgroupid,struct btrfs_qgroup_stats *stats);#endif
Signature · 1 line
2.15.1
PATCH 7/8 btrfs-progs: subvolume: add quota info to btrfs sub show
In reply to [email protected]From: Jeff Mahoney <[email protected]>
This patch reports on the first-level qgroup, if any, associated with a particular subvolume. It displays the usage and limit, subject to the usual unit parameters.
--- cmds-subvolume.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+)
cmds-subvolume.c +46 -0
@@ -972,6 +972,7 @@ static const char * const cmd_subvol_show_usage[] = {"Show more information about the subvolume" ,"-r|--rootid rootid of the subvolume" ,"-u|--uuid uuid of the subvolume" , +HELPINFO_UNITS_SHORT_LONG ,"" ,"If no option is specified, <subvol-path> will be shown, otherwise" , "the rootid or uuid are resolved relative to the <mnt> path.",
@@ -993,6 +994,13 @@ static int cmd_subvol_show(int argc, char **argv)int by_uuid =0 ;u64 rootid_arg;u8 uuid_arg[BTRFS_UUID_SIZE ]; +struct btrfs_qgroup_stats stats; +unsigned int unit_mode; +const char *referenced_size; +const char *referenced_limit_size ="-" ; +unsigned field_width =0 ; + + unit_mode =get_unit_mode_from_arg (&argc, argv,1 );while (1 ) {int c;
@@ -1112,6 +1120,44 @@ static int cmd_subvol_show(int argc, char **argv)btrfs_list_subvols_print (fd, filter_set,NULL ,BTRFS_LIST_LAYOUT_RAW ,1 , raw_prefix); + ret =btrfs_qgroup_query (fd, get_ri.root_id, &stats); +if (ret <0 ) { +if (ret == -ENODATA ) +printf ("Quotas must be enabled for per-subvolume usage\n" ); +else if (ret != -ENOTTY ) +fprintf (stderr, +"\nERROR: BTRFS_IOC_QUOTA_QUERY failed: %s\n" , +strerror (errno)); + goto out; + } + +printf ("\tQuota Usage:\t\t" ); +fflush (stdout); + + referenced_size =pretty_size_mode (stats.info.referenced, unit_mode); +if (stats.limit.max_referenced) + referenced_limit_size =pretty_size_mode ( + stats.limit.max_referenced, + unit_mode); + field_width =max (strlen (referenced_size), +strlen (referenced_limit_size)); + +printf ("%-*s referenced, %s exclusive\n " , field_width, + referenced_size, +pretty_size_mode (stats.info.exclusive, unit_mode)); + +printf ("\tQuota Limits:\t\t" ); +if (stats.limit.max_referenced || stats.limit.max_exclusive) { +const char *excl ="-" ; + +if (stats.limit.max_exclusive) + excl =pretty_size_mode (stats.limit.max_exclusive, + unit_mode); +printf ("%-*s referenced, %s exclusive\n" , field_width, + referenced_limit_size, excl); + }else +printf ("None\n" ); + out:/* clean up */ free (get_ri.path);
Signature · 1 line
2.15.1
PATCH 7/8 btrfs-progs: qgroups: introduce btrfs_qgroup_query
In reply to [email protected]From: Jeff Mahoney <[email protected]>
The only mechanism we have in the progs for searching qgroups is to load all of them and filter the results. This works for qgroup show but to add quota information to 'btrfs subvoluem show' it's pretty wasteful.
This patch splits out setting up the search and performing the search so we can search for a single qgroupid more easily.
--- qgroup.c | 100 +++++++++++++++++++++++++++++++++++++++++++++------------------ qgroup.h | 7 +++++ 2 files changed, 78 insertions(+), 29 deletions(-)
qgroup.c +71 -29
@@ -1150,11 +1150,11 @@ static inline void print_status_flag_warning(u64 flags)warning ("qgroup data inconsistent, rescan recommended" ); } -static int __qgroups_search (int fd,struct qgroup_lookup *qgroup_lookup) +static int __qgroups_search (int fd,struct btrfs_ioctl_search_args *args, +struct qgroup_lookup *qgroup_lookup) {int ret; -struct btrfs_ioctl_search_args args; -struct btrfs_ioctl_search_key *sk = &args.key; +struct btrfs_ioctl_search_key *sk = &args->key;struct btrfs_ioctl_search_header *sh;unsigned long off =0 ;unsigned int i;
@@ -1165,30 +1165,12 @@ static int __qgroups_search(int fd, struct qgroup_lookup *qgroup_lookup)u64 qgroupid;u64 qgroupid1; -memset (&args,0 ,sizeof (args)); - - sk->tree_id =BTRFS_QUOTA_TREE_OBJECTID ; - sk->max_type =BTRFS_QGROUP_RELATION_KEY ; - sk->min_type =BTRFS_QGROUP_STATUS_KEY ; - sk->max_objectid = (u64)-1 ; - sk->max_offset = (u64)-1 ; - sk->max_transid = (u64)-1 ; - sk->nr_items =4096 ; -qgroup_lookup_init (qgroup_lookup);while (1 ) { - ret =ioctl (fd,BTRFS_IOC_TREE_SEARCH , &args); + ret =ioctl (fd,BTRFS_IOC_TREE_SEARCH , args);if (ret <0 ) { -if (errno ==ENOENT ) { -error ("can't list qgroups: quotas not enabled" ); - ret = -ENOTTY ; - }else { -error ("can't list qgroups: %s" , -strerror (errno)); - ret = -errno; - } - + ret = -errno;break ; }
@@ -1202,14 +1184,14 @@ static int __qgroups_search(int fd, struct qgroup_lookup *qgroup_lookup)* read the root_ref item it contains */ for (i =0 ; i < sk->nr_items; i++) { - sh = (struct btrfs_ioctl_search_header *)(args.buf + + sh = (struct btrfs_ioctl_search_header *)(args->buf + off); off +=sizeof (*sh);switch (btrfs_search_header_type (sh)) {case BTRFS_QGROUP_STATUS_KEY : si = (struct btrfs_qgroup_status_item *) - (args.buf + off); + (args->buf + off); flags =btrfs_stack_qgroup_status_flags (si);print_status_flag_warning (flags);
@@ -1217,7 +1199,7 @@ static int __qgroups_search(int fd, struct qgroup_lookup *qgroup_lookup)case BTRFS_QGROUP_INFO_KEY : qgroupid =btrfs_search_header_offset (sh); info = (struct btrfs_qgroup_info_item *) - (args.buf + off); + (args->buf + off); ret =update_qgroup_info (fd, qgroup_lookup, qgroupid, info);
@@ -1225,7 +1207,7 @@ static int __qgroups_search(int fd, struct qgroup_lookup *qgroup_lookup)case BTRFS_QGROUP_LIMIT_KEY : qgroupid =btrfs_search_header_offset (sh); limit = (struct btrfs_qgroup_limit_item *) - (args.buf + off); + (args->buf + off); ret =update_qgroup_limit (fd, qgroup_lookup, qgroupid, limit);
@@ -1271,6 +1253,66 @@ static int __qgroups_search(int fd, struct qgroup_lookup *qgroup_lookup)return ret; } +static int qgroups_search_all (int fd,struct qgroup_lookup *qgroup_lookup) +{ +struct btrfs_ioctl_search_args args = { + .key = { + .tree_id =BTRFS_QUOTA_TREE_OBJECTID , + .max_type =BTRFS_QGROUP_RELATION_KEY , + .min_type =BTRFS_QGROUP_STATUS_KEY , + .max_objectid = (u64)-1 , + .max_offset = (u64)-1 , + .max_transid = (u64)-1 , + .nr_items =4096 , + }, + }; +int ret; + + ret =__qgroups_search (fd, &args, qgroup_lookup); +if (ret == -ENOTTY ) +error ("can't list qgroups: quotas not enabled" ); +else if (ret <0 ) +error ("can't list qgroups: %s" ,strerror (-ret)); +return ret; +} + +int btrfs_qgroup_query (int fd,u64 qgroupid,struct btrfs_qgroup_stats *stats) +{ +struct btrfs_ioctl_search_args args = { + .key = { + .tree_id =BTRFS_QUOTA_TREE_OBJECTID , + .min_type =BTRFS_QGROUP_INFO_KEY , + .max_type =BTRFS_QGROUP_LIMIT_KEY , + .max_objectid =0 , + .max_offset = qgroupid, + .max_transid = (u64)-1 , + .nr_items =4096 ,/* should be 2, i think */ + }, + }; +struct qgroup_lookup qgroup_lookup; +struct btrfs_qgroup *qgroup; +struct rb_node *n; +int ret; + + ret =__qgroups_search (fd, &args, &qgroup_lookup); +if (ret <0 ) +return ret; + + ret = -ENODATA ; + n =rb_first (&qgroup_lookup.root); +if (n) { + qgroup = rb_entry(n, struct btrfs_qgroup, rb_node); + stats->qgroupid = qgroup->qgroupid; + stats->info = qgroup->info; + stats->limit = qgroup->limit; + + ret =0 ; + } + +__free_all_qgroups (&qgroup_lookup); +return ret; +} +static void print_all_qgroups (struct qgroup_lookup *qgroup_lookup,bool verbose) {
@@ -1297,7 +1339,7 @@ int btrfs_show_qgroups(int fd,struct qgroup_lookup sort_tree;int ret; - ret =__qgroups_search (fd, &qgroup_lookup); + ret =qgroups_search_all (fd, &qgroup_lookup);if (ret)return ret; __filter_and_sort_qgroups(&qgroup_lookup, &sort_tree,
@@ -1500,7 +1542,7 @@ int btrfs_export_qgroups_json(int fd,struct qgroup_lookup sort_tree;int ret =0 ; - ret =__qgroups_search (fd, &qgroup_lookup); + ret =qgroups_search_all (fd, &qgroup_lookup);if (ret)return ret; __filter_and_sort_qgroups(&qgroup_lookup, &sort_tree,
qgroup.h +7 -0
@@ -87,6 +87,12 @@ struct btrfs_qgroup_info {u64 exclusive_compressed; }; +struct btrfs_qgroup_stats { +u64 qgroupid; +struct btrfs_qgroup_info info; +struct btrfs_qgroup_limit limit; +}; +int btrfs_qgroup_parse_sort_string (const char *opt_arg,struct btrfs_qgroup_comparer_set **comps); int btrfs_show_qgroups(int fd, struct btrfs_qgroup_filter_set *,
@@ -108,4 +114,5 @@ int qgroup_inherit_add_group(struct btrfs_qgroup_inherit **inherit, char *arg);int qgroup_inherit_add_copy (struct btrfs_qgroup_inherit **inherit,char *arg,int type); +int btrfs_qgroup_query (int fd,u64 qgroupid,struct btrfs_qgroup_stats *stats);#endif
Signature · 1 line
2.15.1
PATCH 8/8 btrfs-progs: add quota info to btrfs sub show
In reply to [email protected]From: Jeff Mahoney <[email protected]>
This patch reports on the first-level qgroup, if any, associated with a particular subvolume. It displays the usage and limit, subject to the usual unit parameters.
--- cmds-subvolume.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+)
cmds-subvolume.c +46 -0
@@ -972,6 +972,7 @@ static const char * const cmd_subvol_show_usage[] = {"Show more information about the subvolume" ,"-r|--rootid rootid of the subvolume" ,"-u|--uuid uuid of the subvolume" , +HELPINFO_UNITS_SHORT_LONG ,"" ,"If no option is specified, <subvol-path> will be shown, otherwise" , "the rootid or uuid are resolved relative to the <mnt> path.",
@@ -993,6 +994,13 @@ static int cmd_subvol_show(int argc, char **argv)int by_uuid =0 ;u64 rootid_arg;u8 uuid_arg[BTRFS_UUID_SIZE ]; +struct btrfs_qgroup_stats stats; +unsigned int unit_mode; +const char *referenced_size; +const char *referenced_limit_size ="-" ; +unsigned field_width =0 ; + + unit_mode =get_unit_mode_from_arg (&argc, argv,1 );while (1 ) {int c;
@@ -1112,6 +1120,44 @@ static int cmd_subvol_show(int argc, char **argv)btrfs_list_subvols_print (fd, filter_set,NULL ,BTRFS_LIST_LAYOUT_RAW ,1 , raw_prefix); + ret =btrfs_qgroup_query (fd, get_ri.root_id, &stats); +if (ret <0 ) { +if (ret == -ENODATA ) +printf ("Quotas must be enabled for per-subvolume usage\n" ); +else if (ret != -ENOTTY ) +fprintf (stderr, +"\nERROR: BTRFS_IOC_QUOTA_QUERY failed: %s\n" , +strerror (errno)); + goto out; + } + +printf ("\tQuota Usage:\t\t" ); +fflush (stdout); + + referenced_size =pretty_size_mode (stats.info.referenced, unit_mode); +if (stats.limit.max_referenced) + referenced_limit_size =pretty_size_mode ( + stats.limit.max_referenced, + unit_mode); + field_width =max (strlen (referenced_size), +strlen (referenced_limit_size)); + +printf ("%-*s referenced, %s exclusive\n " , field_width, + referenced_size, +pretty_size_mode (stats.info.exclusive, unit_mode)); + +printf ("\tQuota Limits:\t\t" ); +if (stats.limit.max_referenced || stats.limit.max_exclusive) { +const char *excl ="-" ; + +if (stats.limit.max_exclusive) + excl =pretty_size_mode (stats.limit.max_exclusive, + unit_mode); +printf ("%-*s referenced, %s exclusive\n" , field_width, + referenced_limit_size, excl); + }else +printf ("None\n" ); + out:/* clean up */ free (get_ri.path);
Signature · 1 line
2.15.1
PATCH 8/8 btrfs-progs: qgroups: export qgroups usage information as JSON
In reply to [email protected]From: Jeff Mahoney <[email protected]>
One of the common requests I receive is for 'df' like facilities for subvolume usage. Really, the request is for monitoring tools to be able to understand when subvolumes may be approaching quota in the same manner traditional file systems approach ENOSPC.
This patch allows us to export the qgroups data in a machine-readable format so that monitoring tools can parse it easily.
There are two modes since JSON can technically handle 64-bit numbers but JavaScript proper cannot. show -j enables JSON mode using 64-bit integers directly. --json-compat presents 64-bit numbers as an array of two 32-bit numbers (high followed by low).
--- Documentation/btrfs-qgroup.asciidoc | 4 + Makefile.inc.in | 4 +- cmds-qgroup.c | 36 +++++- configure.ac | 6 + qgroup.c | 211 ++++++++++++++++++++++++++++++++++++ qgroup.h | 3 + 6 files changed, 258 insertions(+), 6 deletions(-)
Documentation/btrfs-qgroup.asciidoc +4 -0
@@ -105,6 +105,10 @@ list all qgroups which impact the given path(include ancestral qgroups) list all qgroups which impact the given path(exclude ancestral qgroups) -v:::: Be more verbose. Print pathnames of member qgroups when nested. +-j:::: +If enabled, export qgroup usage information in JSON format. This implies --raw. +--json-compat:::: +By default, JSON output contains full 64-bit integers, which may be incompatible with some JSON parsers. This option exports those values as an array of 32-bit numbers in [high, low] format. --raw:::: raw numbers in bytes, without the 'B' suffix. --human-readable::::
Makefile.inc.in +2 -2
@@ -18,9 +18,9 @@ BTRFSRESTORE_ZSTD = @BTRFSRESTORE_ZSTD@ SUBST_CFLAGS = @CFLAGS@ SUBST_LDFLAGS = @LDFLAGS@ -LIBS_BASE = @UUID_LIBS@ @BLKID_LIBS@ -L. -pthread +LIBS_BASE = @UUID_LIBS@ @BLKID_LIBS@ @JSON_LIBS@ -L. -pthread LIBS_COMP = @ZLIB_LIBS@ @LZO2_LIBS@ @ZSTD_LIBS@ -STATIC_LIBS_BASE = @UUID_LIBS_STATIC@ @BLKID_LIBS_STATIC@ -L. -pthread +STATIC_LIBS_BASE = @UUID_LIBS_STATIC@ @BLKID_LIBS_STATIC@ @JSON_LIBS_STATIC@ -L. -pthread STATIC_LIBS_COMP = @ZLIB_LIBS_STATIC@ @LZO2_LIBS_STATIC@ @ZSTD_LIBS_STATIC@ prefix ?= @prefix@
cmds-qgroup.c +32 -4
@@ -282,6 +282,10 @@ static const char * const cmd_qgroup_show_usage[] = {" (excluding ancestral qgroups)" ,"-P print first-level qgroups using pathname" ,"-v verbose, prints all nested subvolumes" , +#ifdef HAVE_JSON +"-j export in JSON format" , +"--json-compat export in JSON compatibility mode" , +#endifHELPINFO_UNITS_LONG ,"--sort=qgroupid,rfer,excl,max_rfer,max_excl,pathname" , " list qgroups sorted by specified items",
@@ -302,6 +306,8 @@ static int cmd_qgroup_show(int argc, char **argv)unsigned unit_mode;int sync =0 ;bool verbose = false; +bool export_json = false; +bool compat_json = false;struct btrfs_qgroup_comparer_set *comparer_set;struct btrfs_qgroup_filter_set *filter_set;
@@ -314,16 +320,26 @@ static int cmd_qgroup_show(int argc, char **argv)int c;enum {GETOPT_VAL_SORT =256 , -GETOPT_VAL_SYNC +GETOPT_VAL_SYNC , +GETOPT_VAL_JSCOMPAT , };static const struct option long_options[] = { {"sort" , required_argument,NULL ,GETOPT_VAL_SORT }, {"sync" , no_argument,NULL ,GETOPT_VAL_SYNC }, {"verbose" , no_argument,NULL ,'v' }, +#ifdef HAVE_JSON + {"json-compat" , no_argument,NULL ,GETOPT_VAL_JSCOMPAT }, +#endif {NULL ,0 ,NULL ,0 } }; - - c =getopt_long (argc, argv,"pPcreFfv" , long_options,NULL ); +const char getopt_chars[] = { +'p' ,'P' ,'c' ,'r' ,'e' ,'F' ,'f' ,'v' , +#ifdef HAVE_JSON +'j' , +#endif +'\0' }; + + c =getopt_long (argc, argv, getopt_chars, long_options,NULL );if (c <0 )break ;switch (c) {
@@ -353,6 +369,14 @@ static int cmd_qgroup_show(int argc, char **argv)case 'f' : filter_flag |=0x2 ;break ; +#ifdef HAVE_JSON +case GETOPT_VAL_JSCOMPAT : + compat_json = true; +case 'j' : + unit_mode =UNITS_RAW ; + export_json = true; +break ; +#endif case GETOPT_VAL_SORT : ret =btrfs_qgroup_parse_sort_string (optarg, &comparer_set);
@@ -405,7 +429,11 @@ static int cmd_qgroup_show(int argc, char **argv)BTRFS_QGROUP_FILTER_PARENT , qgroupid); } - ret =btrfs_show_qgroups (fd, filter_set, comparer_set, verbose); +if (export_json) + ret =btrfs_export_qgroups_json (fd, filter_set, comparer_set, + compat_json); +else + ret =btrfs_show_qgroups (fd, filter_set, comparer_set, verbose);close_file_or_dir (fd, dirstream);free (filter_set);free (comparer_set);
configure.ac +6 -0
@@ -197,6 +197,12 @@ PKG_STATIC(UUID_LIBS_STATIC, [uuid]) PKG_CHECK_MODULES(ZLIB, [zlib]) PKG_STATIC(ZLIB_LIBS_STATIC, [zlib]) +PKG_CHECK_MODULES(JSON, [json-c], [ + AC_DEFINE(HAVE_JSON, [1], [Have JSON]), + PKG_STATIC(JSON_LIBS_STATIC, [json-c], [ + AC_DEFINE(HAVE_JSON_STATIC, [1], [Have JSON static])], [true]) + ], [true]) + AC_ARG_ENABLE([zstd], AS_HELP_STRING([--disable-zstd], [build without zstd support]), [], [enable_zstd=yes]
qgroup.c +211 -0
229 lines not on this page · raw message
qgroup.h +3 -0
@@ -97,6 +97,9 @@ int btrfs_qgroup_parse_sort_string(const char *opt_arg, struct btrfs_qgroup_comparer_set **comps);int btrfs_show_qgroups (int fd,struct btrfs_qgroup_filter_set *,struct btrfs_qgroup_comparer_set *,bool verbose); +int btrfs_export_qgroups_json (int fd,struct btrfs_qgroup_filter_set *, +struct btrfs_qgroup_comparer_set *, +bool compat);void btrfs_qgroup_setup_print_column (enum btrfs_qgroup_column_enum column);void btrfs_qgroup_setup_units (unsigned unit_mode);struct btrfs_qgroup_filter_set *btrfs_qgroup_alloc_filter_set (void );
Signature · 1 line
2.15.1
Quoted 4 lines · On 3/2/18 1:39 PM, [email protected] wrote:
From: Jeff Mahoney <[email protected]>
Hi all -
Quoted 8 lines
The following series addresses some usability issues with the qgroups UI.
1) Adds -W option so we can wait on a rescan completing without starting one.
2) Adds qgroup information to 'btrfs subvolume show'
3) Adds a -P option to show pathnames for first-level qgroups (or member of nested qgroups with -v)
4) Allows exporting the qgroup table in JSON format for use by external programs/scripts.
Grumble. Ignore this thread. I had reordered the patches and didn't clean up an older git format-patch.
-Jeff
Quoted 32 lines
-Jeff
Jeff Mahoney (8):
btrfs-progs: quota: Add -W option to rescan to wait without starting
rescan
btrfs-progs: qgroups: fix misleading index check
btrfs-progs: constify pathnames passed as arguments
btrfs-progs: qgroups: add pathname to show output
btrfs-progs: qgroups: introduce and use info and limit structures
btrfs-progs: qgroups: introducebtrfs_qgroup_query
btrfs-progs: subvolume: add quota info to btrfs sub show
btrfs-progs: qgroups: export qgroups usage information as JSONDocumentation/btrfs-qgroup.asciidoc | 8 + Documentation/btrfs-quota.asciidoc | 10 +- Makefile.inc.in | 4 +- chunk-recover.c | 4 +- cmds-device.c | 2 +- cmds-fi-usage.c | 6 +- cmds-qgroup.c | 49 +++- cmds-quota.c | 21 +- cmds-rescue.c | 4 +- cmds-subvolume.c | 46 ++++ configure.ac | 6 + kerncompat.h | 1 + qgroup.c | 526 ++++++++++++++++++++++++++++++------ qgroup.h | 22 +- send-utils.c | 4 +- utils.c | 22 +- utils.h | 2 + 17 files changed, 621 insertions(+), 116 deletions(-)
Signature · 2 lines
Jeff Mahoney SUSE Labs