xfsprogs-6.10: filesystem properties
Hi all,
After recent discussions about how to allow sysadmins to opt in or out of automatick fsck for XFS[1][2], I now have a patchset for xfsprogs 6.10 that implements the filesystem properties that we talked about.
As a refresher, the design I settled on is to add ATTR_ROOT (aka "trusted") xattrs to the root directory. ATTR_ROOT xattrs can only be accessed by processes with CAP_SYS_ADMIN, so unprivileged userspace can't mess with the sysadmin's configured preferences.
I decided that all fs properties should have "xfs:" in the name to make them look distinct, and defined "trusted.xfs:autofsck" as the property that controls the behavior of automatic fsck. There's a new wrapper program "xfs_property" that one can use to administer the properties. xfs_scrub{,bed} uses the property; and mkfs.xfs can set it for you at format time.
# mkfs.xfs -m autofsck /dev/sda
# xfs_property /dev/sda get autofsck repair
# mount /dev/sda /mnt
# xfs_property /mnt set autofsck=check
# xfs_scrub -o autofsck /mnt Info: /mnt: Checking per autofsck directive.
--D
[1] https://lore.kernel.org/linux-xfs/20240724213852.GA612460@frogsfrogsfrogs/
[2] https://lore.kernel.org/linux-xfs/20240730031030.GA6333@frogsfrogsfrogs/
PATCH 1/3 xfsprogs: filesystem properties
Hi all,
It would be very useful if system administrators could set properties for a given xfs filesystem to control its behavior. This we can do easily and extensibly by setting ATTR_ROOT (aka "trusted") extended attributes on the root directory. To prevent this from becoming a weird free for all, let's add some library and tooling support so that sysadmins simply run the xfs_property program to administer these properties.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=filesystem-properties-6.10
---
Commits in this patchset:
* libfrog: support editing filesystem property sets
* xfs_io: edit filesystem properties
* xfs_db: improve getting and setting extended attributes
* libxfs: hoist listxattr from xfs_repair
* libxfs: pass a transaction context through listxattr
* xfs_db: add a command to list xattrs
* xfs_property: add a new tool to administer fs properties
--- db/attrset.c | 463 ++++++++++++++++++++++++++++++++++++++++++++++- io/Makefile | 4 io/fsproperties.c | 365 +++++++++++++++++++++++++++++++++++++ io/init.c | 1 io/io.h | 1 io/xfs_property | 77 ++++++++ libfrog/Makefile | 7 + libfrog/fsproperties.c | 39 ++++ libfrog/fsproperties.h | 53 +++++ libfrog/fsprops.c | 202 +++++++++++++++++++++ libfrog/fsprops.h | 34 +++ libxfs/Makefile | 2 libxfs/listxattr.c | 42 ++-- libxfs/listxattr.h | 17 ++ man/man8/xfs_db.8 | 68 +++++++ man/man8/xfs_io.8 | 16 ++ man/man8/xfs_property.8 | 61 ++++++ repair/Makefile | 2 repair/listxattr.h | 15 -- repair/pptr.c | 9 + 20 files changed, 1430 insertions(+), 48 deletions(-) create mode 100644 io/fsproperties.c create mode 100755 io/xfs_property create mode 100644 libfrog/fsproperties.c create mode 100644 libfrog/fsproperties.h create mode 100644 libfrog/fsprops.c create mode 100644 libfrog/fsprops.h rename repair/listxattr.c => libxfs/listxattr.c (84%) create mode 100644 libxfs/listxattr.h create mode 100644 man/man8/xfs_property.8 delete mode 100644 repair/listxattr.h
PATCH 1/7 libfrog: support editing filesystem property sets
From: Darrick J. Wong <[email protected]>
Add some library functions so that spaceman and scrub can share the same code to edit and retrieve filesystem properties.
--- libfrog/Makefile | 7 ++ libfrog/fsproperties.c | 39 +++++++++ libfrog/fsproperties.h | 53 +++++++++++++ libfrog/fsprops.c | 202 ++++++++++++++++++++++++++++++++++++++++++++++++ libfrog/fsprops.h | 34 ++++++++ 5 files changed, 335 insertions(+) create mode 100644 libfrog/fsproperties.c create mode 100644 libfrog/fsproperties.h create mode 100644 libfrog/fsprops.c create mode 100644 libfrog/fsprops.h
libfrog/Makefile +7 -0
@@ -20,6 +20,7 @@ convert.c \ crc32.c \ file_exchange.c \ fsgeom.c \ +fsproperties.c \ getparents.c \ histogram.c \ list_sort.c \
@@ -47,6 +48,7 @@ dahashselftest.h \ div64.h \ file_exchange.h \ fsgeom.h \ +fsproperties.h \ getparents.h \ histogram.h \ logging.h \
@@ -60,6 +62,11 @@ workqueue.h LSRCFILES += gen_crc32table.c +ifeq ($(HAVE_LIBATTR),yes) +CFILES+=fsprops.c +HFILES+=fsprops.h +endif + LDIRT = gen_crc32table crc32table.h default: ltdepend $(LTLIBRARY)
libfrog/fsproperties.c new file +39 -0
@@ -0,0 +1,39 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (c) 2024 Oracle. All Rights Reserved. +* Author: Darrick J. Wong <[email protected]> +*/ +#include <string.h> +#include "xfs.h" +#include "libfrog/fsgeom.h" +#include "libfrog/fsproperties.h" +#include "list.h" + +/* Return the offset of a string in a string table, or -1 if not found. */ +static inline int +__fsprops_lookup ( +const char *values[], +unsigned int nr_values, +const char *value) +{ +unsigned int i; + +for (i =0 ; i < nr_values; i++) { +if (values[i] && !strcmp (value, values[i])) +return i; + } + +return -1 ; +} + +#define fsprops_lookup (values, value) \ + __fsprops_lookup((values), ARRAY_SIZE(values), (value)) + +/* Return true if a fs property name=value tuple is allowed. */ +bool +fsprop_validate ( +const char *name, +const char *value) +{ +return true; +}
libfrog/fsproperties.h new file +53 -0
53 lines not on this page · raw message
libfrog/fsprops.c new file +202 -0
202 lines not on this page · raw message
libfrog/fsprops.h new file +34 -0
@@ -0,0 +1,34 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* +* Copyright (c) 2024 Oracle. All Rights Reserved. +* Author: Darrick J. Wong <[email protected]> +*/ +#ifndef __LIBFROG_FSPROPS_H__ +#define __LIBFROG_FSPROPS_H__ + +/* Edit and view filesystem property sets. */ + +struct fsprops_handle { +void *hanp; +size_t hlen; +}; + +struct xfs_fd ; +struct fs_path ; + +int fsprops_open_handle (struct xfs_fd *xfd,struct fs_path *fspath, +struct fsprops_handle *fph); +void fsprops_free_handle (struct fsprops_handle *fph); + +typedef int (*fsprops_name_walk_fn )(struct fsprops_handle *fph, +const char *name,size_t valuelen,void *priv); + +int fsprops_walk_names (struct fsprops_handle *fph,fsprops_name_walk_fn walk_fn, +void *priv); +int fsprops_get (struct fsprops_handle *fph,const char *name,void *attrbuf, +size_t *attrlen); +int fsprops_set (struct fsprops_handle *fph,const char *name,void *attrbuf, +size_t attrlen); +int fsprops_remove (struct fsprops_handle *fph,const char *name); + +#endif /* __LIBFROG_FSPROPS_H__ */
PATCH 2/7 xfs_io: edit filesystem properties
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Add some new subcommands to xfs_io so that users can administer filesystem properties.
--- io/Makefile | 1 io/fsproperties.c | 365 +++++++++++++++++++++++++++++++++++++++++++++++++++++ io/init.c | 1 io/io.h | 1 man/man8/xfs_io.8 | 16 ++ 5 files changed, 383 insertions(+), 1 deletion(-) create mode 100644 io/fsproperties.c
io/Makefile +1 -0
@@ -20,6 +20,7 @@ CFILES = \ fiemap.c \ file.c \ freeze.c \ + fsproperties.c \ fsuuid.c \ fsync.c \ getrusage.c \
io/fsproperties.c new file +365 -0
365 lines not on this page · raw message
io/init.c +1 -0
@@ -89,6 +89,7 @@ init_commands(void)utimes_init ();crc32cselftest_init ();exchangerange_init (); +fsprops_init (); }/*
io/io.h +1 -0
@@ -150,3 +150,4 @@ extern void repair_init(void);extern void crc32cselftest_init (void );extern void bulkstat_init (void );void exchangerange_init (void ); +void fsprops_init (void );
man/man8/xfs_io.8 +15 -1
@@ -1578,7 +1578,21 @@ Print the sysfs or debugfs path for the mounted filesystem. The .B -d option selects debugfs instead of sysfs. - +.TP +.BI "getfsprops " name " [ " names "... ]" +Retrieve the values of the given filesystem properties. +.TP +.BI "listfsprops [ " \-v " ]" +List all filesystem properties that have been stored in the filesystem. +If the +.B \-v +flag is specified, prints the values of the properties. +.TP +.BI "setfsprops " name = value " [ " name = value "... ]" +Set the given filesystem properties to the specified values. +.TP +.BI "removefsprops " name " [ " names "... ]" +Remove the given filesystem properties. .SH OTHER COMMANDS .TP
Re: [PATCH 2/7] xfs_io: edit filesystem properties
Quoted 4 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Add some new subcommands to
xfs_ioso that users can administer filesystem properties.
s/some //?
Quoted 2 lines
Missing newline here.
io/fsproperties.c line 94
Quoted 1 line
io/fsproperties.c new file +1
-0+if (!(*print_values)) {
The inner set of braces here should not be needed.
Otherwise looks good:
PATCH 3/7 xfs_db: improve getting and setting extended attributes
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Add an attr_get command to retrieve the value of an xattr from a file; and extend the attr_set command to allow passing of string values.
--- db/attrset.c | 262 ++++++++++++++++++++++++++++++++++++++++++++++++++++- man/man8/xfs_db.8 | 40 ++++++++ 2 files changed, 293 insertions(+), 9 deletions(-)
db/attrset.c +255 -7
@@ -17,20 +17,43 @@#include "inode.h" #include "malloc.h" #include <sys/xattr.h> +#include "libfrog/fsproperties.h" +static int attr_get_f (int argc,char **argv);static int attr_set_f (int argc,char **argv);static int attr_remove_f (int argc,char **argv); +static void attrget_help (void );static void attrset_help (void ); +static const cmdinfo_t attr_get_cmd = + {"attr_get" ,"aget" , attr_get_f,1 ,-1 ,0 , +N_ ("[-r|-s|-u|-p|-Z] name" ), +N_ ("get the named attribute on the current inode" ), attrget_help };static const cmdinfo_t attr_set_cmd = {"attr_set" ,"aset" , attr_set_f,1 ,-1 ,0 , -N_ ("[-r|-s|-u|-p] [-n] [-R|-C] [-v n] name" ), +N_ ("[-r|-s|-u|-p|-Z] [-n] [-R|-C] [-v n] name" ),N_ ("set the named attribute on the current inode" ), attrset_help };static const cmdinfo_t attr_remove_cmd = {"attr_remove" ,"aremove" , attr_remove_f,1 ,-1 ,0 , -N_ ("[-r|-s|-u|-p] [-n] name" ), +N_ ("[-r|-s|-u|-p|-Z] [-n] name" ),N_ ("remove the named attribute from the current inode" ), attrset_help }; +static void +attrget_help (void ) +{ +dbprintf (_ ( +"\n" +" The attr_get command provide interfaces for retrieving the values of extended\n" +" attributes of a file. This command requires attribute names to be specified.\n" +" There are 4 namespace flags:\n" +" -r -- 'root'\n" +" -u -- 'user' (default)\n" +" -s -- 'secure'\n" +" -p -- 'parent'\n" +" -Z -- fs property\n" +"\n" )); +} +static void attrset_help (void ) {
@@ -45,10 +68,15 @@ attrset_help(void)" -u -- 'user' (default)\n" " -s -- 'secure'\n" " -p -- 'parent'\n" +" -Z -- fs property\n" "\n" " For attr_set, these options further define the type of set operation:\n" " -C -- 'create' - create attribute, fail if it already exists\n" " -R -- 'replace' - replace attribute, fail if it does not exist\n" +"\n" +" If the attribute value is a string, it can be specified after the\n" +" attribute name.\n" +"\n" " The backward compatibility mode 'noattr2' can be emulated (-n) also.\n" "\n")); }
@@ -59,6 +87,7 @@ attrset_init(void)if (!expert_mode)return ; +add_command (&attr_get_cmd);add_command (&attr_set_cmd);add_command (&attr_remove_cmd); }
@@ -106,6 +135,58 @@ get_buf_from_file( LIBXFS_ATTR_ROOT | \ LIBXFS_ATTR_PARENT) +static bool +adjust_fsprop_attr_name ( +struct xfs_da_args *args, +bool *free_name) +{ +const char *o = (const char *)args->name; +char *p; +int ret; + +if ((args->attr_filter &LIBXFS_ATTR_NS ) !=LIBXFS_ATTR_ROOT ) { +dbprintf (_ ("fs properties must be ATTR_ROOT\n" )); +return false; + } + + ret =fsprop_name_to_attr_name (o, &p); +if (ret <0 ) { +dbprintf (_ ("could not allocate fs property name string\n" )); +return false; + } + args->namelen = ret; + args->name = (const uint8_t *)p; + +if (*free_name) +free ((void *)o); + *free_name = true; + +if (args->namelen >MAXNAMELEN ) { +dbprintf (_ ("%s: name too long\n" ), args->name); +return false; + } + +if (args->valuelen >ATTR_MAX_VALUELEN ) { +dbprintf (_ ("%s: value too long\n" ), args->name); +return false; + } + +return true; +} + +static void +print_fsprop ( +struct xfs_da_args *args) +{ +const char *p = +attr_name_to_fsprop_name ((const char *)args->name); + +if (p) +printf ("%s=%.*s\n" , p, args->valuelen, (char *)args->value); +else +fprintf (stderr,_ ("%s: not a fs property?\n" ), args->name); +} + static int attr_set_f( int argc,
@@ -119,7 +200,9 @@ attr_set_f(char *sp;char *name_from_file =NULL ;char *value_from_file =NULL ; +bool free_name = false;enum xfs_attr_update op =XFS_ATTRUPDATE_UPSERT ; +bool fsprop = false;int c;int error;
@@ -132,9 +215,12 @@ attr_set_f(return 0 ; } -while ((c =getopt (argc, argv,"ruspCRnN:v:V:" )) !=EOF ) { +while ((c =getopt (argc, argv,"ruspCRnN:v:V:Z" )) !=EOF ) {switch (c) {/* namespaces */ +case 'Z' : + fsprop = true; + fallthrough;case 'r' : args.attr_filter &= ~LIBXFS_ATTR_NS ; args.attr_filter |=LIBXFS_ATTR_ROOT ;
@@ -213,9 +299,10 @@ attr_set_f(if (!args.name)return 0 ; + free_name = true; args.namelen = namelen; }else { -if (optind != argc -1 ) { +if (optind != argc -1 && optind != argc -2 ) {dbprintf (_ ("too few options for attr_set (no name given)\n" ));return 0 ; }
@@ -250,6 +337,25 @@ attr_set_f( goto out; }memset (args.value,'v' , args.valuelen); + }else if (optind == argc -2 ) { + args.valuelen =strlen (argv[optind +1 ]); + args.value =strdup (argv[optind +1 ]); +if (!args.value) { +dbprintf (_ ("cannot allocate buffer (%d)\n" ), + args.valuelen); + goto out; + } + } + +if (fsprop) { +if (!fsprop_validate ((const char *)args.name, args.value)) { +dbprintf (_ ("%s: invalid value \"%s\"\n" ), + args.name, args.value); + goto out; + } + +if (!adjust_fsprop_attr_name (&args, &free_name)) + goto out; }if (libxfs_iget (mp,NULL , iocur_top->ino,0 , &args.dp)) {
@@ -269,6 +375,9 @@ attr_set_f( goto out; } +if (fsprop) +print_fsprop (&args); +/* refresh with updated inode contents */ set_cur_inode (iocur_top->ino);
@@ -277,7 +386,7 @@ attr_set_f(libxfs_irele (args.dp);if (args.value)free (args.value); -if (name_from_file) +if (free_name)free ((void *)args.name);return 0 ; }
@@ -293,6 +402,8 @@ attr_remove_f( .op_flags = XFS_DA_OP_OKNOENT, };char *name_from_file =NULL ; +bool free_name = false; +bool fsprop = false;int c;int error;
@@ -305,9 +416,12 @@ attr_remove_f(return 0 ; } -while ((c =getopt (argc, argv,"ruspnN:" )) !=EOF ) { +while ((c =getopt (argc, argv,"ruspnN:Z" )) !=EOF ) {switch (c) {/* namespaces */ +case 'Z' : + fsprop = true; + fallthrough;case 'r' : args.attr_filter &= ~LIBXFS_ATTR_NS ; args.attr_filter |=LIBXFS_ATTR_ROOT ;
@@ -354,6 +468,7 @@ attr_remove_f(if (!args.name)return 0 ; + free_name = true; args.namelen = namelen; }else {if (optind != argc -1 ) {
@@ -374,6 +489,9 @@ attr_remove_f( } } +if (fsprop && !adjust_fsprop_attr_name (&args, &free_name)) + goto out; +if (libxfs_iget (mp,NULL , iocur_top->ino,0 , &args.dp)) {dbprintf (_ ("failed to iget inode %llu\n" ), (unsigned long long )iocur_top->ino);
@@ -398,7 +516,137 @@ attr_remove_f( out:if (args.dp)libxfs_irele (args.dp); -if (name_from_file) +if (free_name) +free ((void *)args.name); +return 0 ; +} + +static int +attr_get_f ( +int argc, +char **argv) +{ +struct xfs_da_args args = { + .geo = mp->m_attr_geo, + .whichfork =XFS_ATTR_FORK , + .op_flags =XFS_DA_OP_OKNOENT , + }; +char *name_from_file =NULL ; +bool free_name = false; +bool fsprop = false; +int c; +int error; + +if (cur_typ ==NULL ) { +dbprintf (_ ("no current type\n" )); +return 0 ; + } +if (cur_typ->typnm !=TYP_INODE ) { +dbprintf (_ ("current type is not inode\n" )); +return 0 ; + } + +while ((c =getopt (argc, argv,"ruspN:Z" )) !=EOF ) { +switch (c) { +/* namespaces */ +case 'Z' : + fsprop = true; + fallthrough; +case 'r' : + args.attr_filter &= ~LIBXFS_ATTR_NS ; + args.attr_filter |=LIBXFS_ATTR_ROOT ; +break ; +case 'u' : + args.attr_filter &= ~LIBXFS_ATTR_NS ; +break ; +case 's' : + args.attr_filter &= ~LIBXFS_ATTR_NS ; + args.attr_filter |=LIBXFS_ATTR_SECURE ; +break ; +case 'p' : + args.attr_filter &= ~LIBXFS_ATTR_NS ; + args.attr_filter |=XFS_ATTR_PARENT ; +break ; + +case 'N' : + name_from_file = optarg; +break ; +default : +dbprintf (_ ("bad option for attr_get command\n" )); +return 0 ; + } + } + +if (name_from_file) { +int namelen; + +if (optind != argc) { +dbprintf (_ ("too many options for attr_get (no name needed)\n" )); +return 0 ; + } + + args.name =get_buf_from_file (name_from_file,MAXNAMELEN , + &namelen); +if (!args.name) +return 0 ; + + free_name = true; + args.namelen = namelen; + }else { +if (optind != argc -1 ) { +dbprintf (_ ("too few options for attr_get (no name given)\n" )); +return 0 ; + } + + args.name = (const unsigned char *)argv[optind]; +if (!args.name) { +dbprintf (_ ("invalid name\n" )); +return 0 ; + } + + args.namelen =strlen (argv[optind]); +if (args.namelen >=MAXNAMELEN ) { +dbprintf (_ ("name too long\n" )); + goto out; + } + } + +if (libxfs_iget (mp,NULL , iocur_top->ino,0 , &args.dp)) { +dbprintf (_ ("failed to iget inode %llu\n" ), + (unsigned long long )iocur_top->ino); + goto out; + } + +if (fsprop && !adjust_fsprop_attr_name (&args, &free_name)) + goto out; + + args.owner = iocur_top->ino; +libxfs_attr_sethash (&args); + +/* +* Look up attr value with a maximally long length and a null buffer +* to return the value and the correct length. +*/ + args.valuelen =XATTR_SIZE_MAX ; + error = -libxfs_attr_get (&args); +if (error) { +dbprintf (_ ("failed to get attr %s on inode %llu: %s\n" ), + args.name, (unsigned long long )iocur_top->ino, +strerror (error)); + goto out; + } + +if (fsprop) +print_fsprop (&args); +else +printf ("%.*s\n" , args.valuelen, (char *)args.value); + +out: +if (args.dp) +libxfs_irele (args.dp); +if (args.value) +free (args.value); +if (free_name)free ((void *)args.name);return 0 ; }
man/man8/xfs_db.8 +38 -2
@@ -184,7 +184,35 @@ Displays the length, free block count, per-AG reservation size, and per-AG reservation usage for a given AG. If no argument is given, display information for all AGs. .TP -.BI "attr_remove [\-p|\-r|\-u|\-s] [\-n] [\-N " namefile "|" name "] " +.BI "attr_get [\-p|\-r|\-u|\-s|\-Z] [\-N " namefile "|" name "] " +Print the value of the specified extended attribute from the current file. +.RS 1.0i +.TP 0.4i +.B \-p +Sets the attribute in the parent namespace. +Only one namespace option can be specified. +.TP +.B \-r +Sets the attribute in the root namespace. +Only one namespace option can be specified. +.TP +.B \-u +Sets the attribute in the user namespace. +Only one namespace option can be specified. +.TP +.B \-s +Sets the attribute in the secure namespace. +Only one namespace option can be specified. +.TP +.B \-Z +Sets a filesystem property in the root namespace. +Only one namespace option can be specified. +.TP +.B \-N +Read the name from this file. +.RE +.TP +.BI "attr_remove [\-p|\-r|\-u|\-s|\-Z] [\-n] [\-N " namefile "|" name "] " Remove the specified extended attribute from the current file. .RS 1.0i .TP 0.4i
@@ -204,6 +232,10 @@ Only one namespace option can be specified. Sets the attribute in the secure namespace. Only one namespace option can be specified. .TP +.B \-Z +Sets a filesystem property in the root namespace. +Only one namespace option can be specified. +.TP .B \-N Read the name from this file. .TP
@@ -211,7 +243,7 @@ Read the name from this file. Do not enable 'noattr2' mode on V4 filesystems. .RE .TP -.BI "attr_set [\-p\-r|\-u|\-s] [\-n] [\-R|\-C] [\-v " valuelen "|\-V " valuefile "] [\-N " namefile "|" name "] " +.BI "attr_set [\-p\-r|\-u|\-s|\-Z] [\-n] [\-R|\-C] [\-v " valuelen "|\-V " valuefile "] [\-N " namefile "|" name "] [" value "]" Sets an extended attribute on the current file with the given name. .RS 1.0i .TP 0.4i
@@ -231,6 +263,10 @@ Only one namespace option can be specified. Sets the attribute in the secure namespace. Only one namespace option can be specified. .TP +.B \-Z +Sets a filesystem property in the root namespace. +Only one namespace option can be specified. +.TP .B \-N Read the name from this file. .TP
Re: [PATCH 3/7] xfs_db: improve getting and setting extended attributes
Quoted 6 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Add an
attr_getcommand to retrieve the value of an xattr from a file; and extend theattr_setcommand to allow passing of string values.
Missing newline here again (not going to say this again if it shows up on more patches..)
Re: [PATCH 3/7] xfs_db: improve getting and setting extended attributes
Quoted 10 lines · Christoph Hellwig
Quoted 6 lines · Darrick J. Wong
From: Darrick J. Wong <[email protected]>
Add an
attr_getcommand to retrieve the value of an xattr from a file; and extend theattr_setcommand to allow passing of string values.Missing newline here again (not going to say this again if it shows up on more patches..)
Yeah, I don't know why that happened. Something went wrong in my text processing scripts.
--D
PATCH 4/7 libxfs: hoist listxattr from xfs_repair
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Hoist the listxattr code from xfs_repair so that we can use it in xfs_db.
--- libxfs/Makefile | 2 ++ libxfs/listxattr.c | 2 +- libxfs/listxattr.h | 6 +++--- repair/Makefile | 2 -- repair/pptr.c | 2 +- 5 files changed, 7 insertions(+), 7 deletions(-) rename repair/listxattr.c => libxfs/listxattr.c (99%) rename repair/listxattr.h => libxfs/listxattr.h (81%)
libxfs/Makefile +2 -0
@@ -23,6 +23,7 @@ HFILES = \ defer_item.h \ libxfs_io.h \ libxfs_api_defs.h \ + listxattr.h \ init.h \ libxfs_priv.h \ linux-err.h \
@@ -69,6 +70,7 @@ CFILES = buf_mem.c \ defer_item.c \ init.c \ kmem.c \ + listxattr.c \ logitem.c \ rdwr.c \ topology.c \
libxfs/listxattr.c renamed from repair/listxattr.c +1 -1
@@ -6,7 +6,7 @@#include "libxfs.h" #include "libxlog.h" #include "libfrog/bitmap.h" -#include "repair/listxattr.h" +#include "listxattr.h" /* Call a function for every entry in a shortform xattr structure. */ STATIC int
libxfs/listxattr.h renamed from repair/listxattr.h +3 -3
@@ -3,8 +3,8 @@* Copyright (c) 2022-2024 Oracle. All Rights Reserved. * Author: Darrick J. Wong <[email protected]> */ -#ifndef __REPAIR_LISTXATTR_H__ -#define __REPAIR_LISTXATTR_H__ +#ifndef __LIBXFS_LISTXATTR_H__ +#define __LIBXFS_LISTXATTR_H__ typedef int (*xattr_walk_fn)(struct xfs_inode *ip, unsigned int attr_flags, const unsigned char *name, unsigned int namelen,
@@ -12,4 +12,4 @@ typedef int (*xattr_walk_fn)(struct xfs_inode *ip, unsigned int attr_flags,int xattr_walk (struct xfs_inode *ip,xattr_walk_fn attr_fn,void *priv); -#endif /* __REPAIR_LISTXATTR_H__ */ +#endif /* __LIBXFS_LISTXATTR_H__ */
repair/Makefile +0 -2
@@ -24,7 +24,6 @@ HFILES = \ err_protos.h \ globals.h \ incore.h \ - listxattr.h \ pptr.h \ prefetch.h \ progress.h \
@@ -59,7 +58,6 @@ CFILES = \ incore_ext.c \ incore_ino.c \ init.c \ - listxattr.c \ phase1.c \ phase2.c \ phase3.c \
repair/pptr.c +1 -1
@@ -11,7 +11,7 @@#include "repair/globals.h" #include "repair/err_protos.h" #include "repair/slab.h" -#include "repair/listxattr.h" +#include "libxfs/listxattr.h" #include "repair/threads.h" #include "repair/incore.h" #include "repair/pptr.h"
PATCH 5/7 libxfs: pass a transaction context through listxattr
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Pass a transaction context so that a new caller can walk the attr names and query the values all in one go without deadlocking on nested buffer access.
While we're at it, make the existing xfs_repair callers try to use empty transactions so that we don't deadlock on cycles in the xattr structure.
--- libxfs/listxattr.c | 40 +++++++++++++++++++++++----------------- libxfs/listxattr.h | 6 ++++-- repair/pptr.c | 7 ++++++- 3 files changed, 33 insertions(+), 20 deletions(-)
libxfs/listxattr.c +23 -17
@@ -11,6 +11,7 @@/* Call a function for every entry in a shortform xattr structure. */ STATIC intxattr_walk_sf ( +struct xfs_trans *tp,struct xfs_inode *ip,xattr_walk_fn attr_fn,void *priv)
@@ -22,7 +23,7 @@ xattr_walk_sf( sfe =libxfs_attr_sf_firstentry (hdr);for (i =0 ; i < hdr->count; i++) { - error =attr_fn (ip, sfe->flags, sfe->nameval, sfe->namelen, + error =attr_fn (tp, ip, sfe->flags, sfe->nameval, sfe->namelen, &sfe->nameval[sfe->namelen], sfe->valuelen, priv);if (error)
@@ -37,6 +38,7 @@ xattr_walk_sf(/* Call a function for every entry in this xattr leaf block. */ STATIC int xattr_walk_leaf_entries( + struct xfs_trans *tp, struct xfs_inode *ip, xattr_walk_fn attr_fn, struct xfs_buf *bp,
@@ -75,7 +77,7 @@ xattr_walk_leaf_entries( valuelen =be32_to_cpu (name_rmt->valuelen); } - error =attr_fn (ip, entry->flags, name, namelen, value, + error =attr_fn (tp, ip, entry->flags, name, namelen, value, valuelen, priv);if (error)return error;
@@ -91,6 +93,7 @@ xattr_walk_leaf_entries(*/ STATIC intxattr_walk_leaf ( +struct xfs_trans *tp,struct xfs_inode *ip,xattr_walk_fn attr_fn,void *priv)
@@ -98,18 +101,19 @@ xattr_walk_leaf(struct xfs_buf *leaf_bp;int error; - error = -libxfs_attr3_leaf_read (NULL , ip, ip->i_ino,0 , &leaf_bp); + error = -libxfs_attr3_leaf_read (tp, ip, ip->i_ino,0 , &leaf_bp);if (error)return error; - error =xattr_walk_leaf_entries (ip, attr_fn, leaf_bp, priv); -libxfs_trans_brelse (NULL , leaf_bp); + error =xattr_walk_leaf_entries (tp, ip, attr_fn, leaf_bp, priv); +libxfs_trans_brelse (tp, leaf_bp);return error; }/* Find the leftmost leaf in the xattr dabtree. */ STATIC intxattr_walk_find_leftmost_leaf ( +struct xfs_trans *tp,struct xfs_inode *ip,struct bitmap *seen_blocks,struct xfs_buf **leaf_bpp)
@@ -127,7 +131,7 @@ xattr_walk_find_leftmost_leaf(for (;;) {uint16_t magic; - error = -libxfs_da3_node_read (NULL , ip, blkno, &bp, + error = -libxfs_da3_node_read (tp, ip, blkno, &bp,XFS_ATTR_FORK );if (error)return error;
@@ -164,7 +168,7 @@ xattr_walk_find_leftmost_leaf(/* Find the next level towards the leaves of the dabtree. */ btree = nodehdr.btree; blkno =be32_to_cpu (btree->before); -libxfs_trans_brelse (NULL , bp); +libxfs_trans_brelse (tp, bp);/* Make sure we haven't seen this new block already. */ if (bitmap_test (seen_blocks, blkno,1 ))
@@ -184,13 +188,14 @@ xattr_walk_find_leftmost_leaf(return 0 ; out_buf: -libxfs_trans_brelse (NULL , bp); +libxfs_trans_brelse (tp, bp);return error; }/* Call a function for every entry in a node-format xattr structure. */ STATIC intxattr_walk_node ( +struct xfs_trans *tp,struct xfs_inode *ip,xattr_walk_fn attr_fn,void *priv)
@@ -204,12 +209,12 @@ xattr_walk_node(bitmap_alloc (&seen_blocks); - error =xattr_walk_find_leftmost_leaf (ip, seen_blocks, &leaf_bp); + error =xattr_walk_find_leftmost_leaf (tp, ip, seen_blocks, &leaf_bp);if (error) goto out_bitmap;for (;;) { - error =xattr_walk_leaf_entries (ip, attr_fn, leaf_bp, + error =xattr_walk_leaf_entries (tp, ip, attr_fn, leaf_bp, priv);if (error) goto out_leaf;
@@ -220,13 +225,13 @@ xattr_walk_node(if (leafhdr.forw ==0 ) goto out_leaf; -libxfs_trans_brelse (NULL , leaf_bp); +libxfs_trans_brelse (tp, leaf_bp);/* Make sure we haven't seen this new leaf already. */ if (bitmap_test (seen_blocks, leafhdr.forw,1 )) goto out_bitmap; - error = -libxfs_attr3_leaf_read (NULL , ip, ip->i_ino, + error = -libxfs_attr3_leaf_read (tp, ip, ip->i_ino, leafhdr.forw, &leaf_bp);if (error) goto out_bitmap;
@@ -238,7 +243,7 @@ xattr_walk_node( } out_leaf: -libxfs_trans_brelse (NULL , leaf_bp); +libxfs_trans_brelse (tp, leaf_bp); out_bitmap:bitmap_free (&seen_blocks);return error;
@@ -247,6 +252,7 @@ xattr_walk_node(/* Call a function for every extended attribute in a file. */ int xattr_walk ( +struct xfs_trans *tp,struct xfs_inode *ip,xattr_walk_fn attr_fn,void *priv)
@@ -257,15 +263,15 @@ xattr_walk(return 0 ;if (ip->i_af.if_format ==XFS_DINODE_FMT_LOCAL ) -return xattr_walk_sf (ip, attr_fn, priv); +return xattr_walk_sf (tp, ip, attr_fn, priv);/* attr functions require that the attr fork is loaded */ - error = -libxfs_iread_extents (NULL , ip,XFS_ATTR_FORK ); + error = -libxfs_iread_extents (tp, ip,XFS_ATTR_FORK );if (error)return error;if (libxfs_attr_is_leaf (ip)) -return xattr_walk_leaf (ip, attr_fn, priv); +return xattr_walk_leaf (tp, ip, attr_fn, priv); -return xattr_walk_node (ip, attr_fn, priv); +return xattr_walk_node (tp, ip, attr_fn, priv); }
libxfs/listxattr.h +4 -2
@@ -6,10 +6,12 @@#ifndef __LIBXFS_LISTXATTR_H__#define __LIBXFS_LISTXATTR_H__ -typedef int (*xattr_walk_fn )(struct xfs_inode *ip,unsigned int attr_flags, +typedef int (*xattr_walk_fn )(struct xfs_trans *tp,struct xfs_inode *ip, +unsigned int attr_flags,const unsigned char *name,unsigned int namelen,const void *value,unsigned int valuelen,void *priv); -int xattr_walk (struct xfs_inode *ip,xattr_walk_fn attr_fn,void *priv); +int xattr_walk (struct xfs_trans *tp,struct xfs_inode *ip, +xattr_walk_fn attr_fn,void *priv);#endif /* __LIBXFS_LISTXATTR_H__ */
repair/pptr.c +6 -1
@@ -593,6 +593,7 @@ store_file_pptr_name(/* Decide if this is a directory parent pointer and stash it if so. */ static int examine_xattr( + struct xfs_trans *tp, struct xfs_inode *ip, unsigned int attr_flags, const unsigned char *name,
@@ -1205,6 +1206,7 @@ check_file_parent_ptrs(struct xfs_inode *ip, struct file_scan *fscan) { + struct xfs_trans *tp = NULL;int error; error = -init_slab (&fscan->file_pptr_recs,sizeof (struct file_pptr ));
@@ -1215,7 +1217,10 @@ check_file_parent_ptrs( fscan->have_garbage = false; fscan->nr_file_pptrs =0 ; - error =xattr_walk (ip, examine_xattr, fscan); +libxfs_trans_alloc_empty (ip->i_mount, &tp); + error =xattr_walk (tp, ip, examine_xattr, fscan); +if (tp) +libxfs_trans_cancel (tp);if (error && !no_modify) do_error(_("ino %llu parent pointer scan failed: %s\n"), (unsigned long long)ip->i_ino,
PATCH 6/7 xfs_db: add a command to list xattrs
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Add a command to list extended attributes from xfs_db. We'll need this later to manage the fs properties when unmounted.
--- db/attrset.c | 201 +++++++++++++++++++++++++++++++++++++++++++++++++++++ man/man8/xfs_db.8 | 28 +++++++ 2 files changed, 229 insertions(+)
db/attrset.c +201 -0
@@ -18,13 +18,21 @@#include "malloc.h" #include <sys/xattr.h> #include "libfrog/fsproperties.h" +#include "libxfs/listxattr.h" +static int attr_list_f (int argc,char **argv);static int attr_get_f (int argc,char **argv);static int attr_set_f (int argc,char **argv);static int attr_remove_f (int argc,char **argv); + +static void attrlist_help (void );static void attrget_help (void );static void attrset_help (void ); +static const cmdinfo_t attr_list_cmd = + {"attr_list" ,"alist" , attr_list_f,0 ,-1 ,0 , +N_ ("[-r|-s|-u|-p|-Z] [-v]" ), +N_ ("list attributes on the current inode" ), attrlist_help };static const cmdinfo_t attr_get_cmd = {"attr_get" ,"aget" , attr_get_f,1 ,-1 ,0 ,N_ ("[-r|-s|-u|-p|-Z] name" ),
@@ -38,6 +46,24 @@ static const cmdinfo_t attr_remove_cmd =N_ ("[-r|-s|-u|-p|-Z] [-n] name" ), N_("remove the named attribute from the current inode"), attrset_help }; +static void +attrlist_help (void ) +{ +dbprintf (_ ( +"\n" +" The attr_list command provide interfaces for listing all extended attributes\n" +" attached to an inode.\n" +" There are 4 namespace flags:\n" +" -r -- 'root'\n" +" -u -- 'user' (default)\n" +" -s -- 'secure'\n" +" -p -- 'parent'\n" +" -Z -- fs property\n" +"\n" +" -v -- print the value of the attributes\n" +"\n" )); +} +static void attrget_help (void ) {
@@ -87,6 +113,7 @@ attrset_init(void)if (!expert_mode)return ; +add_command (&attr_list_cmd);add_command (&attr_get_cmd);add_command (&attr_set_cmd);add_command (&attr_remove_cmd);
@@ -650,3 +677,177 @@ attr_get_f(free ((void *)args.name);return 0 ; } + +struct attrlist_ctx { +unsigned int attr_filter; +bool print_values; +bool fsprop; +}; + +static int +attrlist_print ( +struct xfs_trans *tp, +struct xfs_inode *ip, +unsigned int attr_flags, +const unsigned char *name, +unsigned int namelen, +const void *value, +unsigned int valuelen, +void *priv) +{ +struct attrlist_ctx *ctx = priv; +struct xfs_da_args args = { + .geo = mp->m_attr_geo, + .whichfork =XFS_ATTR_FORK , + .op_flags =XFS_DA_OP_OKNOENT , + .dp = ip, + .owner = ip->i_ino, + .trans = tp, + .attr_filter = attr_flags &XFS_ATTR_NSP_ONDISK_MASK , + .name = name, + .namelen = namelen, + }; +char namebuf[MAXNAMELEN +1 ]; +const char *print_name = namebuf; +int error; + +if ((attr_flags &XFS_ATTR_NSP_ONDISK_MASK ) != ctx->attr_filter) +return 0 ; + +/* Make sure the name is null terminated. */ +memcpy (namebuf, name, namelen); + namebuf[MAXNAMELEN ] =0 ; + +if (ctx->fsprop) { +const char *p =attr_name_to_fsprop_name (namebuf); + +if (!p) +return 0 ; + + namelen -= (p - namebuf); + print_name = p; + } + +if (!ctx->print_values) { +printf ("%.*s\n" , namelen, print_name); +return 0 ; + } + +if (value) { +printf ("%.*s=%.*s\n" , namelen, print_name, valuelen, + (char *)value); +return 0 ; + } + +libxfs_attr_sethash (&args); + +/* +* Look up attr value with a maximally long length and a null buffer +* to return the value and the correct length. +*/ + args.valuelen =XATTR_SIZE_MAX ; + error = -libxfs_attr_get (&args); +if (error) { +dbprintf (_ ("failed to get attr %s on inode %llu: %s\n" ), + args.name, (unsigned long long )iocur_top->ino, +strerror (error)); +return error; + } + +printf ("%.*s=%.*s\n" , namelen, print_name, args.valuelen, + (char *)args.value); +kfree (args.value); + +return 0 ; +} + +static int +attr_list_f ( +int argc, +char **argv) +{ +struct attrlist_ctx ctx = { }; +struct xfs_trans *tp; +struct xfs_inode *ip; +int c; +int error; + +if (cur_typ ==NULL ) { +dbprintf (_ ("no current type\n" )); +return 0 ; + } +if (cur_typ->typnm !=TYP_INODE ) { +dbprintf (_ ("current type is not inode\n" )); +return 0 ; + } + +while ((c =getopt (argc, argv,"ruspvZ" )) !=EOF ) { +switch (c) { +/* namespaces */ +case 'Z' : + ctx.fsprop = true; + fallthrough; +case 'r' : + ctx.attr_filter &= ~LIBXFS_ATTR_NS ; + ctx.attr_filter |=LIBXFS_ATTR_ROOT ; +break ; +case 'u' : + ctx.attr_filter &= ~LIBXFS_ATTR_NS ; +break ; +case 's' : + ctx.attr_filter &= ~LIBXFS_ATTR_NS ; + ctx.attr_filter |=LIBXFS_ATTR_SECURE ; +break ; +case 'p' : + ctx.attr_filter &= ~LIBXFS_ATTR_NS ; + ctx.attr_filter |=XFS_ATTR_PARENT ; +break ; + +case 'v' : + ctx.print_values = true; +break ; +default : +dbprintf (_ ("bad option for attr_list command\n" )); +return 0 ; + } + } + +if (ctx.fsprop && + (ctx.attr_filter &LIBXFS_ATTR_NS ) !=LIBXFS_ATTR_ROOT ) { +dbprintf (_ ("fs properties must be ATTR_ROOT\n" )); +return false; + } + +if (optind != argc) { +dbprintf (_ ("too many options for attr_list (no name needed)\n" )); +return 0 ; + } + + error = -libxfs_trans_alloc_empty (mp, &tp); +if (error) { +dbprintf (_ ("failed to allocate empty transaction\n" )); +return 0 ; + } + + error = -libxfs_iget (mp,NULL , iocur_top->ino,0 , &ip); +if (error) { +dbprintf (_ ("failed to iget inode %llu: %s\n" ), + (unsigned long long )iocur_top->ino, +strerror (error)); + goto out_trans; + } + + error =xattr_walk (tp, ip, attrlist_print, &ctx); +if (error) { +dbprintf (_ ("walking inode %llu xattrs: %s\n" ), + (unsigned long long )iocur_top->ino, +strerror (error)); + goto out_inode; + } + +out_inode: +libxfs_irele (ip); +out_trans: +libxfs_trans_cancel (tp); +return 0 ; +}
man/man8/xfs_db.8 +28 -0
@@ -212,6 +212,34 @@ Only one namespace option can be specified. Read the name from this file. .RE .TP +.BI "attr_list [\-p|\-r|\-u|\-s|\-Z] [\-v] " +Lists the extended attributes of the current file. +.RS 1.0i +.TP 0.4i +.B \-p +Sets the attribute in the parent namespace. +Only one namespace option can be specified. +.TP +.B \-r +Sets the attribute in the root namespace. +Only one namespace option can be specified. +.TP +.B \-u +Sets the attribute in the user namespace. +Only one namespace option can be specified. +.TP +.B \-s +Sets the attribute in the secure namespace. +Only one namespace option can be specified. +.TP +.B \-Z +Sets a filesystem property in the root namespace. +Only one namespace option can be specified. +.TP +.B \-v +Print the extended attribute values too. +.RE +.TP .BI "attr_remove [\-p|\-r|\-u|\-s|\-Z] [\-n] [\-N " namefile "|" name "] " Remove the specified extended attribute from the current file. .RS 1.0i
PATCH 7/7 xfs_property: add a new tool to administer fs properties
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Create a tool to list, get, set, and remove filesystem properties.
--- io/Makefile | 3 +- io/xfs_property | 77 +++++++++++++++++++++++++++++++++++++++++++++++ man/man8/xfs_property.8 | 61 +++++++++++++++++++++++++++++++++++++ 3 files changed, 140 insertions(+), 1 deletion(-) create mode 100755 io/xfs_property create mode 100644 man/man8/xfs_property.8
io/Makefile +2 -1
@@ -6,7 +6,7 @@ TOPDIR = .. include $(TOPDIR)/include/builddefs LTCOMMAND = xfs_io -LSRCFILES = xfs_bmap.sh xfs_freeze.sh xfs_mkfile.sh +LSRCFILES = xfs_bmap.sh xfs_freeze.sh xfs_mkfile.sh xfs_property HFILES = init.h io.h CFILES = \ attr.c \
@@ -92,6 +92,7 @@ install: default $(LTINSTALL) -m 755 xfs_bmap.sh $(PKG_SBIN_DIR)/xfs_bmap $(LTINSTALL) -m 755 xfs_freeze.sh $(PKG_SBIN_DIR)/xfs_freeze $(LTINSTALL) -m 755 xfs_mkfile.sh $(PKG_SBIN_DIR)/xfs_mkfile + $(LTINSTALL) -m 755 xfs_property $(PKG_SBIN_DIR)/xfs_property install-dev: -include .dep
io/xfs_property new file +77 -0
77 lines not on this page · raw message
man/man8/xfs_property.8 new file +61 -0
61 lines not on this page · raw message
Re: [PATCH 7/7] xfs_property: add a new tool to administer fs properties
Looks good:
PATCH 2/3 xfs_scrub: admin control of automatic fsck
In reply to Darrick J. WongHi all,
Now that we have the ability to set per-filesystem properties, teach the background xfs_scrub service to pick up advice from the filesystem that it wants to examine, and pick a mode from that. We're only going to enable this by default for newer filesystems.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=autofsck-6.10
---
Commits in this patchset:
* libfrog: define a autofsck filesystem property
* xfs_scrub: allow sysadmin to control background scrubs
* xfs_scrub: use the autofsck fsproperty to select mode
* mkfs: set autofsck filesystem property
--- libfrog/fsproperties.c | 38 ++++++++++++ libfrog/fsproperties.h | 13 ++++ man/man8/mkfs.xfs.8.in | 6 ++ man/man8/xfs_property.8 | 8 ++ man/man8/xfs_scrub.8 | 46 ++++++++++++++ mkfs/lts_4.19.conf | 1 mkfs/lts_5.10.conf | 1 mkfs/lts_5.15.conf | 1 mkfs/lts_5.4.conf | 1 mkfs/lts_6.1.conf | 1 mkfs/lts_6.6.conf | 1 mkfs/xfs_mkfs.c | 122 +++++++++++++++++++++++++++++++++++++ scrub/Makefile | 3 - scrub/phase1.c | 91 ++++++++++++++++++++++++++++ scrub/xfs_scrub.c | 14 ++++ scrub/xfs_scrub.h | 7 ++ scrub/[email protected] | 2 - scrub/[email protected] | 2 - 18 files changed, 353 insertions(+), 5 deletions(-)
PATCH 1/4 libfrog: define a autofsck filesystem property
From: Darrick J. Wong <[email protected]>
Now that we have the ability to set properties on filesystems, create an "autofsck" property so that sysadmins can control background xfs_scrub behaviors.
--- libfrog/fsproperties.c | 38 ++++++++++++++++++++++++++++++++++++++ libfrog/fsproperties.h | 13 +++++++++++++ 2 files changed, 51 insertions(+)
libfrog/fsproperties.c +38 -0
@@ -29,11 +29,49 @@ __fsprops_lookup(#define fsprops_lookup (values, value) \ __fsprops_lookup((values), ARRAY_SIZE(values), (value)) +/* Automatic background fsck fs property */ + +static const char *fsprop_autofsck_values[] = { + [FSPROP_AUTOFSCK_UNSET ] =NULL , + [FSPROP_AUTOFSCK_NONE ] ="none" , + [FSPROP_AUTOFSCK_CHECK ] ="check" , + [FSPROP_AUTOFSCK_OPTIMIZE ] ="optimize" , + [FSPROP_AUTOFSCK_REPAIR ] ="repair" , +}; + +/* Convert the autofsck property enum to a string. */ +const char * +fsprop_autofsck_write ( +enum fsprop_autofsck x) +{ +if (x <=FSPROP_AUTOFSCK_UNSET || + x >=ARRAY_SIZE (fsprop_autofsck_values)) +return NULL ; +return fsprop_autofsck_values[x]; +} + +/* +* Turn a autofsck value string into an enumerated value, or _UNSET if it's +* not recognized. +*/ +enum fsprop_autofsck +fsprop_autofsck_read ( +const char *value) +{ +int ret =fsprops_lookup (fsprop_autofsck_values, value); +if (ret <0 ) +return FSPROP_AUTOFSCK_UNSET ; +return ret; +} +/* Return true if a fs property name=value tuple is allowed. */ bool fsprop_validate (const char *name,const char *value) { +if (!strcmp (name,FSPROP_AUTOFSCK_NAME )) +return fsprops_lookup (fsprop_autofsck_values, value) >=0 ; +return true; }
libfrog/fsproperties.h +13 -0
@@ -50,4 +50,17 @@ bool fsprop_validate(const char *name, const char *value);/* Specific Filesystem Properties */ +#define FSPROP_AUTOFSCK_NAME "autofsck" + +enum fsprop_autofsck { +FSPROP_AUTOFSCK_UNSET =0 ,/* do not set property */ +FSPROP_AUTOFSCK_NONE ,/* no background scrubs */ +FSPROP_AUTOFSCK_CHECK ,/* allow only background checking */ +FSPROP_AUTOFSCK_OPTIMIZE ,/* allow background optimization */ +FSPROP_AUTOFSCK_REPAIR ,/* allow background repair & optimization */ +}; + +const char *fsprop_autofsck_write (enum fsprop_autofsck x); +enum fsprop_autofsck fsprop_autofsck_read (const char *value); +#endif /* __LIBFROG_FSPROPERTIES_H__ */
Re: [PATCH 1/4] libfrog: define a autofsck filesystem property
Looks good:
PATCH 2/4 xfs_scrub: allow sysadmin to control background scrubs
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Add an extended option -o autofsck to xfs_scrub so that it selects the operation mode from the "autofsck" filesystem property.
--- man/man8/xfs_property.8 | 8 ++++ man/man8/xfs_scrub.8 | 46 ++++++++++++++++++++++++ scrub/phase1.c | 91 +++++++++++++++++++++++++++++++++++++++++++++++ scrub/xfs_scrub.c | 14 +++++++ scrub/xfs_scrub.h | 7 ++++ 5 files changed, 166 insertions(+)
man/man8/xfs_property.8 +8 -0
@@ -59,3 +59,11 @@ was set. remove .IR name ... Unsets the given filesystem properties. + +.SH FILESYSTEM PROPERTIES +Known filesystem properties for XFS are: + +.I autofsck +See +.BR xfs_scrub (8) +for more information.
man/man8/xfs_scrub.8 +46 -0
58 lines not on this page · raw message
scrub/phase1.c +91 -0
109 lines not on this page · raw message
scrub/xfs_scrub.c +14 -0
@@ -526,6 +526,10 @@ _("Scrub aborted after phase %d."),if (ret)break ; +/* Did background scrub get canceled on us? */ +if (ctx->mode ==SCRUB_MODE_NONE ) +break ; +/* Too many errors? */ if (scrub_excessive_errors (ctx)) { ret =ECANCELED ;
@@ -630,12 +634,14 @@ report_outcome(enum o_opt_nums {IWARN =0 ,FSTRIM_PCT , +AUTOFSCK ,O_MAX_OPTS , };static char *o_opts[] = { [IWARN ] ="iwarn" , [FSTRIM_PCT ] ="fstrim_pct" , + [AUTOFSCK ] ="autofsck" , [O_MAX_OPTS ] =NULL , };
@@ -688,6 +694,14 @@ parse_o_opts( ctx->fstrim_block_pct = dval /100.0 ;break ; +case AUTOFSCK : +if (val) { +fprintf (stderr, +_ ("-o autofsck does not take an argument\n" )); +usage (); + } + ctx->mode =SCRUB_MODE_NONE ; +break ;default :usage ();break ;
scrub/xfs_scrub.h +7 -0
@@ -26,6 +26,13 @@ extern bool use_force_rebuild;extern bool info_is_warning;enum scrub_mode { +/* +* Prior to phase 1, this means that xfs_scrub should read the +* "autofsck" fs property from the mount and set the value +* appropriate. If it's still set after phase 1, this means we should +* exit without doing anything. +*/ +SCRUB_MODE_NONE ,SCRUB_MODE_DRY_RUN ,SCRUB_MODE_PREEN ,SCRUB_MODE_REPAIR ,
Re: [PATCH 2/4] xfs_scrub: allow sysadmin to control background scrubs
Looks good:
PATCH 3/4 xfs_scrub: use the autofsck fsproperty to select mode
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Now that we can set properties on xfs filesystems, make the xfs_scrub background service query the autofsck property to figure out which operating mode it should use.
--- scrub/Makefile | 3 +-- scrub/[email protected] | 2 +- scrub/[email protected] | 2 +- 3 files changed, 3 insertions(+), 4 deletions(-)
scrub/Makefile +1 -2
@@ -17,7 +17,7 @@ INSTALL_SCRUB = install-scrub XFS_SCRUB_ALL_PROG = xfs_scrub_all XFS_SCRUB_FAIL_PROG = xfs_scrub_fail XFS_SCRUB_ARGS = -p -XFS_SCRUB_SERVICE_ARGS = -b +XFS_SCRUB_SERVICE_ARGS = -b -o autofsck ifeq ($(HAVE_SYSTEMD),yes) INSTALL_SCRUB += install-systemd SYSTEMD_SERVICES=\
@@ -144,7 +144,6 @@ install: $(INSTALL_SCRUB) @echo " [SED] $@" $(Q)$(SED) -e "s|@sbindir@|$(PKG_SBIN_DIR)|g" \ -e "s|@scrub_service_args@|$(XFS_SCRUB_SERVICE_ARGS)|g" \ - -e "s|@scrub_args@|$(XFS_SCRUB_ARGS)|g" \ -e "s|@pkg_libexec_dir@|$(PKG_LIBEXEC_DIR)|g" \ -e "s|@pkg_state_dir@|$(PKG_STATE_DIR)|g" \ -e "s|@media_scan_interval@|$(XFS_SCRUB_ALL_AUTO_MEDIA_SCAN_INTERVAL)|g" \
scrub/[email protected] +1 -1
@@ -22,7 +22,7 @@ RequiresMountsFor=%f [Service] Type=oneshot Environment=SERVICE_MODE=1 -ExecStart=@sbindir@/xfs_scrub @scrub_service_args@ @scrub_args@ -M /tmp/scrub/ %f +ExecStart=@sbindir@/xfs_scrub @scrub_service_args@ -M /tmp/scrub/ %f SyslogIdentifier=%N # Run scrub with minimal CPU and IO priority so that nothing else will starve.
scrub/[email protected] +1 -1
@@ -22,7 +22,7 @@ RequiresMountsFor=%f [Service] Type=oneshot Environment=SERVICE_MODE=1 -ExecStart=@sbindir@/xfs_scrub @scrub_service_args@ @scrub_args@ -M /tmp/scrub/ -x %f +ExecStart=@sbindir@/xfs_scrub @scrub_service_args@ -M /tmp/scrub/ -x %f SyslogIdentifier=%N # Run scrub with minimal CPU and IO priority so that nothing else will starve.
Re: [PATCH 3/4] xfs_scrub: use the autofsck fsproperty to select mode
Looks good:
PATCH 4/4 mkfs: set autofsck filesystem property
In reply to Darrick J. WongFrom: Darrick J. Wong <[email protected]>
Add a new mkfs option -m autofsck so that sysadmins can control the background scrubbing behavior of filesystems from the start.
--- man/man8/mkfs.xfs.8.in | 6 ++ mkfs/lts_4.19.conf | 1 mkfs/lts_5.10.conf | 1 mkfs/lts_5.15.conf | 1 mkfs/lts_5.4.conf | 1 mkfs/lts_6.1.conf | 1 mkfs/lts_6.6.conf | 1 mkfs/xfs_mkfs.c | 122 ++++++++++++++++++++++++++++++++++++++++++++++++ 8 files changed, 133 insertions(+), 1 deletion(-)
man/man8/mkfs.xfs.8.in +6 -0
@@ -323,6 +323,12 @@ option set. When the option .B \-m crc=0 is used, the reference count btree feature is not supported and reflink is disabled. +.TP +.BI autofsck= value +Set the autofsck filesystem property to this value. +See the +.BI xfs_scrub (8) +manual page for more information on this property. .RE .PP .PD 0
mkfs/lts_4.19.conf +1 -0
@@ -8,6 +8,7 @@ finobt=1 inobtcount=0 reflink=0 rmapbt=0 +autofsck=0 [inode] sparse=1
mkfs/lts_5.10.conf +1 -0
@@ -8,6 +8,7 @@ finobt=1 inobtcount=0 reflink=1 rmapbt=0 +autofsck=0 [inode] sparse=1
mkfs/lts_5.15.conf +1 -0
@@ -8,6 +8,7 @@ finobt=1 inobtcount=1 reflink=1 rmapbt=0 +autofsck=0 [inode] sparse=1
mkfs/lts_5.4.conf +1 -0
@@ -8,6 +8,7 @@ finobt=1 inobtcount=0 reflink=1 rmapbt=0 +autofsck=0 [inode] sparse=1
mkfs/lts_6.1.conf +1 -0
@@ -8,6 +8,7 @@ finobt=1 inobtcount=1 reflink=1 rmapbt=0 +autofsck=0 [inode] sparse=1
mkfs/lts_6.6.conf +1 -0
@@ -8,6 +8,7 @@ finobt=1 inobtcount=1 reflink=1 rmapbt=1 +autofsck=0 [inode] sparse=1
mkfs/xfs_mkfs.c +121 -1
194 lines not on this page · raw message
Re: [PATCH 4/4] mkfs: set autofsck filesystem property
Looks good:
PATCH 3/3 debian: enable xfs_scrub_all by default
In reply to Darrick J. WongHi all,
Update our packaging to enable the background xfs_scrub timer by default. This won't do much unless the sysadmin sets the autofsck fs property or formats a filesystem with backref metadata.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=debian-autofsck-6.10
---
Commits in this patchset:
* debian: enable xfs_scrub_all systemd timer services by default
--- debian/rules | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
PATCH 1/1 debian: enable xfs_scrub_all systemd timer services by default
From: Darrick J. Wong <[email protected]>
Now that we're finished building online fsck, enable the periodic background scrub service by default. This involves the postinst script starting the resource management slice and the timer.
No other sub-services need to be enabled or unmasked explicitly. They also shouldn't be started or restarted because that might interrupt background operation unnecessarily.
Although the xfs_scrub_all timer is activated by default, the individual xfs_scrub@ services that it spawns will only do real work on filesystems that are new enough to have back reference metadata available. This avoids surprises for people who are upgrading Debian; only new installs with new mkfs will get any automatic fsck behavior.
--- debian/rules | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
debian/rules +1 -1
@@ -114,7 +114,7 @@ binary-arch: checkroot built dh_compress dh_fixperms dh_makeshlibs - dh_installsystemd -p xfsprogs --no-enable --no-start --no-restart-after-upgrade --no-stop-on-upgrade + dh_installsystemd -p xfsprogs --no-restart-after-upgrade --no-stop-on-upgrade system-xfs_scrub.slice xfs_scrub_all.timer dh_installdeb dh_shlibdeps dh_gencontrol
Re: [PATCH 1/1] debian: enable xfs_scrub_all systemd timer services by default
Looks good:
PATCH 1/1 fstests: xfs filesystem properties
In reply to Darrick J. WongHi all,
It would be very useful if system administrators could set properties for a given xfs filesystem to control its behavior. This we can do easily and extensibly by setting ATTR_ROOT (aka "trusted") extended attributes on the root directory. To prevent this from becoming a weird free for all, let's add some library and tooling support so that sysadmins simply run the xfs_property program to administer these properties.
If you're going to start using this code, I strongly recommend pulling from my git trees, which are linked below.
This has been running on the djcloud for months with no problems. Enjoy! Comments and questions are, as always, welcome.
--D
xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=filesystem-properties
fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=filesystem-properties
---
Commits in this patchset:
* xfs: functional testing for filesystem properties
--- common/config | 1 common/xfs | 14 ++++- doc/group-names.txt | 1 tests/generic/062 | 4 + tests/xfs/1886 | 137 +++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1886.out | 53 ++++++++++++++++++++ tests/xfs/1887 | 122 +++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1887.out | 46 +++++++++++++++++ tests/xfs/1888 | 66 +++++++++++++++++++++++++ tests/xfs/1888.out | 9 +++ tests/xfs/1889 | 63 +++++++++++++++++++++++ tests/xfs/1889.out | 8 +++ 12 files changed, 522 insertions(+), 2 deletions(-) create mode 100755 tests/xfs/1886 create mode 100644 tests/xfs/1886.out create mode 100755 tests/xfs/1887 create mode 100644 tests/xfs/1887.out create mode 100755 tests/xfs/1888 create mode 100644 tests/xfs/1888.out create mode 100755 tests/xfs/1889 create mode 100644 tests/xfs/1889.out
PATCH 1/1 xfs: functional testing for filesystem properties
From: Darrick J. Wong <[email protected]>
Make sure that fs property storage and retrieval actually work.
--- common/config | 1 common/xfs | 14 ++++- doc/group-names.txt | 1 tests/generic/062 | 4 + tests/xfs/1886 | 137 +++++++++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1886.out | 53 ++++++++++++++++++++ tests/xfs/1887 | 122 +++++++++++++++++++++++++++++++++++++++++++++ tests/xfs/1887.out | 46 +++++++++++++++++ tests/xfs/1888 | 66 +++++++++++++++++++++++++ tests/xfs/1888.out | 9 +++ tests/xfs/1889 | 63 +++++++++++++++++++++++ tests/xfs/1889.out | 8 +++ 12 files changed, 522 insertions(+), 2 deletions(-) create mode 100755 tests/xfs/1886 create mode 100644 tests/xfs/1886.out create mode 100755 tests/xfs/1887 create mode 100644 tests/xfs/1887.out create mode 100755 tests/xfs/1888 create mode 100644 tests/xfs/1888.out create mode 100755 tests/xfs/1889 create mode 100644 tests/xfs/1889.out
common/config +1 -0
@@ -234,6 +234,7 @@ export GZIP_PROG="$(type -P gzip)" export BTRFS_IMAGE_PROG="$(type -P btrfs-image)" export BTRFS_MAP_LOGICAL_PROG=$(type -P btrfs-map-logical) export PARTED_PROG="$(type -P parted)" +export XFS_PROPERTY_PROG="$(type -P xfs_property)" # use 'udevadm settle' or 'udevsettle' to wait for lv to be settled. # newer systems have udevadm command but older systems like RHEL5 don't.
common/xfs +12 -2
@@ -1318,8 +1318,8 @@ _require_xfs_spaceman_command() testfile=$TEST_DIR/$$.xfs_spaceman touch $testfile case $command in - "health") - testio=`$XFS_SPACEMAN_PROG -c "health $param" $TEST_DIR 2>&1` + "health"|"listfsprops") + testio=`$XFS_SPACEMAN_PROG -c "$command $param" $TEST_DIR 2>&1` param_checked=1 ;; *)
@@ -1812,3 +1812,13 @@ _require_xfs_parent() || _notrun "kernel does not support parent pointers" _scratch_unmount } + +# Wipe all filesystem properties from an xfs filesystem. The sole argument +# must be the root directory of a filesystem. +_wipe_xfs_properties() +{ + getfattr --match="^trusted.xfs:" --absolute-names --dump --encoding=hex "$1" | \ + grep '=' | sed -e 's/=.*$//g' | while read name; do + setfattr --remove="$name" "$1" + done +}
doc/group-names.txt +1 -0
@@ -56,6 +56,7 @@ fiexchange XFS_IOC_EXCHANGE_RANGE ioctl freeze filesystem freeze tests fsck general fsck tests fsmap FS_IOC_GETFSMAP ioctl +fsproperties Filesystem properties fsr XFS free space reorganizer fuzzers filesystem fuzz tests growfs increasing the size of a filesystem
tests/generic/062 +4 -0
@@ -75,6 +75,10 @@ fi _require_attrs $ATTR_MODES +# Wipe all xfs filesystem properties (which are rootdir xattrs) before we dump +# them all later. +test $FSTYP = "xfs" && _wipe_xfs_properties $SCRATCH_MNT + for nsp in $ATTR_MODES; do for inode in reg dir lnk dev/b dev/c dev/p; do
tests/xfs/1886 new file +137 -0
137 lines not on this page · raw message
tests/xfs/1886.out new file +53 -0
53 lines not on this page · raw message
tests/xfs/1887 new file +122 -0
122 lines not on this page · raw message
tests/xfs/1887.out new file +46 -0
46 lines not on this page · raw message
tests/xfs/1888 new file +66 -0
66 lines not on this page · raw message
tests/xfs/1888.out new file +9 -0
@@ -0,0 +1,9 @@ +QA output created by 1888 +autofsck=repair +failed to get attr xfs:autofsck on inode XXX: No data available +autofsck=none +autofsck=check +autofsck=optimize +autofsck=repair +autofsck=none +autofsck=repair
tests/xfs/1889 new file +63 -0
63 lines not on this page · raw message
tests/xfs/1889.out new file +8 -0
@@ -0,0 +1,8 @@ +QA output created by 1889 +Info: TEST_DIR/1889.mount: Checking and repairing per autofsck directive. +Info: TEST_DIR/1889.mount: Disabling scrub per autofsck directive. +Info: TEST_DIR/1889.mount: Checking per autofsck directive. +Info: TEST_DIR/1889.mount: Optimizing per autofsck directive. +Info: TEST_DIR/1889.mount: Checking and repairing per autofsck directive. +Info: TEST_DIR/1889.mount: Disabling scrub per autofsck directive. +Info: TEST_DIR/1889.mount: Checking and repairing per autofsck directive.
Re: [PATCH 1/1] xfs: functional testing for filesystem properties
Looks good: