mkfs: Configuration file defined options
cover mkfs: Configuration file defined options
Hi Folks,
Because needing config files for mkfs came up yet again in discussion, here is a simple implementation of INI format config files. These config files behave identically to options specified on the command line - the do not change defaults, they do not override CLI options, they are not overridden by cli options.
Example:
$ echo -e "[metadata]\ncrc = 0" > foo
$ mkfs/mkfs.xfs -N -c file=foo -d file=1,size=100m blah
Ini debug: file foo, section metadata, name crc, value 0
Parameters parsed from config file foo successfully
meta-data=blah isize=256 agcount=4, agsize=6400 blks
= sectsz=512 attr=2, projid32bit=1
= crc=0 finobt=0, sparse=0, rmapbt=0
= reflink=0
data = bsize=4096 blocks=25600, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0, ftype=1
log =internal log bsize=4096 blocks=853, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
$And there's a V4 filesystem as specified by the option defined in the config file. If we do:
$ mkfs/mkfs.xfs -N -c file=foo -m crc=1 -d file=1,size=100m blah Ini debug: file foo, section metadata, name crc, value 0 -m crc option respecified Usage: mkfs.xfs ..... $
You can see it errors out because the CRC option was specified in both the config file and on the CLI.
There's lots of stuff we can do to make the conflict and respec error messages better, but that doesn't change the basic functionality of config file based mkfs options. To allow for future changes to the way we want to apply the config file, I created a full option subtype for config files. That means we can add another option to say "apply config file as default values rather than as options" if we decide that is functionality that we want to support.
But policy decisions like that are completely separate to the mechanism, so these patches don't try to address desires to ship "tuned" configs, system wide option files, shipping distro specific defaults in config files, etc. This is purely a mechanism to allow users to specify options via files instead of on the CLI. No more, no less.
So to preempt the endless bikeshedding from derailing getting this functionality merged, all I'm going to say is "no". No feature creep, no "but I want to ...", "can you make it pink...", nothing. All I care about is providing a mechanism that people are still asking for, and that's all I'm going to provide here.
I'm happy to fix bugs and code stuff that people want fixed to get it merged, but in terms of functionality being provided I'm essentially saying "take it or leave it" because I'm not going to waste time bikeshedding this whole thing again.
This has only been given a basic smoke test right now (see above! :). I need to get Darrick's tests from the previous round of config file bikeshedding working in my test environment to do more substantial testing of this....
Cheers,
Dave.
Dave Chinner (3):
build: add support for libinih for mkfs
mkfs: add initial ini format config file parsing support
mkfs: hook up suboption parsing to ini files
configure.ac | 3 + include/builddefs.in | 1 + include/linux.h | 2 +- m4/package_inih.m4 | 20 ++++ mkfs/Makefile | 2 +- mkfs/xfs_mkfs.c | 218 +++++++++++++++++++++++++++++++++++++------ 6 files changed, 218 insertions(+), 28 deletions(-) create mode 100644 m4/package_inih.m4
Signature · 1 line
2.28.0
PATCH 1/3 build: add support for libinih for mkfs
From: Dave Chinner <[email protected]>
Need to make sure the library is present so we can build mkfs with config file support.
--- configure.ac | 3 +++ include/builddefs.in | 1 + m4/package_inih.m4 | 20 ++++++++++++++++++++ 3 files changed, 24 insertions(+) create mode 100644 m4/package_inih.m4
configure.ac +3 -0
@@ -145,6 +145,9 @@ AC_PACKAGE_UTILITIES(xfsprogs) AC_MULTILIB($enable_lib64) AC_RT($enable_librt) +AC_PACKAGE_NEED_INI_H +AC_PACKAGE_NEED_LIBINIH + AC_PACKAGE_NEED_UUID_H AC_PACKAGE_NEED_UUIDCOMPARE
include/builddefs.in +1 -0
@@ -27,6 +27,7 @@ LIBTERMCAP = @libtermcap@ LIBEDITLINE = @libeditline@ LIBBLKID = @libblkid@ LIBDEVMAPPER = @libdevmapper@ +LIBINIH = @libinih@ LIBXFS = $(TOPDIR)/libxfs/libxfs.la LIBFROG = $(TOPDIR)/libfrog/libfrog.la LIBXCMD = $(TOPDIR)/libxcmd/libxcmd.la
m4/package_inih.m4 new file +20 -0
@@ -0,0 +1,20 @@ +AC_DEFUN([AC_PACKAGE_NEED_INI_H], + [ AC_CHECK_HEADERS([ini.h]) + if test $ac_cv_header_ini_h = no; then + echo + echo 'FATAL ERROR: could not find a valid ini.h header.' + echo 'Install the libinih development package.' + exit 1 + fi + ]) + +AC_DEFUN([AC_PACKAGE_NEED_LIBINIH], + [ AC_CHECK_LIB(inih, ini_parse,, [ + echo + echo 'FATAL ERROR: could not find a valid inih library.' + echo 'Install the libinih library package.' + exit 1 + ]) + libinih=-linih + AC_SUBST(libinih) + ])
Signature · 1 line
2.28.0
Re: [PATCH 1/3] build: add support for libinih for mkfs
Quoted 4 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Need to make sure the library is present so we can build mkfs with config file support.
Can you add https://github.com/benhoyt/inih to doc/INSTALL as a dependency?
(that's probably pretty out of date now anyway but it seems worth documenting any new requirement)
-Eric
Re: [PATCH 1/3] build: add support for libinih for mkfs
Quoted 8 lines · Eric Sandeen
Quoted 4 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Need to make sure the library is present so we can build mkfs with config file support.
Can you add https://github.com/benhoyt/inih to doc/INSTALL as a dependency?
Distros should be shipping that library, and configure asks you to install it if it isn't present, just like it does for all the other libraries xfsprogs depends on. We want users to install distro libraries, not have to build dependencies from source and install them...
Quoted 2 lines
(that's probably pretty out of date now anyway but it seems worth documenting any new requirement)
Yeah, we need a lot more than just e2fsprogs-devel and UUIDs these days. I'd prefer that doc/INSTALL just says "the configure script will prompt you to install missing build dependencies" rather than iterate them here and have the list be perpetually incomplete....
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
PATCH 2/3 mkfs: add initial ini format config file parsing support
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Add the framework that will allow the config file to be supplied on the CLI and passed to the library that will parse it. This does not yet do any option parsing from the config file.
--- mkfs/Makefile | 2 +- mkfs/xfs_mkfs.c | 101 +++++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 101 insertions(+), 2 deletions(-)
mkfs/Makefile +1 -1
@@ -11,7 +11,7 @@ HFILES = CFILES = proto.c xfs_mkfs.c LLDLIBS += $(LIBXFS) $(LIBXCMD) $(LIBFROG) $(LIBRT) $(LIBPTHREAD) $(LIBBLKID) \ - $(LIBUUID) + $(LIBUUID) $(LIBINIH) LTDEPENDENCIES += $(LIBXFS) $(LIBXCMD) $(LIBFROG) LLDFLAGS = -static-libtool-libs
mkfs/xfs_mkfs.c +100 -1
167 lines not on this page · raw message
Signature · 1 line
2.28.0
Re: [PATCH 2/3] mkfs: add initial ini format config file parsing support
Quoted 5 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Add the framework that will allow the config file to be supplied on the CLI and passed to the library that will parse it. This does not yet do any option parsing from the config file.
so we have "-c $SUBOPT=file"
From what I read in the cover letter, and from checking in IRC it seems like you envision the ability to also specify defaults from a config file in the future; to that end it might be better to name this $SUBOPT "options=" instead of "file=" as the latter is very generic.
Then in the future, we could have one or both of :
-c defaults=file1 -c options=file2
i.e. configure the defaults, then configure the options
I guess this is just RFC but you want probably to drop the "Ini debug:" printf eventually.
This will need a man page update, of course.
I think it should explain where "file" will be looked for; I assume it is either a full path, or a relative path to the current directory.
(In the future it would be nice to have mkfs.xfs search somewhere under /etc for these files as well, but I'm not bikeshedding!)
-Eric
Re: [PATCH 2/3] mkfs: add initial ini format config file parsing support
Quoted 14 lines · Eric Sandeen
Quoted 5 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Add the framework that will allow the config file to be supplied on the CLI and passed to the library that will parse it. This does not yet do any option parsing from the config file.
so we have "-c $SUBOPT=file"
From what I read in the cover letter, and from checking in IRC it seems like you envision the ability to also specify defaults from a config file in the future; to that end it might be better to name this $SUBOPT "options=" instead of "file=" as the latter is very generic.
Quoted 5 lines
Then in the future, we could have one or both of :
-c defaults=file1 -c options=file2
i.e. configure the defaults, then configure the options
Yup, makes sense. Will change.
Quoted 2 lines
I guess this is just RFC but you want probably to drop the "Ini debug:" printf eventually.
Yeah, I've already removed that so I can run fstests....
Quoted 1 line
This will need a man page update, of course.
Eventually, yes :P
Quoted 2 lines
I think it should explain where "file" will be looked for; I assume it is either a full path, or a relative path to the current directory.
It will work with either, just like all the other "file" parameters passed to mkfs....
Quoted 2 lines
(In the future it would be nice to have mkfs.xfs search somewhere under /etc for these files as well, but I'm not bikeshedding!)
Nope, I'm not doing that. Go away. :)
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
PATCH 3/3 mkfs: hook up suboption parsing to ini files
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Now we have the config file parsing hooked up and feeding in parameters to mkfs, wire the parameters up to the existing CLI option parsing functions. THis gives the config file exactly the same capabilities and constraints as the command line option specification.
--- include/linux.h | 2 +- mkfs/xfs_mkfs.c | 121 +++++++++++++++++++++++++++++++++++++----------- 2 files changed, 95 insertions(+), 28 deletions(-)
include/linux.h +1 -1
@@ -92,7 +92,7 @@ static __inline__ void platform_uuid_unparse(uuid_t *uu, char *buffer)uuid_unparse (*uu, buffer); } -static __inline__int platform_uuid_parse (char *buffer,uuid_t *uu) +static __inline__int platform_uuid_parse (const char *buffer,uuid_t *uu) {return uuid_parse (buffer, *uu); }
mkfs/xfs_mkfs.c +94 -27
291 lines not on this page · raw message
Signature · 1 line
2.28.0
Re: [PATCH 3/3] mkfs: hook up suboption parsing to ini files
Quoted 7 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Now we have the config file parsing hooked up and feeding in parameters to mkfs, wire the parameters up to the existing CLI option parsing functions. THis gives the config file exactly the same capabilities and constraints as the command line option specification.
And as such, as you already mentioned, respecifications on the command line will fail. That can be documented in the man page :)
The section names will need to be documented too.
(In a very much not-bikeshedding way, we could consider [b] rather than [block] so you can cover it with "the section names match the option characters, i.e. for -b one would use section name [b]" but I really don't care and will not mention this again because as long as it's documented it's fine.)
Quoted 24 lines
Signed-off-by Dave Chinner <[email protected]>--- include/linux.h | 2 +- mkfs/xfs_mkfs.c | 121 +++++++++++++++++++++++++++++++++++++----------- 2 files changed, 95 insertions(+), 28 deletions(-)include/linux.h +1
-1@@ -92,7 +92,7 @@ static __inline__ void platform_uuid_unparse(uuid_t *uu, char *buffer)uuid_unparse (*uu, buffer); } -static __inline__int platform_uuid_parse (char *buffer,uuid_t *uu) +static __inline__int platform_uuid_parse (const char *buffer,uuid_t *uu) {return uuid_parse (buffer, *uu); }mkfs/xfs_mkfs.c +0
-0
mkfs/xfs_mkfs.c enum { · line 150
Quoted 9 lines
mkfs/xfs_mkfs.c +6
-0@@ -142,6 +142,13 @@ enum {* name MANDATORY * Name is a single char, e.g., for '-d file', name is 'd'. * +* ini_name MANDATORY +* Name is a string, not longer than MAX_INI_NAME_LEN, that is used as the +* section name for this option set in INI format config files. The only +* option set this is not required for is the command line config file +* specification options, everything else must be configurable via config +* files.
Nothing actually enforces this MANDATORY, right, this is just documentation. So the fact that struct opt_params copts doesn't have it, is fine.
mkfs/xfs_mkfs.c respec( · line 988
Quoted 8 lines
mkfs/xfs_mkfs.c +2
-2@@ -967,13 +984,24 @@ respec(static void unknown ( -char opt, -char *s) +const char opt, +const char *s)
(can all of the constification could maybe go in its own patch just to cut down on the patch doomscrolling or does it have to go with the other changes?)
mkfs/xfs_mkfs.c respec( · line 1001
Quoted 13 lines
mkfs/xfs_mkfs.c +8
-0{fprintf (stderr,_ ("unknown option -%c %s\n" ), opt, s);usage (); } +static void +unknown_cfgfile_opt ( +const char *section, +const char *name, +const char *value) +{ +fprintf (stderr,_ ("unknown config file option: [%s]:%s=%s\n" ), + section, name, value);
If we allow more than one -c subopt in the future we might want to print the filename. Wouldn't /hurt/ to do so now, or just remember to do it if we ever add a 2nd -c subopt.
Quoted 16 lines
mkfs/xfs_mkfs.c +4
-1+usage (); +} + static void check_device_type( const char *name,@@ -1379,7 +1407,7 @@ getnum(*/ static char *getstr ( -char *str, +const char *str,struct opt_params *opts,int index) {
mkfs/xfs_mkfs.c getstr( · line 1419
Quoted 5 lines
mkfs/xfs_mkfs.c +1
-1@@ -1388,14 +1416,14 @@ getstr(/* empty strings for string options are not valid */ if (!str || *str =='\0' )reqval (opts->name, opts->subopts, index); -return str; +return (char *)str;
(what's the cast for?)
Quoted 37 lines
mkfs/xfs_mkfs.c +11
-12@@ -1682,23 +1710,22 @@ sector_opts_parser( }static struct subopts { -char opt;struct opt_params *opts;int (*parser)(struct opt_params *opts,int subopt, -char *value, +const char *value,struct cli_params *cli); } subopt_tab[] = { - {'b' , &bopts, block_opts_parser }, - {'c' , &copts, cfgfile_opts_parser }, - {'d' , &dopts, data_opts_parser }, - {'i' , &iopts, inode_opts_parser }, - {'l' , &lopts, log_opts_parser }, - {'m' , &mopts, meta_opts_parser }, - {'n' , &nopts, naming_opts_parser }, - {'r' , &ropts, rtdev_opts_parser }, - {'s' , &sopts, sector_opts_parser }, - {'\0' ,NULL ,NULL }, + { &bopts, block_opts_parser }, + { &copts, cfgfile_opts_parser }, + { &dopts, data_opts_parser }, + { &iopts, inode_opts_parser }, + { &lopts, log_opts_parser }, + { &mopts, meta_opts_parser }, + { &nopts, naming_opts_parser }, + { &ropts, rtdev_opts_parser }, + { &sopts, sector_opts_parser }, + {NULL ,NULL }, }; static void@@ -1712,12 +1739,12 @@ parse_subopts(int ret =0 ;
mkfs/xfs_mkfs.c parse_subopts( · line 1747
Quoted 10 lines
mkfs/xfs_mkfs.c +2
-2while (sop->opts) { -if (sop->opt == opt) +if (opt && sop->opts->name == opt)break ; sop++; } -/* should never happen */ +/* Should not happen */
ok? :)
Overall this seems remarkably tidy, thanks.
-Eric
Re: [PATCH 3/3] mkfs: hook up suboption parsing to ini files
Quoted 12 lines · Eric Sandeen
Quoted 7 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Now we have the config file parsing hooked up and feeding in parameters to mkfs, wire the parameters up to the existing CLI option parsing functions. THis gives the config file exactly the same capabilities and constraints as the command line option specification.
And as such, as you already mentioned, respecifications on the command line will fail. That can be documented in the man page :)
Quoted 7 lines
The section names will need to be documented too.
(In a very much not-bikeshedding way, we could consider [b] rather than [block] so you can cover it with "the section names match the option characters, i.e. for -b one would use section name [b]" but I really don't care and will not mention this again because as long as it's documented it's fine.)
No, I want the ini file to be human readable. The other thing to consider here is that this is the equivalent of the long option CLI parameter definition (i.e. -b size=foo vs --block="size=foo"). They'll get documented in the man page....
Quoted 39 lines
Quoted 24 lines
Signed-off-by Dave Chinner <[email protected]>--- include/linux.h | 2 +- mkfs/xfs_mkfs.c | 121 +++++++++++++++++++++++++++++++++++++----------- 2 files changed, 95 insertions(+), 28 deletions(-)include/linux.h +1
-1@@ -92,7 +92,7 @@ static __inline__ void platform_uuid_unparse(uuid_t *uu, char *buffer)uuid_unparse (*uu, buffer); } -static __inline__int platform_uuid_parse (char *buffer,uuid_t *uu) +static __inline__int platform_uuid_parse (const char *buffer,uuid_t *uu) {return uuid_parse (buffer, *uu); }mkfs/xfs_mkfs.c +0
-0mkfs/xfs_mkfs.c enum { · line 150
Quoted 9 lines
mkfs/xfs_mkfs.c +6
-0@@ -142,6 +142,13 @@ enum {* name MANDATORY * Name is a single char, e.g., for '-d file', name is 'd'. * +* ini_name MANDATORY +* Name is a string, not longer than MAX_INI_NAME_LEN, that is used as the +* section name for this option set in INI format config files. The only +* option set this is not required for is the command line config file +* specification options, everything else must be configurable via config +* files. Nothing actually enforces this MANDATORY, right, this is just documentation. So the fact that struct
opt_paramscopts doesn't have it, is fine.Quoted 2 lines
mkfs/xfs_mkfs.c +0
-0@@ -967,13 +984,24 @@ respec(
Quoted 9 lines
mkfs/xfs_mkfs.c respec( · line 988
Quoted 6 lines
mkfs/xfs_mkfs.c +2
-2static void unknown ( -char opt, -char *s) +const char opt, +const char *s)(can all of the constification could maybe go in its own patch just to cut down on the patch doomscrolling or does it have to go with the other changes?)
It is a result of the ini parser callback using "const char *" for it's variables. I just whack-a-moled it to get it compile.
Quoted 5 lines
Quoted 5 lines
mkfs/xfs_mkfs.c +0
-0{fprintf (stderr,_ ("unknown option -%c %s\n" ), opt, s);usage (); }
Quoted 12 lines
mkfs/xfs_mkfs.c respec( · line 1001
Quoted 8 lines
mkfs/xfs_mkfs.c +8
-0+static void +unknown_cfgfile_opt ( +const char *section, +const char *name, +const char *value) +{ +fprintf (stderr,_ ("unknown config file option: [%s]:%s=%s\n" ), + section, name, value);If we allow more than one -c subopt in the future we might want to print the filename. Wouldn't /hurt/ to do so now, or just remember to do it if we ever add a 2nd -c subopt.
OK.
Quoted 16 lines
Quoted 16 lines
mkfs/xfs_mkfs.c +4
-1+usage (); +} + static void check_device_type( const char *name,@@ -1379,7 +1407,7 @@ getnum(*/ static char *getstr ( -char *str, +const char *str,struct opt_params *opts,int index) {
Quoted 8 lines
mkfs/xfs_mkfs.c getstr( · line 1419
Quoted 6 lines
mkfs/xfs_mkfs.c +1
-1@@ -1388,14 +1416,14 @@ getstr(/* empty strings for string options are not valid */ if (!str || *str =='\0' )reqval (opts->name, opts->subopts, index); -return str; +return (char *)str;(what's the cast for?)
Because I got tired of whack-a-mole and these returned strings are stored in the cli parameter structure and I didn't feel like chasing the const rabbit down that hole.
i.e. the const on the input parameter goes as far as the input parsing to verify it runs, and the verified input string that is returned and stored is no longer const. Perhaps it would be better to just strdup the string here and return that instead?
Quoted 45 lines
Quoted 45 lines
mkfs/xfs_mkfs.c +12
-13@@ -1682,23 +1710,22 @@ sector_opts_parser( }static struct subopts { -char opt;struct opt_params *opts;int (*parser)(struct opt_params *opts,int subopt, -char *value, +const char *value,struct cli_params *cli); } subopt_tab[] = { - {'b' , &bopts, block_opts_parser }, - {'c' , &copts, cfgfile_opts_parser }, - {'d' , &dopts, data_opts_parser }, - {'i' , &iopts, inode_opts_parser }, - {'l' , &lopts, log_opts_parser }, - {'m' , &mopts, meta_opts_parser }, - {'n' , &nopts, naming_opts_parser }, - {'r' , &ropts, rtdev_opts_parser }, - {'s' , &sopts, sector_opts_parser }, - {'\0' ,NULL ,NULL }, + { &bopts, block_opts_parser }, + { &copts, cfgfile_opts_parser }, + { &dopts, data_opts_parser }, + { &iopts, inode_opts_parser }, + { &lopts, log_opts_parser }, + { &mopts, meta_opts_parser }, + { &nopts, naming_opts_parser }, + { &ropts, rtdev_opts_parser }, + { &sopts, sector_opts_parser }, + {NULL ,NULL }, }; static void@@ -1712,12 +1739,12 @@ parse_subopts(int ret =0 ;while (sop->opts) { -if (sop->opt == opt) +if (opt && sop->opts->name == opt)break ; sop++; }
Quoted 4 lines
mkfs/xfs_mkfs.c parse_subopts( · line 1747
Quoted 2 lines
mkfs/xfs_mkfs.c +1
-1-/* should never happen */ +/* Should not happen */ ok? :)
Ah, I modified this function first, then realised that it would be better to keep the subopt table iteration separate because we didn't need to use getsubopt() to separate "name=value" strings. I'll clean that up.
Quoted 1 line
Overall this seems remarkably tidy, thanks.
I've been saying config file parsing is simple and easy to fit into the existing option parsing infrastructure for a long time. :)
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]