xfsprogs: add mkfs.xfs configuration file parsing support
v2 cover xfsprogs: add mkfs.xfs configuration file parsing support
Eric,
This v2 series addresses the feedback from the last single patch. The syntax is kept from what we last discussed, I think we're done bikeshedding there. As a compromise we keep the bools to start off with as recently discussed on the mailing list.
This is also availabe in git form on my git.kernel.org xfsprogs-dev tree on the 20180517-own-parser branch [0].
Lemme know what you'all think.
[0] https://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/xfsprogs-dev.git/log/?h=20180517-own-parser
Luis
Chnages from v1:
- Stayed with our own parser as its the smallest and we're willing to maintain it as its simple and clear.
- Use -c for the configuration file, and drop the "type" nomenclature to avoid confusion on the interwebs.
- Start to split files off
- Duplicate a bit of items as suggested at LSFMM for the configuration parser structures. We can later consolidate if we think its really needed, however we want the freedom to change these as we see fit and most importantly keep the code apart.
- Conflict resolution and validation is managed now by piggy backing off of the idea of using the defaults to instantiate CLI parameters. CLI always overrides.
Luis R. Rodriguez (5):
mkfs: distinguish between struct sb_feat_args and struct cli_params
mkfs: move shared structs and cli params into their own headers
mkfs: replace defaults source with an enum
mkfs: add helpers to process defaults
mkfs.xfs: add configuration file parsing support using our own parser
include/builddefs.in | 2 + man/man5/mkfs.xfs.d.5 | 121 ++++++++++ man/man8/mkfs.xfs.8 | 26 +++ mkfs/Makefile | 2 +- mkfs/xfs_mkfs.c | 235 ++++++++------------ mkfs/xfs_mkfs_cli.h | 65 ++++++ mkfs/xfs_mkfs_common.h | 87 ++++++++ mkfs/xfs_mkfs_config.c | 591 +++++++++++++++++++++++++++++++++++++++++++++++++ mkfs/xfs_mkfs_config.h | 11 + 9 files changed, 1002 insertions(+), 138 deletions(-) create mode 100644 man/man5/mkfs.xfs.d.5 create mode 100644 mkfs/xfs_mkfs_cli.h create mode 100644 mkfs/xfs_mkfs_common.h create mode 100644 mkfs/xfs_mkfs_config.c create mode 100644 mkfs/xfs_mkfs_config.h
Signature · 1 line
2.16.3
PATCH v2 1/5 mkfs: distinguish between struct sb_feat_args and struct cli_params
The struct sb_feat_args will actually be shared between the code which processes command line options and the configuration file, as such we need to clarify and reflect this clearly in documentation.
--- mkfs/xfs_mkfs.c | 45 ++++++++++++++++++++++++++++----------------- 1 file changed, 28 insertions(+), 17 deletions(-)
mkfs/xfs_mkfs.c +28 -17
@@ -707,26 +707,18 @@ cli_opt_set( }/* -* Options configured on the command line. -* -* This stores all the specific config parameters the user sets on the command -* line. We do not use these values directly - they are inputs to the mkfs -* geometry validation and override any default configuration value we have. +* Shared superblock configuration options * -* We don't keep flags to indicate what parameters are set - if we need to check -* if an option was set on the command line, we check the relevant entry in the -* option table which records whether it was specified in the .seen and -* .str_seen variables in the table. +* These options provide shared configuration tunables for the filesystem +* superblock. There are three possible sources for these options set, each +* source can overriding the later source: * -* Some parameters are stored as strings for post-parsing after their dependent -* options have been resolved (e.g. block size and sector size have been parsed -* and validated). +* o built-in defaults +* o configuration file (XXX) +* o command line * -* This allows us to check that values have been set without needing separate -* flags for each value, and hence avoids needing to record and check for each -* specific option that can set the value later on in the code. In the cases -* where we don't have a cli_params structure around, the above cli_opt_set() -* function can be used. +* These values are not used directly - they are inputs into the mkfs geometry +* validation. */ struct sb_feat_args {int log_version;
@@ -747,6 +739,25 @@ struct sb_feat_args {bool nortalign; }; +/* +* Options configured on the command line. +* +* This stores all the specific config parameters the user sets on the command +* line. We don't keep flags to indicate what parameters are set - if we need +* to check if an option was set on the command line, we check the relevant +* entry in the option table which records whether it was specified in the +* .seen and .str_seen variables in the table. +* +* Some parameters are stored as strings for post-parsing after their dependent +* options have been resolved (e.g. block size and sector size have been parsed +* and validated). +* +* This allows us to check that values have been set without needing separate +* flags for each value, and hence avoids needing to record and check for each +* specific option that can set the value later on in the code. In the cases +* where we don't have a cli_params structure around, the function cli_opt_set() +* function can be used. +*/ struct cli_params {int sectorsize;int blocksize;
Signature · 1 line
2.16.3
Re: [PATCH v2 1/5] mkfs: distinguish between struct sb_feat_args and struct cli_params
Quoted 5 lines · Luis R. Rodriguez
The struct
sb_feat_argswill actually be shared between the code which processes command line options and the configuration file, as such we need to clarify and reflect this clearly in documentation.Signed-off-by Luis R. Rodriguez <[email protected]>
Sensible.
Signature · 2 lines
Dave Chinner [email protected]
PATCH v2 2/5 mkfs: move shared structs and cli params into their own headers
In reply to Luis R. RodriguezBoth struct sb_feat_args and struct mkfs_default_params will be shared between CLI processing and the configuration file processing added later, so move these to their own header. The struct cli_params are CLI specific so move them to its own CLI header as well.
This will help ensure we split things neatly later and also will help ensure the configuration file processing code from the CLI code are kept separate and cannot touch each other's data structures. This also makes it clear what is actually shared between both.
There are no introduced functional changes in this commit and no documentation changes, this is just code shuffling.
--- mkfs/xfs_mkfs.c | 115 +------------------------------------------------ mkfs/xfs_mkfs_cli.h | 65 ++++++++++++++++++++++++++++ mkfs/xfs_mkfs_common.h | 64 +++++++++++++++++++++++++++ 3 files changed, 131 insertions(+), 113 deletions(-) create mode 100644 mkfs/xfs_mkfs_cli.h create mode 100644 mkfs/xfs_mkfs_common.h
mkfs/xfs_mkfs.c +2 -113
@@ -20,6 +20,8 @@#include <ctype.h> #include "xfs_multidisk.h" #include "libxcmd.h" +#include "xfs_mkfs_common.h" +#include "xfs_mkfs_cli.h"
@@ -706,98 +708,6 @@ cli_opt_set( opts->subopt_params[subopt].str_seen; } -/* -* Shared superblock configuration options -* -* These options provide shared configuration tunables for the filesystem -* superblock. There are three possible sources for these options set, each -* source can overriding the later source: -* -* o built-in defaults -* o configuration file (XXX) -* o command line -* -* These values are not used directly - they are inputs into the mkfs geometry -* validation. -*/ -struct sb_feat_args { -int log_version; -int attr_version; -int dir_version; -bool inode_align;/* XFS_SB_VERSION_ALIGNBIT */ -bool nci;/* XFS_SB_VERSION_BORGBIT */ -bool lazy_sb_counters;/* XFS_SB_VERSION2_LAZYSBCOUNTBIT */ -bool parent_pointers;/* XFS_SB_VERSION2_PARENTBIT */ -bool projid32bit;/* XFS_SB_VERSION2_PROJID32BIT */ -bool crcs_enabled;/* XFS_SB_VERSION2_CRCBIT */ -bool dirftype;/* XFS_SB_VERSION2_FTYPE */ -bool finobt;/* XFS_SB_FEAT_RO_COMPAT_FINOBT */ -bool spinodes;/* XFS_SB_FEAT_INCOMPAT_SPINODES */ -bool rmapbt;/* XFS_SB_FEAT_RO_COMPAT_RMAPBT */ -bool reflink;/* XFS_SB_FEAT_RO_COMPAT_REFLINK */ -bool nodalign; -bool nortalign; -}; - -/* -* Options configured on the command line. -* -* This stores all the specific config parameters the user sets on the command -* line. We don't keep flags to indicate what parameters are set - if we need -* to check if an option was set on the command line, we check the relevant -* entry in the option table which records whether it was specified in the -* .seen and .str_seen variables in the table. -* -* Some parameters are stored as strings for post-parsing after their dependent -* options have been resolved (e.g. block size and sector size have been parsed -* and validated). -* -* This allows us to check that values have been set without needing separate -* flags for each value, and hence avoids needing to record and check for each -* specific option that can set the value later on in the code. In the cases -* where we don't have a cli_params structure around, the function cli_opt_set() -* function can be used. -*/ -struct cli_params { -int sectorsize; -int blocksize; - -/* parameters that depend on sector/block size being validated. */ -char *dsize; -char *agsize; -char *dsu; -char *dirblocksize; -char *logsize; -char *lsu; -char *rtextsize; -char *rtsize; - -/* parameters where 0 is a valid CLI value */ -int dsunit; -int dswidth; -int dsw; -int64_t logagno; -int loginternal; -int lsunit; - -/* parameters where 0 is not a valid value */ -int64_t agcount; -int inodesize; -int inopblock; -int imaxpct; -int lsectorsize; -uuid_t uuid; - -/* feature flags that are set */ -struct sb_feat_args sb_feat; - -/* root inode characteristics */ -struct fsxattr fsx; - -/* libxfs device setup */ -struct libxfs_xinit *xi; -}; -/* * Calculated filesystem feature and geometry information. *
@@ -850,27 +760,6 @@ struct mkfs_params {struct sb_feat_args sb_feat; }; -/* -* Default filesystem features and configuration values -* -* This structure contains the default mkfs values that are to be used when -* a user does not specify the option on the command line. We do not use these -* values directly - they are inputs to the mkfs geometry validation and -* calculations. -*/ -struct mkfs_default_params { -char *source;/* where the defaults came from */ - -int sectorsize; -int blocksize; - -/* feature flags that are set */ -struct sb_feat_args sb_feat; - -/* root inode characteristics */ -struct fsxattr fsx; -}; -static void __attribute__((noreturn))usage (void ) {
mkfs/xfs_mkfs_cli.h new file +65 -0
65 lines not on this page · raw message
mkfs/xfs_mkfs_common.h new file +64 -0
64 lines not on this page · raw message
Signature · 1 line
2.16.3
Re: [PATCH v2 2/5] mkfs: move shared structs and cli params into their own headers
Quoted 4 lines · Luis R. Rodriguez
Both struct
sb_feat_argsand structmkfs_default_paramswill be shared between CLI processing and the configuration file processing added later, so move these to their own header. The structcli_paramsare CLI specific so move them to its own CLI header as well.
I'd separate out the CLI details when the CLI parsing is split into it's own file - it's not necssary to do that here.
Quoted 6 lines
This will help ensure we split things neatly later and also will help ensure the configuration file processing code from the CLI code are kept separate and cannot touch each other's data structures. This also makes it clear what is actually shared between both.
Quoted 8 lines
There are no introduced functional changes in this commit and no documentation changes, this is just code shuffling.
Signed-off-by Luis R. Rodriguez <[email protected]>--- mkfs/xfs_mkfs.c | 115 +------------------------------------------------ mkfs/xfs_mkfs_cli.h | 65 ++++++++++++++++++++++++++++ mkfs/xfs_mkfs_common.h | 64 +++++++++++++++++++++++++++
File names.
xfs_mkfs.c is named that way by an old convention - it's the same as xfs_repair.c, xfs_db.c, etc. Every other file in the directory does not need a "xfs_mkfs_" prefix.
I'd also prefer we don't have "common" as a dumping ground header. The CLI, config files, etc are all part of config processing, so config.h would be more appropriate here. Or perhaps "input.h" because they are both processing external inputs....
Quoted 9 lines
3 files changed, 131 insertions(+), 113 deletions(-) create mode 100644 mkfs/xfs_mkfs_cli.h create mode 100644 mkfs/xfs_mkfs_common.hmkfs/xfs_mkfs.c +0
-0
mkfs/xfs_mkfs.c line 24
Quoted 5 lines
mkfs/xfs_mkfs.c +2
-0@@ -20,6 +20,8 @@#include <ctype.h> #include "xfs_multidisk.h" #include "libxcmd.h" +#include "xfs_mkfs_common.h" +#include "xfs_mkfs_cli.h"
#include "config.h"
mkfs/xfs_mkfs_cli.h line 2
Quoted 8 lines
mkfs/xfs_mkfs_cli.h new file +2
-0@@ -0,0 +1,65 @@ +#ifndef _XFS_MKFS_CLI_H +#define _XFS_MKFS_CLI_H
Missing license and copyright. It'll be the same license as the main xfs_mkfs.c file. Copyright will be Red Hat, Inc (because it's new code I wrote as [email protected]) and you'll need to pull the date from the commit message.
(Yeah, I know it's weird putting someone elses copyright on code you're moving about, but that's how it goes :P)
mkfs/xfs_mkfs_cli.h line 4
Quoted 2 lines
mkfs/xfs_mkfs_cli.h new file +2
-0+ +#include "xfs_mkfs_common.h"
No includes in header files if we can avoid them.
[...]
mkfs/xfs_mkfs_common.h line 6
Quoted 12 lines
mkfs/xfs_mkfs_common.h new file +6
-0@@ -0,0 +1,64 @@ +#ifndef _XFS_MKFS_COMMON_H +#define _XFS_MKFS_COMMON_H + +#include "libxfs.h" + +#include <ctype.h>
Same as above for license, copyright and includes, except the copyright will also need to include the SGI copyright line from xfs_mkfs.c...
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Re: [PATCH v2 2/5] mkfs: move shared structs and cli params into their own headers
Quoted 8 lines · Dave Chinner
Quoted 4 lines · Luis R. Rodriguez
Both struct
sb_feat_argsand structmkfs_default_paramswill be shared between CLI processing and the configuration file processing added later, so move these to their own header. The structcli_paramsare CLI specific so move them to its own CLI header as well.I'd separate out the CLI details when the CLI parsing is split into it's own file - it's not necssary to do that here.
Alright.
Quoted 14 lines
Quoted 5 lines
This will help ensure we split things neatly later and also will help ensure the configuration file processing code from the CLI code are kept separate and cannot touch each other's data structures. This also makes it clear what is actually shared between both.
Quoted 8 lines
There are no introduced functional changes in this commit and no documentation changes, this is just code shuffling.
Signed-off-by Luis R. Rodriguez <[email protected]>--- mkfs/xfs_mkfs.c | 115 +------------------------------------------------ mkfs/xfs_mkfs_cli.h | 65 ++++++++++++++++++++++++++++ mkfs/xfs_mkfs_common.h | 64 +++++++++++++++++++++++++++
Quoted 5 lines
File names.
xfs_mkfs.cis named that way by an old convention - it's the same asxfs_repair.c,xfs_db.c, etc. Every other file in the directory does not need a "xfs_mkfs_" prefix.
Yay.
Quoted 1 line
I'd also prefer we don't have "common" as a dumping ground header.
input.h it is.
Quoted 3 lines
The CLI, config files, etc are all part of config processing, so
config.hwould be more appropriate here. Or perhaps "input.h" because they are both processing external inputs....
Alrighty.
Quoted 4 lines
Quoted 4 lines
3 files changed, 131 insertions(+), 113 deletions(-) create mode 100644 mkfs/xfs_mkfs_cli.h create mode 100644 mkfs/xfs_mkfs_common.h
Quoted 12 lines
mkfs/xfs_mkfs.c line 24
Quoted 10 lines
mkfs/xfs_mkfs.c +2
-0@@ -20,6 +20,8 @@#include <ctype.h> #include "xfs_multidisk.h" #include "libxcmd.h" +#include "xfs_mkfs_common.h" +#include "xfs_mkfs_cli.h" #include "
config.h"
You mean input.h ? The config.h would be for the configuration file, no?
Quoted 14 lines
mkfs/xfs_mkfs_cli.h line 2
Quoted 8 lines
mkfs/xfs_mkfs_cli.h new file +2
-0@@ -0,0 +1,65 @@ +#ifndef _XFS_MKFS_CLI_H +#define _XFS_MKFS_CLI_HMissing license and copyright. It'll be the same license as the main
xfs_mkfs.cfile. Copyright will be Red Hat, Inc (because it's new code I wrote as [email protected]) and you'll need to pull the date from the commit message.
Quoted 2 lines
(Yeah, I know it's weird putting someone elses copyright on code you're moving about, but that's how it goes :P)
I'll ignore cli.h for now. The comment below applies to the future input.h (currently the xfs_mkfs_common.h)
Both data structures on input.h (sb features and the defaults) were introduced in 2017 so using that. Is the following header OK?
/*
* Copyright (c) 2017 Red Hat, Inc.
* All Rights Reserved.
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License as
* published by the Free Software Foundation.
*
* This program is distributed in the hope that it would be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write the Free Software Foundation,
* Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA */
Took it as template from a random existing header file.
I'd prefer if we just went with this as the last paragraph:
* You should have received a copy of the GNU General Public License
* along with this program; if not, see <http://www.gnu.org/licenses/>.
It used in the kernel as well, and checkpatch now asks you to consider this.
Lemme know your preference.
Quoted 4 lines
mkfs/xfs_mkfs_cli.h line 4
Quoted 2 lines
mkfs/xfs_mkfs_cli.h new file +2
-0+ +#include "xfs_mkfs_common.h" No includes in header files if we can avoid them.
OK! This cli.h file will not be added in my next iteration though. I'll avoid any header file not needed though.
Quoted 18 lines
[...]
mkfs/xfs_mkfs_common.h line 6
Quoted 12 lines
mkfs/xfs_mkfs_common.h new file +6
-0@@ -0,0 +1,64 @@ +#ifndef _XFS_MKFS_COMMON_H +#define _XFS_MKFS_COMMON_H + +#include "libxfs.h" + +#include <ctype.h> Same as above for license, copyright and includes, except the copyright will also need to include the SGI copyright line from
xfs_mkfs.c...
Why SGI? I don't see anything from SGI on input.h.
Luis
Re: [PATCH v2 2/5] mkfs: move shared structs and cli params into their own headers
Quoted 61 lines · Luis R. Rodriguez
Quoted 8 lines · Dave Chinner
Quoted 4 lines · Luis R. Rodriguez
Both struct
sb_feat_argsand structmkfs_default_paramswill be shared between CLI processing and the configuration file processing added later, so move these to their own header. The structcli_paramsare CLI specific so move them to its own CLI header as well.I'd separate out the CLI details when the CLI parsing is split into it's own file - it's not necssary to do that here.
Alright.
Quoted 14 lines
Quoted 5 lines
This will help ensure we split things neatly later and also will help ensure the configuration file processing code from the CLI code are kept separate and cannot touch each other's data structures. This also makes it clear what is actually shared between both.
Quoted 8 lines
There are no introduced functional changes in this commit and no documentation changes, this is just code shuffling.
Signed-off-by Luis R. Rodriguez <[email protected]>--- mkfs/xfs_mkfs.c | 115 +------------------------------------------------ mkfs/xfs_mkfs_cli.h | 65 ++++++++++++++++++++++++++++ mkfs/xfs_mkfs_common.h | 64 +++++++++++++++++++++++++++Quoted 5 lines
File names.
xfs_mkfs.cis named that way by an old convention - it's the same asxfs_repair.c,xfs_db.c, etc. Every other file in the directory does not need a "xfs_mkfs_" prefix.Yay.
Quoted 1 line
I'd also prefer we don't have "common" as a dumping ground header.
input.hit is.Quoted 3 lines
The CLI, config files, etc are all part of config processing, so
config.hwould be more appropriate here. Or perhaps "input.h" because they are both processing external inputs....Alrighty.
Quoted 4 lines
Quoted 4 lines
3 files changed, 131 insertions(+), 113 deletions(-) create mode 100644 mkfs/xfs_mkfs_cli.h create mode 100644 mkfs/xfs_mkfs_common.hQuoted 12 lines
mkfs/xfs_mkfs.c line 24
Quoted 10 lines
mkfs/xfs_mkfs.c +2
-0@@ -20,6 +20,8 @@#include <ctype.h> #include "xfs_multidisk.h" #include "libxcmd.h" +#include "xfs_mkfs_common.h" +#include "xfs_mkfs_cli.h" #include "
config.h"
Quoted 1 line
You mean
input.h? Theconfig.hwould be for the configuration file, no?
I suggested both config.h and input.h as potential candidates. Given that you use config.h in a later patch, i'd suggest that config.h is the right name for this.
Quoted 25 lines
Both data structures on
input.h(sb features and the defaults) were introduced in 2017 so using that. Is the following header OK?/*
* Copyright (c) 2017 Red Hat, Inc.
* All Rights Reserved.
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License as
* published by the Free Software Foundation.
*
* This program is distributed in the hope that it would be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write the Free Software Foundation,
* Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA */Took it as template from a random existing header file.
I'd prefer if we just went with this as the last paragraph:
Quoted 7 lines
* You should have received a copy of the GNU General Public License
* along with this program; if not, see <http://www.gnu.org/licenses/>.It used in the kernel as well, and checkpatch now asks you to consider this.
Lemme know your preference.
Use the same as existing files. If we're going to update the licensing info, then we need to do it for everything, move to spdx identifiers in the code and use a single copy of each license in the LICENSE file. That's for another day, though....
Quoted 12 lines
Quoted 11 lines
mkfs/xfs_mkfs.c line 6
Quoted 7 lines
mkfs/xfs_mkfs.c +6
-0@@ -0,0 +1,64 @@ +#ifndef _XFS_MKFS_COMMON_H +#define _XFS_MKFS_COMMON_H + +#include "libxfs.h" + +#include <ctype.h> Same as above for license, copyright and includes, except the copyright will also need to include the SGI copyright line from
xfs_mkfs.c...
Quoted 1 line
Why SGI? I don't see anything from SGI on
input.h.
Because the sb_feats structure is derived from much older code abstractions - it pre-existed all refactoring work I did....
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Re: [PATCH v2 2/5] mkfs: move shared structs and cli params into their own headers
Quoted 14 lines · Dave Chinner
Quoted 13 lines · Luis R. Rodriguez
Quoted 12 lines
mkfs/xfs_mkfs.c line 24
Quoted 10 lines
mkfs/xfs_mkfs.c +2
-0@@ -20,6 +20,8 @@#include <ctype.h> #include "xfs_multidisk.h" #include "libxcmd.h" +#include "xfs_mkfs_common.h" +#include "xfs_mkfs_cli.h" #include "
config.h"
Quoted 5 lines
Quoted 1 line
You mean
input.h? Theconfig.hwould be for the configuration file, no?I suggested both
config.handinput.has potential candidates. Given that you useconfig.hin a later patch, i'd suggest thatconfig.his the right name for this.
And the configuration header name?
Luis
PATCH v2 3/5 mkfs: replace defaults source with an enum
In reply to Luis R. RodriguezUsing an enum will let us later just use a switch statement to print out the source, this makes sources easier to document, update and manage.
--- mkfs/xfs_mkfs.c | 5 +++-- mkfs/xfs_mkfs_common.h | 13 ++++++++++++- 2 files changed, 15 insertions(+), 3 deletions(-)
mkfs/xfs_mkfs.c +3 -2
@@ -3697,7 +3697,7 @@ main(/* build time defaults */ struct mkfs_default_params dft = { - .source =_ ("package build definitions" ), + .type =DEFAULTS_BUILTIN , .sectorsize =XFS_MIN_SECTORSIZE , .blocksize =1 <<XFS_DFL_BLOCKSIZE_LOG , .sb_feat = {
@@ -3737,7 +3737,8 @@ main(* implemented, emit a message to indicate where the defaults being * used came from. * -* printf(_("Default configuration sourced from %s\n"), dft.source); +* printf(_("Default configuration sourced from %s\n"), +* default_type_str(dft.type)); */ /* copy new defaults into CLI parsing structure */
mkfs/xfs_mkfs_common.h +12 -1
@@ -40,6 +40,17 @@ struct sb_feat_args {bool nortalign; }; +/* +* File configuration type settings +* +* These are the different possibilities by which you can end up parsing +* default settings with. DEFAULTS_BUILTIN indicates there was no configuration +* file parsed and we are using the built-in defaults on this code. +*/ +enum default_params_type { +DEFAULTS_BUILTIN =0 , +}; +/* * Default filesystem features and configuration values *
@@ -49,7 +60,7 @@ struct sb_feat_args {* calculations. */ struct mkfs_default_params { -char *source;/* where the defaults came from */ +enum default_params_type type;/* where the defaults came from */ int sectorsize;int blocksize;
Signature · 1 line
2.16.3
Re: [PATCH v2 3/5] mkfs: replace defaults source with an enum
Quoted 5 lines · Luis R. Rodriguez
Using an enum will let us later just use a switch statement to print out the source, this makes sources easier to document, update and manage.
Signed-off-by Luis R. Rodriguez <[email protected]>
This is incomplete. :(
Quoted 19 lines
--- mkfs/xfs_mkfs.c | 5 +++-- mkfs/xfs_mkfs_common.h | 13 ++++++++++++- 2 files changed, 15 insertions(+), 3 deletions(-)mkfs/xfs_mkfs.c +1
-1@@ -3697,7 +3697,7 @@ main(/* build time defaults */ struct mkfs_default_params dft = { - .source =_ ("package build definitions" ), + .type =DEFAULTS_BUILTIN , .sectorsize =XFS_MIN_SECTORSIZE , .blocksize =1 <<XFS_DFL_BLOCKSIZE_LOG , .sb_feat = {
mkfs/xfs_mkfs.c main( · line 3741
Quoted 6 lines
mkfs/xfs_mkfs.c +2
-1@@ -3737,7 +3737,8 @@ main(* implemented, emit a message to indicate where the defaults being * used came from. * -* printf(_("Default configuration sourced from %s\n"), dft.source); +* printf(_("Default configuration sourced from %s\n"), +* default_type_str(dft.type));
This function does not exist in the patch. If you are going to add functionality, first turn on that functionality so you can test the patch actually works...
Quoted 11 lines
mkfs/xfs_mkfs.c +0
-0*/ /* copy new defaults into CLI parsing structure */ mkfs/xfs_mkfs_common.h +0
-0@@ -40,6 +40,17 @@ struct sb_feat_args {bool nortalign; };
mkfs/xfs_mkfs_common.h struct sb_feat_args { · line 52
Quoted 10 lines
mkfs/xfs_mkfs_common.h +10
-0+/* +* File configuration type settings +* +* These are the different possibilities by which you can end up parsing +* default settings with. DEFAULTS_BUILTIN indicates there was no configuration +* file parsed and we are using the built-in defaults on this code. +*/ +enum default_params_type { +DEFAULTS_BUILTIN =0 , +};
Please add all the new types here, the functions to print the names, etc.
mkfs/xfs_mkfs_common.h struct sb_feat_args { · line 66
Quoted 12 lines
mkfs/xfs_mkfs_common.h +1
-1/* * Default filesystem features and configuration values * @@ -49,7 +60,7 @@ struct sb_feat_args {* calculations. */ struct mkfs_default_params { -char *source;/* where the defaults came from */ +enum default_params_type type;/* where the defaults came from */ int sectorsize;int blocksize;
As it is, I don't see why this change it necessary - you can just store the appropriate string (as the code currently does) into the structure once the source is known. Why do we need infrastructure to abstract printing a string when we set it directly, anyway?
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Re: [PATCH v2 3/5] mkfs: replace defaults source with an enum
Quoted 8 lines · Dave Chinner
Quoted 5 lines · Luis R. Rodriguez
Using an enum will let us later just use a switch statement to print out the source, this makes sources easier to document, update and manage.
Signed-off-by Luis R. Rodriguez <[email protected]>This is incomplete. :(
It builds, the comment was intentional.
Quoted 21 lines
Quoted 21 lines
--- mkfs/xfs_mkfs.c | 5 +++-- mkfs/xfs_mkfs_common.h | 13 ++++++++++++- 2 files changed, 15 insertions(+), 3 deletions(-)mkfs/xfs_mkfs.c +1
-1@@ -3697,7 +3697,7 @@ main(/* build time defaults */ struct mkfs_default_params dft = { - .source =_ ("package build definitions" ), + .type =DEFAULTS_BUILTIN , .sectorsize =XFS_MIN_SECTORSIZE , .blocksize =1 <<XFS_DFL_BLOCKSIZE_LOG , .sb_feat = {@@ -3737,7 +3737,8 @@ main(* implemented, emit a message to indicate where the defaults being * used came from.
Quoted 8 lines
mkfs/xfs_mkfs.c main( · line 3741
Quoted 4 lines
mkfs/xfs_mkfs.c +2
-1* -* printf(_("Default configuration sourced from %s\n"), dft.source); +* printf(_("Default configuration sourced from %s\n"), +* default_type_str(dft.type)); This function does not exist in the patch. If you are going to add functionality, first turn on that functionality so you can test the patch actually works...
The function is in a comment though. I'll go ahead and add it along with the other enums then.
Quoted 11 lines
Quoted 11 lines
mkfs/xfs_mkfs.c +0
-0*/ /* copy new defaults into CLI parsing structure */ mkfs/xfs_mkfs_common.h +0
-0@@ -40,6 +40,17 @@ struct sb_feat_args {bool nortalign; };
Quoted 13 lines
mkfs/xfs_mkfs_common.h struct sb_feat_args { · line 52
Quoted 10 lines
mkfs/xfs_mkfs_common.h +10
-0+/* +* File configuration type settings +* +* These are the different possibilities by which you can end up parsing +* default settings with. DEFAULTS_BUILTIN indicates there was no configuration +* file parsed and we are using the built-in defaults on this code. +*/ +enum default_params_type { +DEFAULTS_BUILTIN =0 , +};Please add all the new types here, the functions to print the names, etc.
Ok.
Quoted 10 lines
Quoted 10 lines
mkfs/xfs_mkfs_common.h +1
-1/* * Default filesystem features and configuration values * @@ -49,7 +60,7 @@ struct sb_feat_args {* calculations. */ struct mkfs_default_params { -char *source;/* where the defaults came from */ +enum default_params_type type;/* where the defaults came from */
Quoted 7 lines
mkfs/xfs_mkfs_common.h struct sb_feat_args { · line 66
Quoted 2 lines
mkfs/xfs_mkfs_common.h +0
-0int sectorsize;int blocksize;As it is, I don't see why this change it necessary - you can just store the appropriate string (as the code currently does) into the structure once the source is known. Why do we need infrastructure to abstract printing a string when we set it directly, anyway?
Using an enum we get to document the different sources clearly, and we also get to have an enum to compare against for conditionals later, instead of having to strcmp().
Luis
Re: [PATCH v2 3/5] mkfs: replace defaults source with an enum
Quoted 11 lines · Luis R. Rodriguez
Quoted 8 lines · Dave Chinner
Quoted 5 lines · Luis R. Rodriguez
Using an enum will let us later just use a switch statement to print out the source, this makes sources easier to document, update and manage.
Signed-off-by Luis R. Rodriguez <[email protected]>This is incomplete. :(
It builds, the comment was intentional.
That doesn't make it complete. Usually when introducing new code one part at a time, the entire functionality of that part is separated out so it can be reviewed whole, not split across multilple patches and intermingled with other new functionality....
Quoted 18 lines
Quoted 10 lines
Quoted 10 lines
/*
* Default filesystem features and configuration values *mkfs/xfs_mkfs_common.h +1
-1@@ -49,7 +60,7 @@ struct sb_feat_args {* calculations. */ struct mkfs_default_params { -char *source;/* where the defaults came from */ +enum default_params_type type;/* where the defaults came from */ Quoted 7 lines
mkfs/xfs_mkfs_common.h struct sb_feat_args { · line 66
Quoted 2 lines
mkfs/xfs_mkfs_common.h +0
-0int sectorsize;int blocksize;As it is, I don't see why this change it necessary - you can just store the appropriate string (as the code currently does) into the structure once the source is known. Why do we need infrastructure to abstract printing a string when we set it directly, anyway?
Quoted 3 lines
Using an enum we get to document the different sources clearly, and we also get to have an enum to compare against for conditionals later, instead of having to
strcmp().
See my comments in later patches, where I suggest all that gets removed because i don't think it works. :P
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
PATCH v2 4/5 mkfs: add helpers to process defaults
In reply to Luis R. RodriguezMove the built-in defaults globally and add helpers to reset and process the defaults. This will be expanded on later. The commented out print of the defaults source is moved below CLI processing to acknowledge that one will later want to be able to specify a different configuration file to be used through the CLI.
--- mkfs/xfs_mkfs.c | 93 ++++++++++++++++++++++++++++++++++++++------------------- 1 file changed, 63 insertions(+), 30 deletions(-)
mkfs/xfs_mkfs.c +63 -30
@@ -3662,6 +3662,61 @@ rewrite_secondary_superblocks(libxfs_writebuf (buf,LIBXFS_EXIT_ON_FAILURE ); } +/* build time defaults */ +struct mkfs_default_params built_in_dft = { + .type =DEFAULTS_BUILTIN , + .sectorsize =XFS_MIN_SECTORSIZE , + .blocksize =1 <<XFS_DFL_BLOCKSIZE_LOG , + .sb_feat = { + .log_version =2 , + .attr_version =2 , + .dir_version =2 , + .inode_align = true, + .nci = false, + .lazy_sb_counters = true, + .projid32bit = true, + .crcs_enabled = true, + .dirftype = true, + .finobt = true, + .spinodes = true, + .rmapbt = false, + .reflink = false, + .parent_pointers = false, + .nodalign = false, + .nortalign = false, + }, +}; + +/* installs new defaults into the CLI parsing structure */ +static void install_defaults ( +struct mkfs_default_params *dft, +struct cli_params *cli) +{ +memcpy (&cli->sb_feat, &dft->sb_feat,sizeof (cli->sb_feat)); +memcpy (&cli->fsx, &dft->fsx,sizeof (cli->fsx)); +} + +/* +* Reset defaults first to built-in defaults. Then resets cli opts to start +* with these built-in defaults. All previously set CLI options will be ignored. +*/ +static void reset_defaults_and_cli ( +struct mkfs_default_params *dft, +struct cli_params *cli) +{ + *dft = built_in_dft; + +install_defaults (dft, cli); +} + +/* Does the required work to process a new set of defaults */ +static void process_defaults ( +struct mkfs_default_params *dft, +struct cli_params *cli) +{ +install_defaults (dft, cli); +} + int main( int argc,
@@ -3694,31 +3749,9 @@ main( .loginternal = 1, };struct mkfs_params cfg = {}; +struct mkfs_default_params dft; -/* build time defaults */ -struct mkfs_default_params dft = { - .type =DEFAULTS_BUILTIN , - .sectorsize =XFS_MIN_SECTORSIZE , - .blocksize =1 <<XFS_DFL_BLOCKSIZE_LOG , - .sb_feat = { - .log_version =2 , - .attr_version =2 , - .dir_version =2 , - .inode_align = true, - .nci = false, - .lazy_sb_counters = true, - .projid32bit = true, - .crcs_enabled = true, - .dirftype = true, - .finobt = true, - .spinodes = true, - .rmapbt = false, - .reflink = false, - .parent_pointers = false, - .nodalign = false, - .nortalign = false, - }, - }; +reset_defaults_and_cli (&dft, &cli);platform_uuid_generate (&cli.uuid); progname =basename (argv[0 ]);
@@ -3736,14 +3769,9 @@ main(* still be able to override them. When more than one source is * implemented, emit a message to indicate where the defaults being * used came from. -* -* printf(_("Default configuration sourced from %s\n"), -* default_type_str(dft.type)); */ -/* copy new defaults into CLI parsing structure */ -memcpy (&cli.sb_feat, &dft.sb_feat,sizeof (cli.sb_feat)); -memcpy (&cli.fsx, &dft.fsx,sizeof (cli.fsx)); +process_defaults (&dft, &cli);while ((c =getopt (argc, argv,"b:d:i:l:L:m:n:KNp:qr:s:CfV" )) !=EOF ) {switch (c) {
@@ -3795,6 +3823,11 @@ main( }else dfile = xi.dname; +/* +* printf(_("Default configuration sourced from %s\n"), +* default_type_str(dft.type)); +*/ + protostring =setup_proto (protofile);/*
Signature · 1 line
2.16.3
Re: [PATCH v2 4/5] mkfs: add helpers to process defaults
Quoted 33 lines · Luis R. Rodriguez
Move the built-in defaults globally and add helpers to reset and process the defaults. This will be expanded on later. The commented out print of the defaults source is moved below CLI processing to acknowledge that one will later want to be able to specify a different configuration file to be used through the CLI.
Signed-off-by Luis R. Rodriguez <[email protected]>--- mkfs/xfs_mkfs.c | 93 ++++++++++++++++++++++++++++++++++++++------------------- 1 file changed, 63 insertions(+), 30 deletions(-)mkfs/xfs_mkfs.c +14
-0@@ -3662,6 +3662,61 @@ rewrite_secondary_superblocks(libxfs_writebuf (buf,LIBXFS_EXIT_ON_FAILURE ); } +/* build time defaults */ +struct mkfs_default_params built_in_dft = { + .type =DEFAULTS_BUILTIN , + .sectorsize =XFS_MIN_SECTORSIZE , + .blocksize =1 <<XFS_DFL_BLOCKSIZE_LOG , + .sb_feat = { + .log_version =2 , + .attr_version =2 , + .dir_version =2 , + .inode_align = true, + .nci = false, + .lazy_sb_counters = true, + .projid32bit = true, + .crcs_enabled = true,
mkfs/xfs_mkfs.c rewrite_secondary_superblocks( · line 3688
Quoted 10 lines
mkfs/xfs_mkfs.c +10
-0+ .dirftype = true, + .finobt = true, + .spinodes = true, + .rmapbt = false, + .reflink = false, + .parent_pointers = false, + .nodalign = false, + .nortalign = false, + }, +};
Why? We've already got a perfectly good structure initialiser for the default settings in the main() function. Why do we need to create a new structure and a bunch of infrastructure to update it?
Quoted 54 lines
mkfs/xfs_mkfs.c +31
-15+/* installs new defaults into the CLI parsing structure */ +static void install_defaults ( +struct mkfs_default_params *dft, +struct cli_params *cli) +{ +memcpy (&cli->sb_feat, &dft->sb_feat,sizeof (cli->sb_feat)); +memcpy (&cli->fsx, &dft->fsx,sizeof (cli->fsx)); +} + +/* +* Reset defaults first to built-in defaults. Then resets cli opts to start +* with these built-in defaults. All previously set CLI options will be ignored. +*/ +static void reset_defaults_and_cli ( +struct mkfs_default_params *dft, +struct cli_params *cli) +{ + *dft = built_in_dft; + +install_defaults (dft, cli); +} + +/* Does the required work to process a new set of defaults */ +static void process_defaults ( +struct mkfs_default_params *dft, +struct cli_params *cli) +{ +install_defaults (dft, cli); +} + int main( int argc,@@ -3694,31 +3749,9 @@ main( .loginternal = 1, };struct mkfs_params cfg = {}; +struct mkfs_default_params dft; -/* build time defaults */ -struct mkfs_default_params dft = { - .type =DEFAULTS_BUILTIN , - .sectorsize =XFS_MIN_SECTORSIZE , - .blocksize =1 <<XFS_DFL_BLOCKSIZE_LOG , - .sb_feat = { - .log_version =2 , - .attr_version =2 , - .dir_version =2 , - .inode_align = true, - .nci = false, - .lazy_sb_counters = true, - .projid32bit = true, - .crcs_enabled = true, - .dirftype = true,
mkfs/xfs_mkfs.c main( · line 3754
Quoted 10 lines
mkfs/xfs_mkfs.c +1
-9- .finobt = true, - .spinodes = true, - .rmapbt = false, - .reflink = false, - .parent_pointers = false, - .nodalign = false, - .nortalign = false, - }, - }; +reset_defaults_and_cli (&dft, &cli);
I just don't seee what this abstraction improves over just declaring dft directly like this. It's also not explained why the CLI structure needs to be initialised here, because we're going to overwrite it again as soon as we've selected the default value source....
Quoted 6 lines
mkfs/xfs_mkfs.c +0
-0platform_uuid_generate (&cli.uuid); progname =basename (argv[0 ]);@@ -3736,14 +3769,9 @@ main(* still be able to override them. When more than one source is * implemented, emit a message to indicate where the defaults being
mkfs/xfs_mkfs.c main( · line 3774
Quoted 10 lines
mkfs/xfs_mkfs.c +1
-6* used came from. -* -* printf(_("Default configuration sourced from %s\n"), -* default_type_str(dft.type)); */ -/* copy new defaults into CLI parsing structure */ -memcpy (&cli.sb_feat, &dft.sb_feat,sizeof (cli.sb_feat)); -memcpy (&cli.fsx, &dft.fsx,sizeof (cli.fsx)); +process_defaults (&dft, &cli);
i.e. here.
mkfs/xfs_mkfs.c main( · line 3829
Quoted 11 lines
mkfs/xfs_mkfs.c +4
-0while ((c =getopt (argc, argv,"b:d:i:l:L:m:n:KNp:qr:s:CfV" )) !=EOF ) {switch (c) {@@ -3795,6 +3823,11 @@ main( }else dfile = xi.dname; +/* +* printf(_("Default configuration sourced from %s\n"), +* default_type_str(dft.type)); +*/
What does moving this acheive?
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Re: [PATCH v2 4/5] mkfs: add helpers to process defaults
Quoted 40 lines · Dave Chinner
Quoted 39 lines · Luis R. Rodriguez
Move the built-in defaults globally and add helpers to reset and process the defaults. This will be expanded on later. The commented out print of the defaults source is moved below CLI processing to acknowledge that one will later want to be able to specify a different configuration file to be used through the CLI.
Signed-off-by Luis R. Rodriguez <[email protected]>--- mkfs/xfs_mkfs.c | 93 ++++++++++++++++++++++++++++++++++++++------------------- 1 file changed, 63 insertions(+), 30 deletions(-)mkfs/xfs_mkfs.c +20
-0@@ -3662,6 +3662,61 @@ rewrite_secondary_superblocks(libxfs_writebuf (buf,LIBXFS_EXIT_ON_FAILURE ); } +/* build time defaults */ +struct mkfs_default_params built_in_dft = { + .type =DEFAULTS_BUILTIN , + .sectorsize =XFS_MIN_SECTORSIZE , + .blocksize =1 <<XFS_DFL_BLOCKSIZE_LOG , + .sb_feat = { + .log_version =2 , + .attr_version =2 , + .dir_version =2 , + .inode_align = true, + .nci = false, + .lazy_sb_counters = true, + .projid32bit = true, + .crcs_enabled = true, + .dirftype = true, + .finobt = true, + .spinodes = true, + .rmapbt = false, + .reflink = false, + .parent_pointers = false,
Quoted 8 lines
mkfs/xfs_mkfs.c rewrite_secondary_superblocks( · line 3688
Quoted 4 lines
mkfs/xfs_mkfs.c +4
-0+ .nodalign = false, + .nortalign = false, + }, +};Why? We've already got a perfectly good structure initialiser for the default settings in the
main()function. Why do we need to create a new structure
Its the same structure. No changes were made to it, so really the question is why make it global. The answer is there is only one built-in default. I found it odd passing built-in as an argument to a function.
Quoted 1 line
and a bunch of infrastructure to update it?
This is not easy to see from the patch, I'll explain. In short, this was the cleanest solution to allowing us to not have to re-implement and duplicate a lot of conflict checker/value checkers on the config parsing code.
You have to consider that we can use the built-in defaults on the CLI as we do today. That's easy to grasp... But when we add configuration file support, we now have to consider cases such as -- what to do if you have /etc/mkfs.xfs.d/default present, but actually use -c foo. In terms of top down functionality of the code -- since you said we *don't* want to process the parameters in the beginning and then again later, a clean way to deal with this is to have a:
a) built-in --> default resetter c) CLI reseter
The later would also neext to be extended eventually with the option to use defaults from a configuration file instead. So instead we end up with:
a) built-in --> default resetter b) config parser --> config default structure builder c) CLI reseter
Where b) is optional. And when the CLI is used, it can work off of
b) and c) -- essentially *resetting* all values. Ie, when -c is used we'd reset all values previously set on the CLI. Arguments *after* -c would be parsed and apply.
So although it looks a bit odd, this patch does not implement b), it is left to the next patch. But since resettting to built-in and doing a CLI reset with a defaults structure passed are two separate tasks, I've decided its cleaner to make two functions which make this crystal clear.
Quoted 59 lines
Quoted 59 lines
mkfs/xfs_mkfs.c +31
-20+/* installs new defaults into the CLI parsing structure */ +static void install_defaults ( +struct mkfs_default_params *dft, +struct cli_params *cli) +{ +memcpy (&cli->sb_feat, &dft->sb_feat,sizeof (cli->sb_feat)); +memcpy (&cli->fsx, &dft->fsx,sizeof (cli->fsx)); +} + +/* +* Reset defaults first to built-in defaults. Then resets cli opts to start +* with these built-in defaults. All previously set CLI options will be ignored. +*/ +static void reset_defaults_and_cli ( +struct mkfs_default_params *dft, +struct cli_params *cli) +{ + *dft = built_in_dft; + +install_defaults (dft, cli); +} + +/* Does the required work to process a new set of defaults */ +static void process_defaults ( +struct mkfs_default_params *dft, +struct cli_params *cli) +{ +install_defaults (dft, cli); +} + int main( int argc,@@ -3694,31 +3749,9 @@ main( .loginternal = 1, };struct mkfs_params cfg = {}; +struct mkfs_default_params dft; -/* build time defaults */ -struct mkfs_default_params dft = { - .type =DEFAULTS_BUILTIN , - .sectorsize =XFS_MIN_SECTORSIZE , - .blocksize =1 <<XFS_DFL_BLOCKSIZE_LOG , - .sb_feat = { - .log_version =2 , - .attr_version =2 , - .dir_version =2 , - .inode_align = true, - .nci = false, - .lazy_sb_counters = true, - .projid32bit = true, - .crcs_enabled = true, - .dirftype = true, - .finobt = true, - .spinodes = true, - .rmapbt = false, - .reflink = false, - .parent_pointers = false,
Quoted 8 lines
mkfs/xfs_mkfs.c main( · line 3754
Quoted 5 lines
mkfs/xfs_mkfs.c +1
-4- .nodalign = false, - .nortalign = false, - }, - }; +reset_defaults_and_cli (&dft, &cli);I just don't seee what this abstraction improves over just declaring dft directly like this.
Right now it doesn't buy us anything. It is until the next patch which hopefully the reasoning to this would become evident.
Quoted 4 lines
It's also not explained why the CLI
structure needs to be initialised here, because we're going to overwrite it again as soon as we've selected the default value source....
We are -- but these two are actually separate tasks. They are *now* functionally the same, but after the next patch it becomes clear that they are two separate tasks.
And since resetting a default structure to use built-in values is *one* task which I have to re-use in the net patch, I decided to add a helper here.
Quoted 26 lines
Quoted 6 lines
mkfs/xfs_mkfs.c +0
-0platform_uuid_generate (&cli.uuid); progname =basename (argv[0 ]);@@ -3736,14 +3769,9 @@ main(* still be able to override them. When more than one source is * implemented, emit a message to indicate where the defaults being mkfs/xfs_mkfs.c main( · line 3774
Quoted 10 lines
mkfs/xfs_mkfs.c +1
-6* used came from. -* -* printf(_("Default configuration sourced from %s\n"), -* default_type_str(dft.type)); */ -/* copy new defaults into CLI parsing structure */ -memcpy (&cli.sb_feat, &dft.sb_feat,sizeof (cli.sb_feat)); -memcpy (&cli.fsx, &dft.fsx,sizeof (cli.fsx)); +process_defaults (&dft, &cli);i.e. here.
Quoted 7 lines
mkfs/xfs_mkfs.c +0
-0while ((c =getopt (argc, argv,"b:d:i:l:L:m:n:KNp:qr:s:CfV" )) !=EOF ) {switch (c) {@@ -3795,6 +3823,11 @@ main( }else dfile = xi.dname;
Quoted 6 lines
mkfs/xfs_mkfs.c main( · line 3829
Quoted 4 lines
mkfs/xfs_mkfs.c +4
-0+/* +* printf(_("Default configuration sourced from %s\n"), +* default_type_str(dft.type)); +*/ What does moving this acheive?
My hope was that this would make it clear to the reader that the source of the configuration can only be known *after* CLI processing since we want to support -c.
Luis
PATCH v2 5/5 mkfs.xfs: add configuration file parsing support using our own parser
In reply to Luis R. RodriguezYou may want to stick to specific set of configuration options when creating filesystems with mkfs.xfs -- sometimes due to pure technical reasons, but some other times to ensure systems remain compatible as new features are introduced with older kernels, or if you always want to take advantage of some new feature which would otherwise typically be disruptive.
This adds support for parsing a configuration file to override defaults parameters to be used for mkfs.xfs.
We define an XFS configuration directory, /etc/mkfs.xfs.d/ and allow for different configuration files, if none is specified we look for the default configuration file, /etc/mkfs.xfs.d/default. You can override with -c. For instance, if you specify:
mkfs.xfs -c experimental -f /dev/loop0
The file /etc/mkfs.xfs.d/experimental will be used as your configuration file. If you really need to override the full path of the configuration file you may use the MKFS_XFS_CONFIG environment variable.
To verify what configuration file is used on a system use the typical:
mkfs.xfs -N
There is only a subset of options allowed to be set on the configuration file, and currently only 1 or 0 are acceptable values. The default parameters you can override on a configuration file and their current built-in default settings are:
[data]
noalign=0
[inode]
align=1
projid32bit=1
sparse=0
[log]
lazy-count=1
[metadata]
crc=1
finobt=1
rmapbt=0
reflink=0
[naming]
ftype=1
[rtdev]
noalign=0
--- include/builddefs.in | 2 + man/man5/mkfs.xfs.d.5 | 121 ++++++++++ man/man8/mkfs.xfs.8 | 26 +++ mkfs/Makefile | 2 +- mkfs/xfs_mkfs.c | 47 +++- mkfs/xfs_mkfs_common.h | 12 + mkfs/xfs_mkfs_config.c | 591 +++++++++++++++++++++++++++++++++++++++++++++++++ mkfs/xfs_mkfs_config.h | 11 + 8 files changed, 801 insertions(+), 11 deletions(-) create mode 100644 man/man5/mkfs.xfs.d.5 create mode 100644 mkfs/xfs_mkfs_config.c create mode 100644 mkfs/xfs_mkfs_config.h
include/builddefs.in +2 -0
@@ -62,6 +62,7 @@ PKG_LIB_DIR = @libdir@@libdirsuffix@ PKG_INC_DIR = @includedir@/xfs DK_INC_DIR = @includedir@/disk PKG_MAN_DIR = @mandir@ +PKG_ETC_DIR = @sysconfdir@ PKG_DOC_DIR = @datadir@/doc/@pkg_name@ PKG_LOCALE_DIR = @datadir@/locale
@@ -196,6 +197,7 @@ endif GCFLAGS = $(DEBUG) \ -DVERSION=\"$(PKG_VERSION)\" -DLOCALEDIR=\"$(PKG_LOCALE_DIR)\" \ + -DROOT_SYSCONFDIR=\"$(PKG_ETC_DIR)\" \ -DPACKAGE=\"$(PKG_NAME)\" -I$(TOPDIR)/include -I$(TOPDIR)/libxfs ifeq ($(ENABLE_GETTEXT),yes)
man/man5/mkfs.xfs.d.5 new file +121 -0
121 lines not on this page · raw message
man/man8/mkfs.xfs.8 +26 -0
@@ -83,6 +83,26 @@ and .B \-l internal \-l size=10m are equivalent. .PP +An optional XFS configuration file directory +.B mkfs.xfs.d (5) +exists to help fine tune default parameters which can be used when calling +.B mkfs.xfs (8). +If present, the default file /etc/mkfs.xfs.d/default +will be used to override system build-in defaults. Refer to mkfs.xfs.d (5) +for a list of current defaults. +Command line arguments directly passed to +.B mkfs.xfs (8) +will always override parameters set in the configuration file. +You can override the configuration file used within the +.B mkfs.xfs.d (5) +directory by using the -c parameter. Alternatively +you can set and use the MKFS_XFS_CONFIG environment variable to override +the default full path of the configuration file which +.B mkfs.xfs (8) +looks for. +If you use -c the configuration file must be present under +.B mkfs.xfs.d (8). +.PP In the descriptions below, sizes are given in sectors, bytes, blocks, kilobytes, megabytes, gigabytes, etc. Sizes are treated as hexadecimal if prefixed by 0x or 0X,
@@ -123,6 +143,11 @@ Many feature options allow an optional argument of 0 or 1, to explicitly disable or enable the functionality. .SH OPTIONS .TP +.BI \-c " configuration-file" +Override the configuration file used under the +.B mkfs.xfs.d +directory. +.TP .BI \-b " block_size_options" This option specifies the fundamental block size of the filesystem. The valid
@@ -923,6 +948,7 @@ Prints the version number and exits. .SH SEE ALSO .BR xfs (5), .BR mkfs (8), +.BR mkfs.xfs.d (5), .BR mount (8), .BR xfs_info (8), .BR xfs_admin (8).
mkfs/Makefile +1 -1
@@ -8,7 +8,7 @@ include $(TOPDIR)/include/builddefs LTCOMMAND = mkfs.xfs HFILES = -CFILES = proto.c xfs_mkfs.c +CFILES = proto.c xfs_mkfs.c xfs_mkfs_config.c LLDLIBS += $(LIBXFS) $(LIBXCMD) $(LIBFROG) $(LIBRT) $(LIBPTHREAD) $(LIBBLKID) \ $(LIBUUID)
mkfs/xfs_mkfs.c +37 -10
116 lines not on this page · raw message
mkfs/xfs_mkfs_common.h +12 -0
@@ -46,9 +46,20 @@ struct sb_feat_args {* These are the different possibilities by which you can end up parsing * default settings with. DEFAULTS_BUILTIN indicates there was no configuration * file parsed and we are using the built-in defaults on this code. +* DEFAULTS_CONFIG means the default configuration file was found and used. +* DEFAULTS_TYPE_CONFIG means the user asked for a custom configuration type +* and it was used, this means the default directory for mkfs.xfs +* configuration files was used to look for the type specified. If you need +* to override the default mkfs.xfs directory for configuration file you can +* use the MKFS_XFS_CONFIG environment variable to set the absolute path to +* your custom configuration file, when this is set the type is set to +* DEFAULTS_ENVIRONMENT_CONFIG. */ enum default_params_type {DEFAULTS_BUILTIN =0 , +DEFAULTS_CONFIG , +DEFAULTS_ENVIRONMENT_CONFIG , +DEFAULTS_TYPE_CONFIG , };/*
@@ -61,6 +72,7 @@ enum default_params_type {*/ struct mkfs_default_params {enum default_params_type type;/* where the defaults came from */ +const char *config_file;int sectorsize;int blocksize;
mkfs/xfs_mkfs_config.c new file +591 -0
591 lines not on this page · raw message
mkfs/xfs_mkfs_config.h new file +11 -0
@@ -0,0 +1,11 @@ +#ifndef _XFS_MKFS_CONFIG_H +#define _XFS_MKFS_CONFIG_H + +#include "xfs_mkfs_common.h" + +#define MKFS_XFS_CONF_DIR ROOT_SYSCONFDIR "/mkfs.xfs.d/" + +const char *default_type_str (enum default_params_type type); +int parse_config_init (struct mkfs_default_params *dft); + +#endif /* _XFS_MKFS_CONFIG_H */
Signature · 1 line
2.16.3
Re: [PATCH v2 5/5] mkfs.xfs: add configuration file parsing support using our own parser
Quoted 47 lines · Luis R. Rodriguez
You may want to stick to specific set of configuration options when creating filesystems with mkfs.xfs -- sometimes due to pure technical reasons, but some other times to ensure systems remain compatible as new features are introduced with older kernels, or if you always want to take advantage of some new feature which would otherwise typically be disruptive.
This adds support for parsing a configuration file to override defaults parameters to be used for mkfs.xfs.
We define an XFS configuration directory,
/etc/mkfs.xfs.d/and allow for different configuration files, if none is specified we look for the default configuration file,/etc/mkfs.xfs.d/default. You can override with -c. For instance, if you specify:mkfs.xfs -c experimental -f
/dev/loop0The file
/etc/mkfs.xfs.d/experimentalwill be used as your configuration file. If you really need to override the full path of the configuration file you may use theMKFS_XFS_CONFIGenvironment variable.To verify what configuration file is used on a system use the typical:
mkfs.xfs -N
There is only a subset of options allowed to be set on the configuration file, and currently only 1 or 0 are acceptable values. The default parameters you can override on a configuration file and their current built-in default settings are:
[data]
noalign=0[inode]
align=1
projid32bit=1
sparse=0[log]
lazy-count=1[metadata]
crc=1
finobt=1
rmapbt=0
reflink=0
Quoted 5 lines
[naming]
ftype=1[rtdev]
noalign=0
Separate patch, but should there be a way to spit out the current mkfs settings as a config file?
Quoted 41 lines
Signed-off-by Luis R. Rodriguez <[email protected]>--- include/builddefs.in | 2 + man/man5/mkfs.xfs.d.5 | 121 ++++++++++ man/man8/mkfs.xfs.8 | 26 +++ mkfs/Makefile | 2 +- mkfs/xfs_mkfs.c | 47 +++- mkfs/xfs_mkfs_common.h | 12 + mkfs/xfs_mkfs_config.c | 591 +++++++++++++++++++++++++++++++++++++++++++++++++ mkfs/xfs_mkfs_config.h | 11 + 8 files changed, 801 insertions(+), 11 deletions(-) create mode 100644 man/man5/mkfs.xfs.d.5 create mode 100644 mkfs/xfs_mkfs_config.c create mode 100644 mkfs/xfs_mkfs_config.hinclude/builddefs.in +2
-0@@ -62,6 +62,7 @@ PKG_LIB_DIR = @libdir@@libdirsuffix@ PKG_INC_DIR = @includedir@/xfs DK_INC_DIR = @includedir@/disk PKG_MAN_DIR = @mandir@ +PKG_ETC_DIR = @sysconfdir@ PKG_DOC_DIR = @datadir@/doc/@pkg_name@ PKG_LOCALE_DIR = @datadir@/locale@@ -196,6 +197,7 @@ endif GCFLAGS = $(DEBUG) \ -DVERSION=\"$(PKG_VERSION)\" -DLOCALEDIR=\"$(PKG_LOCALE_DIR)\" \ + -DROOT_SYSCONFDIR=\"$(PKG_ETC_DIR)\" \ -DPACKAGE=\"$(PKG_NAME)\" -I$(TOPDIR)/include -I$(TOPDIR)/libxfs ifeq ($(ENABLE_GETTEXT),yes)man/man5/mkfs.xfs.d.5 new file +0
-0
man/man5/mkfs.xfs.d.5 line 8
Quoted 8 lines
man/man5/mkfs.xfs.d.5 new file +8
-0@@ -0,0 +1,121 @@ +.TH mkfs.xfs.d 5 +.SH NAME +mkfs.xfs.d \- mkfs.xfs configuration directory +.SH DESCRIPTION +.B mkfs.xfs (8) +uses a set of initial default parameters for configuration. These defaults +are conservatively decided by the community as xfsprogs and features for XFS +in the kernel advance. One can override these defaults on the
I'm not sure what the sentence "These defaults...in the kernel advance." means... is this close to what you meant?:
"The builtin mkfs defaults are decided by the XFS community. New features are only enabled by default when the community consider it stable."
man/man5/mkfs.xfs.d.5 line 11
Quoted 3 lines
man/man5/mkfs.xfs.d.5 new file +3
-0+.B mkfs.xfs (8) +command line, but there are cases where it is desirable for sensible defaults +to always be specified by the system where
"...where it is desirable for the distro or the system administrator to establish their own supported defaults in a uniform manner." ?
man/man5/mkfs.xfs.d.5 line 16
Quoted 5 lines
man/man5/mkfs.xfs.d.5 new file +5
-0+.B mkfs.xfs (8) +runs on. This may desirable for example on systems with old kernels where the +built-in default parameters on +.B mkfs.xfs (8) +may not be able to create a filesystem which the old kernel supports and it
"...where the built-in default mkfs.xfs parameters create a filesystem that is not supported by the old kernel."
Quoted 5 lines
man/man5/mkfs.xfs.d.5 new file +5
-0+would be unclear what parameters are needed to produce a compatible filesystem. +Overriding +.B mkfs.xfs (8) +built-in defaults may also be desirable if you have a series of systems with +different kernels and want to be able to create filesystems which all systems
man/man5/mkfs.xfs.d.5 line 31
Quoted 10 lines
man/man5/mkfs.xfs.d.5 new file +10
-0+are able to support properly. +.PP +The XFS configuration directory +.B mkfs.xfs.d (5) +can be used to define different configuration files which can be used to +override the built-in default parameters by +.B mkfs.xfs (8). +Different configuration files are supported, the default +configuration file, +.I /etc/mkfs.xfs.d/default
The "/etc" part of the path should be sysconfdir and replaced by the build script to match whatever we're encoding into the mkfs.xfs binary.
Quoted 12 lines
man/man5/mkfs.xfs.d.5 new file +12
-0+, will be looked for first and if present will be used to override +.B mkfs.xfs (8) +built-in default parameters. You can override the configuration file by +specifying its name when using +.B mkfs.xfs (8) +by using the +.B -c +parameter. For example: +.I mkfs.xfs -c experimental -f /dev/sda1 +will make +.B mkfs.xfs (8) +look for and use the configuration file
man/man5/mkfs.xfs.d.5
Quoted 10 lines
man/man5/mkfs.xfs.d.5 new file +10
-0+.I /etc/mkfs.xfs.d/experimental +to override +.B mkfs.xfs (8) +default parameters. If you need to override the full path for a configuration +file you can use the +.I MKFS_XFS_CONFIG +environment variable prior to calling +.B mkfs.xfs (8) +to define the +full path to the configuration file to be used.
Does the environment variable override -c, or the other way around?
man/man5/mkfs.xfs.d.5 line 57
Quoted 5 lines
If you used the
+.B -cman/man5/mkfs.xfs.d.5 new file +3
-0+parameter or if you set the +.I MKFS_XFS_CONFIG +environment variable the configuration file must be present and should parse
"...must parse..."
man/man5/mkfs.xfs.d.5 line 60
Quoted 3 lines
man/man5/mkfs.xfs.d.5 new file +3
-0+correctly. +.PP +Parameters passed to to the
"...passed to the..."
Quoted 66 lines
+.B mkfs.xfs (8)
man/man5/mkfs.xfs.d.5 new file +60
-0+command line always override any defaults set on the configuration file used. +.PP +.B mkfs.xfs (8) +will always describe what configuration file was used, if any +was used at all. To verify which configuration file would be used prior to +execution of +.B mkfs.xfs (8) +you can use +.I mkfs.xfs -N. +.PP +.SH DEFAULT PARAMETERS +Default parameters for +.B mkfs.xfs (8) +consists of a small subset of the parameters one can set with on the command +line. Currently all default parameters can only be either enabled or disabled, +you can set their value to 1 to enable or 0 to disable. Below we list the +different supported default parameters which can be defined on configuration +files, along with the current built-in setting. +.PP +.BI [data] +.br +.BI noalign=0 +.PP +.BI [inode] +.br +.BI align=1 +.br +.BI projid32bit=1 +.br +.BI sparse=0 +.PP +.BI [log] +.br +.BI lazy-count=1 +.PP +.BI [metadata] +.br +.BI crc=1 +.br +.BI finobt=1 +.br +.BI rmapbt=0 +.br +.BI reflink=0 +.PP +.BI [naming] +.br +.BI ftype=1 +.PP +.BI [rtdev] +.br +.BI noalign=0 +.PP +.SH SEE ALSO +.BR mkfs.xfs (8), +.BR xfsctl (3), +.BR xfs_info (8), +.BR xfs_admin (8), +.BR xfsdump (8), +.BR xfsrestore (8).man/man8/mkfs.xfs.8 +0
-0
man/man8/mkfs.xfs.8 and · line 87
Quoted 5 lines
man/man8/mkfs.xfs.8 +2
-0@@ -83,6 +83,26 @@ and .B \-l internal \-l size=10m are equivalent. .PP +An optional XFS configuration file directory +.B mkfs.xfs.d (5)
%sysconfigdir%/mkfs.xfs.d ?
Quoted 82 lines
man/man8/mkfs.xfs.8 +24
-0+exists to help fine tune default parameters which can be used when calling +.B mkfs.xfs (8). +If present, the default file /etc/mkfs.xfs.d/default +will be used to override system build-in defaults. Refer to mkfs.xfs.d (5) +for a list of current defaults. +Command line arguments directly passed to +.B mkfs.xfs (8) +will always override parameters set in the configuration file. +You can override the configuration file used within the +.B mkfs.xfs.d (5) +directory by using the -c parameter. Alternatively +you can set and use the MKFS_XFS_CONFIG environment variable to override +the default full path of the configuration file which +.B mkfs.xfs (8) +looks for. +If you use -c the configuration file must be present under +.B mkfs.xfs.d (8). +.PP In the descriptions below, sizes are given in sectors, bytes, blocks, kilobytes, megabytes, gigabytes, etc. Sizes are treated as hexadecimal if prefixed by 0x or 0X,@@ -123,6 +143,11 @@ Many feature options allow an optional argument of 0 or 1, to explicitly disable or enable the functionality. .SH OPTIONS .TP +.BI \-c " configuration-file" +Override the configuration file used under the +.B mkfs.xfs.d +directory. +.TP .BI \-b " block_size_options" This option specifies the fundamental block size of the filesystem. The valid@@ -923,6 +948,7 @@ Prints the version number and exits. .SH SEE ALSO .BR xfs (5), .BR mkfs (8), +.BR mkfs.xfs.d (5), .BR mount (8), .BR xfs_info (8), .BR xfs_admin (8).mkfs/Makefile +1
-1@@ -8,7 +8,7 @@ include $(TOPDIR)/include/builddefs LTCOMMAND = mkfs.xfs HFILES = -CFILES = proto.c xfs_mkfs.c +CFILES = proto.c xfs_mkfs.c xfs_mkfs_config.c LLDLIBS += $(LIBXFS) $(LIBXCMD) $(LIBFROG) $(LIBRT) $(LIBPTHREAD) $(LIBBLKID) \ $(LIBUUID)mkfs/xfs_mkfs.c +4
-1@@ -21,6 +21,7 @@#include "xfs_multidisk.h" #include "libxcmd.h" #include "xfs_mkfs_common.h" +#include "xfs_mkfs_config.h" #include "xfs_mkfs_cli.h" @@ -3077,11 +3078,13 @@ print_mkfs_cfg(struct mkfs_params *cfg, char *dfile, char *logfile, - char *rtfile) + char *rtfile, + struct mkfs_default_params *dft) { struct sb_feat_args *fp = &cfg->sb_feat; printf(_( +"config-file=%-22s\n" "meta-data=%-22s isize=%-6d agcount=%lld, agsize=%lld blks\n" " =%-22s sectsz=%-5u attr=%u, projid32bit=%u\n" " =%-22s crc=%-8u finobt=%u, sparse=%u, rmapbt=%u, reflink=%u\n"
mkfs/xfs_mkfs.c print_mkfs_cfg( · line 3100
Quoted 7 lines
mkfs/xfs_mkfs.c +1
-0@@ -3091,6 +3094,7 @@ print_mkfs_cfg("log =%-22s bsize=%-6d blocks=%lld, version=%d\n" " =%-22s sectsz=%-5u sunit=%d blks, lazy-count=%d\n" "realtime =%-22s extsz=%-6d blocks=%lld, rtextents=%lld\n"), + dft->type !=DEFAULTS_BUILTIN ? dft->config_file :"empty" , dfile, cfg->inodesize, (long long )cfg->agcount, (long long )cfg->agsize, "", cfg->sectorsize, fp->attr_version, fp->projid32bit,
FWIW all this geometry printing was refactored into libfrog.
Though, you already print where we got the configuration data, so just print dft->config_file there.
Quoted 55 lines
mkfs/xfs_mkfs.c +16
-4@@ -3665,6 +3669,7 @@ rewrite_secondary_superblocks(/* build time defaults */ struct mkfs_default_params built_in_dft = { .type =DEFAULTS_BUILTIN , + .config_file =MKFS_XFS_CONF_DIR "default" , .sectorsize =XFS_MIN_SECTORSIZE , .blocksize =1 <<XFS_DFL_BLOCKSIZE_LOG , .sb_feat = {@@ -3714,6 +3719,13 @@ static void process_defaults(struct mkfs_default_params *dft, struct cli_params *cli) { + int ret; + + ret =parse_config_init (dft); + +if (ret && dft->type !=DEFAULTS_BUILTIN ) +usage (); +install_defaults (dft, cli); }@@ -3750,6 +3762,8 @@ main( };struct mkfs_params cfg = {};struct mkfs_default_params dft; +char *tmp_config; +char custom_config[PATH_MAX ];reset_defaults_and_cli (&dft, &cli);@@ -3760,21 +3774,36 @@ main(textdomain (PACKAGE );/* -* TODO: Sourcing defaults from a config file -* * Before anything else, see if there's a config file with different -* defaults. If a file exists in <package location>, read in the new +* defaults. If a file exists in MKFS_XFS_CONF_DIR/default, read the new * default values and overwrite them in the &dft structure. This way the * new defaults will apply before we parse the CLI, and the CLI will * still be able to override them. When more than one source is * implemented, emit a message to indicate where the defaults being * used came from. */ + tmp_config =getenv ("MKFS_XFS_CONFIG" ); +if (tmp_config !=NULL ) { + dft.config_file = tmp_config; + dft.type =DEFAULTS_ENVIRONMENT_CONFIG ; + }process_defaults (&dft, &cli); -while ((c =getopt (argc, argv,"b:d:i:l:L:m:n:KNp:qr:s:CfV" )) !=EOF ) {
mkfs/xfs_mkfs.c main( · line 3802
Quoted 10 lines
mkfs/xfs_mkfs.c +9
-0+while ((c =getopt (argc, argv,"b:c:d:i:l:L:m:n:KNp:qr:s:CfV" )) !=EOF ) {switch (c) { +case 'c' : +reset_defaults_and_cli (&dft, &cli); + +memset (custom_config,0 ,sizeof (custom_config)); +snprintf (custom_config,sizeof (custom_config),"%s%s" , +MKFS_XFS_CONF_DIR , optarg); + dft.config_file = custom_config; + dft.type =DEFAULTS_TYPE_CONFIG ;
Ok, so we parse $MKFS_XFS_CONFIG but then if the user specifies -c we reset all that and parse that config file.
Just for fun I decided to run:
$ cat > /tmp/butts.lol [gugugugug] bye = 1 [metadata] hork = 1 $ ./build-x64/mkfs/mkfs.xfs -c ../../../../../../../tmp/butts.lol -N /dev/sda Error parsine line: bye=1
I think we need to error out on unrecognized section names:
"Unrecognized section name 'gugugugug'."
And probably report the section name for unrecognized keys:
"Unrecognized name 'metadata.hork"."
Quoted 80 lines
mkfs/xfs_mkfs.c +7
-5+ +process_defaults (&dft, &cli); + +break ;case 'C' :case 'f' : force_overwrite =1 ;@@ -3823,10 +3852,8 @@ main( }else dfile = xi.dname; -/* -* printf(_("Default configuration sourced from %s\n"), -* default_type_str(dft.type)); -*/ +printf (_ ("Default configuration sourced from %s\n" ), +default_type_str (dft.type)); protostring =setup_proto (protofile);@@ -3902,7 +3929,7 @@ main(calculate_log_size (&cfg, &cli, mp);if (!quiet || dry_run) { -print_mkfs_cfg (&cfg, dfile, logfile, rtfile); +print_mkfs_cfg (&cfg, dfile, logfile, rtfile, &dft);if (dry_run)exit (0 ); }mkfs/xfs_mkfs_common.h +12
-0@@ -46,9 +46,20 @@ struct sb_feat_args {* These are the different possibilities by which you can end up parsing * default settings with. DEFAULTS_BUILTIN indicates there was no configuration * file parsed and we are using the built-in defaults on this code. +* DEFAULTS_CONFIG means the default configuration file was found and used. +* DEFAULTS_TYPE_CONFIG means the user asked for a custom configuration type +* and it was used, this means the default directory for mkfs.xfs +* configuration files was used to look for the type specified. If you need +* to override the default mkfs.xfs directory for configuration file you can +* use the MKFS_XFS_CONFIG environment variable to set the absolute path to +* your custom configuration file, when this is set the type is set to +* DEFAULTS_ENVIRONMENT_CONFIG. */ enum default_params_type {DEFAULTS_BUILTIN =0 , +DEFAULTS_CONFIG , +DEFAULTS_ENVIRONMENT_CONFIG , +DEFAULTS_TYPE_CONFIG , };/* @@ -61,6 +72,7 @@ enum default_params_type {*/ struct mkfs_default_params {enum default_params_type type;/* where the defaults came from */ +const char *config_file;int sectorsize;int blocksize;mkfs/xfs_mkfs_config.c new file +12
-0@@ -0,0 +1,591 @@ +/* +* Copyright (c) 2018 Luis R. Rodriguez <[email protected]> +* +* This program is free software; you can redistribute it and/or +* modify it under the terms of the GNU General Public License as +* published by the Free Software Foundation. +* +* This program is distributed in the hope that it would be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU General Public License for more details. +*
mkfs/xfs_mkfs_config.c line 22
Quoted 10 lines
mkfs/xfs_mkfs_config.c new file +10
-0+* You should have received a copy of the GNU General Public License +* along with this program; if not, write the Free Software Foundation, +* Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +*/ + +#include "xfs_mkfs_common.h" +#include "xfs_mkfs_config.h" + +#define CONFIG_MAX_KEY 1024 +#define CONFIG_MAX_VALUE PATH_MAX
That's a pretty big size considering we only allow 0 and 1...
mkfs/xfs_mkfs_config.c line 30
Quoted 8 lines
mkfs/xfs_mkfs_config.c new file +8
-0+#define CONFIG_MAX_BUFFER CONFIG_MAX_KEY + CONFIG_MAX_VALUE + 3 + +/* +* Enums for each configuration option. All these currently match the CLI +* parameters for now but this may change later, so we keep all this code +* and definitions separate. The rules for configuration parameters may also +* differ. +*/
So... I think these enums should be shared between cli and config file processing, or if the config file parser retains its own private definitions then it should only have the options that we support in the config file. It's weird how things like data.sunit are specified here but the config file doesn't actually do anything with it?
$ cat > /tmp/butts.lol
[data]
sunit = 5
$ ./build-x64/mkfs/mkfs.xfs -c ../../../tmp/butts.lol -N /dev/sda | grep sunit
= sunit=0 swidth=0 blksThat really ought to error out, since we don't support setting sunit?
Quoted 195 lines
mkfs/xfs_mkfs_config.c new file +195
-0+ +enum { +B_SIZE =0 , +B_MAX_OPTS , +}; + +enum { +D_AGCOUNT =0 , +D_FILE , +D_NAME , +D_SIZE , +D_SUNIT , +D_SWIDTH , +D_AGSIZE , +D_SU , +D_SW , +D_SECTSIZE , +D_NOALIGN , +D_RTINHERIT , +D_PROJINHERIT , +D_EXTSZINHERIT , +D_COWEXTSIZE , +D_MAX_OPTS , +}; + +enum { +I_ALIGN =0 , +I_MAXPCT , +I_PERBLOCK , +I_SIZE , +I_ATTR , +I_PROJID32BIT , +I_SPINODES , +I_MAX_OPTS , +}; + +enum { +L_AGNUM =0 , +L_INTERNAL , +L_SIZE , +L_VERSION , +L_SUNIT , +L_SU , +L_DEV , +L_SECTSIZE , +L_FILE , +L_NAME , +L_LAZYSBCNTR , +L_MAX_OPTS , +}; + +enum { +N_SIZE =0 , +N_VERSION , +N_FTYPE , +N_MAX_OPTS , +}; + +enum { +R_EXTSIZE =0 , +R_SIZE , +R_DEV , +R_FILE , +R_NAME , +R_NOALIGN , +R_MAX_OPTS , +}; + +enum { +S_SIZE =0 , +S_SECTSIZE , +S_MAX_OPTS , +}; + +enum { +M_CRC =0 , +M_FINOBT , +M_UUID , +M_RMAPBT , +M_REFLINK , +M_MAX_OPTS , +}; + +/* Just define the max options array size manually right now */ +#define MAX_SUBOPTS D_MAX_OPTS + +enum mkfs_config_section { +MKFS_CONFIG_SECTION_BLOCK =0 , +MKFS_CONFIG_SECTION_DATA , +MKFS_CONFIG_SECTION_INODE , +MKFS_CONFIG_SECTION_LOG , +MKFS_CONFIG_SECTION_METADATA , +MKFS_CONFIG_SECTION_NAMING , +MKFS_CONFIG_SECTION_RTDEV , +MKFS_CONFIG_SECTION_SECTOR , + +MKFS_CONFIG_SECTION_INVALID , +}; + +struct opt_params { +const enum mkfs_config_section section; +const char *subopts[MAX_SUBOPTS ]; +}; + +extern struct opt_params config_sopts; + +struct opt_params config_bopts = { + .section ='b' , + .subopts = { + [B_SIZE ] ="size" , + }, +}; + +struct opt_params config_dopts = { + .section ='d' , + .subopts = { + [D_AGCOUNT ] ="agcount" , + [D_FILE ] ="file" , + [D_NAME ] ="name" , + [D_SIZE ] ="size" , + [D_SUNIT ] ="sunit" , + [D_SWIDTH ] ="swidth" , + [D_AGSIZE ] ="agsize" , + [D_SU ] ="su" , + [D_SW ] ="sw" , + [D_SECTSIZE ] ="sectsize" , + [D_NOALIGN ] ="noalign" , + [D_RTINHERIT ] ="rtinherit" , + [D_PROJINHERIT ] ="projinherit" , + [D_EXTSZINHERIT ] ="extszinherit" , + [D_COWEXTSIZE ] ="cowextsize" , + }, +}; + +struct opt_params config_iopts = { + .section ='i' , + .subopts = { + [I_ALIGN ] ="align" , + [I_MAXPCT ] ="maxpct" , + [I_PERBLOCK ] ="perblock" , + [I_SIZE ] ="size" , + [I_ATTR ] ="attr" , + [I_PROJID32BIT ] ="projid32bit" , + [I_SPINODES ] ="sparse" , + }, +}; + +struct opt_params config_lopts = { + .section ='l' , + .subopts = { + [L_AGNUM ] ="agnum" , + [L_INTERNAL ] ="internal" , + [L_SIZE ] ="size" , + [L_VERSION ] ="version" , + [L_SUNIT ] ="sunit" , + [L_SU ] ="su" , + [L_DEV ] ="logdev" , + [L_SECTSIZE ] ="sectsize" , + [L_FILE ] ="file" , + [L_NAME ] ="name" , + [L_LAZYSBCNTR ] ="lazy-count" , + }, +}; + +struct opt_params config_nopts = { + .section ='n' , + .subopts = { + [N_SIZE ] ="size" , + [N_VERSION ] ="version" , + [N_FTYPE ] ="ftype" , + }, +}; + +struct opt_params config_ropts = { + .section ='r' , + .subopts = { + [R_EXTSIZE ] ="extsize" , + [R_SIZE ] ="size" , + [R_DEV ] ="rtdev" , + [R_FILE ] ="file" , + [R_NAME ] ="name" , + [R_NOALIGN ] ="noalign" , + }, +}; + +struct opt_params config_sopts = { + .section ='s' , + .subopts = { + [S_SIZE ] ="size" , + [S_SECTSIZE ] ="sectsize" , + }, +}; + +struct opt_params config_mopts = { + .section ='m' ,
mkfs/xfs_mkfs_config.c line 235
Quoted 10 lines
mkfs/xfs_mkfs_config.c new file +10
-0+ .subopts = { + [M_CRC] = "crc", + [M_FINOBT] = "finobt", + [M_UUID] = "uuid", + [M_RMAPBT] = "rmapbt", + [M_REFLINK] = "reflink", + }, +}; + +const char *default_type_str (enum default_params_type type)
const char *
default_type_str(
enum default_params_type type)
{
...
}Please follow regular xfs formatting conventions.
mkfs/xfs_mkfs_config.c line 247
Quoted 12 lines
mkfs/xfs_mkfs_config.c new file +12
-0+{ +switch (type) { +case DEFAULTS_BUILTIN : +return _ ("package built-in definitions" ); +case DEFAULTS_CONFIG : +return _ ("default configuration system file" ); +case DEFAULTS_ENVIRONMENT_CONFIG : +return _ ("custom configuration file set by environment" ); +case DEFAULTS_TYPE_CONFIG : +return _ ("custom configuration type file" ); + } +return _ ("Unkown\n" );
"Unknown"
Quoted 189 lines
mkfs/xfs_mkfs_config.c new file +189
-0+} + +static int block_config_parser (struct mkfs_default_params *dft,int subopt, +bool value) +{ +return -EINVAL ; +} + +static int data_config_parser (struct mkfs_default_params *dft,int subopt, +bool value) +{ +switch (subopt) { +case D_NOALIGN : + dft->sb_feat.nodalign = value; +break ; +default : +return -EINVAL ; + } +return 0 ; +} + +static int inode_config_parser (struct mkfs_default_params *dft,int subopt, +bool value) +{ +switch (subopt) { +case I_ALIGN : + dft->sb_feat.inode_align = value; +break ; +case I_PROJID32BIT : + dft->sb_feat.projid32bit = value; +break ; +case I_SPINODES : + dft->sb_feat.spinodes = value; +break ; +default : +return -EINVAL ; + } +return 0 ; +} + +static int log_config_parser (struct mkfs_default_params *dft,int subopt, +bool value) +{ +switch (subopt) { +case L_LAZYSBCNTR : + dft->sb_feat.lazy_sb_counters = value; +break ; +default : +return -EINVAL ; + } +return 0 ; +} + +static int meta_config_parser (struct mkfs_default_params *dft,int subopt, +bool value) +{ +switch (subopt) { +case M_CRC : + dft->sb_feat.crcs_enabled = value; +if (dft->sb_feat.crcs_enabled) + dft->sb_feat.dirftype = true; +break ; +case M_FINOBT : + dft->sb_feat.finobt = value; +break ; +case M_RMAPBT : + dft->sb_feat.rmapbt = value; +break ; +case M_REFLINK : + dft->sb_feat.reflink = value; +break ; +default : +return -EINVAL ; + } +return 0 ; +} + +static int naming_config_parser (struct mkfs_default_params *dft,int subopt, +bool value) +{ +switch (subopt) { +case N_FTYPE : + dft->sb_feat.dirftype = value; +break ; +default : +return -EINVAL ; + } +return 0 ; +} + +static int rtdev_config_parser (struct mkfs_default_params *dft,int subopt, +bool value) +{ +switch (subopt) { +case R_NOALIGN : + dft->sb_feat.nortalign = value; +break ; +default : +return -EINVAL ; + } +return 0 ; +} + +static int sector_config_parser (struct mkfs_default_params *dft,int subopt, +bool value) +{ +return -EINVAL ; +} + +struct config_subopts { +enum mkfs_config_section section; +struct opt_params *opts; +int (*config_parser)(struct mkfs_default_params *dft, +int subop,bool value); +} config_subopt_tab[] = { + {MKFS_CONFIG_SECTION_BLOCK , &config_bopts, block_config_parser }, + {MKFS_CONFIG_SECTION_DATA , &config_dopts, data_config_parser }, + {MKFS_CONFIG_SECTION_INODE , &config_iopts, inode_config_parser }, + {MKFS_CONFIG_SECTION_LOG , &config_lopts, log_config_parser }, + {MKFS_CONFIG_SECTION_METADATA , &config_mopts, meta_config_parser }, + {MKFS_CONFIG_SECTION_NAMING , &config_nopts, naming_config_parser }, + {MKFS_CONFIG_SECTION_RTDEV , &config_ropts, rtdev_config_parser }, + {MKFS_CONFIG_SECTION_SECTOR , &config_sopts, sector_config_parser }, + {'\0' ,NULL }, +}; + +static int +parse_config_subopts ( +enum mkfs_config_section section, +bool value, +char *line, +struct mkfs_default_params *dft) +{ +struct config_subopts *sop = &config_subopt_tab[0 ]; +char **subopts = (char **)sop->opts->subopts; +int subopt; +char *val; + +while (sop->opts) { +if (sop->section == section) +break ; + sop++; + } + +/* should never happen */ +if (!sop->opts) +return -EINVAL ; + + subopts = (char **)sop->opts->subopts; + subopt =getsubopt (&line, subopts, &val); +return (sop->config_parser)(dft, subopt, value); +} + +static enum mkfs_config_section get_config_section (const char *buffer) +{ +if (strncmp ("block" , buffer,5 ) ==0 ) +return MKFS_CONFIG_SECTION_BLOCK ; +if (strncmp ("data" , buffer,4 ) ==0 ) +return MKFS_CONFIG_SECTION_DATA ; +if (strncmp ("inode" , buffer,5 ) ==0 ) +return MKFS_CONFIG_SECTION_INODE ; +if (strncmp ("log" , buffer,3 ) ==0 ) +return MKFS_CONFIG_SECTION_LOG ; +if (strncmp ("metadata" , buffer,8 ) ==0 ) +return MKFS_CONFIG_SECTION_METADATA ; +if (strncmp ("naming" , buffer,6 ) ==0 ) +return MKFS_CONFIG_SECTION_NAMING ; +if (strncmp ("rtdev" , buffer,5 ) ==0 ) +return MKFS_CONFIG_SECTION_RTDEV ; +if (strncmp ("sector" , buffer,6 ) ==0 ) +return MKFS_CONFIG_SECTION_SECTOR ; + +return MKFS_CONFIG_SECTION_INVALID ; +} + +static int mkfs_check_config_file_limits (const char *filename, +const struct stat *sb, +unsigned int max_entries) +{ +unsigned long long size_limit; + + size_limit =CONFIG_MAX_BUFFER * max_entries; + +/* +* Detect wrap around -- if max_entries * size_limit goves over +* unsigned long long we detect that there. Note some libraries, +* like libiniconfig, only groks max INT_MAX entries anyway, so +* if we ever use a library for parsing we'd be restricted to that +* limit.
mkfs/xfs_mkfs_config.c line 446
Quoted 10 lines
mkfs/xfs_mkfs_config.c new file +10
-0+*/ +if (size_limit < max_entries || max_entries >INT_MAX ) +return -E2BIG ; + +if (sb->st_size > size_limit) { +fprintf (stderr, +"File %s is too big! File size limit: %llu\n" , + filename, size_limit); +return -E2BIG ; + }
Not sure why we care about the file size? If the config file respecifies options a trillion times then let it.
Quoted 41 lines
mkfs/xfs_mkfs_config.c new file +41
-0+ +return 0 ; +} + +enum parse_line_type { +PARSE_COMMENT =0 , +PARSE_SECTION , +PARSE_TAG_VALUE , +PARSE_INVALID , +PARSE_EOF , +}; + +static int parse_line_section (const char *line,char *tag) +{ +return sscanf (line," [%[^]]s]" , tag); +} + +static int parse_line_tag_value (const char *line,char *tag, +unsigned int *value) +{ +return sscanf (line," %[^ \t=]" +" = " +"%u " , + tag, value); +} + +static enum parse_line_type parse_get_line_type (const char *line,char *tag, +bool *value) +{ +int ret; +unsigned int uint_value; + +memset (tag,0 ,80 ); + +if (strlen (line) <2 ) +return PARSE_INVALID ; + +if (line[0 ] =='#' ) +return PARSE_COMMENT ; + + ret =parse_line_section (line, tag);
mkfs/xfs_mkfs_config.c line 497
Quoted 10 lines
mkfs/xfs_mkfs_config.c new file +10
-0+if (ret ==1 ) +return PARSE_SECTION ; + +if (ret ==EOF ) +return PARSE_EOF ; + + ret =parse_line_tag_value (line, tag, &uint_value); +if (ret ==2 ) { +if (uint_value !=1 && uint_value !=0 ) +return -EINVAL ;
If I set data.sunit = 5 this will return -EINVAL to the switch in parse_config_init. However, the switch lacks a default clause so it silently drops invalid values and proceeds with the format.
Quoted 11 lines
mkfs/xfs_mkfs_config.c new file +11
-0+ + *value = !!uint_value; + +return PARSE_TAG_VALUE ; + } + +if (ret ==EOF ) +return PARSE_EOF ; + +return PARSE_INVALID ; +}
mkfs/xfs_mkfs_config.c line 518
Quoted 10 lines
mkfs/xfs_mkfs_config.c new file +10
-0+ +int parse_config_init (struct mkfs_default_params *dft) +{ +int ret; +FILE *fp; +struct stat sp; +unsigned int num_subopts_sections =sizeof (config_subopt_tab) / +sizeof (struct config_subopts ) -1 ; +char *p; +char line[80 ], tag[80 ];
Waitaminute, didn't we set CONFIG_MAX_KEY = 1024 and CONFIG_MAX_VALUE to PATH_MAX?
mkfs/xfs_mkfs_config.c line 525
Quoted 7 lines
mkfs/xfs_mkfs_config.c new file +7
-0+bool value; +enum parse_line_type parse_type; +enum mkfs_config_section section =MKFS_CONFIG_SECTION_INVALID ; + + fp =fopen (dft->config_file,"r" ); +if (!fp) { +if (dft->type !=DEFAULTS_BUILTIN ) {
Why would we be parsing a config file if dft->type == DEFAULTS_BUILTIN?
Quoted 4 lines
mkfs/xfs_mkfs_config.c new file +4
-0+fprintf (stderr,_ ("could not open config file: %s\n" ), + dft->config_file); + ret = -ENOENT ; + }else
mkfs/xfs_mkfs_config.c line 539
Quoted 10 lines
mkfs/xfs_mkfs_config.c new file +10
-0+ ret =0 ; +return ret; + } + +/* +* If we found a file without it being specified on the command line, +* it would be the default configuration file. +*/ +if (dft->type ==DEFAULTS_BUILTIN ) + dft->type =DEFAULTS_CONFIG ;
Oh, I see, type == _BUILTIN really means "we picked up /etc/mkfs.xfs.d/defaults"... shouldn't the caller set this?
mkfs/xfs_mkfs_config.c line 541
Quoted 2 lines
mkfs/xfs_mkfs_config.c new file +2
-0+ + ret =stat (dft->config_file, &sp);
fstat(fileno(fp), &sp); ?
mkfs/xfs_mkfs_config.c line 543
Quoted 2 lines
mkfs/xfs_mkfs_config.c new file +2
-0+if (ret) { +if (dft->type !=DEFAULTS_BUILTIN )
Didn't we just set this to _CONFIG?
mkfs/xfs_mkfs_config.c line 546
Quoted 3 lines
mkfs/xfs_mkfs_config.c new file +3
-0+fprintf (stderr,_ ("could not stat() config file: %s: %s\n" ), + dft->config_file,strerror (ret)); +return ret;
We just leaked fp...
mkfs/xfs_mkfs_config.c line 550
Quoted 4 lines
mkfs/xfs_mkfs_config.c new file +4
-0+ } + +if (!sp.st_size) +return 0 ;
Can't we just keep going? If the file size is zero we never execute the loop.
Quoted 18 lines
mkfs/xfs_mkfs_config.c new file +18
-0+ ret =mkfs_check_config_file_limits (dft->config_file, &sp, + num_subopts_sections); +if (ret) +return ret; + +while (!feof (fp)) { + p =fgets (line,sizeof (line), fp); +if (!p) +continue ; + + parse_type =parse_get_line_type (line, tag, &value); + +switch (parse_type) { +case PARSE_COMMENT : +case PARSE_INVALID : +case PARSE_EOF : +break ; +case PARSE_SECTION :
mkfs/xfs_mkfs_config.c line 579
Quoted 10 lines
mkfs/xfs_mkfs_config.c new file +10
-0+ section =get_config_section (tag); +if (!section) { +fprintf (stderr,_ ("Invalid section: %s\n" ), + tag); +return -EINVAL ; + } +break ; +case PARSE_TAG_VALUE : +/* Trims white spaces */ +snprintf (line,sizeof (line),"%s=%u" , tag, value);
So we reconstruct the line (without the whitespaces) so that we can use getsubopt to map the tag to an integer value (e.g. D_AGCOUNT)... we already isolated *tag, so can't we map it directly?
mkfs/xfs_mkfs_config.c line 586
Quoted 7 lines
mkfs/xfs_mkfs_config.c new file +7
-0+ ret =parse_config_subopts (section, value, line, dft); +if (ret) { +fprintf (stderr,_ ("Error parsine line: %s\n" ), + line); +return ret; + } +break ;
case -EINVAL?
--D
Quoted 29 lines
mkfs/xfs_mkfs_config.c new file +5
-0+ } + } + +return 0 ; +}mkfs/xfs_mkfs_config.h new file +11
-0@@ -0,0 +1,11 @@ +#ifndef _XFS_MKFS_CONFIG_H +#define _XFS_MKFS_CONFIG_H + +#include "xfs_mkfs_common.h" + +#define MKFS_XFS_CONF_DIR ROOT_SYSCONFDIR "/mkfs.xfs.d/" + +const char *default_type_str (enum default_params_type type); +int parse_config_init (struct mkfs_default_params *dft); + +#endif /* _XFS_MKFS_CONFIG_H */ Signature · 6 lines
2.16.3 -- To unsubscribe from this list: send the line "unsubscribe linux-xfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 5/5] mkfs.xfs: add configuration file parsing support using our own parser
Quoted 48 lines · Darrick J. Wong
Quoted 47 lines · Luis R. Rodriguez
You may want to stick to specific set of configuration options when creating filesystems with mkfs.xfs -- sometimes due to pure technical reasons, but some other times to ensure systems remain compatible as new features are introduced with older kernels, or if you always want to take advantage of some new feature which would otherwise typically be disruptive.
This adds support for parsing a configuration file to override defaults parameters to be used for mkfs.xfs.
We define an XFS configuration directory,
/etc/mkfs.xfs.d/and allow for different configuration files, if none is specified we look for the default configuration file,/etc/mkfs.xfs.d/default. You can override with -c. For instance, if you specify:mkfs.xfs -c experimental -f
/dev/loop0The file
/etc/mkfs.xfs.d/experimentalwill be used as your configuration file. If you really need to override the full path of the configuration file you may use theMKFS_XFS_CONFIGenvironment variable.To verify what configuration file is used on a system use the typical:
mkfs.xfs -N
There is only a subset of options allowed to be set on the configuration file, and currently only 1 or 0 are acceptable values. The default parameters you can override on a configuration file and their current built-in default settings are:
[data]
noalign=0[inode]
align=1
projid32bit=1
sparse=0[log]
lazy-count=1[metadata]
crc=1
finobt=1
rmapbt=0
reflink=0
Quoted 8 lines
Quoted 5 lines
[naming]
ftype=1[rtdev]
noalign=0Separate patch, but should there be a way to spit out the current mkfs settings as a config file?
Sure, I have that in my mind but definitely separate patch. As can be seen, there are still quite a bit of minor things to wrap up. No point in getting ahead of ourselves.
Quoted 53 lines
Quoted 41 lines
Signed-off-by Luis R. Rodriguez <[email protected]>--- include/builddefs.in | 2 + man/man5/mkfs.xfs.d.5 | 121 ++++++++++ man/man8/mkfs.xfs.8 | 26 +++ mkfs/Makefile | 2 +- mkfs/xfs_mkfs.c | 47 +++- mkfs/xfs_mkfs_common.h | 12 + mkfs/xfs_mkfs_config.c | 591 +++++++++++++++++++++++++++++++++++++++++++++++++ mkfs/xfs_mkfs_config.h | 11 + 8 files changed, 801 insertions(+), 11 deletions(-) create mode 100644 man/man5/mkfs.xfs.d.5 create mode 100644 mkfs/xfs_mkfs_config.c create mode 100644 mkfs/xfs_mkfs_config.hinclude/builddefs.in +2
-0@@ -62,6 +62,7 @@ PKG_LIB_DIR = @libdir@@libdirsuffix@ PKG_INC_DIR = @includedir@/xfs DK_INC_DIR = @includedir@/disk PKG_MAN_DIR = @mandir@ +PKG_ETC_DIR = @sysconfdir@ PKG_DOC_DIR = @datadir@/doc/@pkg_name@ PKG_LOCALE_DIR = @datadir@/locale@@ -196,6 +197,7 @@ endif GCFLAGS = $(DEBUG) \ -DVERSION=\"$(PKG_VERSION)\" -DLOCALEDIR=\"$(PKG_LOCALE_DIR)\" \ + -DROOT_SYSCONFDIR=\"$(PKG_ETC_DIR)\" \ -DPACKAGE=\"$(PKG_NAME)\" -I$(TOPDIR)/include -I$(TOPDIR)/libxfs ifeq ($(ENABLE_GETTEXT),yes)man/man5/mkfs.xfs.d.5 new file +0
-0man/man5/mkfs.xfs.d.5 line 8
Quoted 8 lines
man/man5/mkfs.xfs.d.5 new file +8
-0@@ -0,0 +1,121 @@ +.TH mkfs.xfs.d 5 +.SH NAME +mkfs.xfs.d \- mkfs.xfs configuration directory +.SH DESCRIPTION +.B mkfs.xfs (8) +uses a set of initial default parameters for configuration. These defaults +are conservatively decided by the community as xfsprogs and features for XFS +in the kernel advance. One can override these defaults on theI'm not sure what the sentence "These defaults...in the kernel advance." means... is this close to what you meant?:
Quoted 2 lines
"The builtin mkfs defaults are decided by the XFS community. New features are only enabled by default when the community consider it stable."
That reads much better. Amended.
Quoted 6 lines
man/man5/mkfs.xfs.d.5 line 11
Quoted 3 lines
man/man5/mkfs.xfs.d.5 new file +3
-0+.B mkfs.xfs (8) +command line, but there are cases where it is desirable for sensible defaults +to always be specified by the system where"...where it is desirable for the distro or the system administrator to establish their own supported defaults in a uniform manner." ?
Sure, I'll also add at the end:
"regardless of the version of mkfs.xfs used"
Quoted 9 lines
man/man5/mkfs.xfs.d.5 line 16
Quoted 5 lines
man/man5/mkfs.xfs.d.5 new file +5
-0+.B mkfs.xfs (8) +runs on. This may desirable for example on systems with old kernels where the +built-in default parameters on +.B mkfs.xfs (8) +may not be able to create a filesystem which the old kernel supports and it"...where the built-in default mkfs.xfs parameters create a filesystem that is not supported by the old kernel."
Quoted 6 lines
man/man5/mkfs.xfs.d.5 line 22
Quoted 6 lines
man/man5/mkfs.xfs.d.5 new file +6
-0+would be unclear what parameters are needed to produce a compatible filesystem. +Overriding +.B mkfs.xfs (8) +built-in defaults may also be desirable if you have a series of systems with +different kernels and want to be able to create filesystems which all systems +are able to support properly.
I've moved this to another sentence as well then:
In such situations it would also be unclear what parameters are needed to produce a compatible filesystem, having a configuration file present ensures that if newer versions of
.B mkfs.xfs (8)
are deployed, creating a filesystem will remain compatible.
Quoted 4 lines
Quoted 4 lines
man/man5/mkfs.xfs.d.5 new file +4
-0+.PP +The XFS configuration directory +.B mkfs.xfs.d (5) +can be used to define different configuration files which can be used to
Quoted 8 lines
man/man5/mkfs.xfs.d.5 line 31
Quoted 5 lines
man/man5/mkfs.xfs.d.5 new file +5
-0+override the built-in default parameters by +.B mkfs.xfs (8). +Different configuration files are supported, the default +configuration file, +.I /etc/mkfs.xfs.d/defaultThe "/etc" part of the path should be sysconfdir and replaced by the build script to match whatever we're encoding into the mkfs.xfs binary.
Alrighty .. and all other "etc" references I take it.. sure.
Quoted 16 lines
Quoted 16 lines
man/man5/mkfs.xfs.d.5 new file +16
-0+, will be looked for first and if present will be used to override +.B mkfs.xfs (8) +built-in default parameters. You can override the configuration file by +specifying its name when using +.B mkfs.xfs (8) +by using the +.B -c +parameter. For example: +.I mkfs.xfs -c experimental -f /dev/sda1 +will make +.B mkfs.xfs (8) +look for and use the configuration file +.I /etc/mkfs.xfs.d/experimental +to override +.B mkfs.xfs (8) +default parameters. If you need to override the full path for a configuration
Quoted 8 lines
man/man5/mkfs.xfs.d.5
Quoted 6 lines
man/man5/mkfs.xfs.d.5 new file +6
-0+file you can use the +.I MKFS_XFS_CONFIG +environment variable prior to calling +.B mkfs.xfs (8) +to define the +full path to the configuration file to be used.Does the environment variable override -c, or the other way around?
The other way around.
Will continue the rest tomorrow.
Luis
Re: [PATCH v2 5/5] mkfs.xfs: add configuration file parsing support using our own parser
In reply to Darrick J. WongQuoted 8 lines · Darrick J. Wong
man/man5/mkfs.xfs.d.5 line 57
Quoted 5 lines · Luis R. Rodriguez
If you used the
+.B -cman/man5/mkfs.xfs.d.5 new file +3
-0+parameter or if you set the +.I MKFS_XFS_CONFIG +environment variable the configuration file must be present and should parse"...must parse..."
Fixed and dropped the MKFS_XFS_CONFIG environment variable reference.
Quoted 5 lines
man/man5/mkfs.xfs.d.5 line 60
Quoted 3 lines
man/man5/mkfs.xfs.d.5 new file +3
-0+correctly. +.PP +Parameters passed to to the"...passed to the..."
Fixed.
Quoted 4 lines
Quoted 4 lines
man/man8/mkfs.xfs.8 +0
-0
Quoted 8 lines
man/man8/mkfs.xfs.8 and · line 87
Quoted 6 lines
man/man8/mkfs.xfs.8 +2
-0@@ -83,6 +83,26 @@ and .B \-l internal \-l size=10m are equivalent. .PP +An optional XFS configuration file directory +.B mkfs.xfs.d (5)%sysconfigdir%/mkfs.xfs.d ?
I rather refer to the man page using the short name. When referring to the full path though I will add the %sysconfigdir%/mkfs.xfs.d and have this parse.
Quoted 26 lines
Quoted 16 lines
man/man8/mkfs.xfs.8 +3
-1@@ -3077,11 +3078,13 @@ print_mkfs_cfg( struct mkfs_params *cfg, char *dfile, char *logfile, - char *rtfile) + char *rtfile, + struct mkfs_default_params *dft) { struct sb_feat_args *fp = &cfg->sb_feat; printf(_( +"config-file=%-22s\n" "meta-data=%-22s isize=%-6d agcount=%lld, agsize=%lld blks\n" " =%-22s sectsz=%-5u attr=%u, projid32bit=%u\n" " =%-22s crc=%-8u finobt=%u, sparse=%u, rmapbt=%u, reflink=%u\n"man/man8/mkfs.xfs.8 print_mkfs_cfg( · line 3100
Quoted 7 lines
man/man8/mkfs.xfs.8 +1
-0@@ -3091,6 +3094,7 @@ print_mkfs_cfg( "log =%-22s bsize=%-6d blocks=%lld, version=%d\n" " =%-22s sectsz=%-5u sunit=%d blks, lazy-count=%d\n" "realtime =%-22s extsz=%-6d blocks=%lld, rtextents=%lld\n"), + dft->type != DEFAULTS_BUILTIN ? dft->config_file : "empty", dfile, cfg->inodesize, (long long)cfg->agcount, (long long)cfg->agsize, "", cfg->sectorsize, fp->attr_version, fp->projid32bit,FWIW all this geometry printing was refactored into libfrog.
Quoted 2 lines
Though, you already print where we got the configuration data, so just print
dft->config_filethere.
True, will do.
Quoted 6 lines
Quoted 6 lines
man/man8/mkfs.xfs.8 +4
-1- while ((c = getopt(argc, argv, "b:d:i:l:L:m:n:KNp:qr:s:CfV")) != EOF) { + while ((c = getopt(argc, argv, "b:c:d:i:l:L:m:n:KNp:qr:s:CfV")) != EOF) { switch (c) { + case 'c': + reset_defaults_and_cli(&dft, &cli); +
Quoted 8 lines
man/man8/mkfs.xfs.8
Quoted 5 lines
man/man8/mkfs.xfs.8 +5
-0+ memset(custom_config, 0, sizeof(custom_config)); + snprintf(custom_config, sizeof(custom_config), "%s%s", + MKFS_XFS_CONF_DIR, optarg); + dft.config_file = custom_config; + dft.type = DEFAULTS_TYPE_CONFIG;Ok, so we parse $MKFS_XFS_CONFIG but then if the user specifies -c we reset all that and parse that config file.
Right.
Quoted 10 lines
Just for fun I decided to run:
$ cat > /tmp/butts.lol [gugugugug] bye = 1 [metadata] hork = 1 $ ./build-x64/mkfs/mkfs.xfs -c ../../../../../../../tmp/butts.lol -N /dev/sda Error parsine line: bye=1
Quoted 3 lines
I think we need to error out on unrecognized section names:
"Unrecognized section name 'gugugugug'."
Will fix.
Quoted 3 lines
And probably report the section name for unrecognized keys:
"Unrecognized name 'metadata.hork"."
Alright.
Quoted 4 lines
man/man8/mkfs.xfs.8
Quoted 2 lines
man/man8/mkfs.xfs.8 +2
-0+#define CONFIG_MAX_KEY 1024 +#define CONFIG_MAX_VALUE PATH_MAXThat's a pretty big size considering we only allow 0 and 1...
Heh true, as per Chinner will just check for a single max file size for about 1MiB or so, so we won't need these as it was just used to compute a max theoretical file size. By using a simple reasonable file size we remove the need for the above.
Quoted 11 lines
man/man8/mkfs.xfs.8
Quoted 8 lines
man/man8/mkfs.xfs.8 +8
-0+#define CONFIG_MAX_BUFFER CONFIG_MAX_KEY + CONFIG_MAX_VALUE + 3 + +/* + * Enums for each configuration option. All these currently match the CLI + * parameters for now but this may change later, so we keep all this code + * and definitions separate. The rules for configuration parameters may also + * differ. + */So... I think these enums should be shared between cli and config file processing,
As per Chinner's request we'd duplicate for now as we may want to support alternative config options which would differ from the CLI options. This is why the comment above it notes this.
Quoted 4 lines
or if the config file parser retains its own private definitions then it should only have the options that we support in the config file. It's weird how things like data.sunit are specified here but the config file doesn't actually do anything with it?
The enum is defined for it, the parser however lacks a switch entry for it. I'll just trim the enums to what we currently parse only then.
Quoted 7 lines
$ cat > /tmp/butts.lol [data] sunit = 5 $ ./build-x64/mkfs/mkfs.xfs -c ../../../tmp/butts.lol -N /dev/sda | grep sunit = sunit=0 swidth=0 blksThat really ought to error out, since we don't support setting sunit?
Will verify and fix.
Quoted 10 lines
Quoted 1 line
+const char *default_type_str(enum
default_params_typetype)const char *
default_type_str(
enumdefault_params_typetype){ ... }Please follow regular xfs formatting conventions.
Will do.
Quoted 6 lines
Quoted 6 lines
man/man8/mkfs.xfs.8 +6
-0+{ + switch (type) { + case DEFAULTS_BUILTIN: + return _("package built-in definitions"); + case DEFAULTS_CONFIG: + return _("default configuration system file");
Quoted 8 lines
man/man8/mkfs.xfs.8
Quoted 6 lines
man/man8/mkfs.xfs.8 +6
-0+ case DEFAULTS_ENVIRONMENT_CONFIG: + return _("custom configuration file set by environment"); + case DEFAULTS_TYPE_CONFIG: + return _("custom configuration type file"); + } + return _("Unkown\n");"Unknown"
This call is being removed as per Chinner's request.
Quoted 19 lines
Quoted 19 lines
man/man8/mkfs.xfs.8 +19
-0+static int mkfs_check_config_file_limits(const char *filename, + const struct stat *sb, + unsigned int max_entries) +{ + unsigned long long size_limit; + + size_limit = CONFIG_MAX_BUFFER * max_entries; + + /* + * Detect wrap around -- if max_entries * size_limit goves over + * unsigned long long we detect that there. Note some libraries, + * like libiniconfig, only groks max INT_MAX entries anyway, so + * if we ever use a library for parsing we'd be restricted to that + * limit. + */ + if (size_limit < max_entries || max_entries > INT_MAX) + return -E2BIG; + + if (sb->st_size > size_limit) {
Quoted 8 lines
man/man8/mkfs.xfs.8
Quoted 5 lines
man/man8/mkfs.xfs.8 +5
-0+ fprintf(stderr, + "File %s is too big! File size limit: %llu\n", + filename, size_limit); + return -E2BIG; + }Not sure why we care about the file size? If the config file respecifies options a trillion times then let it.
I'll simplify the check to 1MiB file size alone.
Chinner requested we actually don't allow respecification on the configuration file. Let me know what you two decide.
Quoted 47 lines
Quoted 47 lines
man/man8/mkfs.xfs.8 +47
-0+ + return 0; +} + +enum parse_line_type { + PARSE_COMMENT = 0, + PARSE_SECTION, + PARSE_TAG_VALUE, + PARSE_INVALID, + PARSE_EOF, +}; + +static int parse_line_section(const char *line, char *tag) +{ + return sscanf(line, " [%[^]]s]", tag); +} + +static int parse_line_tag_value(const char *line, char *tag, + unsigned int *value) +{ + return sscanf(line, " %[^ \t=]" + " = " + "%u ", + tag, value); +} + +static enum parse_line_type parse_get_line_type(const char *line, char *tag, + bool *value) +{ + int ret; + unsigned int uint_value; + + memset(tag, 0, 80); + + if (strlen(line) < 2) + return PARSE_INVALID; + + if (line[0] == '#') + return PARSE_COMMENT; + + ret = parse_line_section(line, tag); + if (ret == 1) + return PARSE_SECTION; + + if (ret == EOF) + return PARSE_EOF; +
Quoted 8 lines
man/man8/mkfs.xfs.8
Quoted 4 lines
man/man8/mkfs.xfs.8 +4
-0+ ret = parse_line_tag_value(line, tag, &uint_value); + if (ret == 2) { + if (uint_value != 1 && uint_value != 0) + return -EINVAL;If I set data.sunit = 5 this will return
-EINVALto the switch inparse_config_init. However, the switch lacks a default clause so it silently drops invalid values and proceeds with the format.
Odd, in this case sunit is a valid tag, so it shoudl have errored out, but will re-test and also ensure invalid tags fail accordingly.
Quoted 16 lines
Quoted 16 lines
man/man8/mkfs.xfs.8 +16
-0+ + *value = !!uint_value; + + return PARSE_TAG_VALUE; + } + + if (ret == EOF) + return PARSE_EOF; + + return PARSE_INVALID; +} + +int parse_config_init(struct mkfs_default_params *dft) +{ + int ret; + FILE *fp;
Quoted 8 lines
man/man8/mkfs.xfs.8
Quoted 5 lines
man/man8/mkfs.xfs.8 +5
-0+ struct stat sp; + unsigned int num_subopts_sections = sizeof(config_subopt_tab) / + sizeof(struct config_subopts) - 1; + char *p; + char line[80], tag[80];Waitaminute, didn't we set
CONFIG_MAX_KEY= 1024 andCONFIG_MAX_VALUEtoPATH_MAX?
Heh yeah, I'll just drop those and keep 80.
Quoted 9 lines
man/man8/mkfs.xfs.8
Quoted 7 lines
man/man8/mkfs.xfs.8 +7
-0+ bool value; + enum parse_line_type parse_type; + enum mkfs_config_section section = MKFS_CONFIG_SECTION_INVALID; + + fp = fopen(dft->config_file, "r"); + if (!fp) { + if (dft->type != DEFAULTS_BUILTIN) {Why would we be parsing a config file if
dft->type==DEFAULTS_BUILTIN?
If the user specified an environment variable thent he error should be fatal, otherwise, if the type is built-in then the default is being picked up, and in such case an error on the file not existing is non-fatal.
But all this will be simplified now that we'll just deal with *one* choice of parsing early.
Quoted 9 lines
Quoted 9 lines
man/man8/mkfs.xfs.8 +9
-0+ fprintf(stderr, _("could not open config file: %s\n"), + dft->config_file); + ret = -ENOENT; + } else + ret = 0; + return ret; + } + + /*
Quoted 8 lines
man/man8/mkfs.xfs.8
Quoted 5 lines
man/man8/mkfs.xfs.8 +5
-0+ * If we found a file without it being specified on the command line, + * it would be the default configuration file. + */ + if (dft->type == DEFAULTS_BUILTIN) + dft->type = DEFAULTS_CONFIG;Oh, I see, type ==
_BUILTINreally means "we picked up/etc/mkfs.xfs.d/defaults"... shouldn't the caller set this?
No, it means the built-in defaults were used, but since the config file exists, it means either the deault config file was found or the user specified one.
But again, this will all be simplified further.
Quoted 4 lines
man/man8/mkfs.xfs.8
Quoted 2 lines
man/man8/mkfs.xfs.8 +2
-0+ + ret = stat(dft->config_file, &sp);
fstat(fileno(fp), &sp); ?
Sure!
Quoted 4 lines
man/man8/mkfs.xfs.8
Quoted 2 lines
man/man8/mkfs.xfs.8 +2
-0+ if (ret) { + if (dft->type != DEFAULTS_BUILTIN)Didn't we just set this to
_CONFIG?
Only if it was not built-in, ie, if the environment variable was used nope.
Quoted 5 lines
man/man8/mkfs.xfs.8
Quoted 3 lines
man/man8/mkfs.xfs.8 +3
-0+ fprintf(stderr, _("could not stat() config file: %s: %s\n"), + dft->config_file, strerror(ret)); + return ret;We just leaked fp...
I've sprinkled goto outs, thanks.
Quoted 7 lines
man/man8/mkfs.xfs.8
Quoted 4 lines
man/man8/mkfs.xfs.8 +4
-0+ } + + if (!sp.st_size) + return 0;Can't we just keep going? If the file size is zero we never execute the loop.
Either way is fine, less code is better, so will remove this.
Quoted 23 lines
Quoted 23 lines
man/man8/mkfs.xfs.8 +23
-0+ ret = mkfs_check_config_file_limits(dft->config_file, &sp, + num_subopts_sections); + if (ret) + return ret; + + while (!feof(fp)) { + p = fgets(line, sizeof(line), fp); + if (!p) + continue; + + parse_type = parse_get_line_type(line, tag, &value); + + switch (parse_type) { + case PARSE_COMMENT: + case PARSE_INVALID: + case PARSE_EOF: + break; + case PARSE_SECTION: + section = get_config_section(tag); + if (!section) { + fprintf(stderr, _("Invalid section: %s\n"), + tag); + return -EINVAL;
Quoted 8 lines
man/man8/mkfs.xfs.8
Quoted 5 lines
man/man8/mkfs.xfs.8 +5
-0+ } + break; + case PARSE_TAG_VALUE: + /* Trims white spaces */ + snprintf(line, sizeof(line), "%s=%u", tag, value);So we reconstruct the line (without the whitespaces) so that we can use getsubopt to map the tag to an integer value (e.g.
D_AGCOUNT)...
Right.
Quoted 1 line
we already isolated *tag, so can't we map it directly?
What do you mean by map it directly?
Quoted 9 lines
man/man8/mkfs.xfs.8
Quoted 7 lines
man/man8/mkfs.xfs.8 +7
-0+ ret = parse_config_subopts(section, value, line, dft); + if (ret) { + fprintf(stderr, _("Error parsine line: %s\n"), + line); + return ret; + } + break;case
-EINVAL?
I take it you meant case default: return -EINVAL? Since I use enums on the return value from parse_get_line_type() and on the switch statement, the compiler checks for any known enums which were not on the switch and complains if one is found.
So long as parse_get_line_type() returns an PARSE_INVALID or PARSE_IGNORE (which I'll have to add) then we're OK without it, and are better off without a default case to ensure we cover all parse type conditions we add.
Luis
Re: [PATCH v2 5/5] mkfs.xfs: add configuration file parsing support using our own parser
In reply to Luis R. RodriguezQuoted 9 lines · Luis R. Rodriguez
You may want to stick to specific set of configuration options when creating filesystems with mkfs.xfs -- sometimes due to pure technical reasons, but some other times to ensure systems remain compatible as new features are introduced with older kernels, or if you always want to take advantage of some new feature which would otherwise typically be disruptive.
This adds support for parsing a configuration file to override defaults parameters to be used for mkfs.xfs.
.....
Just looking at code right now...
Quoted 26 lines
mkfs/xfs_mkfs.c +4
-1@@ -21,6 +21,7 @@#include "xfs_multidisk.h" #include "libxcmd.h" #include "xfs_mkfs_common.h" +#include "xfs_mkfs_config.h" #include "xfs_mkfs_cli.h" @@ -3077,11 +3078,13 @@ print_mkfs_cfg(struct mkfs_params *cfg, char *dfile, char *logfile, - char *rtfile) + char *rtfile, + struct mkfs_default_params *dft) { struct sb_feat_args *fp = &cfg->sb_feat; printf(_( +"config-file=%-22s\n" "meta-data=%-22s isize=%-6d agcount=%lld, agsize=%lld blks\n" " =%-22s sectsz=%-5u attr=%u, projid32bit=%u\n" " =%-22s crc=%-8u finobt=%u, sparse=%u, rmapbt=%u, reflink=%u\n"
mkfs/xfs_mkfs.c print_mkfs_cfg( · line 3100
Quoted 7 lines
mkfs/xfs_mkfs.c +1
-0@@ -3091,6 +3094,7 @@ print_mkfs_cfg("log =%-22s bsize=%-6d blocks=%lld, version=%d\n" " =%-22s sectsz=%-5u sunit=%d blks, lazy-count=%d\n" "realtime =%-22s extsz=%-6d blocks=%lld, rtextents=%lld\n"), + dft->type !=DEFAULTS_BUILTIN ? dft->config_file :"empty" , dfile, cfg->inodesize, (long long )cfg->agcount, (long long )cfg->agsize, "", cfg->sectorsize, fp->attr_version, fp->projid32bit,
Haven't we already printed where the config was sourced from? Why do we need to print it again here?
Quoted 9 lines
mkfs/xfs_mkfs.c +1
-0@@ -3665,6 +3669,7 @@ rewrite_secondary_superblocks(/* build time defaults */ struct mkfs_default_params built_in_dft = { .type =DEFAULTS_BUILTIN , + .config_file =MKFS_XFS_CONF_DIR "default" , .sectorsize =XFS_MIN_SECTORSIZE , .blocksize =1 <<XFS_DFL_BLOCKSIZE_LOG , .sb_feat = {
mkfs/xfs_mkfs.c static void process_defaults( · line 3727
Quoted 9 lines
mkfs/xfs_mkfs.c +6
-0@@ -3714,6 +3719,13 @@ static void process_defaults(struct mkfs_default_params *dft, struct cli_params *cli) { + int ret; + + ret =parse_config_init (dft); + +if (ret && dft->type !=DEFAULTS_BUILTIN ) +usage ();
That doesn't make any sense in isolation.
Quoted 17 lines
mkfs/xfs_mkfs.c +2
-1install_defaults (dft, cli); }@@ -3750,6 +3762,8 @@ main( };struct mkfs_params cfg = {};struct mkfs_default_params dft; +char *tmp_config; +char custom_config[PATH_MAX ];reset_defaults_and_cli (&dft, &cli);@@ -3760,21 +3774,36 @@ main(textdomain (PACKAGE );/* -* TODO: Sourcing defaults from a config file
mkfs/xfs_mkfs.c main( · line 3784
Quoted 10 lines
mkfs/xfs_mkfs.c +1
-2-* * Before anything else, see if there's a config file with different -* defaults. If a file exists in <package location>, read in the new +* defaults. If a file exists in MKFS_XFS_CONF_DIR/default, read the new * default values and overwrite them in the &dft structure. This way the * new defaults will apply before we parse the CLI, and the CLI will * still be able to override them. When more than one source is * implemented, emit a message to indicate where the defaults being * used came from. */
This comment makes no mention of environment variables, or how they interact with other sources of config files.
mkfs/xfs_mkfs.c main( · line 3791
Quoted 7 lines
mkfs/xfs_mkfs.c +5
-0+ tmp_config =getenv ("MKFS_XFS_CONFIG" ); +if (tmp_config !=NULL ) { + dft.config_file = tmp_config; + dft.type =DEFAULTS_ENVIRONMENT_CONFIG ; + }process_defaults (&dft, &cli);
So why are we processing the defaults before we've checked if a default file is specified on the command line? All this should do is set up the config file to be read, and set the dft.source = _("Environment Variable");
mkfs/xfs_mkfs.c main( · line 3796
Quoted 6 lines
mkfs/xfs_mkfs.c +3
-1-while ((c =getopt (argc, argv,"b:d:i:l:L:m:n:KNp:qr:s:CfV" )) !=EOF ) { +while ((c =getopt (argc, argv,"b:c:d:i:l:L:m:n:KNp:qr:s:CfV" )) !=EOF ) {switch (c) { +case 'c' : +reset_defaults_and_cli (&dft, &cli);
This will trash the existing CLI inputs that have already been parsed. All this code here should do is set up the config file to be read.
mkfs/xfs_mkfs.c main( · line 3804
Quoted 8 lines
mkfs/xfs_mkfs.c +8
-0+ +memset (custom_config,0 ,sizeof (custom_config)); +snprintf (custom_config,sizeof (custom_config),"%s%s" , +MKFS_XFS_CONF_DIR , optarg); + dft.config_file = custom_config; + dft.type =DEFAULTS_TYPE_CONFIG ; + +process_defaults (&dft, &cli);
Again, what happens if the user specifies multiple config files? The second config file will trash everything from the first, as well as any other options between them.
I think that we need to pull the "-c <file>" option from the command line and process all the defaults /before/ we enter the main command line processing loop.
IOWs:
config_file = getenv("MKFS_XFS_CONFIG");
if (config_file)
dft.source = _("Environment Variable");
/*
* Pull config line options from command line
*/
while ((c = getopt(argc, argv, "c:")) != EOF) {
switch (c) {
case 'c':
/* XXX: fail if already seen a CLI option */
config_file = optarg;
dft.source = _("Command Line");
break;
default:
continue;
}
}
if (config_file) {
/*
* parse_defaults_file() knows about config file
* search paths, so just pass in the raw,
* unvalidated filename and let it do all the work
* finding the file containing the default values.
*/
error = parse_defaults_file(&dft, config_file);
if (error) {
/* fail! */
}
}
printf(_("Default configuration sourced from %s\n"), dft.source);
/*
* Done parsing defaults now, so memcpy defaults into CLI
* structure, reset getopt and start parsing CLI options
*/
memcpy(/* sb_feats */);
memcpy(/* fsxattr */);
optind = 1;
while ((c = getopt(argc, argv, "b:c:d:i:l:L:m:n:KNp:qr:s:CfV")) != EOF) {
switch (c) {
case 'c':
/* already validated and parsed, ignore */
break;
....Quoted 13 lines
mkfs/xfs_mkfs_common.h +5
-0@@ -46,9 +46,20 @@ struct sb_feat_args {* These are the different possibilities by which you can end up parsing * default settings with. DEFAULTS_BUILTIN indicates there was no configuration * file parsed and we are using the built-in defaults on this code. +* DEFAULTS_CONFIG means the default configuration file was found and used. +* DEFAULTS_TYPE_CONFIG means the user asked for a custom configuration type +* and it was used, this means the default directory for mkfs.xfs +* configuration files was used to look for the type specified. If you need +* to override the default mkfs.xfs directory for configuration file you can
mkfs/xfs_mkfs_common.h struct sb_feat_args { · line 63
Quoted 10 lines
mkfs/xfs_mkfs_common.h +6
-0+* use the MKFS_XFS_CONFIG environment variable to set the absolute path to +* your custom configuration file, when this is set the type is set to +* DEFAULTS_ENVIRONMENT_CONFIG. */ enum default_params_type {DEFAULTS_BUILTIN =0 , +DEFAULTS_CONFIG , +DEFAULTS_ENVIRONMENT_CONFIG , +DEFAULTS_TYPE_CONFIG , };
Again, I just don't see the need for this...
mkfs/xfs_mkfs_common.h enum default_params_type { · line 75
Quoted 6 lines
mkfs/xfs_mkfs_common.h +1
-0/* @@ -61,6 +72,7 @@ enum default_params_type {*/ struct mkfs_default_params {enum default_params_type type;/* where the defaults came from */ +const char *config_file;
Or this (see above :).
Quoted 29 lines
mkfs/xfs_mkfs_common.h +0
-0int sectorsize;int blocksize;mkfs/xfs_mkfs_config.c new file +20
-0@@ -0,0 +1,591 @@ +/* +* Copyright (c) 2018 Luis R. Rodriguez <[email protected]> +* +* This program is free software; you can redistribute it and/or +* modify it under the terms of the GNU General Public License as +* published by the Free Software Foundation. +* +* This program is distributed in the hope that it would be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with this program; if not, write the Free Software Foundation, +* Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +*/ + +#include "xfs_mkfs_common.h" +#include "xfs_mkfs_config.h" +
mkfs/xfs_mkfs_config.c line 30
Quoted 10 lines
mkfs/xfs_mkfs_config.c new file +10
-0+#define CONFIG_MAX_KEY 1024 +#define CONFIG_MAX_VALUE PATH_MAX +#define CONFIG_MAX_BUFFER CONFIG_MAX_KEY + CONFIG_MAX_VALUE + 3 + +/* +* Enums for each configuration option. All these currently match the CLI +* parameters for now but this may change later, so we keep all this code +* and definitions separate. The rules for configuration parameters may also +* differ. +*/
Why define them all when we are only going to parse a small subset?
Quoted 9 lines
mkfs/xfs_mkfs_config.c new file +9
-0+enum mkfs_config_section { +MKFS_CONFIG_SECTION_BLOCK =0 , +MKFS_CONFIG_SECTION_DATA , +MKFS_CONFIG_SECTION_INODE , +MKFS_CONFIG_SECTION_LOG , +MKFS_CONFIG_SECTION_METADATA , +MKFS_CONFIG_SECTION_NAMING , +MKFS_CONFIG_SECTION_RTDEV , +MKFS_CONFIG_SECTION_SECTOR ,
mkfs/xfs_mkfs_config.c line 135
Quoted 10 lines
mkfs/xfs_mkfs_config.c new file +10
-0+ + MKFS_CONFIG_SECTION_INVALID, +}; + +struct opt_params { +const enum mkfs_config_section section; +const char *subopts[MAX_SUBOPTS ]; +}; + +extern struct opt_params config_sopts;
Why is this declared here?
mkfs/xfs_mkfs_config.c line 138
Quoted 3 lines
mkfs/xfs_mkfs_config.c new file +3
-0+ +struct opt_params config_bopts = { + .section ='b' ,
That's not an enum, looks broken. Why isn't this the ascii name of the config section in the config file?
Quoted 15 lines
mkfs/xfs_mkfs_config.c new file +15
-0+ .subopts = { + [B_SIZE] = "size", + }, +}; + +struct opt_params config_dopts = { + .section ='d' , + .subopts = { + [D_AGCOUNT ] ="agcount" , + [D_FILE ] ="file" , + [D_NAME ] ="name" , + [D_SIZE ] ="size" , + [D_SUNIT ] ="sunit" , + [D_SWIDTH ] ="swidth" , + [D_AGSIZE ] ="agsize" ,
mkfs/xfs_mkfs_config.c line 163
Quoted 10 lines
mkfs/xfs_mkfs_config.c new file +10
-0+ [D_SU] = "su", + [D_SW] = "sw", + [D_SECTSIZE] = "sectsize", + [D_NOALIGN] = "noalign", + [D_RTINHERIT] = "rtinherit", + [D_PROJINHERIT] = "projinherit", + [D_EXTSZINHERIT] = "extszinherit", + [D_COWEXTSIZE] = "cowextsize", + }, +};
So the parser will accept names we can't configure?
[...]
Quoted 4 lines
mkfs/xfs_mkfs_config.c new file +4
-0+const char *default_type_str (enum default_params_type type) +{ +switch (type) { +case DEFAULTS_BUILTIN :
mkfs/xfs_mkfs_config.c line 248
Quoted 10 lines
mkfs/xfs_mkfs_config.c new file +10
-0+return _ ("package built-in definitions" ); +case DEFAULTS_CONFIG : +return _ ("default configuration system file" ); +case DEFAULTS_ENVIRONMENT_CONFIG : +return _ ("custom configuration file set by environment" ); +case DEFAULTS_TYPE_CONFIG : +return _ ("custom configuration type file" ); + } +return _ ("Unkown\n" ); +}
This function can go away.
mkfs/xfs_mkfs_config.c line 254
Quoted 5 lines
mkfs/xfs_mkfs_config.c new file +5
-0+static int block_config_parser (struct mkfs_default_params *dft,int subopt, +bool value) +{ +return -EINVAL ; +}
Why is the value passed to the parser a bool type and not a uint64_t?
Quoted 33 lines
mkfs/xfs_mkfs_config.c new file +33
-0+struct config_subopts { +enum mkfs_config_section section; +struct opt_params *opts; +int (*config_parser)(struct mkfs_default_params *dft, +int subop,bool value); +} config_subopt_tab[] = { + {MKFS_CONFIG_SECTION_BLOCK , &config_bopts, block_config_parser }, + {MKFS_CONFIG_SECTION_DATA , &config_dopts, data_config_parser }, + {MKFS_CONFIG_SECTION_INODE , &config_iopts, inode_config_parser }, + {MKFS_CONFIG_SECTION_LOG , &config_lopts, log_config_parser }, + {MKFS_CONFIG_SECTION_METADATA , &config_mopts, meta_config_parser }, + {MKFS_CONFIG_SECTION_NAMING , &config_nopts, naming_config_parser }, + {MKFS_CONFIG_SECTION_RTDEV , &config_ropts, rtdev_config_parser }, + {MKFS_CONFIG_SECTION_SECTOR , &config_sopts, sector_config_parser }, + {'\0' ,NULL }, +}; + +static int +parse_config_subopts ( +enum mkfs_config_section section, +bool value, +char *line, +struct mkfs_default_params *dft) +{ +struct config_subopts *sop = &config_subopt_tab[0 ]; +char **subopts = (char **)sop->opts->subopts; +int subopt; +char *val; + +while (sop->opts) { +if (sop->section == section) +break ; + sop++;
mkfs/xfs_mkfs_config.c line 399
Quoted 10 lines
mkfs/xfs_mkfs_config.c new file +10
-0+ } + +/* should never happen */ +if (!sop->opts) +return -EINVAL ; + + subopts = (char **)sop->opts->subopts; + subopt =getsubopt (&line, subopts, &val); +return (sop->config_parser)(dft, subopt, value); +}
This seems back to front - why do we walk the table to find the entry that corresponds to an enum when we've....
mkfs/xfs_mkfs_config.c line 404
Quoted 4 lines
mkfs/xfs_mkfs_config.c new file +4
-0+static enum mkfs_config_section get_config_section (const char *buffer) +{ +if (strncmp ("block" , buffer,5 ) ==0 ) +return MKFS_CONFIG_SECTION_BLOCK ;
This will match "blocksdfsgdfgd" - strcmp is fine here.
Quoted 7 lines
mkfs/xfs_mkfs_config.c new file +7
-0+if (strncmp ("data" , buffer,4 ) ==0 ) +return MKFS_CONFIG_SECTION_DATA ; +if (strncmp ("inode" , buffer,5 ) ==0 ) +return MKFS_CONFIG_SECTION_INODE ; +if (strncmp ("log" , buffer,3 ) ==0 ) +return MKFS_CONFIG_SECTION_LOG ; +if (strncmp ("metadata" , buffer,8 ) ==0 )
mkfs/xfs_mkfs_config.c line 421
Quoted 10 lines
mkfs/xfs_mkfs_config.c new file +10
-0+return MKFS_CONFIG_SECTION_METADATA ; +if (strncmp ("naming" , buffer,6 ) ==0 ) +return MKFS_CONFIG_SECTION_NAMING ; +if (strncmp ("rtdev" , buffer,5 ) ==0 ) +return MKFS_CONFIG_SECTION_RTDEV ; +if (strncmp ("sector" , buffer,6 ) ==0 ) +return MKFS_CONFIG_SECTION_SECTOR ; + +return MKFS_CONFIG_SECTION_INVALID ; +}
Directly parsed the section name to turn it into an enum? Indeed, this looks all wrong in terms of structure.
static struct confopts blockopts = {
.name = "[block]",
.subopts = {
[B_SIZE] = "size",
},
.parser = block_config_parser,
.seen = false,
};struct confopts *get_confopts(const char *section)
{
if (strcmp(blockopts.name, section) == 0)
return &blockopts;
.....
}.... opts = get_confopts(section); if (!opts) /* fail, invalid section */ if (opts->seen) /* fail, multiple specification */ /* loop extracting and processing sbuopts */
And now the whole section enum construct and theparse_config_subopts() function goes away.
Quoted 17 lines
mkfs/xfs_mkfs_config.c new file +17
-0+static int mkfs_check_config_file_limits (const char *filename, +const struct stat *sb, +unsigned int max_entries) +{ +unsigned long long size_limit; + + size_limit =CONFIG_MAX_BUFFER * max_entries; + +/* +* Detect wrap around -- if max_entries * size_limit goves over +* unsigned long long we detect that there. Note some libraries, +* like libiniconfig, only groks max INT_MAX entries anyway, so +* if we ever use a library for parsing we'd be restricted to that +* limit. +*/ +if (size_limit < max_entries || max_entries >INT_MAX ) +return -E2BIG ;
mkfs/xfs_mkfs_config.c line 449
Quoted 10 lines
mkfs/xfs_mkfs_config.c new file +10
-0+ +if (sb->st_size > size_limit) { +fprintf (stderr, +"File %s is too big! File size limit: %llu\n" , + filename, size_limit); +return -E2BIG ; + } + +return 0 ; +}
What is this supposed to protect again? If somone what to put lots of comments or blank space in the config files, then what's the problem with that? Seems like you jump through a lot of hoops to do just this calculation - why not just say "1MB is more than enough for anyone"?
mkfs/xfs_mkfs_config.c line 459
Quoted 10 lines
mkfs/xfs_mkfs_config.c new file +9
-0+enum parse_line_type { +PARSE_COMMENT =0 , +PARSE_SECTION , +PARSE_TAG_VALUE , +PARSE_INVALID , +PARSE_EOF , +}; + +static int parse_line_section (const char *line,char *tag)
I should have mentioned this before now, but it's finally got to me. :) Function format for XFS code is:
static int
parse_line_section(
const char *line, char *tag)
{
....
mkfs/xfs_mkfs_config.c line 470
Quoted 11 lines
mkfs/xfs_mkfs_config.c new file +11
-0+{ +return sscanf (line," [%[^]]s]" , tag); +} + +static int parse_line_tag_value (const char *line,char *tag, +unsigned int *value) +{ +return sscanf (line," %[^ \t=]" +" = " +"%u " , + tag, value);
Value can be a 64 bit quantity, so needs to be uint64_t....
mkfs/xfs_mkfs_config.c line 479
Quoted 9 lines
mkfs/xfs_mkfs_config.c new file +9
-0+} + +static enum parse_line_type parse_get_line_type (const char *line,char *tag, +bool *value) +{ +int ret; +unsigned int uint_value; + +memset (tag,0 ,80 );
Why? This should be done in the caller, and if there's a fixed buffer size, then please use a #define.
mkfs/xfs_mkfs_config.c line 482
Quoted 3 lines
mkfs/xfs_mkfs_config.c new file +3
-0+ +if (strlen (line) <2 ) +return PARSE_INVALID ;
So empty lines return PARSE_INVALID?
mkfs/xfs_mkfs_config.c line 485
Quoted 3 lines
mkfs/xfs_mkfs_config.c new file +3
-0+ +if (line[0 ] =='#' ) +return PARSE_COMMENT ;
What about lines starting with whitespace? e.g. " # comment"
Quoted 6 lines
mkfs/xfs_mkfs_config.c new file +6
-0+ ret =parse_line_section (line, tag); +if (ret ==1 ) +return PARSE_SECTION ; + +if (ret ==EOF ) +return PARSE_EOF ;
mkfs/xfs_mkfs_config.c line 502
Quoted 10 lines
mkfs/xfs_mkfs_config.c new file +10
-0+ + ret =parse_line_tag_value (line, tag, &uint_value); +if (ret ==2 ) { +if (uint_value !=1 && uint_value !=0 ) +return -EINVAL ; + + *value = !!uint_value; + +return PARSE_TAG_VALUE ; + }
pass the full value back to the caller, let the subopt parser validate it and squash it to the correct type.
mkfs/xfs_mkfs_config.c line 510
Quoted 8 lines
mkfs/xfs_mkfs_config.c new file +8
-0+ +if (ret ==EOF ) +return PARSE_EOF ; + +return PARSE_INVALID ; +} + +int parse_config_init (struct mkfs_default_params *dft)
int
parse_config_file(
struct mkfs_default_params *dft,
const char *config_file)
{
i.e. we only get called if there's a config file configured.
Quoted 12 lines
mkfs/xfs_mkfs_config.c new file +12
-0+{ +int ret; +FILE *fp; +struct stat sp; +unsigned int num_subopts_sections =sizeof (config_subopt_tab) / +sizeof (struct config_subopts ) -1 ; +char *p; +char line[80 ], tag[80 ]; +bool value; +enum parse_line_type parse_type; +enum mkfs_config_section section =MKFS_CONFIG_SECTION_INVALID ; +
mkfs/xfs_mkfs_config.c line 532
Quoted 10 lines
mkfs/xfs_mkfs_config.c new file +10
-0+ fp =fopen (dft->config_file,"r" ); +if (!fp) { +if (dft->type !=DEFAULTS_BUILTIN ) { +fprintf (stderr,_ ("could not open config file: %s\n" ), + dft->config_file); + ret = -ENOENT ; + }else + ret =0 ; +return ret; + }
This should be split out into a separate function that takes the config file and tries to open it. If it's a relative path that was supplied, then this function can also try all the configured search paths for config files.
Quoted 4 lines
mkfs/xfs_mkfs_config.c new file +4
-0+/* +* If we found a file without it being specified on the command line, +* it would be the default configuration file. +*/
mkfs/xfs_mkfs_config.c line 547
Quoted 10 lines
mkfs/xfs_mkfs_config.c new file +10
-0+if (dft->type ==DEFAULTS_BUILTIN ) + dft->type =DEFAULTS_CONFIG ; + + ret =stat (dft->config_file, &sp); +if (ret) { +if (dft->type !=DEFAULTS_BUILTIN ) +fprintf (stderr,_ ("could not stat() config file: %s: %s\n" ), + dft->config_file,strerror (ret)); +return ret; + }
This dft->type futzing can all go away - if the file exists, the mkfs will already know where it came from.
mkfs/xfs_mkfs_config.c line 558
Quoted 11 lines
mkfs/xfs_mkfs_config.c new file +11
-0+ +if (!sp.st_size) +return 0 ; + + ret =mkfs_check_config_file_limits (dft->config_file, &sp, + num_subopts_sections); +if (ret) +return ret; + +while (!feof (fp)) { + p =fgets (line,sizeof (line), fp);
What if the line is longer than 80 characters? e.g. a long comment In that case, we need a line[79] = '/0';, or a memset of line to clear it before fgets is called. i think I prefer the latter...
Quoted 12 lines
mkfs/xfs_mkfs_config.c new file +12
-0+if (!p) +continue ; + + parse_type =parse_get_line_type (line, tag, &value); + +switch (parse_type) { +case PARSE_COMMENT : +case PARSE_INVALID : +case PARSE_EOF : +break ; +case PARSE_SECTION : + section =get_config_section (tag);
mkfs/xfs_mkfs_config.c line 580
Quoted 10 lines
mkfs/xfs_mkfs_config.c new file +10
-0+if (!section) { +fprintf (stderr,_ ("Invalid section: %s\n" ), + tag); +return -EINVAL ; + } +break ; +case PARSE_TAG_VALUE : +/* Trims white spaces */ +snprintf (line,sizeof (line),"%s=%u" , tag, value); + ret =parse_config_subopts (section, value, line, dft);
We've already got the tag and value as discrete variables - why put them back into an ascii string to pass to another function for it to split them back into discrete variables?
This really seems like it could be improved by making this less abstract. i.e.
while (!feof(fp)) {
memset(line, 0, sizeof(line));
p = fgets(line, sizeof(line), fp);
if (!p)
continue;
token = parse_section(line);
if (token) {
opts = get_confopts(section);
if (!opts)
/* fail, invalid */;
if (opts->seen)
/* fail, respec */ ;
}
if (!opts)
continue;
token = parse_value(line, &value);
if (!token)
continue;
error = opts->parse(opts, token, value);
if (error) {
/* warn, fail, ignore? */ ;
}
}I suspect with this even all the B_SIZE type of enums can go away, too, because all we need to do is string compares of the token directly in the suboption parser to know what option we are about to set....
mkfs/xfs_mkfs_config.h line 6
Quoted 12 lines
mkfs/xfs_mkfs_config.h new file +6
-0@@ -0,0 +1,11 @@ +#ifndef _XFS_MKFS_CONFIG_H +#define _XFS_MKFS_CONFIG_H + +#include "xfs_mkfs_common.h" + +#define MKFS_XFS_CONF_DIR ROOT_SYSCONFDIR "/mkfs.xfs.d/"
This can go in the C file, as there's no need for anything other than the file open routine to know this.
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Re: [PATCH v2 5/5] mkfs.xfs: add configuration file parsing support using our own parser
Quoted 13 lines · Dave Chinner
Quoted 12 lines · Luis R. Rodriguez
mkfs/xfs_mkfs.c +2
-1@@ -3077,11 +3078,13 @@ print_mkfs_cfg(struct mkfs_params *cfg, char *dfile, char *logfile, - char *rtfile) + char *rtfile, + struct mkfs_default_params *dft) { struct sb_feat_args *fp = &cfg->sb_feat;
Quoted 16 lines
mkfs/xfs_mkfs.c print_mkfs_cfg( · line 3100
Quoted 13 lines
mkfs/xfs_mkfs.c +2
-0printf(_( +"config-file=%-22s\n" "meta-data=%-22s isize=%-6d agcount=%lld, agsize=%lld blks\n" " =%-22s sectsz=%-5u attr=%u, projid32bit=%u\n" " =%-22s crc=%-8u finobt=%u, sparse=%u, rmapbt=%u, reflink=%u\n"@@ -3091,6 +3094,7 @@ print_mkfs_cfg("log =%-22s bsize=%-6d blocks=%lld, version=%d\n" " =%-22s sectsz=%-5u sunit=%d blks, lazy-count=%d\n" "realtime =%-22s extsz=%-6d blocks=%lld, rtextents=%lld\n"), + dft->type !=DEFAULTS_BUILTIN ? dft->config_file :"empty" , dfile, cfg->inodesize, (long long )cfg->agcount, (long long )cfg->agsize, "", cfg->sectorsize, fp->attr_version, fp->projid32bit,Haven't we already printed where the config was sourced from? Why do we need to print it again here?
The other print describes the enum, this print prints out the *used* config file path if a config file was actually used. Without the other print this would just print either the config file or empty.
Since we are going to remove the environment variable option, then I think the other print is now not needed anymore and my preference is to keep it here.
Thoughts?
Quoted 12 lines
Quoted 12 lines
mkfs/xfs_mkfs.c +1
-0@@ -3665,6 +3669,7 @@ rewrite_secondary_superblocks(/* build time defaults */ struct mkfs_default_params built_in_dft = { .type =DEFAULTS_BUILTIN , + .config_file =MKFS_XFS_CONF_DIR "default" , .sectorsize =XFS_MIN_SECTORSIZE , .blocksize =1 <<XFS_DFL_BLOCKSIZE_LOG , .sb_feat = {@@ -3714,6 +3719,13 @@ static void process_defaults(struct mkfs_default_params *dft, struct cli_params *cli) {
Quoted 8 lines
mkfs/xfs_mkfs.c static void process_defaults( · line 3727
Quoted 6 lines
mkfs/xfs_mkfs.c +6
-0+int ret; + + ret =parse_config_init (dft); + +if (ret && dft->type !=DEFAULTS_BUILTIN ) +usage ();That doesn't make any sense in isolation.
Not sure what you meant at first, but I see that you prefer this to be hidden from the caller. Will change.
Quoted 15 lines
Quoted 15 lines
mkfs/xfs_mkfs.c +2
-0install_defaults (dft, cli); }@@ -3750,6 +3762,8 @@ main( };struct mkfs_params cfg = {};struct mkfs_default_params dft; +char *tmp_config; +char custom_config[PATH_MAX ];reset_defaults_and_cli (&dft, &cli);@@ -3760,21 +3774,36 @@ main(textdomain (PACKAGE );
Quoted 15 lines
mkfs/xfs_mkfs.c main( · line 3784
Quoted 12 lines
mkfs/xfs_mkfs.c +1
-3/* -* TODO: Sourcing defaults from a config file -* * Before anything else, see if there's a config file with different -* defaults. If a file exists in <package location>, read in the new +* defaults. If a file exists in MKFS_XFS_CONF_DIR/default, read the new * default values and overwrite them in the &dft structure. This way the * new defaults will apply before we parse the CLI, and the CLI will * still be able to override them. When more than one source is * implemented, emit a message to indicate where the defaults being * used came from. */ This comment makes no mention of environment variables, or how they interact with other sources of config files.
As per Eric's request, We're getting rid of the environment variable option, so mentioning it is no longer needed.
Quoted 6 lines
Quoted 6 lines
mkfs/xfs_mkfs.c +5
-0+ tmp_config =getenv ("MKFS_XFS_CONFIG" ); +if (tmp_config !=NULL ) { + dft.config_file = tmp_config; + dft.type =DEFAULTS_ENVIRONMENT_CONFIG ; + }
Quoted 6 lines
mkfs/xfs_mkfs.c main( · line 3791
Quoted 1 line
mkfs/xfs_mkfs.c +0
-0process_defaults (&dft, &cli);So why are we processing the defaults before we've checked if a default file is specified on the command line? All this should do is set up the config file to be read, and set the dft.source =
_("Environment Variable");
This was being done *here* since I previously has setup to process the arguments *early* and check if a -c was specified, and if so use the config file for the defaults, otherwise the environment variable if set. But since the way I had implemented processing the arguments early relied on not well guaranteed mechanisms I resorted to avoid that approach.
This was the alternative I came up with.
The environment variable stuff is going away so this should be much simpler now, however the question of argument processing early still remains.
Quoted 9 lines
mkfs/xfs_mkfs.c main( · line 3796
Quoted 6 lines
mkfs/xfs_mkfs.c +3
-1-while ((c =getopt (argc, argv,"b:d:i:l:L:m:n:KNp:qr:s:CfV" )) !=EOF ) { +while ((c =getopt (argc, argv,"b:c:d:i:l:L:m:n:KNp:qr:s:CfV" )) !=EOF ) {switch (c) { +case 'c' : +reset_defaults_and_cli (&dft, &cli);This will trash the existing CLI inputs that have already been parsed.
Exactly.
Quoted 2 lines
All this code here should do is set up the config file to be read.
Consider the case of /etc/mkfs.d/default existing and it having
[metadata]
crc = 1
[naming]
ftype=1
And suppose the user passed -c foo which had:
[metadata]
crc = 0
Without the reset we'd make /etc/mkfs.d/default the system wide defaults onto which -c parameters overlay on top of. I figured that was not desirable.
With built-in defaults as *base* and with the compromise that if -c is used it will reset to built-in defaults, we remove that overlay problem.
Quoted 10 lines
mkfs/xfs_mkfs.c main( · line 3804
Quoted 8 lines
mkfs/xfs_mkfs.c +8
-0+ +memset (custom_config,0 ,sizeof (custom_config)); +snprintf (custom_config,sizeof (custom_config),"%s%s" , +MKFS_XFS_CONF_DIR , optarg); + dft.config_file = custom_config; + dft.type =DEFAULTS_TYPE_CONFIG ; + +process_defaults (&dft, &cli);Again, what happens if the user specifies multiple config files?
The reset strategy lets the user set -c as many times as they wish and also starts from a clean base always.
Quoted 2 lines
The second config file will trash everything from the first, as well as any other options between them.
Yes! By design as we couldn't easily parse arguments early first and then choose just one strategy.
I liked the simplicity that this brings.
Would you really want multiple -c options to work as overlays, one on top of the other?
Quoted 3 lines
I think that we need to pull the "-c <file>" option from the command line and process all the defaults /before/ we enter the main command line processing loop.
That is what I had done originally but ran into that snag of processing arguments twice and the undocumented functionality I found that worked.
Quoted 12 lines
IOWs:
config_file = getenv("MKFS_XFS_CONFIG"); if (config_file) dft.source = _("Environment Variable"); /* * Pull config line options from command line */ while ((c = getopt(argc, argv, "c:")) != EOF) { switch (c) { case 'c': /* XXX: fail if already seen a CLI option */
BTW isn't my enum here sufficient to check for this easily in a clean short and easy way? Granted we'd have only built-in and config, but still... simple and sweet, no?
And do we want to support multiple -c options?
Quoted 22 lines
config_file = optarg;
dft.source = _("Command Line");
break;
default:
continue;
}
}
if (config_file) {
/*
* parse_defaults_file() knows about config file
* search paths, so just pass in the raw,
* unvalidated filename and let it do all the work
* finding the file containing the default values.
*/
error = parse_defaults_file(&dft, config_file);
if (error) {
/* fail! */
}
}
printf(_("Default configuration sourced from %s\n"), dft.source);Quoted 7 lines
/* * Done parsing defaults now, so memcpy defaults into CLI * structure, reset getopt and start parsing CLI options */ memcpy(/* sb_feats */); memcpy(/* fsxattr */); optind = 1;
My getopt(3) *does* not make mention of any of this. This man page however does:
http://man7.org/linux/man-pages/man3/getopt.3.html
NOTES
A program that scans multiple argument vectors, or rescans the same vector more than once, and wants to make use of GNU extensions such as '+' and '-' at the start of optstring, or changes the value of POSIXLY_CORRECT between scans, must reinitialize getopt() by resetting optind to 0, rather than the traditional value of 1. (Resetting to 0 forces the invocation of an internal initialization routine that rechecks POSIXLY_CORRECT and checks for GNU extensions in optstring.)
But... my man page does say:
The variable optind is the index of the next element of the argv[] vector to be processed. It shall be initialized to 1 by the system, and getopt() shall update it when it finishes with each element of argv[]. If the application sets optind to zero before calling getopt(), the behavior is unspecified.
So as per the above man page URL, setting it to 1 seems OK... but my man page just lacks any documentation over this. I don't think we use the GNU extensions mentioned of '+' or '-'.
POSIXLY_CORRECT seems to be an used on the environment, and helps Bash determine if it should enter into POSIX mode, before reading startup files.
We don't set or use POSIXLY_CORRECT anywhere so I think we're fine with the:
optind = 1;
However perhaps a big fat NOTE is worthy?
Quoted 8 lines
while ((c = getopt(argc, argv, "b:c:d:i:l:L:m:n:KNp:qr:s:CfV")) != EOF) {
switch (c) {
case 'c':
/* already validated and parsed, ignore */
break;
....I can live with the above.
Quoted 18 lines
Quoted 18 lines
mkfs/xfs_mkfs_common.h +8
-0@@ -46,9 +46,20 @@ struct sb_feat_args {* These are the different possibilities by which you can end up parsing * default settings with. DEFAULTS_BUILTIN indicates there was no configuration * file parsed and we are using the built-in defaults on this code. +* DEFAULTS_CONFIG means the default configuration file was found and used. +* DEFAULTS_TYPE_CONFIG means the user asked for a custom configuration type +* and it was used, this means the default directory for mkfs.xfs +* configuration files was used to look for the type specified. If you need +* to override the default mkfs.xfs directory for configuration file you can +* use the MKFS_XFS_CONFIG environment variable to set the absolute path to +* your custom configuration file, when this is set the type is set to +* DEFAULTS_ENVIRONMENT_CONFIG. */
Quoted 8 lines
mkfs/xfs_mkfs_common.h struct sb_feat_args { · line 63
Quoted 6 lines
mkfs/xfs_mkfs_common.h +3
-0enum default_params_type {DEFAULTS_BUILTIN =0 , +DEFAULTS_CONFIG , +DEFAULTS_ENVIRONMENT_CONFIG , +DEFAULTS_TYPE_CONFIG , };Again, I just don't see the need for this...
Given what has decided so far, only documentation and checking for it being set already.
Quoted 8 lines
mkfs/xfs_mkfs_common.h enum default_params_type { · line 75
Quoted 6 lines
mkfs/xfs_mkfs_common.h +1
-0/* @@ -61,6 +72,7 @@ enum default_params_type {*/ struct mkfs_default_params {enum default_params_type type;/* where the defaults came from */ +const char *config_file;Or this (see above :).
Alright.
Quoted 33 lines
Quoted 33 lines
mkfs/xfs_mkfs_common.h +0
-0int sectorsize;int blocksize;mkfs/xfs_mkfs_config.c new file +24
-0@@ -0,0 +1,591 @@ +/* +* Copyright (c) 2018 Luis R. Rodriguez <[email protected]> +* +* This program is free software; you can redistribute it and/or +* modify it under the terms of the GNU General Public License as +* published by the Free Software Foundation. +* +* This program is distributed in the hope that it would be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with this program; if not, write the Free Software Foundation, +* Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +*/ + +#include "xfs_mkfs_common.h" +#include "xfs_mkfs_config.h" + +#define CONFIG_MAX_KEY 1024 +#define CONFIG_MAX_VALUE PATH_MAX +#define CONFIG_MAX_BUFFER CONFIG_MAX_KEY + CONFIG_MAX_VALUE + 3 +
Quoted 8 lines
mkfs/xfs_mkfs_config.c line 30
Quoted 6 lines
mkfs/xfs_mkfs_config.c new file +6
-0+/* +* Enums for each configuration option. All these currently match the CLI +* parameters for now but this may change later, so we keep all this code +* and definitions separate. The rules for configuration parameters may also +* differ. +*/ Why define them all when we are only going to parse a small subset?
I liked enums over random 1 characters which have no meaning for configuration file parsing. I just needed an index symbol.
Quoted 13 lines
Quoted 13 lines
mkfs/xfs_mkfs_config.c new file +13
-0+enum mkfs_config_section { +MKFS_CONFIG_SECTION_BLOCK =0 , +MKFS_CONFIG_SECTION_DATA , +MKFS_CONFIG_SECTION_INODE , +MKFS_CONFIG_SECTION_LOG , +MKFS_CONFIG_SECTION_METADATA , +MKFS_CONFIG_SECTION_NAMING , +MKFS_CONFIG_SECTION_RTDEV , +MKFS_CONFIG_SECTION_SECTOR , + +MKFS_CONFIG_SECTION_INVALID , +}; +
Quoted 8 lines
mkfs/xfs_mkfs_config.c line 135
Quoted 6 lines
mkfs/xfs_mkfs_config.c new file +6
-0+struct opt_params { +const enum mkfs_config_section section; +const char *subopts[MAX_SUBOPTS ]; +}; + +extern struct opt_params config_sopts;Why is this declared here?
Nuked.
Quoted 6 lines
mkfs/xfs_mkfs_config.c line 138
Quoted 3 lines
mkfs/xfs_mkfs_config.c new file +3
-0+ +struct opt_params config_bopts = { + .section ='b' ,That's not an enum, looks broken. Why isn't this the ascii name of the config section in the config file?
Sorry, the entire .section crap can be removed. Its not needed. I had already indexed the array with the above enums.
Yeap, it is not needed.
I forgot to nuke it as I tossed things which were not needed out the window. I forgot the lamp.
Quoted 19 lines
Quoted 19 lines
mkfs/xfs_mkfs_config.c new file +19
-0+ .subopts = { + [B_SIZE] = "size", + }, +}; + +struct opt_params config_dopts = { + .section ='d' , + .subopts = { + [D_AGCOUNT ] ="agcount" , + [D_FILE ] ="file" , + [D_NAME ] ="name" , + [D_SIZE ] ="size" , + [D_SUNIT ] ="sunit" , + [D_SWIDTH ] ="swidth" , + [D_AGSIZE ] ="agsize" , + [D_SU ] ="su" , + [D_SW ] ="sw" , + [D_SECTSIZE ] ="sectsize" , + [D_NOALIGN ] ="noalign" ,
Quoted 8 lines
mkfs/xfs_mkfs_config.c line 163
Quoted 6 lines
mkfs/xfs_mkfs_config.c new file +6
-0+ [D_RTINHERIT] = "rtinherit", + [D_PROJINHERIT] = "projinherit", + [D_EXTSZINHERIT] = "extszinherit", + [D_COWEXTSIZE] = "cowextsize", + }, +};So the parser will accept names we can't configure?
No, it will not, it will depend on the individual parser to handle it on its switch. Default is to return -EINVAL by each of them.
I could remove all unused ones, but figured there was no harm to keep what we know exists.
I'll yield to whatever is desired.
Quoted 18 lines
[...]
Quoted 4 lines
mkfs/xfs_mkfs_config.c new file +4
-0+const char *default_type_str (enum default_params_type type) +{ +switch (type) { +case DEFAULTS_BUILTIN :mkfs/xfs_mkfs_config.c line 248
Quoted 10 lines
mkfs/xfs_mkfs_config.c new file +10
-0+return _ ("package built-in definitions" ); +case DEFAULTS_CONFIG : +return _ ("default configuration system file" ); +case DEFAULTS_ENVIRONMENT_CONFIG : +return _ ("custom configuration file set by environment" ); +case DEFAULTS_TYPE_CONFIG : +return _ ("custom configuration type file" ); + } +return _ ("Unkown\n" ); +}This function can go away.
Well depends, if we keep the enum the no ?
Quoted 8 lines
mkfs/xfs_mkfs_config.c line 254
Quoted 5 lines
mkfs/xfs_mkfs_config.c new file +5
-0+static int block_config_parser (struct mkfs_default_params *dft,int subopt, +bool value) +{ +return -EINVAL ; +}Why is the value passed to the parser a bool type and not a
uint64_t?
Because we decided supporting bool right now is OK in my last iteration. And be we, I mean Eric was fine with this as a first incarnation.
Passing the uint64_t is fine and make that change on the next iteration.
Quoted 38 lines
Quoted 38 lines
mkfs/xfs_mkfs_config.c new file +38
-0+struct config_subopts { +enum mkfs_config_section section; +struct opt_params *opts; +int (*config_parser)(struct mkfs_default_params *dft, +int subop,bool value); +} config_subopt_tab[] = { + {MKFS_CONFIG_SECTION_BLOCK , &config_bopts, block_config_parser }, + {MKFS_CONFIG_SECTION_DATA , &config_dopts, data_config_parser }, + {MKFS_CONFIG_SECTION_INODE , &config_iopts, inode_config_parser }, + {MKFS_CONFIG_SECTION_LOG , &config_lopts, log_config_parser }, + {MKFS_CONFIG_SECTION_METADATA , &config_mopts, meta_config_parser }, + {MKFS_CONFIG_SECTION_NAMING , &config_nopts, naming_config_parser }, + {MKFS_CONFIG_SECTION_RTDEV , &config_ropts, rtdev_config_parser }, + {MKFS_CONFIG_SECTION_SECTOR , &config_sopts, sector_config_parser }, + {'\0' ,NULL }, +}; + +static int +parse_config_subopts ( +enum mkfs_config_section section, +bool value, +char *line, +struct mkfs_default_params *dft) +{ +struct config_subopts *sop = &config_subopt_tab[0 ]; +char **subopts = (char **)sop->opts->subopts; +int subopt; +char *val; + +while (sop->opts) { +if (sop->section == section) +break ; + sop++; + } + +/* should never happen */ +if (!sop->opts) +return -EINVAL ;
Quoted 8 lines
mkfs/xfs_mkfs_config.c line 399
Quoted 5 lines
mkfs/xfs_mkfs_config.c new file +5
-0+ + subopts = (char **)sop->opts->subopts; + subopt =getsubopt (&line, subopts, &val); +return (sop->config_parser)(dft, subopt, value); +}This seems back to front - why do we walk the table to find the entry that corresponds to an enum when we've....
Yeah you're right this is stupid legacy crap, from the CLI stuff, we can just do config_subopt_tab[section] provided its < MAX_SECTION or whatever.
Quoted 6 lines
mkfs/xfs_mkfs_config.c line 404
Quoted 4 lines
mkfs/xfs_mkfs_config.c new file +4
-0+static enum mkfs_config_section get_config_section (const char *buffer) +{ +if (strncmp ("block" , buffer,5 ) ==0 ) +return MKFS_CONFIG_SECTION_BLOCK ;This will match "blocksdfsgdfgd" - strcmp is fine here.
OK!
Quoted 12 lines
Quoted 12 lines
mkfs/xfs_mkfs_config.c new file +12
-0+if (strncmp ("data" , buffer,4 ) ==0 ) +return MKFS_CONFIG_SECTION_DATA ; +if (strncmp ("inode" , buffer,5 ) ==0 ) +return MKFS_CONFIG_SECTION_INODE ; +if (strncmp ("log" , buffer,3 ) ==0 ) +return MKFS_CONFIG_SECTION_LOG ; +if (strncmp ("metadata" , buffer,8 ) ==0 ) +return MKFS_CONFIG_SECTION_METADATA ; +if (strncmp ("naming" , buffer,6 ) ==0 ) +return MKFS_CONFIG_SECTION_NAMING ; +if (strncmp ("rtdev" , buffer,5 ) ==0 ) +return MKFS_CONFIG_SECTION_RTDEV ;
Quoted 8 lines
mkfs/xfs_mkfs_config.c line 421
Quoted 5 lines
mkfs/xfs_mkfs_config.c new file +5
-0+if (strncmp ("sector" , buffer,6 ) ==0 ) +return MKFS_CONFIG_SECTION_SECTOR ; + +return MKFS_CONFIG_SECTION_INVALID ; +}Directly parsed the section name to turn it into an enum? Indeed, this looks all wrong in terms of structure.
Given the above change I noted I should have done for the index, this will give us a 1-1 mapping to that required array.
Quoted 9 lines
static struct confopts blockopts = {
.name = "[block]",
.subopts = {
[B_SIZE] = "size",
},
.parser = block_config_parser,
.seen = false,
};Quoted 7 lines
struct confopts *
get_confopts(const char *section){ if (strcmp(blockopts.name, section) == 0) return &blockopts; ..... }
Yeah that works too.
Quoted 6 lines
.... opts = get_confopts(section); if (!opts) /* fail, invalid section */ if (opts->seen) /* fail, multiple specification */
Good point.
Quoted 5 lines
/* loop extracting and processing sbuopts */And now the whole section enum construct and the
parse_config_subopts()function goes away.
Alright.
Quoted 21 lines
Quoted 21 lines
mkfs/xfs_mkfs_config.c new file +21
-0+static int mkfs_check_config_file_limits (const char *filename, +const struct stat *sb, +unsigned int max_entries) +{ +unsigned long long size_limit; + + size_limit =CONFIG_MAX_BUFFER * max_entries; + +/* +* Detect wrap around -- if max_entries * size_limit goves over +* unsigned long long we detect that there. Note some libraries, +* like libiniconfig, only groks max INT_MAX entries anyway, so +* if we ever use a library for parsing we'd be restricted to that +* limit. +*/ +if (size_limit < max_entries || max_entries >INT_MAX ) +return -E2BIG ; + +if (sb->st_size > size_limit) { + fprintf(stderr, + "File %s is too big! File size limit: %llu\n",
Quoted 8 lines
mkfs/xfs_mkfs_config.c line 449
Quoted 6 lines
mkfs/xfs_mkfs_config.c new file +6
-0+ filename, size_limit); +return -E2BIG ; + } + +return 0 ; +}What is this supposed to protect again?
Just a sanity check, it would be silly to start processing more than we allow.
Quoted 5 lines
If somone what to put lots
of comments or blank space in the config files, then what's the problem with that? Seems like you jump through a lot of hoops to do just this calculation - why not just say "1MB is more than enough for anyone"?
Sure.
Quoted 14 lines
mkfs/xfs_mkfs_config.c line 459
Quoted 10 lines
mkfs/xfs_mkfs_config.c new file +9
-0+enum parse_line_type { +PARSE_COMMENT =0 , +PARSE_SECTION , +PARSE_TAG_VALUE , +PARSE_INVALID , +PARSE_EOF , +}; + +static int parse_line_section (const char *line,char *tag)I should have mentioned this before now, but it's finally got to me. :) Function format for XFS code is:
Quoted 6 lines
static int
parse_line_section(const char *line, char *tag){
....
Alright...
Quoted 5 lines
Quoted 5 lines
mkfs/xfs_mkfs_config.c new file +5
-0+{ +return sscanf (line," [%[^]]s]" , tag); +} + +static int parse_line_tag_value(const char *line, char *tag,
Quoted 8 lines
mkfs/xfs_mkfs_config.c line 470
Quoted 6 lines
mkfs/xfs_mkfs_config.c new file +6
-0+ unsigned int *value) +{ +return sscanf (line," %[^ \t=]" +" = " +"%u " , + tag, value);Value can be a 64 bit quantity, so needs to be
uint64_t....
Alright.
Quoted 7 lines
mkfs/xfs_mkfs_config.c line 477
Quoted 7 lines
mkfs/xfs_mkfs_config.c new file +7
-0+} + +static enum parse_line_type parse_get_line_type (const char *line,char *tag, +bool *value) +{ +int ret; +unsigned int uint_value;
We're gonna want uint64_t here too.
Quoted 11 lines
mkfs/xfs_mkfs_config.c line 479
Quoted 2 lines
mkfs/xfs_mkfs_config.c new file +2
-0+ +memset (tag,0 ,80 );Why? This should be done in the caller, and if there's a fixed buffer size, then please use a #define.
mkfs/xfs_mkfs_config.c line 482
Quoted 3 lines
mkfs/xfs_mkfs_config.c new file +3
-0+ +if (strlen (line) <2 ) +return PARSE_INVALID ;So empty lines return
PARSE_INVALID?
Yes, but that's not fatal, we just discard it.
Quoted 5 lines
mkfs/xfs_mkfs_config.c line 485
Quoted 3 lines
mkfs/xfs_mkfs_config.c new file +3
-0+ +if (line[0 ] =='#' ) +return PARSE_COMMENT ;What about lines starting with whitespace? e.g. " # comment"
parse_line_section() below would not return 1 and its ignored in the end as its also not a proper tag/value pair and the section is not set.
Quoted 11 lines
Quoted 11 lines
mkfs/xfs_mkfs_config.c new file +11
-0+ ret =parse_line_section (line, tag); +if (ret ==1 ) +return PARSE_SECTION ; + +if (ret ==EOF ) +return PARSE_EOF ; + + ret =parse_line_tag_value (line, tag, &uint_value); +if (ret ==2 ) { +if (uint_value !=1 && uint_value !=0 ) +return -EINVAL ;
Quoted 8 lines
mkfs/xfs_mkfs_config.c line 502
Quoted 5 lines
mkfs/xfs_mkfs_config.c new file +5
-0+ + *value = !!uint_value; + +return PARSE_TAG_VALUE ; + }pass the full value back to the caller, let the subopt parser validate it and squash it to the correct type.
I wanted parse_get_line_type() to give me what type of line we are dealing with. What you describe is still possible if I just change the argument to uint64_t and then the parser squashes it.
Quoted 15 lines
mkfs/xfs_mkfs_config.c line 510
Quoted 8 lines
mkfs/xfs_mkfs_config.c new file +8
-0+ +if (ret ==EOF ) +return PARSE_EOF ; + +return PARSE_INVALID ; +} + +int parse_config_init (struct mkfs_default_params *dft)int
parse_config_file(
structmkfs_default_params*dft,
const char*config_file)
{
Quoted 1 line
i.e. we only get called if there's a config file configured.
Alright.
Quoted 19 lines
Quoted 19 lines
mkfs/xfs_mkfs_config.c new file +19
-0+{ +int ret; +FILE *fp; +struct stat sp; +unsigned int num_subopts_sections =sizeof (config_subopt_tab) / +sizeof (struct config_subopts ) -1 ; +char *p; +char line[80 ], tag[80 ]; +bool value; +enum parse_line_type parse_type; +enum mkfs_config_section section =MKFS_CONFIG_SECTION_INVALID ; + + fp =fopen (dft->config_file,"r" ); +if (!fp) { +if (dft->type !=DEFAULTS_BUILTIN ) { +fprintf (stderr,_ ("could not open config file: %s\n" ), + dft->config_file); + ret = -ENOENT ; + }else
Quoted 8 lines
mkfs/xfs_mkfs_config.c line 532
Quoted 3 lines
mkfs/xfs_mkfs_config.c new file +3
-0+ ret =0 ; +return ret; + }This should be split out into a separate function that takes the config file and tries to open it. If it's a relative path that was supplied, then this function can also try all the configured search paths for config files.
Eric asked to support 3 cases:
a) ./ -- current working directory
b) / -- full path
c) no prefix - we use the sysconfigdir/mkfs.d/
but sure, I'll toss it in there.
Quoted 9 lines
Quoted 9 lines
mkfs/xfs_mkfs_config.c new file +9
-0+/* +* If we found a file without it being specified on the command line, +* it would be the default configuration file. +*/ +if (dft->type ==DEFAULTS_BUILTIN ) + dft->type =DEFAULTS_CONFIG ; + + ret =stat (dft->config_file, &sp); +if (ret) {
Quoted 8 lines
mkfs/xfs_mkfs_config.c line 547
Quoted 5 lines
mkfs/xfs_mkfs_config.c new file +5
-0+if (dft->type !=DEFAULTS_BUILTIN ) +fprintf (stderr,_ ("could not stat() config file: %s: %s\n" ), + dft->config_file,strerror (ret)); +return ret; + }This
dft->typefutzing can all go away - if the file exists, the mkfs will already know where it came from.
Not convinced yet but I will shoot for removing it.
Quoted 5 lines
Quoted 5 lines
mkfs/xfs_mkfs_config.c new file +5
-0+ +if (!sp.st_size) +return 0 ; + + ret = mkfs_check_config_file_limits(dft->config_file, &sp,
Quoted 8 lines
mkfs/xfs_mkfs_config.c line 558
Quoted 6 lines
mkfs/xfs_mkfs_config.c new file +6
-0+ num_subopts_sections); +if (ret) +return ret; + +while (!feof (fp)) { + p =fgets (line,sizeof (line), fp);What if the line is longer than 80 characters? e.g. a long comment
If its a comment or spaces with a comment, we still only care for the first 80 characters, no?
Quoted 2 lines
In that case, we need a line[79] = '/0';, or a memset of line to clear it before fgets is called. i think I prefer the latter...
I thought I memset() already - bleh, no, ok fixed, thanks!
Quoted 18 lines
Quoted 18 lines
mkfs/xfs_mkfs_config.c new file +18
-0+if (!p) +continue ; + + parse_type =parse_get_line_type (line, tag, &value); + +switch (parse_type) { +case PARSE_COMMENT : +case PARSE_INVALID : +case PARSE_EOF : +break ; +case PARSE_SECTION : + section =get_config_section (tag); +if (!section) { +fprintf (stderr,_ ("Invalid section: %s\n" ), + tag); +return -EINVAL ; + } +break ;
Quoted 8 lines
mkfs/xfs_mkfs_config.c line 580
Quoted 4 lines
mkfs/xfs_mkfs_config.c new file +4
-0+case PARSE_TAG_VALUE : +/* Trims white spaces */ +snprintf (line,sizeof (line),"%s=%u" , tag, value); + ret =parse_config_subopts (section, value, line, dft);We've already got the tag and value as discrete variables - why put them back into an ascii string to pass to another function for it to split them back into discrete variables?
The comment above it explains, "Trims white spaces"
Quoted 29 lines
This really seems like it could be improved by making this less abstract. i.e.
while (!feof(fp)) { memset(line, 0, sizeof(line)); p = fgets(line, sizeof(line), fp); if (!p) continue; token = parse_section(line); if (token) { opts = get_confopts(section); if (!opts) /* fail, invalid */; if (opts->seen) /* fail, respec */ ; } if (!opts) continue; token = parse_value(line, &value); if (!token) continue; error = opts->parse(opts, token, value); if (error) { /* warn, fail, ignore? */ ; } }
Quoted 4 lines
I suspect with this even all the
B_SIZEtype of enums can go away, too, because all we need to do is string compares of the token directly in the suboption parser to know what option we are about to set....
I'll give it a shot.
Quoted 16 lines
mkfs/xfs_mkfs_config.h line 6
Quoted 13 lines
mkfs/xfs_mkfs_config.h new file +6
-0@@ -0,0 +1,11 @@ +#ifndef _XFS_MKFS_CONFIG_H +#define _XFS_MKFS_CONFIG_H + +#include "xfs_mkfs_common.h" + +#define MKFS_XFS_CONF_DIR ROOT_SYSCONFDIR "/mkfs.xfs.d/"This can go in the C file, as there's no need for anything other than the file open routine to know this.
Sure.
Luis
Re: [PATCH v2 5/5] mkfs.xfs: add configuration file parsing support using our own parser
Quoted 31 lines · Luis R. Rodriguez
Quoted 13 lines · Dave Chinner
Quoted 12 lines · Luis R. Rodriguez
mkfs/xfs_mkfs.c +2
-1@@ -3077,11 +3078,13 @@ print_mkfs_cfg(struct mkfs_params *cfg, char *dfile, char *logfile, - char *rtfile) + char *rtfile, + struct mkfs_default_params *dft) { struct sb_feat_args *fp = &cfg->sb_feat;Quoted 16 lines
mkfs/xfs_mkfs.c print_mkfs_cfg( · line 3100
Quoted 13 lines
mkfs/xfs_mkfs.c +2
-0printf(_( +"config-file=%-22s\n" "meta-data=%-22s isize=%-6d agcount=%lld, agsize=%lld blks\n" " =%-22s sectsz=%-5u attr=%u, projid32bit=%u\n" " =%-22s crc=%-8u finobt=%u, sparse=%u, rmapbt=%u, reflink=%u\n"@@ -3091,6 +3094,7 @@ print_mkfs_cfg("log =%-22s bsize=%-6d blocks=%lld, version=%d\n" " =%-22s sectsz=%-5u sunit=%d blks, lazy-count=%d\n" "realtime =%-22s extsz=%-6d blocks=%lld, rtextents=%lld\n"), + dft->type !=DEFAULTS_BUILTIN ? dft->config_file :"empty" , dfile, cfg->inodesize, (long long )cfg->agcount, (long long )cfg->agsize, "", cfg->sectorsize, fp->attr_version, fp->projid32bit,Haven't we already printed where the config was sourced from? Why do we need to print it again here?
Quoted 3 lines
The other print describes the enum, this print prints out the *used* config file path if a config file was actually used. Without the other print this would just print either the config file or empty.
If you look at the changes I proposed, I suggested we change the initial print out the path of the source file, not how the user specified the source file. So this becomes redundant. And given this code is now shared with xfs_info, it has no idea about mkfs config files, so it's not ideal to be adding mkfs-specific stuff to this output.
Further....
Quoted 5 lines
Since we are going to remove the environment variable option, then I think the other print is now not needed anymore and my preference is to keep it here.
Thoughts?
.... this printout only happens after al input has been validated and we're about to make the filesystem. If there's a validation problem, nobody knows what file the defaults were sourced from. IOWs, the file the default are sourced from needs to be printed even before the file is read...
Quoted 16 lines
Quoted 7 lines
Quoted 7 lines
mkfs/xfs_mkfs.c +0
-0.sectorsize = XFS_MIN_SECTORSIZE, .blocksize = 1 << XFS_DFL_BLOCKSIZE_LOG, .sb_feat = {@@ -3714,6 +3719,13 @@ static void process_defaults(struct mkfs_default_params *dft, struct cli_params *cli) {Quoted 8 lines
mkfs/xfs_mkfs.c static void process_defaults( · line 3727
Quoted 6 lines
mkfs/xfs_mkfs.c +6
-0+int ret; + + ret =parse_config_init (dft); + +if (ret && dft->type !=DEFAULTS_BUILTIN ) +usage ();That doesn't make any sense in isolation.
Quoted 2 lines
Not sure what you meant at first, but I see that you prefer this to be hidden from the caller. Will change.
I meant I have no idea what this is checking and why there's a usage call there because usage() is for CLI input, not setting up defaults before CLI processing. Explanation in comments are needed.
Quoted 31 lines
Quoted 15 lines
Quoted 15 lines
mkfs/xfs_mkfs.c +2
-0install_defaults (dft, cli); }@@ -3750,6 +3762,8 @@ main( };struct mkfs_params cfg = {};struct mkfs_default_params dft; +char *tmp_config; +char custom_config[PATH_MAX ];reset_defaults_and_cli (&dft, &cli);@@ -3760,21 +3774,36 @@ main(textdomain (PACKAGE );Quoted 15 lines
mkfs/xfs_mkfs.c main( · line 3784
Quoted 12 lines
mkfs/xfs_mkfs.c +1
-3/* -* TODO: Sourcing defaults from a config file -* * Before anything else, see if there's a config file with different -* defaults. If a file exists in <package location>, read in the new +* defaults. If a file exists in MKFS_XFS_CONF_DIR/default, read the new * default values and overwrite them in the &dft structure. This way the * new defaults will apply before we parse the CLI, and the CLI will * still be able to override them. When more than one source is * implemented, emit a message to indicate where the defaults being * used came from. */ This comment makes no mention of environment variables, or how they interact with other sources of config files.
Quoted 2 lines
As per Eric's request, We're getting rid of the environment variable option, so mentioning it is no longer needed.
I thought Darrick wanted it kept?
Quoted 46 lines
Quoted 6 lines
Quoted 6 lines
mkfs/xfs_mkfs.c +5
-0+ tmp_config =getenv ("MKFS_XFS_CONFIG" ); +if (tmp_config !=NULL ) { + dft.config_file = tmp_config; + dft.type =DEFAULTS_ENVIRONMENT_CONFIG ; + }Quoted 6 lines
mkfs/xfs_mkfs.c main( · line 3791
Quoted 1 line
mkfs/xfs_mkfs.c +0
-0process_defaults (&dft, &cli);So why are we processing the defaults before we've checked if a default file is specified on the command line? All this should do is set up the config file to be read, and set the dft.source =
_("Environment Variable");This was being done *here* since I previously has setup to process the arguments *early* and check if a -c was specified, and if so use the config file for the defaults, otherwise the environment variable if set. But since the way I had implemented processing the arguments early relied on not well guaranteed mechanisms I resorted to avoid that approach.
This was the alternative I came up with.
The environment variable stuff is going away so this should be much simpler now, however the question of argument processing early still remains.
Quoted 9 lines
mkfs/xfs_mkfs.c main( · line 3796
Quoted 6 lines
mkfs/xfs_mkfs.c +3
-1-while ((c =getopt (argc, argv,"b:d:i:l:L:m:n:KNp:qr:s:CfV" )) !=EOF ) { +while ((c =getopt (argc, argv,"b:c:d:i:l:L:m:n:KNp:qr:s:CfV" )) !=EOF ) {switch (c) { +case 'c' : +reset_defaults_and_cli (&dft, &cli);This will trash the existing CLI inputs that have already been parsed.
Exactly.
Quoted 2 lines
All this code here should do is set up the config file to be read.
Consider the case of
/etc/mkfs.d/defaultexisting and it having[metadata]
crc = 1
[naming]
ftype=1
Quoted 7 lines
And suppose the user passed -c foo which had:
[metadata]
crc = 0Without the reset we'd make
/etc/mkfs.d/defaultthe system wide defaults onto which -c parameters overlay on top of. I figured that was not desirable.
It's not desirable. If the user specified a config file, then /etc/mkfs.d/default *is never read*. The user config file is used instead. If the user supplied file does not exist (even after searching) then we fail, not use some other set of defaults.
Quoted 4 lines
Quoted 1 line
Again, what happens if the user specifies multiple config files?
The reset strategy lets the user set -c as many times as they wish and also starts from a clean base always.
multiple "-c" options is a user error and should error out.
Quoted 10 lines
Quoted 2 lines
The second config file will trash everything from the first, as well as any other options between them.
Yes! By design as we couldn't easily parse arguments early first and then choose just one strategy.
I liked the simplicity that this brings.
Would you really want multiple -c options to work as overlays, one on top of the other?
I don't want multiple -c option to be supported at all. We source defaults from a single file only - either the user specified file, or the default if it exists. Not both, not multiple user specified files. One file.
Quoted 6 lines
Quoted 3 lines
I think that we need to pull the "-c <file>" option from the command line and process all the defaults /before/ we enter the main command line processing loop.
That is what I had done originally but ran into that snag of processing arguments twice and the undocumented functionality I found that worked.
Multiple pass option parsing is documented as supported and has been for a long, long time. And to make that clear, we even have a wrapper in xfsprogs for it:
$ git grep platform_getoptreset db/command.c: platform_getoptreset(); include/darwin.h:static __inline__ void platform_getoptreset(void) include/freebsd.h:static __inline__ void platform_getoptreset(void) include/gnukfreebsd.h:static __inline__ void platform_getoptreset(void) include/linux.h:static __inline__ void platform_getoptreset(void) libxcmd/command.c: platform_getoptreset();
And we use it in libxcmd so that each function in xfs_io, xfs_db, xfs_quota, etc can run their own sub-command getopt calls correctly. It was in the initial commits for xfs_db back in 2001, so we've been using multiple pass CLI option parsing in xfsprogs since 2001....
Quoted 13 lines
Quoted 12 lines
IOWs:
config_file = getenv("MKFS_XFS_CONFIG"); if (config_file) dft.source = _("Environment Variable"); /* * Pull config line options from command line */ while ((c = getopt(argc, argv, "c:")) != EOF) { switch (c) { case 'c': /* XXX: fail if already seen a CLI option */
Quoted 2 lines
BTW isn't my enum here sufficient to check for this easily in a clean short and easy way?
A local boolean variable is all that is ncessary for this....
[....]
Quoted 5 lines
Quoted 3 lines
memcpy(/* sb_feats */); memcpy(/* fsxattr */); optind = 1;My getopt(3) *does* not make mention of any of this. This man page however does:
It's been there as long as I can remember. 2nd paragraph of the description:
$man 3 getopt
[...]
DESCRIPTION
[...]
The variable optind is the index of the next element to be processed in argv. The system initializes this value to 1. The caller can reset it to 1 to restart scanning of the same argv, or when scanning a new argument vector.
Quoted 3 lines
optind = 1;
However perhaps a big fat NOTE is worthy?
Why? It's documented, well known behaviour....
Quoted 19 lines
Quoted 18 lines
[...]
Quoted 4 lines
mkfs/xfs_mkfs.c +4
-0+const char *default_type_str (enum default_params_type type) +{ +switch (type) { +case DEFAULTS_BUILTIN :mkfs/xfs_mkfs.c
Quoted 10 lines
mkfs/xfs_mkfs.c +10
-0+return _ ("package built-in definitions" ); +case DEFAULTS_CONFIG : +return _ ("default configuration system file" ); +case DEFAULTS_ENVIRONMENT_CONFIG : +return _ ("custom configuration file set by environment" ); +case DEFAULTS_TYPE_CONFIG : +return _ ("custom configuration type file" ); + } +return _ ("Unkown\n" ); +}This function can go away.
Quoted 1 line
Well depends, if we keep the enum the no ?
That enum needs to die. It's unnecssary abstraction.
Quoted 6 lines
Quoted 4 lines
mkfs/xfs_mkfs.c
Quoted 2 lines
mkfs/xfs_mkfs.c +2
-0+if (strlen (line) <2 ) +return PARSE_INVALID ;So empty lines return
PARSE_INVALID?Yes, but that's not fatal, we just discard it.
That's "ignore", like comments, not "invalid".
Quoted 10 lines
Quoted 6 lines
mkfs/xfs_mkfs.c
Quoted 4 lines
mkfs/xfs_mkfs.c +3
-0+ +if (line[0 ] =='#' ) +return PARSE_COMMENT ;What about lines starting with whitespace? e.g. " # comment"
parse_line_section()below would not return 1 and its ignored in the end as its also not a proper tag/value pair and the section is not set.
So it silently ignores a valid sections/{tag/value pair} if there's trailing stuff on the line? Shouldn't that throw an error?
Quoted 28 lines
Quoted 19 lines
Quoted 19 lines
mkfs/xfs_mkfs.c +19
-0+{ +int ret; +FILE *fp; +struct stat sp; +unsigned int num_subopts_sections =sizeof (config_subopt_tab) / +sizeof (struct config_subopts ) -1 ; +char *p; +char line[80 ], tag[80 ]; +bool value; +enum parse_line_type parse_type; +enum mkfs_config_section section =MKFS_CONFIG_SECTION_INVALID ; + + fp =fopen (dft->config_file,"r" ); +if (!fp) { +if (dft->type !=DEFAULTS_BUILTIN ) { +fprintf (stderr,_ ("could not open config file: %s\n" ), + dft->config_file); + ret = -ENOENT ; + }else Quoted 8 lines
mkfs/xfs_mkfs.c
Quoted 3 lines
mkfs/xfs_mkfs.c +3
-0+ ret =0 ; +return ret; + }This should be split out into a separate function that takes the config file and tries to open it. If it's a relative path that was supplied, then this function can also try all the configured search paths for config files.
Quoted 5 lines
Eric asked to support 3 cases:
a) ./ -- current working directory
b) / -- full path
c) no prefix - we use the sysconfigdir/mkfs.d/
That's pretty much what I just suggested. :)
Quoted 9 lines
Quoted 6 lines
mkfs/xfs_mkfs.c
Quoted 4 lines
mkfs/xfs_mkfs.c +4
-0+return ret; + +while (!feof (fp)) { + p =fgets (line,sizeof (line), fp);What if the line is longer than 80 characters? e.g. a long comment
If its a comment or spaces with a comment, we still only care for the first 80 characters, no?
What does fgets() do with the remaining characters on the line? They are still parked in the input stream, so won't the next fgets() call read them? And then we parse those bytes as if they are a new config line?
IOWs, shouldn't we be using a line-based input function here? Say, getline(3)?
Quoted 27 lines
Quoted 18 lines
Quoted 18 lines
mkfs/xfs_mkfs.c +18
-0+if (!p) +continue ; + + parse_type =parse_get_line_type (line, tag, &value); + +switch (parse_type) { +case PARSE_COMMENT : +case PARSE_INVALID : +case PARSE_EOF : +break ; +case PARSE_SECTION : + section =get_config_section (tag); +if (!section) { +fprintf (stderr,_ ("Invalid section: %s\n" ), + tag); +return -EINVAL ; + } +break ;Quoted 8 lines
mkfs/xfs_mkfs.c
Quoted 4 lines
mkfs/xfs_mkfs.c +4
-0+case PARSE_TAG_VALUE : +/* Trims white spaces */ +snprintf (line,sizeof (line),"%s=%u" , tag, value); + ret =parse_config_subopts (section, value, line, dft);We've already got the tag and value as discrete variables - why put them back into an ascii string to pass to another function for it to split them back into discrete variables?
Quoted 1 line
The comment above it explains, "Trims white spaces"
Why do we need to do that? Doesn't the token parser trim away excess whitespace around each token it returns?
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Re: [PATCH v2 5/5] mkfs.xfs: add configuration file parsing support using our own parser
Quoted 111 lines · Dave Chinner
Quoted 31 lines · Luis R. Rodriguez
Quoted 13 lines · Dave Chinner
Quoted 12 lines · Luis R. Rodriguez
mkfs/xfs_mkfs.c +2
-1@@ -3077,11 +3078,13 @@ print_mkfs_cfg(struct mkfs_params *cfg, char *dfile, char *logfile, - char *rtfile) + char *rtfile, + struct mkfs_default_params *dft) { struct sb_feat_args *fp = &cfg->sb_feat;Quoted 16 lines
mkfs/xfs_mkfs.c print_mkfs_cfg( · line 3100
Quoted 13 lines
mkfs/xfs_mkfs.c +2
-0printf(_( +"config-file=%-22s\n" "meta-data=%-22s isize=%-6d agcount=%lld, agsize=%lld blks\n" " =%-22s sectsz=%-5u attr=%u, projid32bit=%u\n" " =%-22s crc=%-8u finobt=%u, sparse=%u, rmapbt=%u, reflink=%u\n"@@ -3091,6 +3094,7 @@ print_mkfs_cfg("log =%-22s bsize=%-6d blocks=%lld, version=%d\n" " =%-22s sectsz=%-5u sunit=%d blks, lazy-count=%d\n" "realtime =%-22s extsz=%-6d blocks=%lld, rtextents=%lld\n"), + dft->type !=DEFAULTS_BUILTIN ? dft->config_file :"empty" , dfile, cfg->inodesize, (long long )cfg->agcount, (long long )cfg->agsize, "", cfg->sectorsize, fp->attr_version, fp->projid32bit,Haven't we already printed where the config was sourced from? Why do we need to print it again here?
Quoted 3 lines
The other print describes the enum, this print prints out the *used* config file path if a config file was actually used. Without the other print this would just print either the config file or empty.
If you look at the changes I proposed, I suggested we change the initial print out the path of the source file, not how the user specified the source file. So this becomes redundant. And given this code is now shared with
xfs_info, it has no idea about mkfs config files, so it's not ideal to be adding mkfs-specific stuff to this output.Further....
Quoted 5 lines
Since we are going to remove the environment variable option, then I think the other print is now not needed anymore and my preference is to keep it here.
Thoughts?
.... this printout only happens after al input has been validated and we're about to make the filesystem. If there's a validation problem, nobody knows what file the defaults were sourced from. IOWs, the file the default are sourced from needs to be printed even before the file is read...
Quoted 16 lines
Quoted 7 lines
Quoted 7 lines
mkfs/xfs_mkfs.c +0
-0.sectorsize = XFS_MIN_SECTORSIZE, .blocksize = 1 << XFS_DFL_BLOCKSIZE_LOG, .sb_feat = {@@ -3714,6 +3719,13 @@ static void process_defaults(struct mkfs_default_params *dft, struct cli_params *cli) {Quoted 8 lines
mkfs/xfs_mkfs.c static void process_defaults( · line 3727
Quoted 6 lines
mkfs/xfs_mkfs.c +6
-0+int ret; + + ret =parse_config_init (dft); + +if (ret && dft->type !=DEFAULTS_BUILTIN ) +usage ();That doesn't make any sense in isolation.
Quoted 2 lines
Not sure what you meant at first, but I see that you prefer this to be hidden from the caller. Will change.
I meant I have no idea what this is checking and why there's a usage call there because
usage()is for CLI input, not setting up defaults before CLI processing. Explanation in comments are needed.Quoted 31 lines
Quoted 30 lines
Quoted 17 lines
mkfs/xfs_mkfs.c +2
-1install_defaults (dft, cli); }@@ -3750,6 +3762,8 @@ main( };struct mkfs_params cfg = {};struct mkfs_default_params dft; +char *tmp_config; +char custom_config[PATH_MAX ];reset_defaults_and_cli (&dft, &cli);@@ -3760,21 +3774,36 @@ main(textdomain (PACKAGE );/* -* TODO: Sourcing defaults from a config file mkfs/xfs_mkfs.c main( · line 3784
Quoted 10 lines
mkfs/xfs_mkfs.c +1
-2-* * Before anything else, see if there's a config file with different -* defaults. If a file exists in <package location>, read in the new +* defaults. If a file exists in MKFS_XFS_CONF_DIR/default, read the new * default values and overwrite them in the &dft structure. This way the * new defaults will apply before we parse the CLI, and the CLI will * still be able to override them. When more than one source is * implemented, emit a message to indicate where the defaults being * used came from. */ This comment makes no mention of environment variables, or how they interact with other sources of config files.
Quoted 4 lines
Quoted 2 lines
As per Eric's request, We're getting rid of the environment variable option, so mentioning it is no longer needed.
I thought Darrick wanted it kept?
Eric and I argued about this for a while on irc, I think we settled on allowing a single '-c $foo' arg where we parse openat($foo)...
Quoted 51 lines
Quoted 51 lines
Quoted 6 lines
Quoted 6 lines
mkfs/xfs_mkfs.c +5
-0+ tmp_config =getenv ("MKFS_XFS_CONFIG" ); +if (tmp_config !=NULL ) { + dft.config_file = tmp_config; + dft.type =DEFAULTS_ENVIRONMENT_CONFIG ; + }Quoted 6 lines
mkfs/xfs_mkfs.c main( · line 3791
Quoted 1 line
mkfs/xfs_mkfs.c +0
-0process_defaults (&dft, &cli);So why are we processing the defaults before we've checked if a default file is specified on the command line? All this should do is set up the config file to be read, and set the dft.source =
_("Environment Variable");This was being done *here* since I previously has setup to process the arguments *early* and check if a -c was specified, and if so use the config file for the defaults, otherwise the environment variable if set. But since the way I had implemented processing the arguments early relied on not well guaranteed mechanisms I resorted to avoid that approach.
This was the alternative I came up with.
The environment variable stuff is going away so this should be much simpler now, however the question of argument processing early still remains.
Quoted 9 lines
mkfs/xfs_mkfs.c main( · line 3796
Quoted 6 lines
mkfs/xfs_mkfs.c +3
-1-while ((c =getopt (argc, argv,"b:d:i:l:L:m:n:KNp:qr:s:CfV" )) !=EOF ) { +while ((c =getopt (argc, argv,"b:c:d:i:l:L:m:n:KNp:qr:s:CfV" )) !=EOF ) {switch (c) { +case 'c' : +reset_defaults_and_cli (&dft, &cli);This will trash the existing CLI inputs that have already been parsed.
Exactly.
Quoted 2 lines
All this code here should do is set up the config file to be read.
Consider the case of
/etc/mkfs.d/defaultexisting and it having[metadata]
crc = 1
[naming]
ftype=1And suppose the user passed -c foo which had:
[metadata]
crc = 0
Quoted 7 lines
Quoted 2 lines
Without the reset we'd make
/etc/mkfs.d/defaultthe system wide defaults onto which -c parameters overlay on top of. I figured that was not desirable.It's not desirable. If the user specified a config file, then
/etc/mkfs.d/default*is never read*. The user config file is used instead. If the user supplied file does not exist (even after searching) then we fail, not use some other set of defaults.
Yeah. Less confusing to trace where a particular knobturn came from if we only ever parse one config file.
--D
Quoted 234 lines
Quoted 4 lines
Quoted 1 line
Again, what happens if the user specifies multiple config files?
The reset strategy lets the user set -c as many times as they wish and also starts from a clean base always.
multiple "-c" options is a user error and should error out.
Quoted 10 lines
Quoted 2 lines
The second config file will trash everything from the first, as well as any other options between them.
Yes! By design as we couldn't easily parse arguments early first and then choose just one strategy.
I liked the simplicity that this brings.
Would you really want multiple -c options to work as overlays, one on top of the other?
I don't want multiple -c option to be supported at all. We source defaults from a single file only - either the user specified file, or the default if it exists. Not both, not multiple user specified files. One file.
Quoted 6 lines
Quoted 3 lines
I think that we need to pull the "-c <file>" option from the command line and process all the defaults /before/ we enter the main command line processing loop.
That is what I had done originally but ran into that snag of processing arguments twice and the undocumented functionality I found that worked.
Multiple pass option parsing is documented as supported and has been for a long, long time. And to make that clear, we even have a wrapper in xfsprogs for it:
$ git grep platform_getoptreset db/command.c: platform_getoptreset(); include/darwin.h:static __inline__ void platform_getoptreset(void) include/freebsd.h:static __inline__ void platform_getoptreset(void) include/gnukfreebsd.h:static __inline__ void platform_getoptreset(void) include/linux.h:static __inline__ void platform_getoptreset(void) libxcmd/command.c: platform_getoptreset();And we use it in libxcmd so that each function in
xfs_io,xfs_db,xfs_quota, etc can run their own sub-command getopt calls correctly. It was in the initial commits forxfs_dbback in 2001, so we've been using multiple pass CLI option parsing in xfsprogs since 2001....Quoted 13 lines
Quoted 12 lines
IOWs:
config_file = getenv("MKFS_XFS_CONFIG"); if (config_file) dft.source = _("Environment Variable"); /* * Pull config line options from command line */ while ((c = getopt(argc, argv, "c:")) != EOF) { switch (c) { case 'c': /* XXX: fail if already seen a CLI option */Quoted 2 lines
BTW isn't my enum here sufficient to check for this easily in a clean short and easy way?
A local boolean variable is all that is ncessary for this....
[....]
Quoted 5 lines
Quoted 3 lines
memcpy(/* sb_feats */); memcpy(/* fsxattr */); optind = 1;My getopt(3) *does* not make mention of any of this. This man page however does:
It's been there as long as I can remember. 2nd paragraph of the description:
$man 3 getopt
[...]
DESCRIPTION
[...]
The variable optind is the index of the next element to be processed in argv. The system initializes this value to 1. The caller can reset it to 1 to restart scanning of the same argv, or when scanning a new argument vector.Quoted 3 lines
optind = 1;
However perhaps a big fat NOTE is worthy?
Why? It's documented, well known behaviour....
Quoted 19 lines
Quoted 18 lines
[...]
Quoted 4 lines
mkfs/xfs_mkfs.c +4
-0+const char *default_type_str (enum default_params_type type) +{ +switch (type) { +case DEFAULTS_BUILTIN :mkfs/xfs_mkfs.c
Quoted 10 lines
mkfs/xfs_mkfs.c +10
-0+return _ ("package built-in definitions" ); +case DEFAULTS_CONFIG : +return _ ("default configuration system file" ); +case DEFAULTS_ENVIRONMENT_CONFIG : +return _ ("custom configuration file set by environment" ); +case DEFAULTS_TYPE_CONFIG : +return _ ("custom configuration type file" ); + } +return _ ("Unkown\n" ); +}This function can go away.
Quoted 1 line
Well depends, if we keep the enum the no ?
That enum needs to die. It's unnecssary abstraction.
Quoted 6 lines
Quoted 4 lines
mkfs/xfs_mkfs.c
Quoted 2 lines
mkfs/xfs_mkfs.c +2
-0+if (strlen (line) <2 ) +return PARSE_INVALID ;So empty lines return
PARSE_INVALID?Yes, but that's not fatal, we just discard it.
That's "ignore", like comments, not "invalid".
Quoted 10 lines
Quoted 6 lines
mkfs/xfs_mkfs.c
Quoted 4 lines
mkfs/xfs_mkfs.c +3
-0+ +if (line[0 ] =='#' ) +return PARSE_COMMENT ;What about lines starting with whitespace? e.g. " # comment"
parse_line_section()below would not return 1 and its ignored in the end as its also not a proper tag/value pair and the section is not set.So it silently ignores a valid sections/{tag/value pair} if there's trailing stuff on the line? Shouldn't that throw an error?
Quoted 28 lines
Quoted 19 lines
Quoted 19 lines
mkfs/xfs_mkfs.c +19
-0+{ +int ret; +FILE *fp; +struct stat sp; +unsigned int num_subopts_sections =sizeof (config_subopt_tab) / +sizeof (struct config_subopts ) -1 ; +char *p; +char line[80 ], tag[80 ]; +bool value; +enum parse_line_type parse_type; +enum mkfs_config_section section =MKFS_CONFIG_SECTION_INVALID ; + + fp =fopen (dft->config_file,"r" ); +if (!fp) { +if (dft->type !=DEFAULTS_BUILTIN ) { +fprintf (stderr,_ ("could not open config file: %s\n" ), + dft->config_file); + ret = -ENOENT ; + }else Quoted 8 lines
mkfs/xfs_mkfs.c
Quoted 3 lines
mkfs/xfs_mkfs.c +3
-0+ ret =0 ; +return ret; + }This should be split out into a separate function that takes the config file and tries to open it. If it's a relative path that was supplied, then this function can also try all the configured search paths for config files.
Quoted 5 lines
Eric asked to support 3 cases:
a) ./ -- current working directory
b) / -- full path
c) no prefix - we use the sysconfigdir/mkfs.d/That's pretty much what I just suggested. :)
Quoted 9 lines
Quoted 6 lines
mkfs/xfs_mkfs.c
Quoted 4 lines
mkfs/xfs_mkfs.c +4
-0+return ret; + +while (!feof (fp)) { + p =fgets (line,sizeof (line), fp);What if the line is longer than 80 characters? e.g. a long comment
If its a comment or spaces with a comment, we still only care for the first 80 characters, no?
What does
fgets()do with the remaining characters on the line? They are still parked in the input stream, so won't the nextfgets()call read them? And then we parse those bytes as if they are a new config line?IOWs, shouldn't we be using a line-based input function here? Say, getline(3)?
Quoted 27 lines
Quoted 18 lines
Quoted 18 lines
mkfs/xfs_mkfs.c +18
-0+if (!p) +continue ; + + parse_type =parse_get_line_type (line, tag, &value); + +switch (parse_type) { +case PARSE_COMMENT : +case PARSE_INVALID : +case PARSE_EOF : +break ; +case PARSE_SECTION : + section =get_config_section (tag); +if (!section) { +fprintf (stderr,_ ("Invalid section: %s\n" ), + tag); +return -EINVAL ; + } +break ;Quoted 8 lines
mkfs/xfs_mkfs.c
Quoted 4 lines
mkfs/xfs_mkfs.c +4
-0+case PARSE_TAG_VALUE : +/* Trims white spaces */ +snprintf (line,sizeof (line),"%s=%u" , tag, value); + ret =parse_config_subopts (section, value, line, dft);We've already got the tag and value as discrete variables - why put them back into an ascii string to pass to another function for it to split them back into discrete variables?
Quoted 1 line
The comment above it explains, "Trims white spaces"
Why do we need to do that? Doesn't the token parser trim away excess whitespace around each token it returns?
Cheers,
Dave.
Signature · 6 lines
Dave Chinner [email protected] -- To unsubscribe from this list: send the line "unsubscribe linux-xfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 5/5] mkfs.xfs: add configuration file parsing support using our own parser
In reply to Dave ChinnerQuoted 36 lines · Dave Chinner
Quoted 31 lines · Luis R. Rodriguez
Quoted 13 lines · Dave Chinner
Quoted 12 lines · Luis R. Rodriguez
mkfs/xfs_mkfs.c +2
-1@@ -3077,11 +3078,13 @@ print_mkfs_cfg(struct mkfs_params *cfg, char *dfile, char *logfile, - char *rtfile) + char *rtfile, + struct mkfs_default_params *dft) { struct sb_feat_args *fp = &cfg->sb_feat;Quoted 16 lines
mkfs/xfs_mkfs.c print_mkfs_cfg( · line 3100
Quoted 13 lines
mkfs/xfs_mkfs.c +2
-0printf(_( +"config-file=%-22s\n" "meta-data=%-22s isize=%-6d agcount=%lld, agsize=%lld blks\n" " =%-22s sectsz=%-5u attr=%u, projid32bit=%u\n" " =%-22s crc=%-8u finobt=%u, sparse=%u, rmapbt=%u, reflink=%u\n"@@ -3091,6 +3094,7 @@ print_mkfs_cfg("log =%-22s bsize=%-6d blocks=%lld, version=%d\n" " =%-22s sectsz=%-5u sunit=%d blks, lazy-count=%d\n" "realtime =%-22s extsz=%-6d blocks=%lld, rtextents=%lld\n"), + dft->type !=DEFAULTS_BUILTIN ? dft->config_file :"empty" , dfile, cfg->inodesize, (long long )cfg->agcount, (long long )cfg->agsize, "", cfg->sectorsize, fp->attr_version, fp->projid32bit,Haven't we already printed where the config was sourced from? Why do we need to print it again here?
Quoted 3 lines
The other print describes the enum, this print prints out the *used* config file path if a config file was actually used. Without the other print this would just print either the config file or empty.
Quoted 6 lines
If you look at the changes I proposed, I suggested we change the initial print out the path of the source file, not how the user specified the source file. So this becomes redundant. And given this code is now shared with
xfs_info, it has no idea about mkfs config files, so it's not ideal to be adding mkfs-specific stuff to this output.
Alright.
Quoted 6 lines
Further....
Quoted 4 lines
Since we are going to remove the environment variable option, then I think the other print is now not needed anymore and my preference is to keep it here.
Quoted 7 lines
Quoted 1 line
Thoughts?
.... this printout only happens after al input has been validated and we're about to make the filesystem. If there's a validation problem, nobody knows what file the defaults were sourced from. IOWs, the file the default are sourced from needs to be printed even before the file is read...
Good point.
Quoted 51 lines
Quoted 51 lines
Quoted 6 lines
Quoted 6 lines
mkfs/xfs_mkfs.c +5
-0+ tmp_config =getenv ("MKFS_XFS_CONFIG" ); +if (tmp_config !=NULL ) { + dft.config_file = tmp_config; + dft.type =DEFAULTS_ENVIRONMENT_CONFIG ; + }Quoted 6 lines
Quoted 1 line
process_defaults(&dft, &cli);So why are we processing the defaults before we've checked if a default file is specified on the command line? All this should do is set up the config file to be read, and set the dft.source =
_("Environment Variable");This was being done *here* since I previously has setup to process the arguments *early* and check if a -c was specified, and if so use the config file for the defaults, otherwise the environment variable if set. But since the way I had implemented processing the arguments early relied on not well guaranteed mechanisms I resorted to avoid that approach.
This was the alternative I came up with.
The environment variable stuff is going away so this should be much simpler now, however the question of argument processing early still remains.
Quoted 9 lines
mkfs/xfs_mkfs.c main( · line 3796
Quoted 6 lines
mkfs/xfs_mkfs.c +3
-1-while ((c =getopt (argc, argv,"b:d:i:l:L:m:n:KNp:qr:s:CfV" )) !=EOF ) { +while ((c =getopt (argc, argv,"b:c:d:i:l:L:m:n:KNp:qr:s:CfV" )) !=EOF ) {switch (c) { +case 'c' : +reset_defaults_and_cli (&dft, &cli);This will trash the existing CLI inputs that have already been parsed.
Exactly.
Quoted 2 lines
All this code here should do is set up the config file to be read.
Consider the case of
/etc/mkfs.d/defaultexisting and it having[metadata]
crc = 1
[naming]
ftype=1And suppose the user passed -c foo which had:
[metadata]
crc = 0
Quoted 7 lines
Quoted 2 lines
Without the reset we'd make
/etc/mkfs.d/defaultthe system wide defaults onto which -c parameters overlay on top of. I figured that was not desirable.It's not desirable. If the user specified a config file, then
/etc/mkfs.d/default*is never read*. The user config file is used instead. If the user supplied file does not exist (even after searching) then we fail, not use some other set of defaults.
I didn't overlay and will ensure we don't as I change this to read the command line args first prior to reading any file. The reset should not be needed once that is done then.
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Again, what happens if the user specifies multiple config files?
The reset strategy lets the user set -c as many times as they wish and also starts from a clean base always.
multiple "-c" options is a user error and should error out.
Great.
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
The second config file will trash everything from the first, as well as any other options between them.
Yes! By design as we couldn't easily parse arguments early first and then choose just one strategy.
I liked the simplicity that this brings.
Quoted 7 lines
Quoted 2 lines
Would you really want multiple -c options to work as overlays, one on top of the other?
I don't want multiple -c option to be supported at all. We source defaults from a single file only - either the user specified file, or the default if it exists. Not both, not multiple user specified files. One file.
OK!
Quoted 11 lines
Quoted 6 lines
Quoted 3 lines
I think that we need to pull the "-c <file>" option from the command line and process all the defaults /before/ we enter the main command line processing loop.
That is what I had done originally but ran into that snag of processing arguments twice and the undocumented functionality I found that worked.
Multiple pass option parsing is documented as supported and has been for a long, long time. And to make that clear, we even have a wrapper in xfsprogs for it:
Quoted 6 lines
$ git grep platform_getoptreset db/command.c: platform_getoptreset(); include/darwin.h:static __inline__ void platform_getoptreset(void) include/freebsd.h:static __inline__ void platform_getoptreset(void) include/gnukfreebsd.h:static __inline__ void platform_getoptreset(void) include/linux.h:static __inline__ void platform_getoptreset(void)
That sets optind = 0 but indeed despite my concerns if its used widely for years, let's go with it.
Quoted 6 lines
libxcmd/command.c:platform_getoptreset();And we use it in libxcmd so that each function in
xfs_io,xfs_db,xfs_quota, etc can run their own sub-command getopt calls correctly. It was in the initial commits forxfs_dbback in 2001, so we've been using multiple pass CLI option parsing in xfsprogs since 2001....
Alright!
Quoted 13 lines
Quoted 13 lines
Quoted 12 lines
IOWs:
config_file = getenv("MKFS_XFS_CONFIG"); if (config_file) dft.source = _("Environment Variable"); /* * Pull config line options from command line */ while ((c = getopt(argc, argv, "c:")) != EOF) { switch (c) { case 'c': /* XXX: fail if already seen a CLI option */
Quoted 4 lines
Quoted 2 lines
BTW isn't my enum here sufficient to check for this easily in a clean short and easy way?
A local boolean variable is all that is ncessary for this....
OK!
Quoted 21 lines
[....]
Quoted 5 lines
Quoted 3 lines
memcpy(/* sb_feats */); memcpy(/* fsxattr */); optind = 1;My getopt(3) *does* not make mention of any of this. This man page however does:
It's been there as long as I can remember. 2nd paragraph of the description:
$man 3 getopt
[...]
DESCRIPTION
[...]
The variable optind is the index of the next element to be processed in argv. The system initializes this value to 1. The caller can reset it to 1 to restart scanning of the same argv, or when scanning a new argument vector.
Quoted 5 lines
Quoted 3 lines
optind = 1;
However perhaps a big fat NOTE is worthy?
Why? It's documented, well known behaviour....
Alright.
Quoted 19 lines
Quoted 19 lines
Quoted 18 lines
[...]
Quoted 4 lines
mkfs/xfs_mkfs.c +4
-0+const char *default_type_str (enum default_params_type type) +{ +switch (type) { +case DEFAULTS_BUILTIN :mkfs/xfs_mkfs.c
Quoted 10 lines
mkfs/xfs_mkfs.c +10
-0+return _ ("package built-in definitions" ); +case DEFAULTS_CONFIG : +return _ ("default configuration system file" ); +case DEFAULTS_ENVIRONMENT_CONFIG : +return _ ("custom configuration file set by environment" ); +case DEFAULTS_TYPE_CONFIG : +return _ ("custom configuration type file" ); + } +return _ ("Unkown\n" ); +}This function can go away.
Quoted 3 lines
Quoted 1 line
Well depends, if we keep the enum the no ?
That enum needs to die. It's unnecssary abstraction.
Alrighty.
Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
mkfs/xfs_mkfs.c
Quoted 2 lines
mkfs/xfs_mkfs.c +2
-0+if (strlen (line) <2 ) +return PARSE_INVALID ;So empty lines return
PARSE_INVALID?Yes, but that's not fatal, we just discard it.
That's "ignore", like comments, not "invalid".
Sure that makes sense.
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
mkfs/xfs_mkfs.c
Quoted 4 lines
mkfs/xfs_mkfs.c +3
-0+ +if (line[0 ] =='#' ) +return PARSE_COMMENT ;What about lines starting with whitespace? e.g. " # comment"
Quoted 6 lines
Quoted 3 lines
parse_line_section()below would not return 1 and its ignored in the end as its also not a proper tag/value pair and the section is not set.So it silently ignores a valid sections/{tag/value pair} if there's trailing stuff on the line? Shouldn't that throw an error?
Will fix.
Quoted 28 lines
Quoted 28 lines
Quoted 27 lines
Quoted 12 lines
mkfs/xfs_mkfs.c +12
-0+{ +int ret; +FILE *fp; +struct stat sp; +unsigned int num_subopts_sections =sizeof (config_subopt_tab) / +sizeof (struct config_subopts ) -1 ; +char *p; +char line[80 ], tag[80 ]; +bool value; +enum parse_line_type parse_type; +enum mkfs_config_section section =MKFS_CONFIG_SECTION_INVALID ; +mkfs/xfs_mkfs.c
Quoted 10 lines
mkfs/xfs_mkfs.c +10
-0+ fp =fopen (dft->config_file,"r" ); +if (!fp) { +if (dft->type !=DEFAULTS_BUILTIN ) { +fprintf (stderr,_ ("could not open config file: %s\n" ), + dft->config_file); + ret = -ENOENT ; + }else + ret =0 ; +return ret; + }This should be split out into a separate function that takes the config file and tries to open it. If it's a relative path that was supplied, then this function can also try all the configured search paths for config files.
Quoted 7 lines
Quoted 5 lines
Eric asked to support 3 cases:
a) ./ -- current working directory
b) / -- full path
c) no prefix - we use the sysconfigdir/mkfs.d/That's pretty much what I just suggested. :)
Good, so only look at the current working directory *iff* "./" is passed, right?
Quoted 10 lines
Quoted 9 lines
Quoted 6 lines
mkfs/xfs_mkfs.c
Quoted 4 lines
mkfs/xfs_mkfs.c +4
-0+return ret; + +while (!feof (fp)) { + p =fgets (line,sizeof (line), fp);What if the line is longer than 80 characters? e.g. a long comment
If its a comment or spaces with a comment, we still only care for the first 80 characters, no?
Quoted 7 lines
What does
fgets()do with the remaining characters on the line? They are still parked in the input stream, so won't the nextfgets()call read them? And then we parse those bytes as if they are a new config line?IOWs, shouldn't we be using a line-based input function here? Say, getline(3)?
I suspect you are right, will look into it, and use it unless I find something better.
Quoted 27 lines
Quoted 27 lines
Quoted 26 lines
Quoted 12 lines
mkfs/xfs_mkfs.c +12
-0+if (!p) +continue ; + + parse_type =parse_get_line_type (line, tag, &value); + +switch (parse_type) { +case PARSE_COMMENT : +case PARSE_INVALID : +case PARSE_EOF : +break ; +case PARSE_SECTION : + section =get_config_section (tag);mkfs/xfs_mkfs.c
Quoted 10 lines
mkfs/xfs_mkfs.c +10
-0+if (!section) { +fprintf (stderr,_ ("Invalid section: %s\n" ), + tag); +return -EINVAL ; + } +break ; +case PARSE_TAG_VALUE : +/* Trims white spaces */ +snprintf (line,sizeof (line),"%s=%u" , tag, value); + ret =parse_config_subopts (section, value, line, dft);We've already got the tag and value as discrete variables - why put them back into an ascii string to pass to another function for it to split them back into discrete variables?
Quoted 4 lines
Quoted 1 line
The comment above it explains, "Trims white spaces"
Why do we need to do that? Doesn't the token parser trim away excess whitespace around each token it returns?
No, I don't think it did, but will check again.
Luis
Re: [PATCH v2 5/5] mkfs.xfs: add configuration file parsing support using our own parser
In reply to Dave ChinnerQuoted 13 lines · Dave Chinner
mkfs/xfs_mkfs_config.c line 479
Quoted 9 lines · Luis R. Rodriguez
mkfs/xfs_mkfs_config.c new file +9
-0+} + +static enum parse_line_type parse_get_line_type (const char *line,char *tag, +bool *value) +{ +int ret; +unsigned int uint_value; + +memset (tag,0 ,80 );Why? This should be done in the caller, and if there's a fixed buffer size, then please use a #define.
I've modified to use 'm' specifier on scanf().
Using 'm' tells scanf() to allocate a buffer for us, this is easier as we then enable the parser to do as it sees fit so long as the patterns match. We just have to later free it after use.
That removes the need to memset() and stick to a predefined buffer sizes for the tags (section or subopt name).
Luis
Re: [PATCH v2 5/5] mkfs.xfs: add configuration file parsing support using our own parser
In reply to Luis R. RodriguezQuoted 15 lines · Luis R. Rodriguez
You may want to stick to specific set of configuration options when creating filesystems with mkfs.xfs -- sometimes due to pure technical reasons, but some other times to ensure systems remain compatible as new features are introduced with older kernels, or if you always want to take advantage of some new feature which would otherwise typically be disruptive.
This adds support for parsing a configuration file to override defaults parameters to be used for mkfs.xfs.
We define an XFS configuration directory,/etc/mkfs.xfs.d/ and allow for different configuration files, if none is specified we look for the default configuration file,
/etc/mkfs.xfs.d/default. You can override with -c. For instance, if you specify:
Quoted 5 lines
mkfs.xfs -c experimental -f
/dev/loop0The file
/etc/mkfs.xfs.d/experimentalwill be used as your configuration file. If you really need to override the full path of the configuration file you may use theMKFS_XFS_CONFIGenvironment variable.
I'm swamped under a deadline at work this week so just commenting at a very high level for now, but I'm curious; why use an env var vs providing a full path for -c ? env vars always strike me as magic unexpected behaviors.
# mkfs.xfs -c /my/fancy/path/to/config
seems much clearer than
# export MKFS_XFS_CONFIG=/my/fancy/path/to/ # mkfs.xfs -c config
i.e. if a full path is specified use it, else use the config directory.
Thoughts?
Thanks,
-Eric
Quoted 33 lines
To verify what configuration file is used on a system use the typical:
mkfs.xfs -N
There is only a subset of options allowed to be set on the configuration file, and currently only 1 or 0 are acceptable values. The default parameters you can override on a configuration file and their current built-in default settings are:
[data]
noalign=0[inode]
align=1
projid32bit=1
sparse=0[log]
lazy-count=1[metadata]
crc=1
finobt=1
rmapbt=0
reflink=0[naming]
ftype=1[rtdev]
noalign=0Signed-off-by Luis R. Rodriguez<[email protected]>
Re: [PATCH v2 5/5] mkfs.xfs: add configuration file parsing support using our own parser
Quoted 29 lines · Eric Sandeen
Quoted 15 lines · Luis R. Rodriguez
You may want to stick to specific set of configuration options when creating filesystems with mkfs.xfs -- sometimes due to pure technical reasons, but some other times to ensure systems remain compatible as new features are introduced with older kernels, or if you always want to take advantage of some new feature which would otherwise typically be disruptive.
This adds support for parsing a configuration file to override defaults parameters to be used for mkfs.xfs.
We define an XFS configuration directory,/etc/mkfs.xfs.d/ and allow for different configuration files, if none is specified we look for the default configuration file,
/etc/mkfs.xfs.d/default. You can override with -c. For instance, if you specify:Quoted 5 lines
mkfs.xfs -c experimental -f
/dev/loop0The file
/etc/mkfs.xfs.d/experimentalwill be used as your configuration file. If you really need to override the full path of the configuration file you may use theMKFS_XFS_CONFIGenvironment variable.I'm swamped under a deadline at work this week so just commenting at a very high level for now, but I'm curious; why use an env var vs providing a full path for -c ? env vars always strike me as magic unexpected behaviors.
# mkfs.xfs -c /my/fancy/path/to/config
Quoted 8 lines
seems much clearer than
# export MKFS_XFS_CONFIG=/my/fancy/path/to/ # mkfs.xfs -c configi.e. if a full path is specified use it, else use the config directory.
Thoughts?
In this case your choices are:
MKFS_XFS_CONFIG=/my/fancy/path/to/config mkfs.xfs
or:
cp /my/fancy/path/to/config /etc/mkfs.xfs.d/hoogah mkfs.xfs -c hooga
<shrug> Bikeshedding more, what if either option accepted either an absolute path, or a file in $sysconfdir/etc/mkfs.xfs.d/ ? And the -c option can be specified once to override the environment variable / builtin detaults?
--D
Quoted 41 lines
Thanks,
-EricQuoted 26 lines
To verify what configuration file is used on a system use the typical:
mkfs.xfs -N
There is only a subset of options allowed to be set on the configuration file, and currently only 1 or 0 are acceptable values. The default parameters you can override on a configuration file and their current built-in default settings are:
[data]
noalign=0[inode]
align=1
projid32bit=1
sparse=0[log]
lazy-count=1[metadata]
crc=1
finobt=1
rmapbt=0
reflink=0Quoted 7 lines
[naming]
ftype=1[rtdev]
noalign=0Signed-off-by Luis R. Rodriguez<[email protected]>--
To unsubscribe from this list: send the line "unsubscribe linux-xfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 5/5] mkfs.xfs: add configuration file parsing support using our own parser
Quoted 42 lines · Darrick J. Wong
Quoted 29 lines · Eric Sandeen
Quoted 15 lines · Luis R. Rodriguez
You may want to stick to specific set of configuration options when creating filesystems with mkfs.xfs -- sometimes due to pure technical reasons, but some other times to ensure systems remain compatible as new features are introduced with older kernels, or if you always want to take advantage of some new feature which would otherwise typically be disruptive.
This adds support for parsing a configuration file to override defaults parameters to be used for mkfs.xfs.
We define an XFS configuration directory,/etc/mkfs.xfs.d/ and allow for different configuration files, if none is specified we look for the default configuration file,
/etc/mkfs.xfs.d/default. You can override with -c. For instance, if you specify:Quoted 5 lines
mkfs.xfs -c experimental -f
/dev/loop0The file
/etc/mkfs.xfs.d/experimentalwill be used as your configuration file. If you really need to override the full path of the configuration file you may use theMKFS_XFS_CONFIGenvironment variable.I'm swamped under a deadline at work this week so just commenting at a very high level for now, but I'm curious; why use an env var vs providing a full path for -c ? env vars always strike me as magic unexpected behaviors.
# mkfs.xfs -c /my/fancy/path/to/configQuoted 8 lines
seems much clearer than
# export MKFS_XFS_CONFIG=/my/fancy/path/to/ # mkfs.xfs -c configi.e. if a full path is specified use it, else use the config directory.
Thoughts?
In this case your choices are:
MKFS_XFS_CONFIG=/my/fancy/path/to/config mkfs.xfs
Quoted 6 lines
or:
cp /my/fancy/path/to/config/etc/mkfs.xfs.d/hoogahmkfs.xfs -c hooga<shrug> Bikeshedding more, what if either option accepted either an absolute path, or a file in $sysconfdir/etc/mkfs.xfs.d/ ?
Let us recall that the reason for -c set in stone on /@sysconfigdir@/mkfs.xfs.d/ was to allow clean simple code.
The MKFS_XFS_CONFIG is simply a comopromise to allow flexibility on a full path in case you cannot use the /@sysconfigdir@/mkfs.xfs.d/ directory.
Supporting both remains simple.
If we wanted to support what you suggest, if a user specified -c hoogah we'd have to treat multiple possibilities in code, increasing complexity:
a) Did the user mean the hoogah in the present directory?
b) Did the user mean hoogah in /@sysconfigdir@/mkfs.xfs.d/
Granted, if the user specified a -c /tmp/hoogah its clearer that the full path would be desirable. So, I think what you suggest makes sense provided we get rid of a) option and require only an alternative path *iff* the first character in the path is '/'. Does this work for all cases we wish to support a full path in?
Quoted 3 lines
And the -c
option can be specified once to override the environment variable / builtin detaults?
The -c option as-is in my patch series always overrides the environment variable, but it is currently always tied to /@sysconfigdir@/mkfs.xfs.d/. So if you specify both the -c wins. Its also why I implemented the reset feature / mechanism.
Luis
Re: [PATCH v2 5/5] mkfs.xfs: add configuration file parsing support using our own parser
Quoted 2 lines · Luis R. Rodriguez
Quoted 1 line · Darrick J. Wong
On Thu, May 17, 2018 at 10:24:13PM -0500, Eric Sandeen wrote:
...
Quoted 12 lines
Let us recall that the reason for -c set in stone on /@sysconfigdir@/mkfs.xfs.d/ was to allow clean simple code.
The
MKFS_XFS_CONFIGis simply a comopromise to allow flexibility on a full path in case you cannot use the /@sysconfigdir@/mkfs.xfs.d/ directory.Supporting both remains simple.
If we wanted to support what you suggest, if a user specified -c hoogah we'd have to treat multiple possibilities in code, increasing complexity:
Quoted 8 lines
a) Did the user mean the hoogah in the present directory?
b) Did the user mean hoogah in /@sysconfigdir@/mkfs.xfs.d/Granted, if the user specified a -c
/tmp/hoogahits clearer that the full path would be desirable. So, I think what you suggest makes sense provided we get rid of a) option and require only an alternative path *iff* the first character in the path is '/'. Does this work for all cases we wish to support a full path in?
Possibly include "./" as well. i.e. mkfs.xfs -c ./configdir/myconfig
so:
mkfs.xfs -c hoogah searches /@sysconfigdir@/mkfs.xfs.d/
mkfs.xfs -c ./hoogah or
mkfs.xfs -c /path/to/hoogah do exactly what you'd expect.
Quoted 3 lines
Quoted 3 lines
And the -c
option can be specified once to override the environment variable / builtin detaults?
I'd like to drop the environment variable altogether. If there is a strong case to be made for keeping it, please make it. :) (I don't consider the existence of MKE2FS_CONFIG to be a strong argument, FWIW, because our semantics & mechanisms are quite different.)
Thanks,
-Eric
Re: [PATCH v2 5/5] mkfs.xfs: add configuration file parsing support using our own parser
Quoted 34 lines · Eric Sandeen
Quoted 2 lines · Luis R. Rodriguez
Quoted 1 line · Darrick J. Wong
On Thu, May 17, 2018 at 10:24:13PM -0500, Eric Sandeen wrote:
...
Quoted 12 lines
Let us recall that the reason for -c set in stone on /@sysconfigdir@/mkfs.xfs.d/ was to allow clean simple code.
The
MKFS_XFS_CONFIGis simply a comopromise to allow flexibility on a full path in case you cannot use the /@sysconfigdir@/mkfs.xfs.d/ directory.Supporting both remains simple.
If we wanted to support what you suggest, if a user specified -c hoogah we'd have to treat multiple possibilities in code, increasing complexity:
Quoted 8 lines
a) Did the user mean the hoogah in the present directory?
b) Did the user mean hoogah in /@sysconfigdir@/mkfs.xfs.d/Granted, if the user specified a -c
/tmp/hoogahits clearer that the full path would be desirable. So, I think what you suggest makes sense provided we get rid of a) option and require only an alternative path *iff* the first character in the path is '/'. Does this work for all cases we wish to support a full path in?Possibly include "./" as well. i.e. mkfs.xfs -c ./configdir/myconfig
so:
mkfs.xfs -c hoogah searches /@sysconfigdir@/mkfs.xfs.d/
mkfs.xfs -c ./hoogah or
mkfs.xfs -c /path/to/hoogah do exactly what you'd expect.
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines
And the -c
option can be specified once to override the environment variable / builtin detaults?I'd like to drop the environment variable altogether. If there is a strong case to be made for keeping it, please make it. :) (I don't consider the existence of
MKE2FS_CONFIGto be a strong argument, FWIW, because our semantics & mechanisms are quite different.)
Sounds good. I'll drop the environment variable junk and only support the above 3 cases.
Luis
Re: [PATCH v2 5/5] mkfs.xfs: add configuration file parsing support using our own parser
Quoted 1 line · Luis R. Rodriguez
On Fri, May 18, 2018 at 12:09:23PM -0500, Eric Sandeen wrote:
...
Quoted 5 lines
Quoted 5 lines
mkfs.xfs -c hoogah searches /@sysconfigdir@/mkfs.xfs.d/ mkfs.xfs -c ./hoogah or mkfs.xfs -c /path/to/hoogah do exactly what you'd expect.
...
Quoted 4 lines
Sounds good. I'll drop the environment variable junk and only support the above 3 cases.
mkfs.xfs -c ../hoogah too, please. :-)
Jan
Re: [PATCH v2 5/5] mkfs.xfs: add configuration file parsing support using our own parser
Quoted 8 lines · Jan Tulak
Quoted 1 line · Luis R. Rodriguez
On Fri, May 18, 2018 at 12:09:23PM -0500, Eric Sandeen wrote:
...
Quoted 5 lines
Quoted 5 lines
mkfs.xfs -c hoogah searches /@sysconfigdir@/mkfs.xfs.d/ mkfs.xfs -c ./hoogah or mkfs.xfs -c /path/to/hoogah do exactly what you'd expect.
Quoted 7 lines
...
Quoted 4 lines
Sounds good. I'll drop the environment variable junk and only support the above 3 cases.
mkfs.xfs -c ../hoogah too, please. :-)
Done.
Luis
Re: [PATCH v2 5/5] mkfs.xfs: add configuration file parsing support using our own parser
In reply to Darrick J. WongQuoted 37 lines · Darrick J. Wong
Quoted 36 lines · Eric Sandeen
Quoted 15 lines · Luis R. Rodriguez
You may want to stick to specific set of configuration options when creating filesystems with mkfs.xfs -- sometimes due to pure technical reasons, but some other times to ensure systems remain compatible as new features are introduced with older kernels, or if you always want to take advantage of some new feature which would otherwise typically be disruptive.
This adds support for parsing a configuration file to override defaults parameters to be used for mkfs.xfs.
We define an XFS configuration directory,/etc/mkfs.xfs.d/ and allow for different configuration files, if none is specified we look for the default configuration file,
/etc/mkfs.xfs.d/default. You can override with -c. For instance, if you specify:Quoted 5 lines
mkfs.xfs -c experimental -f
/dev/loop0The file
/etc/mkfs.xfs.d/experimentalwill be used as your configuration file. If you really need to override the full path of the configuration file you may use theMKFS_XFS_CONFIGenvironment variable.I'm swamped under a deadline at work this week so just commenting at a very high level for now, but I'm curious; why use an env var vs providing a full path for -c ? env vars always strike me as magic unexpected behaviors.
# mkfs.xfs -c /my/fancy/path/to/configseems much clearer than
# export MKFS_XFS_CONFIG=/my/fancy/path/to/ # mkfs.xfs -c configi.e. if a full path is specified use it, else use the config directory.
Quoted 8 lines
Quoted 1 line
Thoughts?
In this case your choices are:
MKFS_XFS_CONFIG=/my/fancy/path/to/config mkfs.xfsor:
cp /my/fancy/path/to/config/etc/mkfs.xfs.d/hoogahmkfs.xfs -c hooga
Sorry, why do we need to copy it to /etc/mkfs.xfs.d/?
Quoted 2 lines
<shrug> Bikeshedding more, what if either option accepted either an absolute path, or a file in $sysconfdir/etc/mkfs.xfs.d/ ?
I kinda assumed that config files could be located anywhere, but we only searched the sysconfig path if it didn't point at a local file...
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Re: [PATCH v2 5/5] mkfs.xfs: add configuration file parsing support using our own parser
Quoted 40 lines · Dave Chinner
Quoted 39 lines · Darrick J. Wong
Quoted 37 lines · Eric Sandeen
Quoted 15 lines · Luis R. Rodriguez
You may want to stick to specific set of configuration options when creating filesystems with mkfs.xfs -- sometimes due to pure technical reasons, but some other times to ensure systems remain compatible as new features are introduced with older kernels, or if you always want to take advantage of some new feature which would otherwise typically be disruptive.
This adds support for parsing a configuration file to override defaults parameters to be used for mkfs.xfs.
We define an XFS configuration directory,/etc/mkfs.xfs.d/ and allow for different configuration files, if none is specified we look for the default configuration file,
/etc/mkfs.xfs.d/default. You can override with -c. For instance, if you specify:Quoted 5 lines
mkfs.xfs -c experimental -f
/dev/loop0The file
/etc/mkfs.xfs.d/experimentalwill be used as your configuration file. If you really need to override the full path of the configuration file you may use theMKFS_XFS_CONFIGenvironment variable.I'm swamped under a deadline at work this week so just commenting at a very high level for now, but I'm curious; why use an env var vs providing a full path for -c ? env vars always strike me as magic unexpected behaviors.
# mkfs.xfs -c /my/fancy/path/to/configseems much clearer than
# export MKFS_XFS_CONFIG=/my/fancy/path/to/ # mkfs.xfs -c configi.e. if a full path is specified use it, else use the config directory.
Thoughts?
Quoted 8 lines
Quoted 6 lines
In this case your choices are:
MKFS_XFS_CONFIG=/my/fancy/path/to/config mkfs.xfsor:
cp /my/fancy/path/to/config/etc/mkfs.xfs.d/hoogahmkfs.xfs -c hoogaSorry, why do we need to copy it to
/etc/mkfs.xfs.d/?
No particular reason, just showing off relative pathname interpretation. That could have been
mkfs.xfs -c ../../my/fancy/path/to/config
But that's kinda ugly. $sysconfdir seems to get set to /usr on my local builds, though I think dpkg overrides that back to /... :)
Quoted 6 lines
Quoted 2 lines
<shrug> Bikeshedding more, what if either option accepted either an absolute path, or a file in $sysconfdir/etc/mkfs.xfs.d/ ?
I kinda assumed that config files could be located anywhere, but we only searched the sysconfig path if it didn't point at a local file...
<shrug> openat() semantics are fine enough with me, I think.
--D
Quoted 12 lines
Cheers,
Dave.
Signature · 6 lines
Dave Chinner [email protected] -- To unsubscribe from this list: send the line "unsubscribe linux-xfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 5/5] mkfs.xfs: add configuration file parsing support using our own parser
Quoted 50 lines · Darrick J. Wong
Quoted 40 lines · Dave Chinner
Quoted 39 lines · Darrick J. Wong
Quoted 37 lines · Eric Sandeen
Quoted 15 lines · Luis R. Rodriguez
You may want to stick to specific set of configuration options when creating filesystems with mkfs.xfs -- sometimes due to pure technical reasons, but some other times to ensure systems remain compatible as new features are introduced with older kernels, or if you always want to take advantage of some new feature which would otherwise typically be disruptive.
This adds support for parsing a configuration file to override defaults parameters to be used for mkfs.xfs.
We define an XFS configuration directory,/etc/mkfs.xfs.d/ and allow for different configuration files, if none is specified we look for the default configuration file,
/etc/mkfs.xfs.d/default. You can override with -c. For instance, if you specify:Quoted 5 lines
mkfs.xfs -c experimental -f
/dev/loop0The file
/etc/mkfs.xfs.d/experimentalwill be used as your configuration file. If you really need to override the full path of the configuration file you may use theMKFS_XFS_CONFIGenvironment variable.I'm swamped under a deadline at work this week so just commenting at a very high level for now, but I'm curious; why use an env var vs providing a full path for -c ? env vars always strike me as magic unexpected behaviors.
# mkfs.xfs -c /my/fancy/path/to/configseems much clearer than
# export MKFS_XFS_CONFIG=/my/fancy/path/to/ # mkfs.xfs -c configi.e. if a full path is specified use it, else use the config directory.
Thoughts?
Quoted 8 lines
Quoted 6 lines
In this case your choices are:
MKFS_XFS_CONFIG=/my/fancy/path/to/config mkfs.xfsor:
cp /my/fancy/path/to/config/etc/mkfs.xfs.d/hoogahmkfs.xfs -c hoogaSorry, why do we need to copy it to
/etc/mkfs.xfs.d/?
Quoted 7 lines
No particular reason, just showing off relative pathname interpretation. That could have been
mkfs.xfs -c ../../my/fancy/path/to/config
But that's kinda ugly. $sysconfdir seems to get set to /usr on my local builds, though I think dpkg overrides that back to /... :)
You will need to set:
--sysconfdir=/etc
And the spec file / debian/control file will need to be update to use this on the configure line.
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
<shrug> Bikeshedding more, what if either option accepted either an absolute path, or a file in $sysconfdir/etc/mkfs.xfs.d/ ?
I kinda assumed that config files could be located anywhere, but we only searched the sysconfig path if it didn't point at a local file...
<shrug>
openat()semantics are fine enough with me, I think.
Well this is a big difference, and I think being clear on this would be good. If the user specified:
-c foo
and the file 'foo' is present but also exists on $sysconfdir/etc/mkfs.xfs.d/foo do we use the local file if the user did not pass ./foo ?
Luis
Re: [PATCH v2 5/5] mkfs.xfs: add configuration file parsing support using our own parser
Quoted 12 lines · Luis R. Rodriguez
Quoted 10 lines · Darrick J. Wong
Quoted 7 lines · Dave Chinner
Quoted 2 lines · Darrick J. Wong
<shrug> Bikeshedding more, what if either option accepted either an absolute path, or a file in $sysconfdir/etc/mkfs.xfs.d/ ?
I kinda assumed that config files could be located anywhere, but we only searched the sysconfig path if it didn't point at a local file...
<shrug>
openat()semantics are fine enough with me, I think.
Quoted 8 lines
Well this is a big difference, and I think being clear on this would be good. If the user specified:
-c foo
and the file 'foo' is present but also exists on $sysconfdir/etc/mkfs.xfs.d/foo do we use the local file if the user did not pass ./foo ?
I would have expected "foo" to be considered the same as "./foo". It's a relative path.
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Re: [PATCH v2 5/5] mkfs.xfs: add configuration file parsing support using our own parser
Quoted 16 lines · Dave Chinner
Quoted 15 lines · Luis R. Rodriguez
Quoted 10 lines · Darrick J. Wong
Quoted 7 lines · Dave Chinner
Quoted 2 lines · Darrick J. Wong
<shrug> Bikeshedding more, what if either option accepted either an absolute path, or a file in $sysconfdir/etc/mkfs.xfs.d/ ?
I kinda assumed that config files could be located anywhere, but we only searched the sysconfig path if it didn't point at a local file...
<shrug>
openat()semantics are fine enough with me, I think.Well this is a big difference, and I think being clear on this would be good. If the user specified:
Quoted 8 lines
Quoted 5 lines
-c foo
and the file 'foo' is present but also exists on $sysconfdir/etc/mkfs.xfs.d/foo do we use the local file if the user did not pass ./foo ?
I would have expected "foo" to be considered the same as "./foo". It's a relative path.
Urgh, so now if foo exists in $PWD /and/ in $sysconfdir/etc/mkfs.xfs.d/foo we have to have a hierarchy between the two? :/
Ok. back up. Honestly I think the only good reason to have config files outside of $sysconfdir/etc/mkfs.xfs.d/ is to facilitate testing without perturbing the system's files in order to do so.
In that case maybe we should distill it down to:
1) -c foo looks in $sysconfdir/etc/mkfs.xfs.d
2) -c /absolute/path/to/foo looks there, obviously
and drop relative paths. Absolute paths are more cumbersome but testing scripts won't care about that. mkfs.xfs -c pwd/foo isn't that hard either.
-Eric
Re: [PATCH v2 5/5] mkfs.xfs: add configuration file parsing support using our own parser
Quoted 23 lines · Eric Sandeen
Quoted 22 lines · Dave Chinner
Quoted 20 lines · Luis R. Rodriguez
Quoted 10 lines · Darrick J. Wong
Quoted 7 lines · Dave Chinner
Quoted 2 lines · Darrick J. Wong
<shrug> Bikeshedding more, what if either option accepted either an absolute path, or a file in $sysconfdir/etc/mkfs.xfs.d/ ?
I kinda assumed that config files could be located anywhere, but we only searched the sysconfig path if it didn't point at a local file...
<shrug>
openat()semantics are fine enough with me, I think.Well this is a big difference, and I think being clear on this would be good. If the user specified:
-c foo
and the file 'foo' is present but also exists on $sysconfdir/etc/mkfs.xfs.d/foo do we use the local file if the user did not pass ./foo ?
Quoted 5 lines
Quoted 2 lines
I would have expected "foo" to be considered the same as "./foo". It's a relative path.
Urgh, so now if foo exists in $PWD /and/ in $sysconfdir/etc/mkfs.xfs.d/foo we have to have a hierarchy between the two? :/
Of course there is. The user may not know anything about admin configured defaults, and they are well within their rights to have their own local files that have names that match global admin default files.
Just about every major app with config files searches relative path first, then $USER/.<app>/, then $SYS/<app>/. It's a well known, obvious search path (i.e. local->user default->global default) and we have no valid excuse for making up our own special snowflake that is completely different. We don't need the $USER directory (not every app needs or uses it), but otherwise we should follow convention....
IMO, making a distinction in behaviour between "-c foo" and "-c ./foo" such that they implicitly direct the app between "local" and "global" config file search paths is really bad UI design. It's surprising, and will lead to people complaining about how they needed to strace mkfs.xfs to work out it isn't actually reading their local config file.
Apps doing unnecessarily special UI crap is what makes people like me really, really, really hate software developers. We should not be making up our own special snowflake behaviour just because we think we're special enough we're allowed to do special things that users don't expect.
Quoted 11 lines
Ok. back up. Honestly I think the only good reason to have config files outside of $sysconfdir/etc/mkfs.xfs.d/ is to facilitate testing without perturbing the system's files in order to do so.
In that case maybe we should distill it down to:
1) -c foo looks in $sysconfdir/etc/mkfs.xfs.d
2) -c /absolute/path/to/foo looks there, obviouslyand drop relative paths. Absolute paths are more cumbersome but testing scripts won't care about that. mkfs.xfs -c
pwd/foo isn't that hard either.
IMO, that's just another special snowflake....
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Re: [PATCH v2 5/5] mkfs.xfs: add configuration file parsing support using our own parser
Quoted 27 lines · Dave Chinner
Quoted 26 lines · Eric Sandeen
Quoted 24 lines · Dave Chinner
Quoted 12 lines · Luis R. Rodriguez
Quoted 10 lines · Darrick J. Wong
Quoted 7 lines · Dave Chinner
Quoted 2 lines · Darrick J. Wong
<shrug> Bikeshedding more, what if either option accepted either an absolute path, or a file in $sysconfdir/etc/mkfs.xfs.d/ ?
I kinda assumed that config files could be located anywhere, but we only searched the sysconfig path if it didn't point at a local file...
<shrug>
openat()semantics are fine enough with me, I think.Quoted 8 lines
Well this is a big difference, and I think being clear on this would be good. If the user specified:
-c foo
and the file 'foo' is present but also exists on $sysconfdir/etc/mkfs.xfs.d/foo do we use the local file if the user did not pass ./foo ?
I would have expected "foo" to be considered the same as "./foo". It's a relative path.
Quoted 7 lines
Quoted 2 lines
Urgh, so now if foo exists in $PWD /and/ in $sysconfdir/etc/mkfs.xfs.d/foo we have to have a hierarchy between the two? :/
Of course there is. The user may not know anything about admin configured defaults, and they are well within their rights to have their own local files that have names that match global admin default files.
Chinner's got a point here. If felt silly to not support this so I just added support for it.
Will spin out a new set as I think I've taken care of all comments now.
Luis
Re: [PATCH v2 5/5] mkfs.xfs: add configuration file parsing support using our own parser
In reply to Darrick J. WongQuoted 1 line · Darrick J. Wong
<shrug>
openat()semantics are fine enough with me, I think.
I tried.
With openat() we'd have to go through a bit of hoops to figure out which actual file we ended up using and it was not pretty. And given the other snowflake considerations folks wanted to support as that *is* what typically users support, it turned out much easier to just use good 'ol open().
New patch set coming.
Luis