Skip to contentSkip to conversation

mkfs.xfs: add configuration file parsing support using our own parser

PATCH In review Luis R. Rodriguez 4 participants 16 messages Mar 13, 2018 – May 17, 2018

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 mkfs.xfs: add configuration file parsing support using our own parser

You may want to stick to specific set of configuration options when creating filesystems with mkfs.xfs -- sometimes due to pure technical reasons, but some other times to ensure systems remain compatible as new features are introduced with older kernels, or if you always want to take advantage of some new feature which would otherwise typically be disruptive.

This adds support for parsing a configuration file to override defaults parameters to be used for mkfs.xfs.

We define an XFS configuration directory, /etc/mkfs.xfs.d/ and allow for different types of configuration files, if none is specified we look for the default type, /etc/mkfs.xfs.d/default, and you can override with -T. For instance, if you specify:

mkfs.xfs -T experimental -f /dev/loop0

The file /etc/mkfs.xfs.d/experimental will be used as your configuration file. If you really need to override the full path of the configuration file you may use the MKFS_XFS_CONFIG environment variable.

To use /etc/ be sure to configure xfsprogs with:

./configure --sysconfdir=/etc/

To verify what configuration file is used on a system use the typical:

mkfs.xfs -N

There is only a subset of options allowed to be set on the configuration file, and currently only 1 or 0 are acceptable values. The default parameters you can override on a configuration file and their current built-in default settings are:

[data]
noalign=0

[inode]
align=1
projid32bit=1
sparse=0

[log]
lazy-count=1

[metadata]
crc=1
finobt=1
rmapbt=0
reflink=0

[naming]
ftype=1

[rtdev]
noalign=0

---

We have floated around enough bike shedding emails to have at least reached a consensus on the fact that we'd be only supporting a mimimum set of default parameters and would strive to simplify our parser as much as possible.

We never really decided on what parser to use though. To help with this I've implemented what we have discussed so far with 3 different parsers options and provide below URLs to both just a single patch needed for each on top today's xfsprogs's origin/for-next and also provide a respective xfsprogs branch on kernel.org:

a) Our own parser (this patch) [0] [1]
b) Using libini_config [2] [3]
c) Using e2fsprogs profile parser [4] [5]

I decided to re-evaluate using our own parser as well again, despite the original stated goal to not reinvent the wheel, since we decided to purposely simplify the parser as much as possible. To this end I also limited the default parameters we can override via the configuration file only to settings which we can enable or disable, so default parameters for which we can only set their values to 1 or 0. Final mkfs.xfs size evaluations also reveal carrying our own simple parser may be worth considering for now, provided at the end of this email.

Using our own parser is not a requirement, it just seems to make sense for now given the simplicity of our current parsing needs. We can switch to a library at any time. We can use either libini_config or the e2fsprogs profile parser, if we are willing to depend on an external shared library or use a static library.

Using a library libini_config proved to be the most robust and actively engaged library to consider [6].

Using the e2fsprogs profile parser may make sense if we wanted to groom and optimize a library for sizes *together* with the community. If one of our goals is to use a *shared library*, and if one of the requirements for this library is for it be as small as possible, we may want to consider grooming together with the community a future *shared profile parser*. An independent e2fsprogs profile parser today is about 14.86% the size of libini_config.so.5.2.0 [6], as such those really counting bytes may like this approach.

However implementing our own simple parser still produces the smallest binary, and dependencies today.

I believe that the size gains of using the e2fsprogs profile parser only makes sense if we want to *reduce overall shared space* with other filesystem configuration tools. I've already taken the e2fsprogs profile parser and made it a standalone library which we can use [7] and Ted is willing to help with this, should we want to embrark on that route.

Current file size evaluations using the different parsing options reveal that carrying our own parser produces the smallest binary. Using libini_config actually has an extra penalty of 576 bytes over carrying our own parser.

Using e2fsprogs actually carries an added penalty of 79776 bytes over using and carrying our own simple parser at this point.

Because of all this I've tried to simplify our own parser as much as possible, and strived to keep the code as readible as possible.

Current size evaluations with the different parser options:

own-parser:

$ size mkfs/mkfs.xfs
   text    data     bss     dec     hex filename
 394164   38448     568  433180   69c1c mkfs/mkfs.xfs
$ du -b mkfs/mkfs.xfs
2552400 mkfs/mkfs.xfs

libini_config

$ size mkfs/mkfs.xfs
   text    data     bss     dec     hex filename
 395160   38568     568  434296   6a078 mkfs/mkfs.xfs
$ du -b mkfs/mkfs.xfs
2552976 mkfs/mkfs.xfs

e2fsprogs profile:

$ size mkfs/mkfs.xfs
   text    data     bss     dec     hex filename
 411913   38656     728  451297   6e2e1 mkfs/mkfs.xfs
$ du -b mkfs/mkfs.xfs
2632176 mkfs/mkfs.xfs

Let me know what color of the bike shed you'd like for this.

[0] http://drvbp1.linux-foundation.org/~mcgrof/patches/2018/03/13/own-parser.patch
[1] https://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/xfsprogs-dev.git/log/?h=20180313-own-parser
[2] http://drvbp1.linux-foundation.org/~mcgrof/patches/2018/03/13/libini.patch
[3] https://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/xfsprogs-dev.git/log/?h=20180313-libini_config
[4] http://drvbp1.linux-foundation.org/~mcgrof/patches/2018/03/13/eprofile.patch
[5] https://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/xfsprogs-dev.git/log/?h=20180313-eprofile-parser
[6] https://gitlab.com/mcgrof/filesystem-configuration-paper/blob/master/paper.pdf
[7] https://gitlab.com/mcgrof/libekprofile

Luis

 include/builddefs.in  |   2 +
 man/man5/mkfs.xfs.d.5 | 121 ++++++++++++++
 man/man8/mkfs.xfs.8   |  23 +++
 mkfs/xfs_mkfs.c       | 432 +++++++++++++++++++++++++++++++++++++++++++++++---
 4 files changed, 559 insertions(+), 19 deletions(-)
 create mode 100644 man/man5/mkfs.xfs.d.5
include/builddefs.in +2 -0
index 7a2a62686717..231c96196ab9 100644
@@ -63,6 +63,7 @@ PKG_LIB_DIR	= @libdir@@libdirsuffix@
 PKG_INC_DIR	= @includedir@/xfs
 DK_INC_DIR	= @includedir@/disk
 PKG_MAN_DIR	= @mandir@
+PKG_ETC_DIR	= @sysconfdir@
 PKG_DOC_DIR	= @datadir@/doc/@pkg_name@
 PKG_LOCALE_DIR	= @datadir@/locale
 
@@ -194,6 +195,7 @@ endif
 
 GCFLAGS = $(DEBUG) \
 	  -DVERSION=\"$(PKG_VERSION)\" -DLOCALEDIR=\"$(PKG_LOCALE_DIR)\"  \
+	  -DROOT_SYSCONFDIR=\"$(PKG_ETC_DIR)\"  \
 	  -DPACKAGE=\"$(PKG_NAME)\" -I$(TOPDIR)/include -I$(TOPDIR)/libxfs
 
 ifeq ($(ENABLE_GETTEXT),yes)
man/man5/mkfs.xfs.d.5 new file +121 -0

121 lines not on this page · raw message

man/man8/mkfs.xfs.8 +23 -0
index 4b8c78c37806..18489b3a19ff 100644
@@ -83,6 +83,23 @@ and
 .B \-l internal \-l size=10m
 are equivalent.
 .PP
+An optional XFS configuration type file directory
+.B mkfs.xfs.d (5)
+exists to help fine tune default parameters which can be used when calling
+.B mkfs.xfs (8), by default type will be used by default, /etc/mkfs.xfs.d/default.
+Command line arguments directly passed to
+.B mkfs.xfs (8)
+will always override parameters set it the configuration type file.
+You can override configuration file type on the
+.B mkfs.xfs.d (5)
+directory by using the -t parameter and secifying the type. Alternatively
+you can set and use the MKFS_XFS_CONFIG environment variable to override
+the default full path of the first file
+.B mkfs.xfs (8)
+looks for.
+If you use -t the type configuration file must be present under
+.B mkfs.xfs.d (8).
+.PP
 In the descriptions below, sizes are given in sectors, bytes, blocks,
 kilobytes, megabytes, gigabytes, etc.
 Sizes are treated as hexadecimal if prefixed by 0x or 0X,
@@ -123,6 +140,11 @@ Many feature options allow an optional argument of 0 or 1, to explicitly
 disable or enable the functionality.
 .SH OPTIONS
 .TP
+.BI \-t " configuration-type"
+Override the default type of the configuratio file under
+.B mkfs.xfs.d
+used.
+.TP
 .BI \-b " block_size_options"
 This option specifies the fundamental block size of the filesystem.
 The valid
@@ -923,6 +945,7 @@ Prints the version number and exits.
 .SH SEE ALSO
 .BR xfs (5),
 .BR mkfs (8),
+.BR mkfs.xfs.d (5),
 .BR mount (8),
 .BR xfs_info (8),
 .BR xfs_admin (8).
mkfs/xfs_mkfs.c +413 -19

568 lines not on this page · raw message

Signature · 1 line
2.16.2
#1 lore raw
Quoted 10 lines · Luis R. Rodriguez

You may want to stick to specific set of configuration options when creating filesystems with mkfs.xfs -- sometimes due to pure technical reasons, but some other times to ensure systems remain compatible as new features are introduced with older kernels, or if you always want to take advantage of some new feature which would otherwise typically be disruptive.

This adds support for parsing a configuration file to override defaults parameters to be used for mkfs.xfs.

Quoted 6 lines

We define an XFS configuration directory, /etc/mkfs.xfs.d/ and allow for different types of configuration files, if none is specified we look for the default type, /etc/mkfs.xfs.d/default, and you can override with -T. For instance, if you specify:

mkfs.xfs -T experimental -f /dev/loop0

Just one obvious comment from a brief glance - you changed this from "-t" to "-T", but ....

man/man8/mkfs.xfs.8 and · line 95

Quoted 12 lines

are equivalent.
.PP

man/man8/mkfs.xfs.8 +10 -0
+An optional XFS configuration type file directory
+.B mkfs.xfs.d (5)
+exists to help fine tune default parameters which can be used when calling
+.B mkfs.xfs (8), by default type will be used by default, /etc/mkfs.xfs.d/default.
+Command line arguments directly passed to
+.B mkfs.xfs (8)
+will always override parameters set it the configuration type file.
+You can override configuration file type on the
+.B mkfs.xfs.d (5)
+directory by using the -t parameter and secifying the type. Alternatively

Not here, or ....

man/man8/mkfs.xfs.8 and · line 100

Quoted 5 lines
man/man8/mkfs.xfs.8 +5 -0
+you can set and use the MKFS_XFS_CONFIG environment variable to override
+the default full path of the first file
+.B mkfs.xfs (8)
+looks for.
+If you use -t the type configuration file must be present under

here, or ....

mkfs/xfs_mkfs.c line 28

Quoted 5 lines
mkfs/xfs_mkfs.c +5 -0
+#define MKFS_XFS_CONF_DIR	ROOT_SYSCONFDIR "/mkfs.xfs.d/"
+#define CONFIG_MAX_KEY		1024
+#define CONFIG_MAX_VALUE	PATH_MAX
+#define CONFIG_MAX_BUFFER	CONFIG_MAX_KEY + CONFIG_MAX_VALUE + 3
+#define PARAM_OPTS		"T:b:d:i:l:L:m:n:KNp:qr:s:CfV"

[ Please don't obfuscate parsing options like this ]

Quoted 15 lines
mkfs/xfs_mkfs.c +1 -4
@@ -3827,25 +4198,47 @@ main(
 	textdomain(PACKAGE);
 
 	/*
-	 * TODO: Sourcing defaults from a config file
-	 *
 	 * Before anything else, see if there's a config file with different
-	 * defaults. If a file exists in <package location>, read in the new
+	 * defaults. If a file exists in MKFS_XFS_CONF_DIR/default, read the new
 	 * default values and overwrite them in the &dft structure. This way the
 	 * new defaults will apply before we parse the CLI, and the CLI will
 	 * still be able to override them. When more than one source is
 	 * implemented, emit a message to indicate where the defaults being
 	 * used came from.
-	 *

mkfs/xfs_mkfs.c main( · line 4216

Quoted 10 lines
mkfs/xfs_mkfs.c +8 -1
-	 * printf(_("Default configuration sourced from %s\n"), dft.source);
 	 */
+	tmp_config = getenv("MKFS_XFS_CONFIG");
+	if (tmp_config != NULL) {
+		dft.config_file = tmp_config;
+		dft.type = DEFAULTS_ENVIRONMENT_CONFIG;
+	}
+
+	c = getopt(argc, argv, PARAM_OPTS);
+	if (c != EOF && c == 't') {

Here. Did you test this?

And, well, I have my doubts about this method of option parsing. Where does it say in the getopt(3) man page that CLI options are always parsed and returned in order of the option string rather than the order the appear on the CLI?

I'll spend some more time looking at it, but my initial impression is that there's a bit of work to be done yet...

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#2 lore raw
Quoted 10 lines · Dave Chinner

Just one obvious comment from a brief glance - you changed this from "-t" to "-T", but ....

Quoted 1 line

+directory by using the -t parameter and secifying the type. Alternatively

Not here, or ....

Quoted 1 line

+If you use -t the type configuration file must be present under

here, or ....

Quoted 7 lines

mkfs/xfs_mkfs.c line 28

Quoted 5 lines
mkfs/xfs_mkfs.c +5 -0
+#define MKFS_XFS_CONF_DIR	ROOT_SYSCONFDIR "/mkfs.xfs.d/"
+#define CONFIG_MAX_KEY		1024
+#define CONFIG_MAX_VALUE	PATH_MAX
+#define CONFIG_MAX_BUFFER	CONFIG_MAX_KEY + CONFIG_MAX_VALUE + 3
+#define PARAM_OPTS		"T:b:d:i:l:L:m:n:KNp:qr:s:CfV"

[ Please don't obfuscate parsing options like this ]

The reason was we use them twice.

Quoted 19 lines
Quoted 19 lines
mkfs/xfs_mkfs.c +3 -5
@@ -3827,25 +4198,47 @@ main(
 	textdomain(PACKAGE);
 
 	/*
-	 * TODO: Sourcing defaults from a config file
-	 *
 	 * Before anything else, see if there's a config file with different
-	 * defaults. If a file exists in <package location>, read in the new
+	 * defaults. If a file exists in MKFS_XFS_CONF_DIR/default, read the new
 	 * default values and overwrite them in the &dft structure. This way the
 	 * new defaults will apply before we parse the CLI, and the CLI will
 	 * still be able to override them. When more than one source is
 	 * implemented, emit a message to indicate where the defaults being
 	 * used came from.
-	 *
-	 * printf(_("Default configuration sourced from %s\n"), dft.source);
 	 */
+	tmp_config = getenv("MKFS_XFS_CONFIG");
+	if (tmp_config != NULL) {
Quoted 8 lines

mkfs/xfs_mkfs.c main( · line 4216

Quoted 6 lines
mkfs/xfs_mkfs.c +6 -0
+		dft.config_file = tmp_config;
+		dft.type = DEFAULTS_ENVIRONMENT_CONFIG;
+	}
+
+	c = getopt(argc, argv, PARAM_OPTS);
+	if (c != EOF && c == 't') {

Here. Did you test this?

I forgot to test -T after we bike-shed from -t to -T, sorry. Will fix. I had however tested -t for sure.

Quoted 4 lines

And, well, I have my doubts about this method of option parsing. Where does it say in the getopt(3) man page that CLI options are always parsed and returned in order of the option string rather than the order the appear on the CLI?

You are right. Its rather unclear. If -t is passed at the end this may not work.

Quoted 2 lines

I'll spend some more time looking at it, but my initial impression is that there's a bit of work to be done yet...

You're right, we should decide if we want to allow for -T to be used only in the beginning or if we want to allow for it anywhere on the command line. If we allow for it only at the beginning of the command line it may be easier and cleaner to support: we'd just check argv[1] for -T manually and if passed just bump argv down two slots. So we leave getopt() usage only for the other stuff.

Thoughts?

Luis

#3 lore raw
Quoted 19 lines · Luis R. Rodriguez
Quoted 10 lines · Dave Chinner

Just one obvious comment from a brief glance - you changed this from "-t" to "-T", but ....

Quoted 1 line

+directory by using the -t parameter and secifying the type. Alternatively

Not here, or ....

Quoted 1 line

+If you use -t the type configuration file must be present under

here, or ....

Quoted 7 lines

mkfs/xfs_mkfs.c line 28

Quoted 5 lines
mkfs/xfs_mkfs.c +5 -0
+#define MKFS_XFS_CONF_DIR	ROOT_SYSCONFDIR "/mkfs.xfs.d/"
+#define CONFIG_MAX_KEY		1024
+#define CONFIG_MAX_VALUE	PATH_MAX
+#define CONFIG_MAX_BUFFER	CONFIG_MAX_KEY + CONFIG_MAX_VALUE + 3
+#define PARAM_OPTS		"T:b:d:i:l:L:m:n:KNp:qr:s:CfV"

[ Please don't obfuscate parsing options like this ]

Quoted 1 line

The reason was we use them twice.

Which is not necessary. You don't need to change the option parsing loop at all - the default config file can be parsed at any time as a normal option. The default structure is not used for calculations until after all the CLI options are parsed, so it can just be treated as another CLI option in any location on the CLI.

Quoted 6 lines
Quoted 2 lines

I'll spend some more time looking at it, but my initial impression is that there's a bit of work to be done yet...

You're right, we should decide if we want to allow for -T to be used only in the beginning or if we want to allow for it anywhere on the command line.

That's not what I meant. I meant the code needs a lot of work, not that we needed to decide where on the CLI the config file option should be placed.

Basically:

1. the default file parsing code needs to go into it's own file. xfs_mkfs.c is too large and needs to be split into smaller files, so lets not make it worse by shovelling another 500 lines of code into it...

2. The default option should not share option table parsing with the CLI options, because all that means is you add extra code to convert config file sections to CLI sections before using the common parsing code.

3. it needs to support more than just boolean options e.g. for setting a default log size

4. it needs default values to be range checked for sanity. These values can be different to the min/max values the can be specified on CLI as defaults need to be more forgiving when combined with other random options.

There will be other things when I look at the code, but those are the first ones that come to mind....

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#4 lore raw
Quoted 20 lines · Dave Chinner
Quoted 19 lines · Luis R. Rodriguez
Quoted 17 lines · Dave Chinner

Just one obvious comment from a brief glance - you changed this from "-t" to "-T", but ....

Quoted 1 line

+directory by using the -t parameter and secifying the type. Alternatively

Not here, or ....

Quoted 1 line

+If you use -t the type configuration file must be present under

here, or ....

mkfs/xfs_mkfs.c line 28

Quoted 5 lines
mkfs/xfs_mkfs.c +5 -0
+#define MKFS_XFS_CONF_DIR	ROOT_SYSCONFDIR "/mkfs.xfs.d/"
+#define CONFIG_MAX_KEY		1024
+#define CONFIG_MAX_VALUE	PATH_MAX
+#define CONFIG_MAX_BUFFER	CONFIG_MAX_KEY + CONFIG_MAX_VALUE + 3
+#define PARAM_OPTS		"T:b:d:i:l:L:m:n:KNp:qr:s:CfV"

[ Please don't obfuscate parsing options like this ]

Quoted 7 lines
Quoted 1 line

The reason was we use them twice.

Which is not necessary. You don't need to change the option parsing loop at all - the default config file can be parsed at any time as a normal option. The default structure is not used for calculations until after all the CLI options are parsed, so it can just be treated as another CLI option in any location on the CLI.

Alright.

Quoted 11 lines
Quoted 6 lines
Quoted 2 lines

I'll spend some more time looking at it, but my initial impression is that there's a bit of work to be done yet...

You're right, we should decide if we want to allow for -T to be used only in the beginning or if we want to allow for it anywhere on the command line.

That's not what I meant. I meant the code needs a lot of work, not that we needed to decide where on the CLI the config file option should be placed.

Quoted 6 lines

Basically:

1. the default file parsing code needs to go into it's own file. xfs_mkfs.c is too large and needs to be split into smaller files, so lets not make it worse by shovelling another 500 lines of code into it...

Sure.

Quoted 4 lines

2. The default option should not share option table parsing with the CLI options, because all that means is you add extra code to convert config file sections to CLI sections before using the common parsing code.

Sure.

Quoted 2 lines

3. it needs to support more than just boolean options e.g. for setting a default log size

Log size is not part of struct mkfs_default_params, do we reaally want to be able to set that via the configuration file?

sectorsize and blocksize are though.
Then struct mkfs_default_params has a struct sb_feat_args, of which non-bools are (and their corresponding mapping):

int log_version L_VERSION
int attr_version I_ATTR
int dir_version N_VERSION

So sure, I can add support for these, and sectorsize and blocksize. It just didn't seem worth it given most of the others were bool, so if we restrict the parser to bool its much simpler.

BTW I see we have bool parent_pointers, and that maps to setting XFS_SB_VERSION2_PARENTBIT, but we have no respective specific CLI param, so of these:

[data]
[inode]
[log]
[metadata]
[naming]
[rtdev]

Where does parent_pointers map to? I'd only enable "parent_pointers=0|parent_pointers=1" as its a bool.

Quoted 4 lines

4. it needs default values to be range checked for sanity. These values can be different to the min/max values the can be specified on CLI as defaults need to be more forgiving when combined with other random options.

Alright.

Quoted 2 lines

There will be other things when I look at the code, but those are the first ones that come to mind....

I can address most of what you describe rather easily, the biggest issue is ensuring that our goal here is only to modify struct mkfs_default_params values, inclusive of what is in struct sb_feat_args.

Note that struct mkfs_default_params also has a struct fsxattr... do we want to modify all those (this struct is defined per OS), for Linux we have:

struct fsxattr {

        __u32           fsx_xflags;     /* xflags field value (get/set) */      
        __u32           fsx_extsize;    /* extsize field value (get/set)*/      
        __u32           fsx_nextents;   /* nextents field value (get)   */      
        __u32           fsx_projid;     /* project identifier (get/set) */      
        __u32           fsx_cowextsize; /* cow extsize field value (get/set) */ 
        unsigned char   fsx_pad[8];                                             

};

I didn't enable parsing support for these, do we want to parse each of these as well?

Luis

#5 lore raw
Quoted 56 lines · Luis R. Rodriguez
Quoted 20 lines · Dave Chinner
Quoted 19 lines · Luis R. Rodriguez
Quoted 17 lines · Dave Chinner

Just one obvious comment from a brief glance - you changed this from "-t" to "-T", but ....

Quoted 1 line

+directory by using the -t parameter and secifying the type. Alternatively

Not here, or ....

Quoted 1 line

+If you use -t the type configuration file must be present under

here, or ....

mkfs/xfs_mkfs.c line 28

Quoted 5 lines
mkfs/xfs_mkfs.c +5 -0
+#define MKFS_XFS_CONF_DIR	ROOT_SYSCONFDIR "/mkfs.xfs.d/"
+#define CONFIG_MAX_KEY		1024
+#define CONFIG_MAX_VALUE	PATH_MAX
+#define CONFIG_MAX_BUFFER	CONFIG_MAX_KEY + CONFIG_MAX_VALUE + 3
+#define PARAM_OPTS		"T:b:d:i:l:L:m:n:KNp:qr:s:CfV"

[ Please don't obfuscate parsing options like this ]

Quoted 7 lines
Quoted 1 line

The reason was we use them twice.

Which is not necessary. You don't need to change the option parsing loop at all - the default config file can be parsed at any time as a normal option. The default structure is not used for calculations until after all the CLI options are parsed, so it can just be treated as another CLI option in any location on the CLI.

Alright.

Quoted 11 lines
Quoted 6 lines
Quoted 2 lines

I'll spend some more time looking at it, but my initial impression is that there's a bit of work to be done yet...

You're right, we should decide if we want to allow for -T to be used only in the beginning or if we want to allow for it anywhere on the command line.

That's not what I meant. I meant the code needs a lot of work, not that we needed to decide where on the CLI the config file option should be placed.

Quoted 6 lines

Basically:

1. the default file parsing code needs to go into it's own file. xfs_mkfs.c is too large and needs to be split into smaller files, so lets not make it worse by shovelling another 500 lines of code into it...

Sure.

Quoted 4 lines

2. The default option should not share option table parsing with the CLI options, because all that means is you add extra code to convert config file sections to CLI sections before using the common parsing code.

Quoted 7 lines

Sure.

Quoted 2 lines

3. it needs to support more than just boolean options e.g. for setting a default log size

Log size is not part of struct mkfs_default_params, do we reaally want to be able to set that via the configuration file?

I think you're over-complicating/over-thinking this. setting a default log size is an /example/, not something we /must do/. I could have said "default directory blocksize", which is another thing that is not currently in the defaults structure but is also something we should also consider as a configurable default because there are classes of users that want to use a different default value.

Quoted 11 lines

sectorsize and blocksize are though.
Then struct mkfs_default_params has a struct sb_feat_args, of which non-bools are (and their corresponding mapping):

int log_version L_VERSION
int attr_version I_ATTR
int dir_version N_VERSION

So sure, I can add support for these, and sectorsize and blocksize. It just didn't seem worth it given most of the others were bool, so if we restrict the parser to bool its much simpler.

Again, you're only looking at the current implementation to support what it implements rather than implementing what we need to arbitrary config file parameters.

e.g. Look at validate_dirblocksize() - it's default value is hard coded into the function, rather than getting it from the defaults structure. Same goes for validate_inodesize(), and so on. Those are going to need to be pulled up into the default value structure, because they are things we're going to want to support changing default values for.

Start by looking at all the types we parse on the CLI (bool, flag, int, int64, unit suffices, etc) and implementing those, because they are all going to be needed to set the value of some default parameter....

Quoted 3 lines

BTW I see we have bool parent_pointers, and that maps to setting XFS_SB_VERSION2_PARENTBIT, but we have no respective specific CLI param, so of these:

Again, you're looking at implementation, and asking questions about a function that *isn't yet implemented* and so has no valid value other than 0. When it's actually implemented, then we'll add it to the CLI options and the default parsing into the appropriate section. It's not the job of a "introduce config files for defaults" patch to implement CLI/default parsing support for features that aren't currently implemented.

Quoted 17 lines
Quoted 2 lines

There will be other things when I look at the code, but those are the first ones that come to mind....

I can address most of what you describe rather easily, the biggest issue is ensuring that our goal here is only to modify struct mkfs_default_params values, inclusive of what is in struct sb_feat_args.

Note that struct mkfs_default_params also has a struct fsxattr... do we want to modify all those (this struct is defined per OS), for Linux we have:

struct fsxattr {

        __u32           fsx_xflags;     /* xflags field value (get/set) */      
        __u32           fsx_extsize;    /* extsize field value (get/set)*/      
        __u32           fsx_nextents;   /* nextents field value (get)   */      
        __u32           fsx_projid;     /* project identifier (get/set) */      
        __u32           fsx_cowextsize; /* cow extsize field value (get/set) */ 
        unsigned char   fsx_pad[8];                                             
Quoted 4 lines

};

I didn't enable parsing support for these, do we want to parse each of these as well?

They are set by parsing existing CLI options, so if you enable those options in the config file parsing, then this structure will need to be updated, too.

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#6 lore raw
Quoted 8 lines · Dave Chinner
Quoted 6 lines · Luis R. Rodriguez
Quoted 2 lines · Dave Chinner

3. it needs to support more than just boolean options e.g. for setting a default log size

Log size is not part of struct mkfs_default_params, do we reaally want to be able to set that via the configuration file?

Quoted 7 lines

I think you're over-complicating/over-thinking this. setting a default log size is an /example/, not something we /must do/. I could have said "default directory blocksize", which is another thing that is not currently in the defaults structure but is also something we should also consider as a configurable default because there are classes of users that want to use a different default value.

Just wanted to be sure.

Quoted 8 lines
Quoted 8 lines

sectorsize and blocksize are though.
Then struct mkfs_default_params has a struct sb_feat_args, of which non-bools are (and their corresponding mapping):

int log_version L_VERSION
int attr_version I_ATTR
int dir_version N_VERSION

Quoted 7 lines
Quoted 3 lines

So sure, I can add support for these, and sectorsize and blocksize. It just didn't seem worth it given most of the others were bool, so if we restrict the parser to bool its much simpler.

Again, you're only looking at the current implementation to support what it implements rather than implementing what we need to arbitrary config file parameters.

Sure, from my perspective *just* parsing the bools on struct mkfs_default_params sufficed for my own needs to ensure I can drop in a new xfsprogs on ancient systems. I understand what you mean now though, in that the default structure may *very likely* grow later and we may want to expand on it later. Got it.

Quoted 3 lines

e.g. Look at validate_dirblocksize() - it's default value is hard coded into the function, rather than getting it from the defaults structure.

If nothing is set it depends if dirblocklog is set. And I guess if nothing is set dirblocklog is set to XFS_MIN_REC_DIRSIZE, and dirblocksize would be (1 << XFS_MIN_REC_DIRSIZE).

Quoted 1 line

Same goes for validate_inodesize(), and so on.

This default value will depends on if crcs_enabled which is already part of the default struct. If we enable one to set this via the default struct we'd just have to do similar sanity checks, which I'd hope we can just share across both files then.

Quoted 4 lines

Those are
going to need to be pulled up into the default value structure, because they are things we're going to want to support changing default values for.

Alright...

Quoted 4 lines

Start by looking at all the types we parse on the CLI (bool, flag, int, int64, unit suffices, etc) and implementing those, because they are all going to be needed to set the value of some default parameter....

Last I looked uint64 would suffice for all, modulo we had a few "strings", an odd one was for N_VERSION where it may be a string with "ci" in which case we'd set nci=true, otherwise we assume we're updating the dir_version.

Quoted 11 lines
Quoted 3 lines

BTW I see we have bool parent_pointers, and that maps to setting XFS_SB_VERSION2_PARENTBIT, but we have no respective specific CLI param, so of these:

Again, you're looking at implementation, and asking questions about a function that *isn't yet implemented* and so has no valid value other than 0. When it's actually implemented, then we'll add it to the CLI options and the default parsing into the appropriate section. It's not the job of a "introduce config files for defaults" patch to implement CLI/default parsing support for features that aren't currently implemented.

Great.

Quoted 19 lines
Quoted 19 lines
Quoted 2 lines

There will be other things when I look at the code, but those are the first ones that come to mind....

I can address most of what you describe rather easily, the biggest issue is ensuring that our goal here is only to modify struct mkfs_default_params values, inclusive of what is in struct sb_feat_args.

Note that struct mkfs_default_params also has a struct fsxattr... do we want to modify all those (this struct is defined per OS), for Linux we have:

struct fsxattr {

        __u32           fsx_xflags;     /* xflags field value (get/set) */
        __u32           fsx_extsize;    /* extsize field value (get/set)*/
        __u32           fsx_nextents;   /* nextents field value (get)   */
        __u32           fsx_projid;     /* project identifier (get/set) */
        __u32           fsx_cowextsize; /* cow extsize field value (get/set) */
        unsigned char   fsx_pad[8];

};

Quoted 6 lines
Quoted 2 lines

I didn't enable parsing support for these, do we want to parse each of these as well?

They are set by parsing existing CLI options, so if you enable those options in the config file parsing, then this structure will need to be updated, too.

I don't have a need for this to support old kernels. I'm perfectly happy to support only what we have in the defaults structure for now.

Luis

#7 lore raw
In reply to Dave Chinner
Quoted 5 lines · Dave Chinner
Quoted 4 lines · Luis R. Rodriguez
Quoted 3 lines · Dave Chinner
Quoted 1 line · Luis R. Rodriguez

On Wed, Mar 14, 2018 at 08:39:16AM +1100, Dave Chinner wrote:

Quoted 8 lines
Quoted 8 lines
Quoted 6 lines

Basically:

1. the default file parsing code needs to go into it's own file. xfs_mkfs.c is too large and needs to be split into smaller files, so lets not make it worse by shovelling another 500 lines of code into it...

Sure.

Other than using its own file you had suggested to have the config stuff have its own structure. At LSFMM you had also suggested it should have its own defaults which may change later, however for now they can be the same as the CLI defaults. I suspect likewise applies for maxval, minval.

What about the conflict mapping / checkers? Don't we want to share that? If so it would make sense to share the enums used to describe the individual subopts as well. If so I'll shove the enums into its own file. The conflict stuff is already in its own structure, so if we want to *share* that, I could just move all that into its own set of files and share that.

Luis

#8 lore raw
Quoted 20 lines · Luis R. Rodriguez
Quoted 5 lines · Dave Chinner
Quoted 4 lines · Luis R. Rodriguez
Quoted 3 lines · Dave Chinner
Quoted 1 line · Luis R. Rodriguez

On Wed, Mar 14, 2018 at 08:39:16AM +1100, Dave Chinner wrote:

Quoted 8 lines
Quoted 8 lines
Quoted 6 lines

Basically:

1. the default file parsing code needs to go into it's own file. xfs_mkfs.c is too large and needs to be split into smaller files, so lets not make it worse by shovelling another 500 lines of code into it...

Sure.

Other than using its own file you had suggested to have the config stuff have its own structure. At LSFMM you had also suggested it should have its own defaults which may change later, however for now they can be the same as the CLI defaults. I suspect likewise applies for maxval, minval.

Quoted 5 lines

What about the conflict mapping / checkers? Don't we want to share that? If so it would make sense to share the enums used to describe the individual subopts as well. If so I'll shove the enums into its own file. The conflict stuff is already in its own structure, so if we want to *share* that, I could just move all that into its own set of files and share that.

Re-poke.

Luis

#9 lore raw
Quoted 21 lines · Luis R. Rodriguez
Quoted 20 lines · Luis R. Rodriguez
Quoted 5 lines · Dave Chinner
Quoted 4 lines · Luis R. Rodriguez
Quoted 3 lines · Dave Chinner
Quoted 1 line · Luis R. Rodriguez

On Wed, Mar 14, 2018 at 08:39:16AM +1100, Dave Chinner wrote:

Quoted 8 lines
Quoted 8 lines
Quoted 6 lines

Basically:

1. the default file parsing code needs to go into it's own file. xfs_mkfs.c is too large and needs to be split into smaller files, so lets not make it worse by shovelling another 500 lines of code into it...

Sure.

Other than using its own file you had suggested to have the config stuff have its own structure. At LSFMM you had also suggested it should have its own defaults which may change later, however for now they can be the same as the CLI defaults. I suspect likewise applies for maxval, minval.

Quoted 7 lines
Quoted 5 lines

What about the conflict mapping / checkers? Don't we want to share that? If so it would make sense to share the enums used to describe the individual subopts as well. If so I'll shove the enums into its own file. The conflict stuff is already in its own structure, so if we want to *share* that, I could just move all that into its own set of files and share that.

Re-poke.

Alright... I'm just gonna do what I think is right :)

Luis

#10 lore raw
In reply to Luis R. Rodriguez
Quoted 10 lines · Luis R. Rodriguez

You may want to stick to specific set of configuration options when creating filesystems with mkfs.xfs -- sometimes due to pure technical reasons, but some other times to ensure systems remain compatible as new features are introduced with older kernels, or if you always want to take advantage of some new feature which would otherwise typically be disruptive.

This adds support for parsing a configuration file to override defaults parameters to be used for mkfs.xfs.

Quoted 6 lines

We define an XFS configuration directory, /etc/mkfs.xfs.d/ and allow for different types of configuration files, if none is specified we look for the default type, /etc/mkfs.xfs.d/default, and you can override with -T. For instance, if you specify:

mkfs.xfs -T experimental -f /dev/loop0

Ok, since you have to fix it all anyway, can we please use "-c" for config instead of "-T" for type? Because that's confusing w/ mkfs's '-t' for the filesystem type, and mke2fs's -t for *cough* fs-type ... this isn't a type (xfs,ext4,msdos,f2fs) and it's not an fs-type ala mke2fs (PTSD from big/huge/largefile/largefile4 ... I really don't want The Internet to start writing config files for a "webserver" type for xfs. Can we just call them 'configs'?)

Quoted 7 lines

The file /etc/mkfs.xfs.d/experimental will be used as your configuration file. If you really need to override the full path of the configuration file you may use the MKFS_XFS_CONFIG environment variable.

To use /etc/ be sure to configure xfsprogs with:

./configure --sysconfdir=/etc/

surely this should be made default?

Quoted 3 lines

To verify what configuration file is used on a system use the typical:

mkfs.xfs -N

Nitpick: -N by itself is not valid, it's treated as an error. When it does get a config file, IMHO it should give its name, not just "custom configuration file"

Quoted 2 lines

There is only a subset of options allowed to be set on the configuration file,

I think this is problematic - allowing only booleans is pretty arbitrary.

Quoted 27 lines

and currently only 1 or 0 are acceptable values. The default parameters you can override on a configuration file and their current built-in default settings are:

[data]
noalign=0

[inode]
align=1
projid32bit=1
sparse=0

[log]
lazy-count=1

[metadata]
crc=1
finobt=1
rmapbt=0
reflink=0

[naming]
ftype=1

[rtdev]
noalign=0

Quoted 6 lines

---

We have floated around enough bike shedding emails to have at least reached a consensus on the fact that we'd be only supporting a mimimum set of default parameters and would strive to simplify our parser as much as possible.

See above - sorry for not chiming in sooner, but I don't think an arbitrary restriction to the boolean options will be sufficient in the long run.

...
snip
...

Quoted 27 lines
include/builddefs.in +2 -0
index 7a2a62686717..231c96196ab9 100644
@@ -63,6 +63,7 @@ PKG_LIB_DIR	= @libdir@@libdirsuffix@
 PKG_INC_DIR	= @includedir@/xfs
 DK_INC_DIR	= @includedir@/disk
 PKG_MAN_DIR	= @mandir@
+PKG_ETC_DIR	= @sysconfdir@
 PKG_DOC_DIR	= @datadir@/doc/@pkg_name@
 PKG_LOCALE_DIR	= @datadir@/locale
 
@@ -194,6 +195,7 @@ endif
 
 GCFLAGS = $(DEBUG) \
 	  -DVERSION=\"$(PKG_VERSION)\" -DLOCALEDIR=\"$(PKG_LOCALE_DIR)\"  \
+	  -DROOT_SYSCONFDIR=\"$(PKG_ETC_DIR)\"  \
 	  -DPACKAGE=\"$(PKG_NAME)\" -I$(TOPDIR)/include -I$(TOPDIR)/libxfs
 
 ifeq ($(ENABLE_GETTEXT),yes)
man/man5/mkfs.xfs.d.5 new file +0 -0
new file mode 100644
index 000000000000..89fd28c79139

man/man5/mkfs.xfs.d.5 line 3

Quoted 3 lines
man/man5/mkfs.xfs.d.5 new file +3 -0
@@ -0,0 +1,121 @@
+.TH mkfs.xfs.d 5
+.SH NAME
+mkfs.xfs.d \- mkfs.xfs configuration directory

IMHO this is all far too wordy, will put my editor hat on later.

<snip>

man/man8/mkfs.xfs.8 and · line 89

Quoted 12 lines
man/man8/mkfs.xfs.8 +4 -0
index 4b8c78c37806..18489b3a19ff 100644
@@ -83,6 +83,23 @@ and
 .B \-l internal \-l size=10m
 are equivalent.
 .PP
+An optional XFS configuration type file directory
+.B mkfs.xfs.d (5)
+exists to help fine tune default parameters which can be used when calling
+.B mkfs.xfs (8), by default type will be used by default, /etc/mkfs.xfs.d/default.

Optional mkfs.xfs configuration files may be stored in mkfs.xfs.d, which may be used to override built-in mkfs.xfs default parameters.

man/man8/mkfs.xfs.8 and · line 90

Quoted 1 line
man/man8/mkfs.xfs.8 +1 -0
+Command line arguments directly passed to

drop "directly," commandline args are area always "directly" passed.

man/man8/mkfs.xfs.8 and · line 92

Quoted 2 lines
man/man8/mkfs.xfs.8 +2 -0
+.B mkfs.xfs (8)
+will always override parameters set it the configuration type file.

drop "type."

man/man8/mkfs.xfs.8 and

Quoted 3 lines
man/man8/mkfs.xfs.8 +3 -0
+You can override configuration file type on the
+.B mkfs.xfs.d (5)
+directory by using the -t parameter and secifying the type.

A config file may be selected with the -c option as described below.

man/man8/mkfs.xfs.8 and · line 99

Quoted 5 lines

Alternatively

man/man8/mkfs.xfs.8 +4 -0
+you can set and use the MKFS_XFS_CONFIG environment variable to override
+the default full path of the first file
+.B mkfs.xfs (8)
+looks for.

(How do you envision this env var being used? How is this better than simply allowing -t to specify a full path?)

Quoted 2 lines

+If you use -t the type configuration file must be present under +.B mkfs.xfs.d (8).

Hm I'll try to wordsmith all this as well.

Quoted 5 lines
mkfs/xfs_mkfs.c +0 -0
index 1ca6a2d148c4..e66330a50b68 100644

mkfs/xfs_mkfs.c line 28

Quoted 9 lines
mkfs/xfs_mkfs.c +5 -1
@@ -21,7 +21,11 @@
 #include "xfs_multidisk.h"
 #include "libxcmd.h"
 
-
+#define MKFS_XFS_CONF_DIR	ROOT_SYSCONFDIR "/mkfs.xfs.d/"
+#define CONFIG_MAX_KEY		1024
+#define CONFIG_MAX_VALUE	PATH_MAX
+#define CONFIG_MAX_BUFFER	CONFIG_MAX_KEY + CONFIG_MAX_VALUE + 3
+#define PARAM_OPTS		"T:b:d:i:l:L:m:n:KNp:qr:s:CfV"

what dave said ;)

also re: what dave said - functioning properly only if -t/-T/-c is given first on the commandline is a little odd, though it makes some semantic sense to specify a config file first then additional options so I can live with it, I think.

But it really must then fail, as opposed to silently ignoring it, if the option comes later.

<snip wow a whole lot of lines to parse the configfile>

Ok, so my configfile-containing-actual-commandlines was shot down, but I'm a bit dismayed at the LOC required to re-parse the configfile, especially when it's not even handling all options....

I'll have to take more time to review the bulk of the code, but the behavioral issues above need to be addressed I think.

-Eric

#11 lore raw
Quoted 16 lines · Eric Sandeen
Quoted 15 lines · Luis R. Rodriguez

You may want to stick to specific set of configuration options when creating filesystems with mkfs.xfs -- sometimes due to pure technical reasons, but some other times to ensure systems remain compatible as new features are introduced with older kernels, or if you always want to take advantage of some new feature which would otherwise typically be disruptive.

This adds support for parsing a configuration file to override defaults parameters to be used for mkfs.xfs.

We define an XFS configuration directory, /etc/mkfs.xfs.d/ and allow for different types of configuration files, if none is specified we look for the default type, /etc/mkfs.xfs.d/default, and you can override with -T. For instance, if you specify:

Quoted 4 lines
Quoted 1 line

mkfs.xfs -T experimental -f /dev/loop0

Ok, since you have to fix it all anyway, can we please use "-c" for config instead of "-T" for type?

Sure. Its what I had originally used and I followed Darrick's bike-shed email to use -t, and then -T after. I guess I'll revert back.

Quoted 1 line

Can we just call them 'configs'?)

My pleasure.

Quoted 9 lines
Quoted 7 lines

The file /etc/mkfs.xfs.d/experimental will be used as your configuration file. If you really need to override the full path of the configuration file you may use the MKFS_XFS_CONFIG environment variable.

To use /etc/ be sure to configure xfsprogs with:

./configure --sysconfdir=/etc/

surely this should be made default?

As you wish.

Quoted 7 lines
Quoted 3 lines

To verify what configuration file is used on a system use the typical:

mkfs.xfs -N

Nitpick: -N by itself is not valid, it's treated as an error. When it does get a config file, IMHO it should give its name, not just "custom configuration file"

Sure, will amend.

Quoted 5 lines
Quoted 2 lines

There is only a subset of options allowed to be set on the configuration file,

I think this is problematic - allowing only booleans is pretty arbitrary.

I was trying to simplify things considering how limited the number of uints were on struct mkfs_default_params, and instead how most were just bools. But sure, we can go ahead and parse the uints out of struct mkfs_default_params as well.

Quoted 6 lines
Quoted 3 lines

We have floated around enough bike shedding emails to have at least reached a consensus on the fact that we'd be only supporting a mimimum set of default parameters and would strive to simplify our parser as much as possible.

See above - sorry for not chiming in sooner, but I don't think an arbitrary restriction to the boolean options will be sufficient in the long run.

Sure.

Quoted 8 lines

man/man5/mkfs.xfs.d.5 line 3

Quoted 6 lines
man/man5/mkfs.xfs.d.5 new file +3 -0
@@ -0,0 +1,121 @@
+.TH mkfs.xfs.d 5
+.SH NAME
+mkfs.xfs.d \- mkfs.xfs configuration directory

IMHO this is all far too wordy, will put my editor hat on later.

Thanks, I appreciate the editorial feedback.

Quoted 8 lines

man/man5/mkfs.xfs.d.5

Quoted 5 lines

Alternatively

man/man5/mkfs.xfs.d.5 new file +4 -0
+you can set and use the MKFS_XFS_CONFIG environment variable to override
+the default full path of the first file
+.B mkfs.xfs (8)
+looks for.

(How do you envision this env var being used? How is this better than simply allowing -t to specify a full path?)

-c can specify the full path if we want, its up to us if we want to allow for that. Its simpler to support it, so fine by me to only accept that and remove the environment variable.

Quoted 17 lines
Quoted 5 lines
mkfs/xfs_mkfs.c +0 -0
index 1ca6a2d148c4..e66330a50b68 100644

mkfs/xfs_mkfs.c line 28

Quoted 9 lines
mkfs/xfs_mkfs.c +5 -1
@@ -21,7 +21,11 @@
 #include "xfs_multidisk.h"
 #include "libxcmd.h"
 
-
+#define MKFS_XFS_CONF_DIR	ROOT_SYSCONFDIR "/mkfs.xfs.d/"
+#define CONFIG_MAX_KEY		1024
+#define CONFIG_MAX_VALUE	PATH_MAX
+#define CONFIG_MAX_BUFFER	CONFIG_MAX_KEY + CONFIG_MAX_VALUE + 3
+#define PARAM_OPTS		"T:b:d:i:l:L:m:n:KNp:qr:s:CfV"

what dave said ;)

Quoted 7 lines

also re: what dave said - functioning properly only if -t/-T/-c is given first on the commandline is a little odd, though it makes some semantic sense to specify a config file first then additional options so I can live with it, I think.

But it really must then fail, as opposed to silently ignoring it, if the option comes later.

I'll just enable -c to be parsed via the command line interface options so it can be defined anywhere.

Quoted 5 lines

<snip wow a whole lot of lines to parse the configfile>

Ok, so my configfile-containing-actual-commandlines was shot down, but I'm a bit dismayed at the LOC required to re-parse the configfile, especially when it's not even handling all options....

Adding uint shouldn't be too much extra code, it just means the callbacks will be passing a uint and each option will have also implement a boundary check. With the way Dave is suggesting this to be on its own file and set of data structures or range checks, this shouldn't be too hard to do, its just yet-more code.

Quoted 2 lines

I'll have to take more time to review the bulk of the code, but the behavioral issues above need to be addressed I think.

Makes sense, thanks for the review.

Luis

#12 lore raw
In reply to Eric Sandeen
Quoted 5 lines · Eric Sandeen
Quoted 2 lines

There is only a subset of options allowed to be set on the configuration file,

I think this is problematic - allowing only booleans is pretty arbitrary.

...

Quoted 6 lines
Quoted 6 lines

and currently only 1 or 0 are acceptable values. The default parameters you can override on a configuration file and their current built-in default settings are:

[data]
noalign=0

...

Quoted 6 lines
Quoted 3 lines

We have floated around enough bike shedding emails to have at least reached a consensus on the fact that we'd be only supporting a mimimum set of default parameters and would strive to simplify our parser as much as possible.

See above - sorry for not chiming in sooner, but I don't think an arbitrary restriction to the boolean options will be sufficient in the long run.

Ok, I'm rethinking this concern now.

It seems like we may have two separate but related purposes for a mkfs config file:

1) Set system-wide defaults based on kernel and/or OS support.
2) Set specific-use defaults, such as "mkfs.xfs -c gluster" etc.

For the former, it may well only be the booleans that matter. For the latter, it may be more, like setting inode size, or directory block size, etc.

Is there anything we'd reasonably want to set system-wide that's /not/ a boolean? If not, the maybe that can be Phase 1 as long as the config file format doesn't need to change; we can just add to the list of supported config options at a later date, perhaps?

The system-wide, supported-features default config may be the most pressing need at this point, the rest is just nice to have.

Thanks,
-Eric

#13 lore raw
Quoted 32 lines · Eric Sandeen
Quoted 5 lines · Eric Sandeen
Quoted 2 lines

There is only a subset of options allowed to be set on the configuration file,

I think this is problematic - allowing only booleans is pretty arbitrary.

...

Quoted 6 lines
Quoted 6 lines

and currently only 1 or 0 are acceptable values. The default parameters you can override on a configuration file and their current built-in default settings are:

[data]
noalign=0

...

Quoted 6 lines
Quoted 3 lines

We have floated around enough bike shedding emails to have at least reached a consensus on the fact that we'd be only supporting a mimimum set of default parameters and would strive to simplify our parser as much as possible.

See above - sorry for not chiming in sooner, but I don't think an arbitrary restriction to the boolean options will be sufficient in the long run.

Ok, I'm rethinking this concern now.

It seems like we may have two separate but related purposes for a mkfs config file:

1) Set system-wide defaults based on kernel and/or OS support.
2) Set specific-use defaults, such as "mkfs.xfs -c gluster" etc.

Quoted 7 lines

For the former, it may well only be the booleans that matter. For the latter, it may be more, like setting inode size, or directory block size, etc.

Is there anything we'd reasonably want to set system-wide that's /not/ a boolean? If not, the maybe that can be Phase 1 as long as the config file format doesn't need to change; we can just add to the list of supported config options at a later date, perhaps?

Looking at the *documented* mkfs options, I think it'd be useful for cloud users to be able to set cowextsize and extsize from mkfs.

As I was saying on irc, I /think/ the options (Allison feel free to chime in here) that we want are....

-m crc,finobt,rmapbt,reflink,projinherit,extszinherit,cowextsize,extsize -i maxpct,sparse
-n ftype

--D

Quoted 9 lines

The system-wide, supported-features default config may be the most pressing need at this point, the rest is just nice to have.

Thanks,
-Eric
--
To unsubscribe from this list: send the line "unsubscribe linux-xfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#14 lore raw
Quoted 36 lines · Darrick J. Wong
Quoted 35 lines · Eric Sandeen
Quoted 5 lines · Eric Sandeen
Quoted 2 lines

There is only a subset of options allowed to be set on the configuration file,

I think this is problematic - allowing only booleans is pretty arbitrary.

...

Quoted 6 lines
Quoted 6 lines

and currently only 1 or 0 are acceptable values. The default parameters you can override on a configuration file and their current built-in default settings are:

[data]
noalign=0

...

Quoted 6 lines
Quoted 3 lines

We have floated around enough bike shedding emails to have at least reached a consensus on the fact that we'd be only supporting a mimimum set of default parameters and would strive to simplify our parser as much as possible.

See above - sorry for not chiming in sooner, but I don't think an arbitrary restriction to the boolean options will be sufficient in the long run.

Ok, I'm rethinking this concern now.

It seems like we may have two separate but related purposes for a mkfs config file:

1) Set system-wide defaults based on kernel and/or OS support.
2) Set specific-use defaults, such as "mkfs.xfs -c gluster" etc.

For the former, it may well only be the booleans that matter. For the latter, it may be more, like setting inode size, or directory block size, etc.

Quoted 7 lines
Quoted 4 lines

Is there anything we'd reasonably want to set system-wide that's /not/ a boolean? If not, the maybe that can be Phase 1 as long as the config file format doesn't need to change; we can just add to the list of supported config options at a later date, perhaps?

Looking at the *documented* mkfs options, I think it'd be useful for cloud users to be able to set cowextsize and extsize from mkfs.

Fair, but I guess my point is that that falls more into special-use config optimization, vs. supported/not-supported defaults.

And as such could maybe come in Phase 2 of the grand plan. ;)

-Eric

Quoted 8 lines

As I was saying on irc, I /think/ the options (Allison feel free to chime in here) that we want are....

(quoted diff) +0 -4
-m crc,finobt,rmapbt,reflink,projinherit,extszinherit,cowextsize,extsize
-i maxpct,sparse
-n ftype
 
--D
#15 lore raw
In reply to Darrick J. Wong
Quoted 47 lines · Darrick J. Wong
Quoted 32 lines · Eric Sandeen
Quoted 5 lines · Eric Sandeen
Quoted 2 lines

There is only a subset of options allowed to be set on the configuration file,

I think this is problematic - allowing only booleans is pretty arbitrary.

...

Quoted 6 lines
Quoted 6 lines

and currently only 1 or 0 are acceptable values. The default parameters you can override on a configuration file and their current built-in default settings are:

[data]
noalign=0

...

Quoted 6 lines
Quoted 3 lines

We have floated around enough bike shedding emails to have at least reached a consensus on the fact that we'd be only supporting a mimimum set of default parameters and would strive to simplify our parser as much as possible.

See above - sorry for not chiming in sooner, but I don't think an arbitrary restriction to the boolean options will be sufficient in the long run.

Ok, I'm rethinking this concern now.

It seems like we may have two separate but related purposes for a mkfs config file:

1) Set system-wide defaults based on kernel and/or OS support.
2) Set specific-use defaults, such as "mkfs.xfs -c gluster" etc.

Quoted 7 lines

For the former, it may well only be the booleans that matter. For the latter, it may be more, like setting inode size, or directory block size, etc.

Is there anything we'd reasonably want to set system-wide that's /not/ a boolean? If not, the maybe that can be Phase 1 as long as the config file format doesn't need to change; we can just add to the list of supported config options at a later date, perhaps?

Looking at the *documented* mkfs options, I think it'd be useful for cloud users to be able to set cowextsize and extsize from mkfs.

As I was saying on irc, I /think/ the options (Allison feel free to chime in here) that we want are....

Review

Quoted 3 lines
(quoted diff) +0 -3
-m crc,finobt,rmapbt,reflink,projinherit,extszinherit,cowextsize,extsize
-i maxpct,sparse
-n ftype

Of these, the following would need respective defaults added:

-m extszinherit,cowextsize,extsize
-i maxpct,

Once defaults are added, the respective configuration entry can be added. The rest would be supported from the start, and from what I have seen all that would go in would enable support for use on older kernels.

Luis

Quoted 13 lines

--D

Quoted 9 lines

The system-wide, supported-features default config may be the most pressing need at this point, the rest is just nice to have.

Thanks,
-Eric
--
To unsubscribe from this list: send the line "unsubscribe linux-xfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Signature · 1 line
Do not panic