Skip to contentSkip to conversation

mkfs: Configuration file defined options

PATCH In review Dave Chinner 2 participants 9 messages Oct 27, 2020 – Oct 29, 2020

lore
Share & tools
lore
#1 lore raw

PATCH v3 1/5 build: add support for libinih for mkfs

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

Need to make sure the library is present so we can build mkfs with config file support.

---
 configure.ac         |  3 +++
 doc/INSTALL          |  5 +++++
 include/builddefs.in |  1 +
 m4/package_inih.m4   | 20 ++++++++++++++++++++
 4 files changed, 29 insertions(+)
 create mode 100644 m4/package_inih.m4
configure.ac +3 -0
index 345d90be8b8c..c419f16c39c2 100644
@@ -145,6 +145,9 @@ AC_PACKAGE_UTILITIES(xfsprogs)
 AC_MULTILIB($enable_lib64)
 AC_RT($enable_librt)
 
+AC_PACKAGE_NEED_INI_H
+AC_PACKAGE_NEED_LIBINIH
+
 AC_PACKAGE_NEED_UUID_H
 AC_PACKAGE_NEED_UUIDCOMPARE
 
doc/INSTALL +5 -0
index d4395eefa834..2b04f9cfe108 100644
@@ -28,6 +28,11 @@ Linux Instructions
    (on an RPM based system) or the uuid-dev package (on a Debian system)
    as some of the commands make use of the UUID library provided by these.
 
+   If your distro does not provide a libinih package, you can download and build
+   it from source from the upstream repository found at:
+
+	https://github.com/benhoyt/inih
+
    To build the package and install it manually, use the following steps:
 
 	# make
include/builddefs.in +1 -0
index 30b2727a8db4..e8f447f92baf 100644
@@ -27,6 +27,7 @@ LIBTERMCAP = @libtermcap@
 LIBEDITLINE = @libeditline@
 LIBBLKID = @libblkid@
 LIBDEVMAPPER = @libdevmapper@
+LIBINIH = @libinih@
 LIBXFS = $(TOPDIR)/libxfs/libxfs.la
 LIBFROG = $(TOPDIR)/libfrog/libfrog.la
 LIBXCMD = $(TOPDIR)/libxcmd/libxcmd.la
m4/package_inih.m4 new file +20 -0
new file mode 100644
index 000000000000..a2775592e09d
@@ -0,0 +1,20 @@
+AC_DEFUN([AC_PACKAGE_NEED_INI_H],
+  [ AC_CHECK_HEADERS([ini.h])
+    if test $ac_cv_header_ini_h = no; then
+	echo
+	echo 'FATAL ERROR: could not find a valid ini.h header.'
+	echo 'Install the libinih development package.'
+	exit 1
+    fi
+  ])
+
+AC_DEFUN([AC_PACKAGE_NEED_LIBINIH],
+  [ AC_CHECK_LIB(inih, ini_parse,, [
+	echo
+	echo 'FATAL ERROR: could not find a valid inih library.'
+	echo 'Install the libinih library package.'
+	exit 1
+    ])
+    libinih=-linih
+    AC_SUBST(libinih)
+  ])
Signature · 1 line
2.28.0
#0 lore raw

v3 cover mkfs: Configuration file defined options

Version 3:

- really do "-c file=xxx" > "-c options=xxx" everywhere
- update libinih parser error handling to be more robust
- ini_name -> ini_section and comment updates
- man page layout and grammar improvements
- Updated Darrick's old config file fstests to exercise the new code.

Version 2:

- "-c file=xxx" > "-c options=xxx"
- split out constification into new patch
- removed debug output
- fixed some comments
- added man page stuff

Hi Folks,

Because needing config files for mkfs came up yet again in discussion, here is a simple implementation of INI format config files. These config files behave identically to options specified on the command line - the do not change defaults, they do not override CLI options, they are not overridden by cli options.

Example:

$ echo -e "[metadata]\ncrc = 0" > foo
$ mkfs/mkfs.xfs -N -c options=foo -d file=1,size=100m blah
Parameters parsed from config file foo successfully
meta-data=blah                   isize=256    agcount=4, agsize=6400 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=0        finobt=0, sparse=0, rmapbt=0
         =                       reflink=0
data     =                       bsize=4096   blocks=25600, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0, ftype=1
log      =internal log           bsize=4096   blocks=853, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
$

And there's a V4 filesystem as specified by the option defined in the config file. If we do:

$ mkfs/mkfs.xfs -N -c options=foo -m crc=1 -d file=1,size=100m blah
-m crc option respecified
Usage: mkfs.xfs
.....
$

You can see it errors out because the CRC option was specified in both the config file and on the CLI.

There's lots of stuff we can do to make the conflict and respec error messages better, but that doesn't change the basic functionality of config file based mkfs options. To allow for future changes to the way we want to apply config files, I created a full option subtype for config files. That means we can add another option to say "apply config file as default values rather than as options" if we decide that is functionality that we want to support.

However, policy decisions like that are completely separate to the mechanism, so these patches don't try to address desires to ship "tuned" configs, system wide option files, shipping distro specific defaults in config files, etc. This is purely a mechanism to allow users to specify options via files instead of on the CLI. No more, no less.

Git branch can be found here:

git://git.kernel.org/pub/scm/linux/kernel/git/dgc/xfsprogs-dev.git mkfs-config-file

Cheers,

Dave.

Dave Chinner (5):
build: add support for libinih for mkfs
mkfs: add initial ini format config file parsing support
mkfs: constify various strings
mkfs: hook up suboption parsing to ini files
mkfs: document config files in mkfs.xfs(8)

 configure.ac         |   3 +
 doc/INSTALL          |   5 +
 include/builddefs.in |   1 +
 include/linux.h      |   2 +-
 m4/package_inih.m4   |  20 ++++
 man/man8/mkfs.xfs.8  | 113 +++++++++++++++++++--
 mkfs/Makefile        |   2 +-
 mkfs/xfs_mkfs.c      | 231 ++++++++++++++++++++++++++++++++++++++-----
 8 files changed, 343 insertions(+), 34 deletions(-)
 create mode 100644 m4/package_inih.m4
Signature · 1 line
2.28.0
#4 lore raw

PATCH v3 3/5 mkfs: constify various strings

From: Dave Chinner <[email protected]>

Because the ini parser uses const strings and so the opt parsing needs to be told about it to avoid compiler warnings.

---
 include/linux.h |  2 +-
 mkfs/xfs_mkfs.c | 28 ++++++++++++++--------------
 2 files changed, 15 insertions(+), 15 deletions(-)
include/linux.h +1 -1
index 57726bb12b74..03b3278bb895 100644
@@ -92,7 +92,7 @@ static __inline__ void platform_uuid_unparse(uuid_t *uu, char *buffer)
 	uuid_unparse(*uu, buffer);
 }
 
-static __inline__ int platform_uuid_parse(char *buffer, uuid_t *uu)
+static __inline__ int platform_uuid_parse(const char *buffer, uuid_t *uu)
 {
 	return uuid_parse(buffer, *uu);
 }
mkfs/xfs_mkfs.c +14 -14
index 33be9ba16c90..829f0383b602 100644
@@ -975,8 +975,8 @@ respec(
 
 static void
 unknown(
-	char		opt,
-	char		*s)
+	const char	opt,
+	const char	*s)
 {
 	fprintf(stderr, _("unknown option -%c %s\n"), opt, s);
 	usage();
@@ -1387,7 +1387,7 @@ getnum(
  */
 static char *
 getstr(
-	char			*str,
+	const char		*str,
 	struct opt_params	*opts,
 	int			index)
 {
@@ -1396,14 +1396,14 @@ getstr(
 	/* empty strings for string options are not valid */
 	if (!str || *str == '\0')
 		reqval(opts->name, opts->subopts, index);
-	return str;
+	return (char *)str;
 }
 
 static int
 block_opts_parser(
 	struct opt_params	*opts,
 	int			subopt,
-	char			*value,
+	const char		*value,
 	struct cli_params	*cli)
 {
 	switch (subopt) {
@@ -1420,7 +1420,7 @@ static int
 cfgfile_opts_parser(
 	struct opt_params	*opts,
 	int			subopt,
-	char			*value,
+	const char		*value,
 	struct cli_params	*cli)
 {
 	switch (subopt) {
@@ -1437,7 +1437,7 @@ static int
 data_opts_parser(
 	struct opt_params	*opts,
 	int			subopt,
-	char			*value,
+	const char		*value,
 	struct cli_params	*cli)
 {
 	switch (subopt) {
@@ -1506,7 +1506,7 @@ static int
 inode_opts_parser(
 	struct opt_params	*opts,
 	int			subopt,
-	char			*value,
+	const char		*value,
 	struct cli_params	*cli)
 {
 	switch (subopt) {
@@ -1541,7 +1541,7 @@ static int
 log_opts_parser(
 	struct opt_params	*opts,
 	int			subopt,
-	char			*value,
+	const char		*value,
 	struct cli_params	*cli)
 {
 	switch (subopt) {
@@ -1587,7 +1587,7 @@ static int
 meta_opts_parser(
 	struct opt_params	*opts,
 	int			subopt,
-	char			*value,
+	const char		*value,
 	struct cli_params	*cli)
 {
 	switch (subopt) {
@@ -1621,7 +1621,7 @@ static int
 naming_opts_parser(
 	struct opt_params	*opts,
 	int			subopt,
-	char			*value,
+	const char		*value,
 	struct cli_params	*cli)
 {
 	switch (subopt) {
@@ -1650,7 +1650,7 @@ static int
 rtdev_opts_parser(
 	struct opt_params	*opts,
 	int			subopt,
-	char			*value,
+	const char		*value,
 	struct cli_params	*cli)
 {
 	switch (subopt) {
@@ -1680,7 +1680,7 @@ static int
 sector_opts_parser(
 	struct opt_params	*opts,
 	int			subopt,
-	char			*value,
+	const char		*value,
 	struct cli_params	*cli)
 {
 	switch (subopt) {
@@ -1700,7 +1700,7 @@ static struct subopts {
 	struct opt_params *opts;
 	int		(*parser)(struct opt_params	*opts,
 				  int			subopt,
-				  char			*value,
+				  const char		*value,
 				  struct cli_params	*cli);
 } subopt_tab[] = {
 	{ 'b', &bopts, block_opts_parser },
Signature · 1 line
2.28.0
#5 lore raw

PATCH v3 4/5 mkfs: hook up suboption parsing to ini files

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

Now we have the config file parsing hooked up and feeding in parameters to mkfs, wire the parameters up to the existing CLI option parsing functions. THis gives the config file exactly the same capabilities and constraints as the command line option specification.

---
 mkfs/xfs_mkfs.c | 94 +++++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 79 insertions(+), 15 deletions(-)
mkfs/xfs_mkfs.c +79 -15
index 829f0383b602..635a36f393b7 100644
@@ -143,6 +143,12 @@ enum {
  * name MANDATORY
  *   Name is a single char, e.g., for '-d file', name is 'd'.
  *
+ * ini_section MANDATORY
+ *   This field is required to connect each opt_params (that is to say, each
+ *   option class) to a section in the config file. The only option class this
+ *   is not required for is the config file specification class itself.
+ *   The section name is a string, not longer than MAX_INI_NAME_LEN.
+ *
  * subopts MANDATORY
  *   Subopts is a list of strings naming suboptions. In the example above,
  *   it would contain "file". The last entry of this list has to be NULL.
@@ -201,6 +207,8 @@ enum {
  */
 struct opt_params {
 	const char	name;
+#define MAX_INI_NAME_LEN	32
+	const char	ini_section[MAX_INI_NAME_LEN];
 	const char	*subopts[MAX_SUBOPTS];
 
 	struct subopt_param {
@@ -228,6 +236,7 @@ static struct opt_params sopts;
 
 static struct opt_params bopts = {
 	.name = 'b',
+	.ini_section = "block",
 	.subopts = {
 		[B_SIZE] = "size",
 	},
@@ -267,6 +276,7 @@ static struct opt_params copts = {
 
 static struct opt_params dopts = {
 	.name = 'd',
+	.ini_section = "data",
 	.subopts = {
 		[D_AGCOUNT] = "agcount",
 		[D_FILE] = "file",
@@ -411,6 +421,7 @@ static struct opt_params dopts = {
 
 static struct opt_params iopts = {
 	.name = 'i',
+	.ini_section = "inode",
 	.subopts = {
 		[I_ALIGN] = "align",
 		[I_MAXPCT] = "maxpct",
@@ -472,6 +483,7 @@ static struct opt_params iopts = {
 
 static struct opt_params lopts = {
 	.name = 'l',
+	.ini_section = "log",
 	.subopts = {
 		[L_AGNUM] = "agnum",
 		[L_INTERNAL] = "internal",
@@ -571,6 +583,7 @@ static struct opt_params lopts = {
 
 static struct opt_params nopts = {
 	.name = 'n',
+	.ini_section = "naming",
 	.subopts = {
 		[N_SIZE] = "size",
 		[N_VERSION] = "version",
@@ -602,6 +615,7 @@ static struct opt_params nopts = {
 
 static struct opt_params ropts = {
 	.name = 'r',
+	.ini_section = "realtime",
 	.subopts = {
 		[R_EXTSIZE] = "extsize",
 		[R_SIZE] = "size",
@@ -652,6 +666,7 @@ static struct opt_params ropts = {
 
 static struct opt_params sopts = {
 	.name = 's',
+	.ini_section = "sector",
 	.subopts = {
 		[S_SIZE] = "size",
 		[S_SECTSIZE] = "sectsize",
@@ -682,6 +697,7 @@ static struct opt_params sopts = {
 
 static struct opt_params mopts = {
 	.name = 'm',
+	.ini_section = "metadata",
 	.subopts = {
 		[M_CRC] = "crc",
 		[M_FINOBT] = "finobt",
@@ -982,6 +998,17 @@ unknown(
 	usage();
 }
 
+static void
+invalid_cfgfile_opt(
+	const char	*filename,
+	const char	*section,
+	const char	*name,
+	const char	*value)
+{
+	fprintf(stderr, _("%s: invalid config file option: [%s]:%s=%s\n"),
+		filename, section, name, value);
+}
+
 static void
 check_device_type(
 	const char	*name,
@@ -1696,23 +1723,22 @@ sector_opts_parser(
 }
 
 static struct subopts {
-	char		opt;
 	struct opt_params *opts;
 	int		(*parser)(struct opt_params	*opts,
 				  int			subopt,
 				  const char		*value,
 				  struct cli_params	*cli);
 } subopt_tab[] = {
-	{ 'b', &bopts, block_opts_parser },
-	{ 'c', &copts, cfgfile_opts_parser },
-	{ 'd', &dopts, data_opts_parser },
-	{ 'i', &iopts, inode_opts_parser },
-	{ 'l', &lopts, log_opts_parser },
-	{ 'm', &mopts, meta_opts_parser },
-	{ 'n', &nopts, naming_opts_parser },
-	{ 'r', &ropts, rtdev_opts_parser },
-	{ 's', &sopts, sector_opts_parser },
-	{ '\0', NULL, NULL },
+	{ &bopts, block_opts_parser },
+	{ &copts, cfgfile_opts_parser },
+	{ &dopts, data_opts_parser },
+	{ &iopts, inode_opts_parser },
+	{ &lopts, log_opts_parser },
+	{ &mopts, meta_opts_parser },
+	{ &nopts, naming_opts_parser },
+	{ &ropts, rtdev_opts_parser },
+	{ &sopts, sector_opts_parser },
+	{ NULL, NULL },
 };
 
 static void
@@ -1726,7 +1752,7 @@ parse_subopts(
 	int		ret = 0;
 
 	while (sop->opts) {
-		if (sop->opt == opt)
+		if (opt && sop->opts->name == opt)
 			break;
 		sop++;
 	}
@@ -1749,6 +1775,45 @@ parse_subopts(
 	}
 }
 
+static bool
+parse_cfgopt(
+	const char	*section,
+	const char	*name,
+	const char	*value,
+	struct cli_params *cli)
+{
+	struct subopts	*sop = &subopt_tab[0];
+	char		**subopts;
+	int		ret = 0;
+	int		i;
+
+	while (sop->opts) {
+		if (sop->opts->ini_section[0] != '\0' &&
+		    strcasecmp(section, sop->opts->ini_section) == 0)
+			break;
+		sop++;
+	}
+
+	/* Config files with unknown sections get caught here. */
+	if (!sop->opts)
+		goto invalid_opt;
+
+	subopts = (char **)sop->opts->subopts;
+	for (i = 0; i < MAX_SUBOPTS; i++) {
+		if (!subopts[i])
+			break;
+		if (strcasecmp(name, subopts[i]) == 0) {
+			ret = (sop->parser)(sop->opts, i, value, cli);
+			if (ret)
+				goto invalid_opt;
+			return true;
+		}
+	}
+invalid_opt:
+	invalid_cfgfile_opt(cli->cfgfile, section, name, value);
+	return false;
+}
+
 static void
 validate_sectorsize(
 	struct mkfs_params	*cfg,
@@ -3630,9 +3695,8 @@ cfgfile_parse_ini(
 {
 	struct cli_params	*cli = user;
 
-	fprintf(stderr, "Ini debug: file %s, section %s, name %s, value %s\n",
-		cli->cfgfile, section, name, value);
-
+	if (!parse_cfgopt(section, name, value, cli))
+		return 0;
 	return 1;
 }
 
Signature · 1 line
2.28.0
#7 lore raw

PATCH v3 5/5 mkfs: document config files in mkfs.xfs(8)

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

So people know it exists.

---
 man/man8/mkfs.xfs.8 | 113 +++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 106 insertions(+), 7 deletions(-)
man/man8/mkfs.xfs.8 +106 -7
index 0a7858748457..b959f293edb5 100644
@@ -122,8 +122,46 @@ If the size of the block or sector is not specified, the default sizes
 Many feature options allow an optional argument of 0 or 1, to explicitly
 disable or enable the functionality.
 .SH OPTIONS
+Options may be specified either on the command line or in a configuration file.
+Not all command line options can be specified in configuration files; only the
+command line options followed by a
+.B [section]
+label can be used in a configuration file.
+.PP
+Options that can be used in configuration files are grouped into related
+sections containing multiple options.
+The command line options and configuration files use the same option
+sections and grouping.
+Configuration file section names are listed in the command line option
+sections below.
+Option names and values are the same for both command line
+and configuration file specification.
+.PP
+Options specified are the combined set of command line parameters and
+configuration file parameters.
+Duplicated options will result in a respecification error, regardless of the
+location they were specified at.
+.TP
+.BI \-c " configuration_file_option"
+This option specifies the files that mkfs configuration will be obtained from.
+The valid
+.I configuration_file_option
+is:
+.RS 1.2i
 .TP
+.BI options= name
+The configuration options will be sourced from the file specified by the
+.I name
+option string.
+This option can be use either an absolute or relative path to the configuration
+file to be read.
+.RE
+.PP
+.PD 0
 .BI \-b " block_size_options"
+.TP
+.BI "Section Name: " [block]
+.PD
 This option specifies the fundamental block size of the filesystem.
 The valid
 .I block_size_option
@@ -141,8 +179,12 @@ Although
 will accept any of these values and create a valid filesystem,
 XFS on Linux can only mount filesystems with pagesize or smaller blocks.
 .RE
-.TP
+.PP
+.PD 0
 .BI \-m " global_metadata_options"
+.TP
+.BI "Section Name: " [metadata]
+.PD
 These options specify metadata format options that either apply to the entire
 filesystem or aren't easily characterised by a specific functionality group. The
 valid
@@ -243,8 +285,12 @@ reflink-enabled XFS filesystems.  To use filesystem DAX with XFS, specify the
 .B \-m reflink=0
 option to mkfs.xfs to disable the reflink feature.
 .RE
-.TP
+.PP
+.PD 0
 .BI \-d " data_section_options"
+.TP
+.BI "Section Name: " [data]
+.PD
 These options specify the location, size, and other parameters of the
 data section of the filesystem. The valid
 .I data_section_options
@@ -416,8 +462,12 @@ By default,
 .B mkfs.xfs
 will not write to the device if it suspects that there is a filesystem
 or partition table on the device already.
-.TP
+.PP
+.PD 0
 .BI \-i " inode_options"
+.TP
+.BI "Section Name: " [inode]
+.PD
 This option specifies the inode size of the filesystem, and other
 inode allocation parameters.
 The XFS inode contains a fixed-size part and a variable-size part.
@@ -537,8 +587,12 @@ accommodate a chunk of 64 inodes. Without this feature enabled, inode
 allocations can fail with out of space errors under severe fragmented
 free space conditions.
 .RE
-.TP
+.PP
+.PD 0
 .BI \-l " log_section_options"
+.TP
+.BI "Section Name: " [log]
+.PD
 These options specify the location, size, and other parameters of the
 log section of the filesystem. The valid
 .I log_section_options
@@ -651,8 +705,12 @@ is 1 (on) so you must specify
 if you want to disable this feature for older kernels which don't support
 it.
 .RE
-.TP
+.PP
+.PD 0
 .BI \-n " naming_options"
+.TP
+.BI "Section Name: " [naming]
+.PD
 These options specify the version and size parameters for the naming
 (directory) area of the filesystem. The valid
 .I naming_options
@@ -858,8 +916,12 @@ to be constructed;
 the
 .B \-q
 flag suppresses this.
-.TP
+.PP
+.PD 0
 .BI \-r " realtime_section_options"
+.TP
+.BI "Section Name: " [realtime]
+.PD
 These options specify the location, size, and other parameters of the
 real-time section of the filesystem. The valid
 .I realtime_section_options
@@ -893,8 +955,12 @@ or logical volume containing the section.
 This option disables stripe size detection, enforcing a realtime device with no
 stripe geometry.
 .RE
-.TP
+.PP
+.PD 0
 .BI \-s " sector_size_options"
+.TP
+.BI "Section Name: " [sector]
+.PD
 This option specifies the fundamental sector size of the filesystem.
 The valid
 .I sector_size_option
@@ -933,6 +999,39 @@ Do not attempt to discard blocks at mkfs time.
 .TP
 .B \-V
 Prints the version number and exits.
+.SH Configuration File Format
+The configuration file uses a basic INI format to specify sections and options
+within a section.
+Section and option names are case sensitive.
+Section names must not contain whitespace.
+Options are name-value pairs, ended by the first whitespace in the line.
+Option names cannot contain whitespace.
+Full line comments can be added by starting a line with a # symbol.
+If values contain whitespace, then it must be quoted.
+.PP
+The following example configuration file sets the block size to 4096 bytes,
+turns on reverse mapping btrees and sets the inode size to 2048 bytes.
+.PP
+.PD 0
+# Example mkfs.xfs configuration file
+.HP
+.HP
+[block]
+.HP
+size=4k
+.HP
+.HP
+[metadata]
+.HP
+rmapbt=1
+.HP
+.HP
+[inode]
+.HP
+size=2048
+.HP
+.PD
+.PP
 .SH SEE ALSO
 .BR xfs (5),
 .BR mkfs (8),
Signature · 1 line
2.28.0
#2 lore raw

PATCH v3 2/5 mkfs: add initial ini format config file parsing support

In reply to Dave Chinner

From: Dave Chinner <[email protected]>

Add the framework that will allow the config file to be supplied on the CLI and passed to the library that will parse it. This does not yet do any option parsing from the config file.

---
 mkfs/Makefile   |   2 +-
 mkfs/xfs_mkfs.c | 119 +++++++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 119 insertions(+), 2 deletions(-)
mkfs/Makefile +1 -1
index 31482b08d559..b8805f7e1ea1 100644
@@ -11,7 +11,7 @@ HFILES =
 CFILES = proto.c xfs_mkfs.c
 
 LLDLIBS += $(LIBXFS) $(LIBXCMD) $(LIBFROG) $(LIBRT) $(LIBPTHREAD) $(LIBBLKID) \
-	$(LIBUUID)
+	$(LIBUUID) $(LIBINIH)
 LTDEPENDENCIES += $(LIBXFS) $(LIBXCMD) $(LIBFROG)
 LLDFLAGS = -static-libtool-libs
 
mkfs/xfs_mkfs.c +118 -1

185 lines not on this page · raw message

Signature · 1 line
2.28.0
Reviewed-by on v3 2/5
#3 lore raw

Re: [PATCH v3 2/5] mkfs: add initial ini format config file parsing support

Quoted 7 lines · Dave Chinner

From: Dave Chinner <[email protected]>

Add the framework that will allow the config file to be supplied on the CLI and passed to the library that will parse it. This does not yet do any option parsing from the config file.

Looks ok,

--D

Quoted 220 lines
---
 mkfs/Makefile   |   2 +-
 mkfs/xfs_mkfs.c | 119 +++++++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 119 insertions(+), 2 deletions(-)
mkfs/Makefile +1 -1
index 31482b08d559..b8805f7e1ea1 100644
@@ -11,7 +11,7 @@ HFILES =
 CFILES = proto.c xfs_mkfs.c
 
 LLDLIBS += $(LIBXFS) $(LIBXCMD) $(LIBFROG) $(LIBRT) $(LIBPTHREAD) $(LIBBLKID) \
-	$(LIBUUID)
+	$(LIBUUID) $(LIBINIH)
 LTDEPENDENCIES += $(LIBXFS) $(LIBXCMD) $(LIBFROG)
 LLDFLAGS = -static-libtool-libs
 
mkfs/xfs_mkfs.c +118 -1
index 8fe149d74b0a..33be9ba16c90 100644
@@ -11,6 +11,7 @@
 #include "libfrog/fsgeom.h"
 #include "libfrog/topology.h"
 #include "libfrog/convert.h"
+#include <ini.h>
 
 #define TERABYTES(count, blog)	((uint64_t)(count) << (40 - (blog)))
 #define GIGABYTES(count, blog)	((uint64_t)(count) << (30 - (blog)))
@@ -44,6 +45,11 @@ enum {
 	B_MAX_OPTS,
 };
 
+enum {
+	C_OPTFILE = 0,
+	C_MAX_OPTS,
+};
+
 enum {
 	D_AGCOUNT = 0,
 	D_FILE,
@@ -237,6 +243,28 @@ static struct opt_params bopts = {
 	},
 };
 
+/*
+ * Config file specification. Usage is:
+ *
+ * mkfs.xfs -c options=<name>
+ *
+ * A subopt is used for the filename so in future we can extend the behaviour
+ * of the config file (e.g. specified defaults rather than options) if we ever
+ * have a need to do that sort of thing.
+ */
+static struct opt_params copts = {
+	.name = 'c',
+	.subopts = {
+		[C_OPTFILE] = "options",
+	},
+	.subopt_params = {
+		{ .index = C_OPTFILE,
+		  .conflicts = { { NULL, LAST_CONFLICT } },
+		  .defaultval = SUBOPT_NEEDS_VAL,
+		},
+	},
+};
+
 static struct opt_params dopts = {
 	.name = 'd',
 	.subopts = {
@@ -748,6 +776,8 @@ struct cli_params {
 	int	sectorsize;
 	int	blocksize;
 
+	char	*cfgfile;
+
 	/* parameters that depend on sector/block size being validated. */
 	char	*dsize;
 	char	*agsize;
@@ -862,6 +892,7 @@ usage( void )
 {
 	fprintf(stderr, _("Usage: %s\n\
 /* blocksize */		[-b size=num]\n\
+/* config file */	[-c options=xxx]\n\
 /* metadata */		[-m crc=0|1,finobt=0|1,uuid=xxx,rmapbt=0|1,reflink=0|1]\n\
 /* data subvol */	[-d agcount=n,agsize=n,file,name=xxx,size=num,\n\
 			    (sunit=value,swidth=value|su=num,sw=num|noalign),\n\
@@ -1385,6 +1416,23 @@ block_opts_parser(
 	return 0;
 }
 
+static int
+cfgfile_opts_parser(
+	struct opt_params	*opts,
+	int			subopt,
+	char			*value,
+	struct cli_params	*cli)
+{
+	switch (subopt) {
+	case C_OPTFILE:
+		cli->cfgfile = getstr(value, opts, subopt);
+		break;
+	default:
+		return -EINVAL;
+	}
+	return 0;
+}
+
 static int
 data_opts_parser(
 	struct opt_params	*opts,
@@ -1656,6 +1704,7 @@ static struct subopts {
 				  struct cli_params	*cli);
 } subopt_tab[] = {
 	{ 'b', &bopts, block_opts_parser },
+	{ 'c', &copts, cfgfile_opts_parser },
 	{ 'd', &dopts, data_opts_parser },
 	{ 'i', &iopts, inode_opts_parser },
 	{ 'l', &lopts, log_opts_parser },
@@ -3562,6 +3611,65 @@ check_root_ino(
 	}
 }
 
+/*
+ * INI file format option parser.
+ *
+ * This is called by the file parser library for every valid option it finds in
+ * the config file. The option is already broken down into a
+ * {section,name,value} tuple, so all we need to do is feed it to the correct
+ * suboption parser function and translate the return value.
+ *
+ * Returns 0 on failure, 1 for success.
+ */
+static int
+cfgfile_parse_ini(
+	void			*user,
+	const char		*section,
+	const char		*name,
+	const char		*value)
+{
+	struct cli_params	*cli = user;
+
+	fprintf(stderr, "Ini debug: file %s, section %s, name %s, value %s\n",
+		cli->cfgfile, section, name, value);
+
+	return 1;
+}
+
+void
+cfgfile_parse(
+	struct cli_params	*cli)
+{
+	int			error;
+
+	if (!cli->cfgfile)
+		return;
+
+	error = ini_parse(cli->cfgfile, cfgfile_parse_ini, cli);
+	if (error) {
+		if (error > 0) {
+			fprintf(stderr,
+		_("%s: Unrecognised input on line %d. Aborting.\n"),
+				cli->cfgfile, error);
+		} else if (error == -1) {
+			fprintf(stderr,
+		_("Unable to open config file %s. Aborting.\n"),
+				cli->cfgfile);
+		} else if (error == -2) {
+			fprintf(stderr,
+		_("Memory allocation failure parsing %s. Aborting.\n"),
+				cli->cfgfile);
+		} else {
+			fprintf(stderr,
+		_("Unknown error %d opening config file %s. Aborting.\n"),
+				error, cli->cfgfile);
+		}
+		exit(1);
+	}
+	printf(_("Parameters parsed from config file %s successfully\n"),
+		cli->cfgfile);
+}
+
 int
 main(
 	int			argc,
@@ -3648,13 +3756,14 @@ main(
 	memcpy(&cli.sb_feat, &dft.sb_feat, sizeof(cli.sb_feat));
 	memcpy(&cli.fsx, &dft.fsx, sizeof(cli.fsx));
 
-	while ((c = getopt(argc, argv, "b:d:i:l:L:m:n:KNp:qr:s:CfV")) != EOF) {
+	while ((c = getopt(argc, argv, "b:c:d:i:l:L:m:n:KNp:qr:s:CfV")) != EOF) {
 		switch (c) {
 		case 'C':
 		case 'f':
 			force_overwrite = 1;
 			break;
 		case 'b':
+		case 'c':
 		case 'd':
 		case 'i':
 		case 'l':
@@ -3698,6 +3807,14 @@ main(
 	} else
 		dfile = xi.dname;
 
+	/*
+	 * Now we have all the options parsed, we can read in the option file
+	 * specified on the command line via "-c options=xxx". Once we have all
+	 * the options from this file parsed, we can then proceed with parameter
+	 * and bounds checking and making the filesystem.
+	 */
+	cfgfile_parse(&cli);
+
 	protostring = setup_proto(protofile);
 
 	/*
Signature · 1 line
2.28.0
#6 lore raw

Re: [PATCH v3 4/5] mkfs: hook up suboption parsing to ini files

In reply to Dave Chinner
Quoted 106 lines · Dave Chinner

From: Dave Chinner <[email protected]>

Now we have the config file parsing hooked up and feeding in parameters to mkfs, wire the parameters up to the existing CLI option parsing functions. THis gives the config file exactly the same capabilities and constraints as the command line option specification.

---
 mkfs/xfs_mkfs.c | 94 +++++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 79 insertions(+), 15 deletions(-)
mkfs/xfs_mkfs.c +16 -0
index 829f0383b602..635a36f393b7 100644
@@ -143,6 +143,12 @@ enum {
  * name MANDATORY
  *   Name is a single char, e.g., for '-d file', name is 'd'.
  *
+ * ini_section MANDATORY
+ *   This field is required to connect each opt_params (that is to say, each
+ *   option class) to a section in the config file. The only option class this
+ *   is not required for is the config file specification class itself.
+ *   The section name is a string, not longer than MAX_INI_NAME_LEN.
+ *
  * subopts MANDATORY
  *   Subopts is a list of strings naming suboptions. In the example above,
  *   it would contain "file". The last entry of this list has to be NULL.
@@ -201,6 +207,8 @@ enum {
  */
 struct opt_params {
 	const char	name;
+#define MAX_INI_NAME_LEN	32
+	const char	ini_section[MAX_INI_NAME_LEN];
 	const char	*subopts[MAX_SUBOPTS];
 
 	struct subopt_param {
@@ -228,6 +236,7 @@ static struct opt_params sopts;
 
 static struct opt_params bopts = {
 	.name = 'b',
+	.ini_section = "block",
 	.subopts = {
 		[B_SIZE] = "size",
 	},
@@ -267,6 +276,7 @@ static struct opt_params copts = {
 
 static struct opt_params dopts = {
 	.name = 'd',
+	.ini_section = "data",
 	.subopts = {
 		[D_AGCOUNT] = "agcount",
 		[D_FILE] = "file",
@@ -411,6 +421,7 @@ static struct opt_params dopts = {
 
 static struct opt_params iopts = {
 	.name = 'i',
+	.ini_section = "inode",
 	.subopts = {
 		[I_ALIGN] = "align",
 		[I_MAXPCT] = "maxpct",
@@ -472,6 +483,7 @@ static struct opt_params iopts = {
 
 static struct opt_params lopts = {
 	.name = 'l',
+	.ini_section = "log",
 	.subopts = {
 		[L_AGNUM] = "agnum",
 		[L_INTERNAL] = "internal",
@@ -571,6 +583,7 @@ static struct opt_params lopts = {
 
 static struct opt_params nopts = {
 	.name = 'n',
+	.ini_section = "naming",
 	.subopts = {
 		[N_SIZE] = "size",
 		[N_VERSION] = "version",
@@ -602,6 +615,7 @@ static struct opt_params nopts = {
 
 static struct opt_params ropts = {
 	.name = 'r',
+	.ini_section = "realtime",
 	.subopts = {
 		[R_EXTSIZE] = "extsize",
 		[R_SIZE] = "size",
@@ -652,6 +666,7 @@ static struct opt_params ropts = {
 
 static struct opt_params sopts = {
 	.name = 's',
+	.ini_section = "sector",
 	.subopts = {
 		[S_SIZE] = "size",
 		[S_SECTSIZE] = "sectsize",
@@ -682,6 +697,7 @@ static struct opt_params sopts = {
 
 static struct opt_params mopts = {
 	.name = 'm',
+	.ini_section = "metadata",
 	.subopts = {
 		[M_CRC] = "crc",
 		[M_FINOBT] = "finobt",
@@ -982,6 +998,17 @@ unknown(
 	usage();
 }

mkfs/xfs_mkfs.c unknown( · line 1009

Quoted 10 lines
mkfs/xfs_mkfs.c +9 -0
 
+static void
+invalid_cfgfile_opt(
+	const char	*filename,
+	const char	*section,
+	const char	*name,
+	const char	*value)
+{
+	fprintf(stderr, _("%s: invalid config file option: [%s]:%s=%s\n"),
+		filename, section, name, value);

Silly nit: space after the last colon.

"/etc/xfs.conf: invalid config file option: [metadata]: crc=0"

(A pity libinih doesn't tell you the line number...)

Also, does this really need a separate function since it's only called from one place?

Quoted 40 lines
mkfs/xfs_mkfs.c +12 -11
+}
+
 static void
 check_device_type(
 	const char	*name,
@@ -1696,23 +1723,22 @@ sector_opts_parser(
 }
 
 static struct subopts {
-	char		opt;
 	struct opt_params *opts;
 	int		(*parser)(struct opt_params	*opts,
 				  int			subopt,
 				  const char		*value,
 				  struct cli_params	*cli);
 } subopt_tab[] = {
-	{ 'b', &bopts, block_opts_parser },
-	{ 'c', &copts, cfgfile_opts_parser },
-	{ 'd', &dopts, data_opts_parser },
-	{ 'i', &iopts, inode_opts_parser },
-	{ 'l', &lopts, log_opts_parser },
-	{ 'm', &mopts, meta_opts_parser },
-	{ 'n', &nopts, naming_opts_parser },
-	{ 'r', &ropts, rtdev_opts_parser },
-	{ 's', &sopts, sector_opts_parser },
-	{ '\0', NULL, NULL },
+	{ &bopts, block_opts_parser },
+	{ &copts, cfgfile_opts_parser },
+	{ &dopts, data_opts_parser },
+	{ &iopts, inode_opts_parser },
+	{ &lopts, log_opts_parser },
+	{ &mopts, meta_opts_parser },
+	{ &nopts, naming_opts_parser },
+	{ &ropts, rtdev_opts_parser },
+	{ &sopts, sector_opts_parser },
+	{ NULL, NULL },
 };
 
 static void

mkfs/xfs_mkfs.c parse_subopts( · line 1755

Quoted 5 lines
mkfs/xfs_mkfs.c +1 -1
@@ -1726,7 +1752,7 @@ parse_subopts(
 	int		ret = 0;
 
 	while (sop->opts) {
-		if (sop->opt == opt)
+		if (opt && sop->opts->name == opt)

When does parse_subopts get passed a opt==0? AFAICT the only caller is the switch statement under getopt, which constrains c to one of 'bdilmnrsc'.

--D

Quoted 64 lines
mkfs/xfs_mkfs.c +41 -3
 			break;
 		sop++;
 	}
@@ -1749,6 +1775,45 @@ parse_subopts(
 	}
 }
 
+static bool
+parse_cfgopt(
+	const char	*section,
+	const char	*name,
+	const char	*value,
+	struct cli_params *cli)
+{
+	struct subopts	*sop = &subopt_tab[0];
+	char		**subopts;
+	int		ret = 0;
+	int		i;
+
+	while (sop->opts) {
+		if (sop->opts->ini_section[0] != '\0' &&
+		    strcasecmp(section, sop->opts->ini_section) == 0)
+			break;
+		sop++;
+	}
+
+	/* Config files with unknown sections get caught here. */
+	if (!sop->opts)
+		goto invalid_opt;
+
+	subopts = (char **)sop->opts->subopts;
+	for (i = 0; i < MAX_SUBOPTS; i++) {
+		if (!subopts[i])
+			break;
+		if (strcasecmp(name, subopts[i]) == 0) {
+			ret = (sop->parser)(sop->opts, i, value, cli);
+			if (ret)
+				goto invalid_opt;
+			return true;
+		}
+	}
+invalid_opt:
+	invalid_cfgfile_opt(cli->cfgfile, section, name, value);
+	return false;
+}
+
 static void
 validate_sectorsize(
 	struct mkfs_params	*cfg,
@@ -3630,9 +3695,8 @@ cfgfile_parse_ini(
 {
 	struct cli_params	*cli = user;
 
-	fprintf(stderr, "Ini debug: file %s, section %s, name %s, value %s\n",
-		cli->cfgfile, section, name, value);
-
+	if (!parse_cfgopt(section, name, value, cli))
+		return 0;
 	return 1;
 }
 
Signature · 1 line
2.28.0
Reviewed-by on v3 5/5
#8 lore raw

Re: [PATCH v3 5/5] mkfs: document config files in mkfs.xfs(8)

In reply to Dave Chinner
Quoted 5 lines · Dave Chinner

From: Dave Chinner <[email protected]>

So people know it exists.

Yay!

--D

Quoted 196 lines
---
 man/man8/mkfs.xfs.8 | 113 +++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 106 insertions(+), 7 deletions(-)
man/man8/mkfs.xfs.8 +106 -7
index 0a7858748457..b959f293edb5 100644
@@ -122,8 +122,46 @@ If the size of the block or sector is not specified, the default sizes
 Many feature options allow an optional argument of 0 or 1, to explicitly
 disable or enable the functionality.
 .SH OPTIONS
+Options may be specified either on the command line or in a configuration file.
+Not all command line options can be specified in configuration files; only the
+command line options followed by a
+.B [section]
+label can be used in a configuration file.
+.PP
+Options that can be used in configuration files are grouped into related
+sections containing multiple options.
+The command line options and configuration files use the same option
+sections and grouping.
+Configuration file section names are listed in the command line option
+sections below.
+Option names and values are the same for both command line
+and configuration file specification.
+.PP
+Options specified are the combined set of command line parameters and
+configuration file parameters.
+Duplicated options will result in a respecification error, regardless of the
+location they were specified at.
+.TP
+.BI \-c " configuration_file_option"
+This option specifies the files that mkfs configuration will be obtained from.
+The valid
+.I configuration_file_option
+is:
+.RS 1.2i
 .TP
+.BI options= name
+The configuration options will be sourced from the file specified by the
+.I name
+option string.
+This option can be use either an absolute or relative path to the configuration
+file to be read.
+.RE
+.PP
+.PD 0
 .BI \-b " block_size_options"
+.TP
+.BI "Section Name: " [block]
+.PD
 This option specifies the fundamental block size of the filesystem.
 The valid
 .I block_size_option
@@ -141,8 +179,12 @@ Although
 will accept any of these values and create a valid filesystem,
 XFS on Linux can only mount filesystems with pagesize or smaller blocks.
 .RE
-.TP
+.PP
+.PD 0
 .BI \-m " global_metadata_options"
+.TP
+.BI "Section Name: " [metadata]
+.PD
 These options specify metadata format options that either apply to the entire
 filesystem or aren't easily characterised by a specific functionality group. The
 valid
@@ -243,8 +285,12 @@ reflink-enabled XFS filesystems.  To use filesystem DAX with XFS, specify the
 .B \-m reflink=0
 option to mkfs.xfs to disable the reflink feature.
 .RE
-.TP
+.PP
+.PD 0
 .BI \-d " data_section_options"
+.TP
+.BI "Section Name: " [data]
+.PD
 These options specify the location, size, and other parameters of the
 data section of the filesystem. The valid
 .I data_section_options
@@ -416,8 +462,12 @@ By default,
 .B mkfs.xfs
 will not write to the device if it suspects that there is a filesystem
 or partition table on the device already.
-.TP
+.PP
+.PD 0
 .BI \-i " inode_options"
+.TP
+.BI "Section Name: " [inode]
+.PD
 This option specifies the inode size of the filesystem, and other
 inode allocation parameters.
 The XFS inode contains a fixed-size part and a variable-size part.
@@ -537,8 +587,12 @@ accommodate a chunk of 64 inodes. Without this feature enabled, inode
 allocations can fail with out of space errors under severe fragmented
 free space conditions.
 .RE
-.TP
+.PP
+.PD 0
 .BI \-l " log_section_options"
+.TP
+.BI "Section Name: " [log]
+.PD
 These options specify the location, size, and other parameters of the
 log section of the filesystem. The valid
 .I log_section_options
@@ -651,8 +705,12 @@ is 1 (on) so you must specify
 if you want to disable this feature for older kernels which don't support
 it.
 .RE
-.TP
+.PP
+.PD 0
 .BI \-n " naming_options"
+.TP
+.BI "Section Name: " [naming]
+.PD
 These options specify the version and size parameters for the naming
 (directory) area of the filesystem. The valid
 .I naming_options
@@ -858,8 +916,12 @@ to be constructed;
 the
 .B \-q
 flag suppresses this.
-.TP
+.PP
+.PD 0
 .BI \-r " realtime_section_options"
+.TP
+.BI "Section Name: " [realtime]
+.PD
 These options specify the location, size, and other parameters of the
 real-time section of the filesystem. The valid
 .I realtime_section_options
@@ -893,8 +955,12 @@ or logical volume containing the section.
 This option disables stripe size detection, enforcing a realtime device with no
 stripe geometry.
 .RE
-.TP
+.PP
+.PD 0
 .BI \-s " sector_size_options"
+.TP
+.BI "Section Name: " [sector]
+.PD
 This option specifies the fundamental sector size of the filesystem.
 The valid
 .I sector_size_option
@@ -933,6 +999,39 @@ Do not attempt to discard blocks at mkfs time.
 .TP
 .B \-V
 Prints the version number and exits.
+.SH Configuration File Format
+The configuration file uses a basic INI format to specify sections and options
+within a section.
+Section and option names are case sensitive.
+Section names must not contain whitespace.
+Options are name-value pairs, ended by the first whitespace in the line.
+Option names cannot contain whitespace.
+Full line comments can be added by starting a line with a # symbol.
+If values contain whitespace, then it must be quoted.
+.PP
+The following example configuration file sets the block size to 4096 bytes,
+turns on reverse mapping btrees and sets the inode size to 2048 bytes.
+.PP
+.PD 0
+# Example mkfs.xfs configuration file
+.HP
+.HP
+[block]
+.HP
+size=4k
+.HP
+.HP
+[metadata]
+.HP
+rmapbt=1
+.HP
+.HP
+[inode]
+.HP
+size=2048
+.HP
+.PD
+.PP
 .SH SEE ALSO
 .BR xfs (5),
 .BR mkfs (8),
Signature · 1 line
2.28.0