Skip to contentSkip to conversation

mkfs config file bikeshed now!

RFC Darrick J. Wong 6 participants 15 messages Feb 26, 2018 – Feb 27, 2018

lore
Share & tools
lore
#0 lore raw
Quoted 3 lines · Darrick J. Wong
Quoted 3 lines

Clearly. Its still a nice sensible feature to address any defaults, and now that I see chinner/mkfs-refactor is merged I'll go work off of that and post patches again soon.

I'll help you start the config file format bikeshedding:

So here at $employer we have to support a bunch of different base system and kernel configurations. Some of our customers run heterogeneous environments where they have to be able to go back and forth between recent and old systems (stock OL6.9 <-> OL7 + UEK4) whereas others only care about formatting to whatever runs on the deployed system. Right now we ship xfsprogs 4.5 with very conservative defaults, but this has become a hassle to keep patching mkfs, to remember which option sets correspond with which config, and to communicate that whole mess to customers. Shipping multiple xfsprogs packages seems gross.

What we'd prefer to do is ship a single up to date xfsprogs package along with a mkfs config file containing distro-supported filesystem profiles. As a side note, if there's no config file then we'd use whatever defaults are coded into xfs_mkfs.c.

For example, say we had a /etc/xfs.conf file shipping with xfsprogs:

[defaults]
metadata.crc = false
naming.ftype = false

[rhel7]
metadata.crc = true

[uek3]
naming.ftype = true

[uek4]
metadata.crc = true
naming.ftype = true
metdata.finobt = true

[experimental]
metadata.rmapbt = true
metadata.reflink = true
inode.sparse = true
data.agcount = 32

The "defaults" section enables the distro vendor to patch in whatever default config they want to support. In the example above the distro is not yet ready to support ftype or v5 filesystems everywhere because they want maximum compatibility, even though xfsprogs (by this point in the future) enables them by default.

# mkfs.xfs /dev/sda <-- formats a boring v4 filesystem

However, if a customer only runs "uek4" on their infrastructure, they might want to take advantage of the superior performance of ftype and finobt. They can elect to trade compatibility for speed:

# mkfs.xfs -t uek4 /dev/sda <-- v5 fs with ftype and finobt

The truly crazy can set their warranty cards on fire:

# mkfs.xfs -t experimental /dev/sda

Under this scheme, we take the suboption names directly from the command line parser and expand the section names in some fashion:

b -> block
d -> data
i -> inode
l -> log
m -> metadata
n -> naming
r -> realtime
s -> sector

And throw in "label" and "protofile" for -L and -p.

Note that a libconfig file (Dave suggested libconfig ages ago) might look more like this:

defaults {
	metadata {
		crc = false
	}
	naming {
		ftype = false
	}
}
rhel7 {
	metadata {
		crc = true
	}
}
uek4 {
	metadata {
		crc = true
		finobt = true
	}
	naming {
		ftype = true
	}
}
experimental {
	metadata {
		rmapbt = true
		reflink = true
	}
	inode {
		sparse = true
	}
	data {
		agcount = 32
	}
}

One downside of the libconfig API unfortunately is that we'd have to manually iterate every option group/line to detect garbage / unknown key values like:

broken {
	metadata {
		klsldgasldgjslgdajldsjgaldsgs = 98272352
	}
}

since there's no way to give the parser a higher level schema that constricts the allowable keywords. But that's (easily/clumsily) worked around.

Ok, what does everyone else think?

--D

#1 lore raw
Quoted 18 lines · Darrick J. Wong
Quoted 3 lines · Darrick J. Wong
Quoted 3 lines

Clearly. Its still a nice sensible feature to address any defaults, and now that I see chinner/mkfs-refactor is merged I'll go work off of that and post patches again soon.

I'll help you start the config file format bikeshedding:

So here at $employer we have to support a bunch of different base system and kernel configurations. Some of our customers run heterogeneous environments where they have to be able to go back and forth between recent and old systems (stock OL6.9 <-> OL7 + UEK4) whereas others only care about formatting to whatever runs on the deployed system. Right now we ship xfsprogs 4.5 with very conservative defaults, but this has become a hassle to keep patching mkfs, to remember which option sets correspond with which config, and to communicate that whole mess to customers. Shipping multiple xfsprogs packages seems gross.

Quoted 4 lines

What we'd prefer to do is ship a single up to date xfsprogs package along with a mkfs config file containing distro-supported filesystem profiles. As a side note, if there's no config file then we'd use whatever defaults are coded into xfs_mkfs.c.

I think that is an extremely important note, and the only sane behavior. :)

Quoted 1 line

For example, say we had a /etc/xfs.conf file shipping with xfsprogs:

/etc/mkfs.xfs.conf please, this configures mkfs and nothing else, right?

Quoted 22 lines

[defaults]
metadata.crc = false
naming.ftype = false

[rhel7]
metadata.crc = true

[uek3]
naming.ftype = true

[uek4]
metadata.crc = true
naming.ftype = true
metdata.finobt = true

[experimental]
metadata.rmapbt = true
metadata.reflink = true
inode.sparse = true
data.agcount = 32

Quoted 5 lines

The "defaults" section enables the distro vendor to patch in whatever default config they want to support. In the example above the distro is not yet ready to support ftype or v5 filesystems everywhere because they want maximum compatibility, even though xfsprogs (by this point in the future) enables them by default.

Do the subsequent sections each independently specify the behavior or do they build on the defaults? I'd prefer the former (do not build on [defaults]), I think.

Quoted 1 line
# mkfs.xfs /dev/sda <-- formats a boring v4 filesystem

IIRC Dave had a patch that emitted what was being used for defaults if not the code - also IIRC I removed it but we should put that back in with any config file parsing, I think.

"Using defaults from config file [defaults]" or somesuch.

I had also asked whether upstream would ship a config file at all. I'd suggest that we should not, because we wouldn't do anything other than restate code defaults, and I'm not interested in keeping that consistent.

(OTOH a mkfs.xfs --config-file-defaults to spit out a code-defaults configfile for editing might be neato) ;)

-Eric

#2 lore raw
Quoted 26 lines · Eric Sandeen
Quoted 18 lines · Darrick J. Wong
Quoted 3 lines · Darrick J. Wong
Quoted 3 lines

Clearly. Its still a nice sensible feature to address any defaults, and now that I see chinner/mkfs-refactor is merged I'll go work off of that and post patches again soon.

I'll help you start the config file format bikeshedding:

So here at $employer we have to support a bunch of different base system and kernel configurations. Some of our customers run heterogeneous environments where they have to be able to go back and forth between recent and old systems (stock OL6.9 <-> OL7 + UEK4) whereas others only care about formatting to whatever runs on the deployed system. Right now we ship xfsprogs 4.5 with very conservative defaults, but this has become a hassle to keep patching mkfs, to remember which option sets correspond with which config, and to communicate that whole mess to customers. Shipping multiple xfsprogs packages seems gross.

Quoted 4 lines

What we'd prefer to do is ship a single up to date xfsprogs package along with a mkfs config file containing distro-supported filesystem profiles. As a side note, if there's no config file then we'd use whatever defaults are coded into xfs_mkfs.c.

Quoted 5 lines

I think that is an extremely important note, and the only sane behavior. :)

Quoted 1 line

For example, say we had a /etc/xfs.conf file shipping with xfsprogs:

/etc/mkfs.xfs.conf please, this configures mkfs and nothing else, right?

Ok. I mean, I might let that be $sysconfdir/mkfs.xfs.conf or something, but I agree about the filename component of the path.

Quoted 11 lines
Quoted 7 lines

[defaults]
metadata.crc = false
naming.ftype = false

[rhel7]
metadata.crc = true

Oh, one more thing. Let's not arbitrarily decide that the commandline uses 1/0 but the configfile uses true/false. I'd really like as much 1:1 consistency here as possible.

Oops. I of course meant 0 / 1 here, since the goal is to use the existing mkfs parsing system here.

Quoted 20 lines
Quoted 14 lines

[uek3]
naming.ftype = true

[uek4]
metadata.crc = true
naming.ftype = true
metdata.finobt = true

[experimental]
metadata.rmapbt = true
metadata.reflink = true
inode.sparse = true
data.agcount = 32

Quoted 5 lines

The "defaults" section enables the distro vendor to patch in whatever default config they want to support. In the example above the distro is not yet ready to support ftype or v5 filesystems everywhere because they want maximum compatibility, even though xfsprogs (by this point in the future) enables them by default.

Quoted 3 lines

Do the subsequent sections each independently specify the behavior or do they build on the defaults? I'd prefer the former (do not build on [defaults]), I think.

Not sure either. It would be useful to let the non-defaults profile inherit off of the defaults profile, but otoh to an administrator it's a lot clearer what they're getting with a profile if we force the config file writer to specify the entire configuration.

Hmmm, I am drifting towards "do not build on defaults".

Quoted 7 lines
Quoted 1 line
# mkfs.xfs /dev/sda <-- formats a boring v4 filesystem

IIRC Dave had a patch that emitted what was being used for defaults if not the code - also IIRC I removed it but we should put that back in with any config file parsing, I think.

"Using defaults from config file [defaults]" or somesuch.

Yes. How about:

"Using profile 'rhel7' from /etc/mkfs.xfs.conf."

Quoted 3 lines

I had also asked whether upstream would ship a config file at all. I'd suggest that we should not, because we wouldn't do anything other than restate code defaults, and I'm not interested in keeping that consistent.

I don't see much of a point since we'd be shipping

# mkfs.xfs --config-file-defaults > /etc/mkfs.xfs.conf
Quoted 2 lines

(OTOH a mkfs.xfs --config-file-defaults to spit out a code-defaults configfile for editing might be neato) ;)

--generate-config-file?

--D

Quoted 5 lines

-Eric
--
To unsubscribe from this list: send the line "unsubscribe linux-xfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#3 lore raw
Quoted 3 lines · Darrick J. Wong
Quoted 2 lines

(OTOH a mkfs.xfs --config-file-defaults to spit out a code-defaults configfile for editing might be neato) ;)

--generate-config-file?

Yeah that's what I meant. I don't think it's that important, tho.

-Eric

#4 lore raw
In reply to Darrick J. Wong
Quoted 5 lines · Darrick J. Wong
Quoted 2 lines
Quoted 1 line

For example, say we had a /etc/xfs.conf file shipping with xfsprogs:

/etc/mkfs.xfs.conf please, this configures mkfs and nothing else, right?

Ok. I mean, I might let that be $sysconfdir/mkfs.xfs.conf or something, but I agree about the filename component of the path.

*nod*

Also a $sysconfdir/mkfs.xfs.conf.d/ scanning scheme might be a useful feature.

Or if we ban inheritance between sections (please), as darrick stated offline maybe just allowing configfile specification on the commandline so that you could point to /etc/xfs/mkfs/$FOO_OS.conf

The point would be that over-arching OS config file package could drop in all the relevant config files for its defaults rather than munging it all together in one config file. I.e. if ceph wants one thing and glusterfs wants another they could each supply their own mkfs.xfs config file w/o hassle.

-Eric

#5 lore raw
In reply to Darrick J. Wong
Quoted 8 lines · Darrick J. Wong

For example, say we had a /etc/xfs.conf file shipping with xfsprogs:

[defaults]
metadata.crc = false
naming.ftype = false

[rhel7]
metadata.crc = true

Oh, one more thing. Let's not arbitrarily decide that the commandline uses 1/0 but the configfile uses true/false. I'd really like as much 1:1 consistency here as possible.

-Eric

#6 lore raw
In reply to Darrick J. Wong
Quoted 2 lines · Darrick J. Wong

Note that a libconfig file (Dave suggested libconfig ages ago) might look more like this:

No, Dave *only hinted* towards libconfig ;)

I ran into grotesque issues with libconfig, and quickly moved to evaluate libini_config (used for SSSD) as an alternative. IMHO it had the best and up to date implementation, had great community traction, and I saw support for it on a slew of old older distros.

I also evaluated Ted Tso's e2fprogs profile parser (based on MIT kerberos), and it turns out that its 14.86% the size of libini_config.so.5.2.0, which is an incredible bonus. It also supports to work with a slew of different OSes (which we probably don't care for), but the gains in size are perhaps worth considering. That and the fact that it will likely build on probably even more ancient distros. The e2fsprogs profile parser also lacks uint64_t support but adding support is very trivial. Another thing with e2fprogs's profile parser is we'd have to come to an agreement with Ted and the e2fsprogs community on sharing the library code. He seems to be open to that if we go this route. To help I already did a lot of the work of extracting what we'd need and also threw in a demo program:

https://gitlab.com/mcgrof/libekprofile

Similar review / demo is available with the other libraries:

https://gitlab.com/mcgrof/libconfig-int64-issues
https://gitlab.com/mcgrof/libini_config-demo

For boring details on further evaluation with other filesystems and what they do and nitty gritty details of the above evaluation you can read my paper [0]. I even had evaluated NFS's configuration parser... at this point I was surprised everyone keeps re-inventing the wheel...

One reason to consider sharing also is to help other filesystems as they likely will run into similar requirements. Why does everyone keep re-inventing this simple wheel on filesystem configuration parsers? Yes I suspect btrfs may want something similar.

So in terms of bikeshedding -- its probably important also to at least zero down on a library we want to work with. The rest I suspect will actually be pretty trivial bikeshedding.

FWIW both libini_config and the e2fsprogs profile parser could cope with namespaces on the configuration file as well. The syntax just varies slightly. IIRC with e2fsprogs profile parsers we could end up with something like:

[defaults]
foo=0
bar=0
[stuff]

	some_release =  {
		foo=1
		bar=2
	}
	new_release = {
		foo=2
	}

Meanwhile I think with libini_config it was something like:

[defaults]
foo=0
bar=0
[stuff/some_release]
foo=1
bar=2
[stuff/new_release]
foo=2

[0] https://gitlab.com/mcgrof/filesystem-configuration-paper/blob/master/paper.pdf

Luis

#7 lore raw
Quoted 7 lines · Luis R. Rodriguez

IIRC with e2fsprogs profile parsers we could end up with something
like:

[defaults]
foo=0
bar=0
[stuff]

Quoted 7 lines
	some_release =  {
		foo=1
		bar=2
	}
	new_release = {
		foo=2
	}

I can't tell what that means. What's "stuff?" Why would we need this sort of nesting?

I /really/ want to adhere to the KISS principle here as much as possible.

The last thing I want is for the config file structure to make it difficult for admins to understand, and easy to mis-write, resulting in unexpected configs deployed...

There are all kinds of clever things we could do (see previous question about inheritance) but config files are going to be write-once-use-many for the most part, and I really think that complexity would be a net negative here. I'd much rather have explicit cut and paste in each section than confusing nesting of option specifications.

-Eric

#8 lore raw
Quoted 18 lines · Eric Sandeen
Quoted 7 lines · Luis R. Rodriguez

IIRC with e2fsprogs profile parsers we could end up with something
like:

[defaults]
foo=0
bar=0
[stuff]

Quoted 7 lines
      some_release =  {
              foo=1
              bar=2
      }
      new_release = {
              foo=2
      }
Quoted 2 lines

I can't tell what that means. What's "stuff?" Why would we need this sort of nesting?

I'll leave the crafty examples to Darrick as he had some complex ideas in mind. On my part I was happy to not have to deal with the namespace stuff -- however I realized some may want it, so I had to mention both libraries libini_config and e2fsprogs profile parser do support it, just in different ways.

Luis

#9 lore raw
Quoted 19 lines · Luis R. Rodriguez
Quoted 18 lines · Eric Sandeen
Quoted 14 lines · Luis R. Rodriguez

IIRC with e2fsprogs profile parsers we could end up with something
like:

[defaults]
foo=0
bar=0
[stuff]

      some_release =  {
              foo=1
              bar=2
      }
      new_release = {
              foo=2
      }
Quoted 8 lines
Quoted 2 lines

I can't tell what that means. What's "stuff?" Why would we need this sort of nesting?

I'll leave the crafty examples to Darrick as he had some complex ideas in mind. On my part I was happy to not have to deal with the namespace stuff -- however I realized some may want it, so I had to mention both libraries libini_config and e2fsprogs profile parser do support it, just in different ways.

Well I /did/ have an even stupider parser in mind if segmented config files got shot down...

/etc/xfs/mkfs/uek5.conf:
data.agcount 32
metadata.reflink 0

# mkfs.xfs -t uek5 /dev/sda

open("/etc/xfs/mkfs/${t_optarg}.conf")...

Then we can parse it with a simple fgets/strtok loop to tokenize the input lines and feed them to the geometry arg parser. But I'll go read your paper all the way through first before I scribble more. :)

--D

Quoted 5 lines

Luis
--
To unsubscribe from this list: send the line "unsubscribe linux-xfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#10 lore raw

So number one, unlike everyone else, I didn't reinvent the wheel. I grabbed the same wheel I wrote for MIT Kerberos years before. :-)

Number two, for examples for I wanted the nesting, from MIT Kerberos:

[realms]
ATHENA.MIT.EDU = {
kdc = kerberos.mit.edu
kdc = kerberos-1.mit.edu
kdc = kerberos-2.mit.edu:750
admin_server = kerberos.mit.edu
master_kdc = kerberos.mit.edu
default_domain = mit.edu
}
EXAMPLE.COM = {
kdc = kerberos.example.com
kdc = kerberos-1.example.com
admin_server = kerberos.example.com
}

And from the file system world, from /etc/mke2fs.conf:

[fs_types]

	ext3 = {
		features = has_journal
	}
	ext4 = {
		features = has_journal,extent,huge_file,flex_bg,metadata_csum,64bit,dir_nlink,extra_isize
		inode_size = 256
	}
	small = {
		blocksize = 1024
		inode_size = 128
		inode_ratio = 4096
	}
	...

It's been handy inside Google since we have different file system configs for different use cases, so we do things like:

mke2fs -T foo-use /dev/sdX

and
mke2fs -T bar-use /dev/sdX

... where foo-use and bar-use would be subsections under fs_types.

But hey, to be clear, I'm not the one trying to claim everyone should use my library (even though it's the best looking and above-average, ala Lake Wobegon. :-)

- Ted

#11 lore raw

One other thing I'll mention is that it was really important to me that it was very easy to look up values from the config file. So for me it was a lot more than just the parser. So I can fetch a read a config parameter from the config file like this:

	profile_get_boolean(ctx->profile, "options", "no_optimize_extents",
			    0, 0, &c);

That's because I use this for a lot more than just file system tuning parametres, but also for changing how e2fsck / mke2fs behaves. So trying to parse all of the config file at startup time and having to stash that in some global context structure is just lot of extra work that isn't necessary with the profile library compared with other libraries which are focused exclusively on the parsing aspect of things (which is actually pretty trivial).

Maybe this doesn't matter for XFS, but other file system utilities may find this issue to be something worthy of consideration when trying to pick a config file library.

- Ted

#12 lore raw
Quoted 16 lines · Theodore Ts'o

One other thing I'll mention is that it was really important to me that it was very easy to look up values from the config file. So for me it was a lot more than just the parser. So I can fetch a read a config parameter from the config file like this:

        profile_get_boolean(ctx->profile, "options", "no_optimize_extents",
                            0, 0, &c);

That's because I use this for a lot more than just file system tuning parametres, but also for changing how e2fsck / mke2fs behaves. So trying to parse all of the config file at startup time and having to stash that in some global context structure is just lot of extra work that isn't necessary with the profile library compared with other libraries which are focused exclusively on the parsing aspect of things (which is actually pretty trivial).

Quoted 3 lines

Maybe this doesn't matter for XFS, but other file system utilities may find this issue to be something worthy of consideration when trying to pick a config file library.

Neat. But yeah, I don't think the nesting would be useful for xfs. Especially as I prefer the /etc/mkfs.xfs.d/ idea. It might make sense to use something like this, instead of foo.bar style prefixing, though.

/etc/mkfs.xfs.d/default:

[metadata]
crc = 0

[naming]
ftype = 0

Cheers,
Jan

Quoted 6 lines

- Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-xfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#13 lore raw
Quoted 8 lines · Jan Tulak
Quoted 7 lines · Theodore Ts'o

One other thing I'll mention is that it was really important to me that it was very easy to look up values from the config file. So for me it was a lot more than just the parser. So I can fetch a read a config parameter from the config file like this:

        profile_get_boolean(ctx->profile, "options", "no_optimize_extents",
                            0, 0, &c);

That's not the way we've set up mkfs to deal with config file defaults.

i.e. the config file creates a "default values" structure that gets used when no CLI parameter was specified. IOWs, all we need the config file to do is to be parsed into a pre-defined structure and the rest of mkfs will already DTRT.

i.e. We don't need (or want) mkfs to be directly pulling random values from config files depending on the current option context.

Quoted 17 lines
Quoted 11 lines

That's because I use this for a lot more than just file system tuning parametres, but also for changing how e2fsck / mke2fs behaves. So trying to parse all of the config file at startup time and having to stash that in some global context structure is just lot of extra work that isn't necessary with the profile library compared with other libraries which are focused exclusively on the parsing aspect of things (which is actually pretty trivial).

Maybe this doesn't matter for XFS, but other file system utilities may find this issue to be something worthy of consideration when trying to pick a config file library.

Neat. But yeah, I don't think the nesting would be useful for xfs. Especially as I prefer the /etc/mkfs.xfs.d/ idea. It might make sense to use something like this, instead of foo.bar style prefixing, though.

Quoted 7 lines

/etc/mkfs.xfs.d/default:

[metadata]
crc = 0

[naming]
ftype = 0

Yup, I like the simplicity of this approach. One config per file, with simple config groups that mirror the CLI grouping and options. Simple for admins, simple for developers, simple to implement and test.

We really don't need any more complexity than this, so lets not make it any more complex than it needs to be - that mistake has already been made in recent times with mkfs.xfs modifications, and I don't want to dig us out of that hole again...

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#14 lore raw
In reply to Theodore Ts'o
Quoted 2 lines · Theodore Ts'o

So number one, unlike everyone else, I didn't reinvent the wheel. I grabbed the same wheel I wrote for MIT Kerberos years before. :-)

Yes thank you for not re-inventing the wheel.

Quoted 3 lines

But hey, to be clear, I'm not the one trying to claim everyone should use my library (even though it's the best looking and above-average, ala Lake Wobegon. :-)

I started out with libconfig. Clearly that's crap. As I waited for the facelift of mkfs/xfs_mkfs.c I found libini_config and was supportive of it as the best in the community given community traction and good code. As I poked further and extracted e2fsprogs profile library into its own repo and sized it up against libini_config, I'll let it be on me to say:

I think its *very* sensible for us to consider using it *provided* we're up to co-maintain a future small shared library in the hopes others will stop also re-inventing the wheel further.

Unless I hear otherwise I'll give e2fsprogs profile library a shot for my next iteration under the assumption we can share the library long term.

Luis