Skip to contentSkip to conversation

Using git to store /etc, redux

DISCUSSION David Härdeman 3 participants 5 messages Jun 15, 2016

lore
Share & tools
lore
#0 lore raw

I recently had the idea to store and track /etc using git. When googling the topic I came across the "Using git to store /etc" thread from the end of last year which provided some interesting details on what would be necessary.

It seems the file metadata (owner, group, mode, xattrs, etc) was the big stumbling point, so I wrote up a tool over the last few days which allows the metadata to be stored in a separate file which can be stored along with the rest of the data in the repo (or separately).

This is also useful for tripwire type checks and for other types of storage which drops some of the metadata (tar comes to mind)...

The tool (metastore) is available from: git://git.hardeman.nu/metastore.git

Not completely cleaned up yet (it lacks a real README and some Makefile targets) but I hope it might be useful to others (it sure is to me).

Please CC me on any replies.

Signature · 1 line
David Härdeman
#1 lore raw
Quoted 13 lines · David Härdeman

I recently had the idea to store and track /etc using git. When googling the topic I came across the "Using git to store /etc" thread from the end of last year which provided some interesting details on what would be necessary.

It seems the file metadata (owner, group, mode, xattrs, etc) was the big stumbling point, so I wrote up a tool over the last few days which allows the metadata to be stored in a separate file which can be stored along with the rest of the data in the repo (or separately).

This is also useful for tripwire type checks and for other types of storage which drops some of the metadata (tar comes to mind)...

Quoted 7 lines

The tool (metastore) is available from:
git://git.hardeman.nu/metastore.git

Not completely cleaned up yet (it lacks a real README and some Makefile targets) but I hope it might be useful to others (it sure is to me).

Please CC me on any replies.

Have you looked at IsiSetup (http://www.isisetup.ch/, linked from
http://git.or.cz/gitwiki/InterfacesFrontendsAndTools) yet? It's a front-end to git specifically targeted for versioning configuration. From a quick glance at it's web I don't see whether it already stores the metadata you describe, but in either case it could be interesting for you.

Signature · 1 line
						 Jan 'Bulb' Hudec <[email protected]>
#2 lore raw
Quoted 6 lines · Jan Hudec

Have you looked at IsiSetup (http://www.isisetup.ch/, linked from
http://git.or.cz/gitwiki/InterfacesFrontendsAndTools) yet? It's a front-end
to git specifically targeted for versioning configuration. From a quick glance at it's web I don't see whether it already stores the metadata you describe, but in either case it could be interesting for you.

Yes, it was mentioned in the previous thread about tracking /etc in git. I've looked at it but it seemed to not store any metadata. Also, it seemed like overkill for just keeping a history of my /etc.

It also used a lot of hairy "grep -v" and similar tricks to run git but to mangle the output before it's shown to the user. I'd prefer to interact directly with git which I'm already familiar with...and git already has everything I need for tracking /etc with the exception of the metadata.

Signature · 1 line
David Härdeman
#3 lore raw
In reply to David Härdeman
Quoted 11 lines · David Härdeman

I recently had the idea to store and track /etc using git. When googling the topic I came across the "Using git to store /etc" thread from the end of last year which provided some interesting details on what would be necessary.

It seems the file metadata (owner, group, mode, xattrs, etc) was the big stumbling point, so I wrote up a tool over the last few days which allows the metadata to be stored in a separate file which can be stored along with the rest of the data in the repo (or separately).

This is also useful for tripwire type checks and for other types of storage which drops some of the metadata (tar comes to mind)...

The tool (metastore) is available from: git://git.hardeman.nu/metastore.git

Not completely cleaned up yet (it lacks a real README and some Makefile targets) but I hope it might be useful to others (it sure is to me).

Please CC me on any replies.

as I understand the issue, the problem isn't creating a tool to store the metadata, but in integrating things with git.

when checking something in a pre-commit hook needs to run the tool to store the data.

git supports this and it's pretty simple to do this.

however when checking things out there are approaches

1. modify git to have a post-checkout hook to set the metadata to match what was stored at checkin and accept the fact that this leaves a window where the file has the wrong metadata on it (between when the file is written and when the hook runs), or use a staging area to have copies of the files during check-in and check-out

2. modify git to know that it needs to check some files out before any others and use an expernal program to write the files to disk. Then this program can use the data stored at checkin to write the files with the appropriate metadata

unforutnantly until one of these is done by someone the utility of programs like your metastore are limited.

David Lang

#4 lore raw
Quoted 8 lines · On Sat, May 19, 2007 at 04:37:54PM -0700, [email protected] wrote:
Quoted 6 lines · David Härdeman

I recently had the idea to store and track /etc using git. When googling the topic I came across the "Using git to store /etc" thread from the end of last year which provided some interesting details on what would be necessary.

It seems the file metadata (owner, group, mode, xattrs, etc) was the big stumbling point, so I wrote up a tool over the last few days which allows the metadata to be stored in a separate file which can be stored along with the rest of the data in the repo (or separately).

This is also useful for tripwire type checks and for other types of storage which drops some of the metadata (tar comes to mind)...

Quoted 7 lines
Quoted 5 lines

The tool (metastore) is available from: git://git.hardeman.nu/metastore.git

Not completely cleaned up yet (it lacks a real README and some Makefile targets) but I hope it might be useful to others (it sure is to me).

Please CC me on any replies.

as I understand the issue, the problem isn't creating a tool to store the metadata, but in integrating things with git.

That is also important of course, the problem is that there are many different scenarios for how people might want to work with the metadata (e.g. whether changed metadata should be stored automatically or only with user interaction, etc).

For the "store /etc in git" solution which is what got me into this, it might be enough to have a pre-commit hook if all changes are made in /etc and committed to /etc/.git periodically (meaning there are no real checkouts to speak of).

Quoted 3 lines

when checking something in a pre-commit hook needs to run the tool to store the data.

git supports this and it's pretty simple to do this.

Yes, I already have hook scripts in my local setup which does this, it was not clear whether pre-commit hooks could change the commit by adding more files to be committed but it seems to work, essentially the pre-commit hook is just:

metastore -s
git-add .metadata

Quoted 7 lines

however when checking things out there are approaches

1. modify git to have a post-checkout hook to set the metadata to match what was stored at checkin and accept the fact that this leaves a window where the file has the wrong metadata on it (between when the file is written and when the hook runs), or use a staging area to have copies of the files during check-in and check-out

Right, I use a non-hook script for this right now which changes umask to 0077, pulls the changes, shows the difference in metadata and asks for confirmation and then applies the metadata (which undoes the effects of the umask setting).

I think I'll add both scripts to my git repo as examples soon.

Signature · 1 line
David Härdeman