Commit ID in exported Tar Ball

Hello,
Michae, a friend of mine, is in the phase of migrating from git to cvs. He releases tar balls of his software project using gitweb. He would love to have a way to have the commit-id of HEAD of the export contained in one of the files he exported that way. Is there infrastructure in git that makes that already possible or does he need to some kind of gerneration tool by himself? Maybe it would be helpful if the git-tar-tree would generate a file .commitid or something like that in the generated tar tree.

Thomas

                                                         ^^^^^^^^^^^^^^^

Man... You are _definitely_ on the wrong list. ;)

Hmm, doesn't seem like git-tar-tree (or git-archive for that matter) supports this out of the box. Maybe it's possible to achieve in combination with the $Id$ construct?

I guess it depends on whether git-tar-tree/git-archive actually does a checkout from the repo from which the archive is made. If so, it should be possible to store "$Id$" in .commitid, and check it in, and it should automagically appear with the correct commit-id in your archive.

Of course, it all depends on whether the $Id$ conversion is triggered by git-archive...

Have fun!

...Johan

Johan Herland, <[email protected]>
www.herland.net

Another possibility might be to add a commandline switch to git-archive so you can decide whether the commit id should be added as a header to the tar file (which it already supports) or as a ordinary file (which should be reasonable trivial to implement). The question if whether it would be worth to add that feature. Don't know if there are many other users out there that need it.

Gruesse,

Frank Lichtenheld <[email protected]>
www: http://www.djpig.de/

Hello,

that would be very good aproach I guess. At least from my point of view. Because it doesn't break diffs, it concentrates on what matters and if you don't like it you don't use it.

Thomas

Although this efficiently solves Michael's problem, I still think the ideal solution would be for git-archive to do the same conversions/filters as a regular checkout would. Otherwise, we'll easily get into situations where a git-archive tree is different enough from a "regular" working tree to cause annoying differences in behaviour.

Have fun!

...Johan

Johan Herland, <[email protected]>
www.herland.net

As noted by Johan Herland, git-archive is a kind of checkout and needs to apply any checkout filters that might be configured.

This patch adds the convenience function convert_sha1_file which returns a buffer containing the object's contents, after converting, if necessary (i.e. it's a combination of read_sha1_file and convert_to_working_tree). Direct calls to read_sha1_file in git-archive are then replaced by calls to convert_sha1_file.

Since convert_sha1_file expects its path argument to be NUL-terminated -- a convention it inherits from convert_to_working_tree -- the patch also changes the path handling in archive-tar.c to always NUL-terminate the string. It used to solely rely on the len field of struct strbuf before.

archive-zip.c already NUL-terminates the path and thus needs no such change.

---
 archive-tar.c |   12 +++++++-----
 archive-zip.c |    2 +-
 cache.h       |    1 +
 convert.c     |   15 +++++++++++++++
 4 files changed, 24 insertions(+), 6 deletions(-)

index 56ff356..eb0abc7 100644
@@ -83,11 +83,12 @@ static void strbuf_append_string(struct strbuf *sb, const char *s)
 	int slen = strlen(s);
 	int total = sb->len + slen;
 	if (total > sb->alloc) {
-		sb->buf = xrealloc(sb->buf, total);
-		sb->alloc = total;
+		sb->buf = xrealloc(sb->buf, total + 1);
+		sb->alloc = total + 1;
 	}
 	memcpy(sb->buf + sb->len, s, slen);
 	sb->len = total;
+	sb->buf[total] = '\0';
 }
 
 /*

@@ -272,18 +273,19 @@ static int write_tar_entry(const unsigned char *sha1,
 	}
 	if (path.alloc < baselen + filenamelen) {
 		free(path.buf);
-		path.buf = xmalloc(baselen + filenamelen);
-		path.alloc = baselen + filenamelen;
+		path.buf = xmalloc(baselen + filenamelen + 1);
+		path.alloc = baselen + filenamelen + 1;
 	}
 	memcpy(path.buf, base, baselen);
 	memcpy(path.buf + baselen, filename, filenamelen);
 	path.len = baselen + filenamelen;
+	path.buf[path.len] = '\0';
 	if (S_ISDIR(mode) || S_ISDIRLNK(mode)) {
 		strbuf_append_string(&path, "/");
 		buffer = NULL;
 		size = 0;
 	} else {
-		buffer = read_sha1_file(sha1, &type, &size);
+		buffer = convert_sha1_file(path.buf, sha1, mode, &type, &size);
 		if (!buffer)
 			die("cannot read %s", sha1_to_hex(sha1));
 	}

index 1eaf262..3cbf6bb 100644
@@ -195,7 +195,7 @@ static int write_zip_entry(const unsigned char *sha1,
 		if (S_ISREG(mode) && zlib_compression_level != 0)
 			method = 8;
 		result = 0;
-		buffer = read_sha1_file(sha1, &type, &size);
+		buffer = convert_sha1_file(path, sha1, mode, &type, &size);
 		if (!buffer)
 			die("cannot read %s", sha1_to_hex(sha1));
 		crc = crc32(crc, buffer, size);

index aaeb04a..4204bc1 100644
@@ -548,6 +548,7 @@ extern void trace_argv_printf(const char **argv, int count, const char *format,
 /* convert.c */
 extern char *convert_to_git(const char *path, const char *src, unsigned long *sizep);
 extern char *convert_to_working_tree(const char *path, const char *src, unsigned long *sizep);
+extern void *convert_sha1_file(const char *path, const unsigned char *sha1, unsigned int mode, enum object_type *type, unsigned long *size);
 
 /* match-trees.c */
 void shift_tree(const unsigned char *, const unsigned char *, unsigned char *, int);

index 12abdaf..c64880b 100644
@@ -652,3 +652,18 @@ char *convert_to_working_tree(const char *path, const char *src, unsigned long *
 
 	return buf;
 }
+
+void *convert_sha1_file(const char *path, const unsigned char *sha1,
+                        unsigned int mode, enum object_type *type,
+                        unsigned long *size)
+{
+	void *buffer = read_sha1_file(sha1, type, size);
+	if (S_ISREG(mode) && buffer) {
+		void *converted = convert_to_working_tree(path, buffer, size);
+		if (converted) {
+			free(buffer);
+			buffer = converted;
+		}
+	}
+	return buffer;
+}

[...]

Thank you for spotting this. Fix-up patch below.

---
Embarrassing. I'm off to go to sleep now.

 archive-tar.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

index eb0abc7..33e7657 100644
@@ -82,7 +82,7 @@ static void strbuf_append_string(struct strbuf *sb, const char *s)
 {
 	int slen = strlen(s);
 	int total = sb->len + slen;
-	if (total > sb->alloc) {
+	if (total + 1 > sb->alloc) {
 		sb->buf = xrealloc(sb->buf, total + 1);
 		sb->alloc = total + 1;
 	}

@@ -271,7 +271,7 @@ static int write_tar_entry(const unsigned char *sha1,
 		path.alloc = PATH_MAX;
 		path.len = path.eof = 0;
 	}
-	if (path.alloc < baselen + filenamelen) {
+	if (path.alloc < baselen + filenamelen + 1) {
 		free(path.buf);
 		path.buf = xmalloc(baselen + filenamelen + 1);
 		path.alloc = baselen + filenamelen + 1;

Something like the following patch? Since we're already embedding the commit ID in a comment, we might as well offer creating a synthetic file for it, too, if that solves a user's problem that might be difficult to work around otherwise.

René

 Documentation/git-archive.txt |    4 ++++
 archive-tar.c                 |    7 +++++++
 archive-zip.c                 |    7 +++++++
 archive.h                     |    1 +
 builtin-archive.c             |   11 +++++++++++
 5 files changed, 30 insertions(+), 0 deletions(-)

index 721e035..ac31aac 100644
@@ -43,6 +43,10 @@ OPTIONS
 --prefix=<prefix>/::
 	Prepend <prefix>/ to each filename in the archive.
 
+--commit-id-file=<filename>::
+	Adds a file to the archive containing the commit ID.  This option
+	is ignored if <tree-ish> references a tree instead of a commit.
+
 <extra>::
 	This can be any options that the archiver backend understand.
 	See next section.

index 33e7657..555850a 100644
@@ -319,6 +319,13 @@ int write_tar_archive(struct archiver_args *args)
 	}
 	read_tree_recursive(args->tree, args->base, plen, 0,
 			    args->pathspec, write_tar_entry);
+	if (args->commit_sha1 && args->commit_sha1_file) {
+		unsigned char fake_sha1[20];
+		pretend_sha1_file(sha1_to_hex(args->commit_sha1), 40,
+		                  OBJ_BLOB, fake_sha1);
+		write_tar_entry(fake_sha1, args->base, plen,
+		                args->commit_sha1_file, 0100666, 0);
+	}
 	write_trailer();
 
 	return 0;

index 3cbf6bb..88c5dfa 100644
@@ -328,6 +328,13 @@ int write_zip_archive(struct archiver_args *args)
 	}
 	read_tree_recursive(args->tree, args->base, plen, 0,
 			    args->pathspec, write_zip_entry);
+	if (args->commit_sha1 && args->commit_sha1_file) {
+		unsigned char fake_sha1[20];
+		pretend_sha1_file(sha1_to_hex(args->commit_sha1), 40,
+		                  OBJ_BLOB, fake_sha1);
+		write_zip_entry(fake_sha1, args->base, plen,
+		                args->commit_sha1_file, 0100666, 0);
+	}
 	write_zip_trailer(args->commit_sha1);
 
 	free(zip_dir);

index 6838dc7..020f82f 100644
@@ -8,6 +8,7 @@ struct archiver_args {
 	const char *base;
 	struct tree *tree;
 	const unsigned char *commit_sha1;
+	const char *commit_sha1_file;
 	time_t time;
 	const char **pathspec;
 	unsigned int verbose : 1;

index 7f4e409..e58ea16 100644
@@ -151,6 +151,7 @@ int parse_archive_args(int argc, const char **argv, struct archiver *ar)
 	int extra_argc = 0;
 	const char *format = "tar";
 	const char *base = "";
+	const char *commit_sha1_file = NULL;
 	int verbose = 0;
 	int i;
 

@@ -174,6 +175,10 @@ int parse_archive_args(int argc, const char **argv, struct archiver *ar)
 			base = arg + 9;
 			continue;
 		}
+		if (!prefixcmp(arg, "--commit-id-file=")) {
+			commit_sha1_file = arg + 17;
+			continue;
+		}
 		if (!strcmp(arg, "--")) {
 			i++;
 			break;

@@ -192,6 +197,11 @@ int parse_archive_args(int argc, const char **argv, struct archiver *ar)
 		usage(archive_usage);
 	if (init_archiver(format, ar) < 0)
 		die("Unknown archive format '%s'", format);
+	if (commit_sha1_file) {
+		size_t namelen = strlen(commit_sha1_file);
+		if (namelen == 0 || commit_sha1_file[namelen - 1] == '/')
+			die("Invalid commit ID file name: %s", commit_sha1_file);
+	}
 
 	if (extra_argc) {
 		if (!ar->parse_extra)

@@ -201,6 +211,7 @@ int parse_archive_args(int argc, const char **argv, struct archiver *ar)
 	}
 	ar->args.verbose = verbose;
 	ar->args.base = base;
+	ar->args.commit_sha1_file = commit_sha1_file;
 
 	return i;
 }

Are you detecting and erroring out if the named file already exists in the tree being archived?

Should we? Maybe we should just replace with warning?

Also should we silently ignore the request if the tree-ish is not a commit-ish, or error out?

What about being able to get the output of git-describe embedded into an archive file? Doesn't git.git do that in its Makefile? ;-)

git-describe is more human-friendly than a SHA-1...

Shawn.

Hello,

For me git-describe does the following:

        (thinkpad) [~/work/slides] git-describe --all HEAD
        heads/master

I don't see how Michael is able to tell from this output what version his users is using. Do I miss something? I really do like the patch Rene has submitted. And it is exactly what is missing. A unique identifier which tells the exact version of the tree the user is using.

Thomas

For version information it is far more useful to use --tags or no options (annotated tags only) instead of --all.

# On git.git's master this morning:

$ git describe HEAD
v1.5.2
$ git describe HEAD^^
v1.5.2-rc3-97-g03f6db0

~~ Brian

Hello,

so this output is useless if you don't have tagged the commit which isn't the case. But thanks for the awareness.

Thomas

No. Just _ONE_ tagged commit should be enough. In a project of mine I tagged the root commit and as I am the only committer and this project is so simple that I don't have to do a lot of branching/merging, I get a single line of history

	t - o1 - o2 - o3 - o4 <- master
	^
	|- tagged root commit

This gives me very nicely enumerated commits like tag-1-g<sha1-abbrev> for the commit o1 and tag-2-g<sha1-abbrev> for o2 ... you get the idea.

-Peter

Hello,

I get the idea but I don't like it very much. From my point of view the commit id makes much more sense. The output is cut&pasted anyway.

Thomas

Thanks for not quoting Brian's reply. Because I had to go and quote it manually, so I can say its *NOT* useless...

Here whatever HEAD's commit is is exactly the commit that the tag v1.5.2 points at. This commit is definately v1.5.2.

Here whatever commit is 2 commits earlier than HEAD is 97 commits *after* v1.5.2-rc3 was tagged. That's a good deal of information right there. I know its v1.5.2-rc3 plus a bunch of additional commits (97 to be exact). Add another commit and that 97 will go to 98. Wow, look, an automatic version counter! No user intervention required!

Sometimes I don't even bother tagging git-gui fixes, for exactly that reason. The output of git-describe is giving me a count along my maint branch, or my master branch.

Now that g03f6db0 suffix is also very useful, it means its the commit whose SHA- starts with 03f6db0. That abbreviated SHA-1 is unique at the time that git-describe ran. At 8 hex digits it will probably also stay unique for quite some time, even in large projects like the kernel.

And even if that isn't unique later on, I doubt there will be another commit with the same leading hex digits that is also 97 commits after v1.5.2, as counted by git-rev-list v1.5.2..$it | wc -l. So even in the case of a later duplicate, we can get back a full SHA-1.

And did you know that Git knows how to parse those, and can checkout that commit?

  $ git checkout v1.5.2-rc3-97-g03f6db0
  Note: moving to "v1.5.2-rc3-97-g03f6db0" which isn't a local branch
  If you want to create a new branch from this checkout, you may do so
  (now or later) by using -b with the checkout command again. Example:
    git checkout -b <new_branch_name>
  HEAD is now at 03f6db0... Merge branch 'maint' to synchronize with 1.5.1.6

Wow. Magic! Not useless!

Shawn.

Hello,
so maybe it isn't that useless as I thought it initial is. But my point still stands. I want the commit id of the HEAD in a _file within_ the tarball and I definitively don't want to tag my project before I get a unique identifier.

Thomas

So what, a magic flag to git-describe like:

	git-describe --untagged HEAD
	1a8213...

Where --untagged just means "give me back the raw commit SHA-1 if the input ref(s) aren't tagged"?

Shawn.

Hello,

that is something I would love to live with. But it should also get the primary objective done which is: Get the commit-id in the tar archive.

Thomas

If you haven't tagged anything, then git can't give you anything better than the SHA-1. Giving a reference relative to a branch isn't useful, as the branch can change. If the branch doesn't change, it's really a tag and you should mark it as such.

Hopefully the patch for the "in a file" flag for git-archive will go in soon (or something like it). I don't think we should make it any more complex than just the hash, myself. If you want something more complex a Makefile rule like

dist:
git archive HEAD > dist.tar
git describe HEAD > version-file
tar rf dist.tar version-file
gzip dist.tar

should do the trick. (And you can replace "git describe" with "git parse-rev" for a bare hash if you'd like.)

~~ Brian

Hello Brian,

to be precise that was my first idea when I started the whole discussion. But I like the patch from Rene much more than that. Because I want to use it with gitweb and it is definitevly easier to use realize that with Renes patch.

Thomas

Yes, and the Makefile does even more than that: it adds a version file, a spec file and another version file for git-gui.

The first two are probably useful for most projects that actually do versioned releases. We could have a simple parser that reads a template, replaces @@VERSION@@ with a git-describe output string and adds the result as a synthetic file to the archive. It's not exactly trivial -- e.g., how to specify git-describe options, template file and synthetic name, all in one command line parameter? -- but it's doable.

I'm not sure how the git-gui version file fits in. I guess it's just a special case and doesn't need git-archive support?

René

Maybe something just as simple as allowing the user to specify a shell script in-tree that we unpack and run for them? That script prints to stdout the content of the file to include.

Well, if you look at git-gui's own version script it really just wants to do git-describe like git.git's script, but it cannot as when its hosted in git.git git-describe gives us back Git's version number, not git-gui's version number. So we get cute and look for the merge commit, and take the second parent, and describe that. That's (by convention of how Junio works) always a true git-gui commit.

In other words, git-gui.git gets a little whacky when Junio distributes it in git.git. git-gui really needs to become a subproject. When that happens its git-describe will become much easier.

So now we're also really talking about, what should git-archive do for a subproject? Sometimes you really do want to repackage and redistribute the subproject as part of the superproject's tarball. Sometimes you don't. I think in the case of git.git and git-gui.git we want to include the subproject. ;-)

Shawn.

Specify how? At the point of git-archive execution? At that point you usually can append the file to the archive as well.

And if you make it somehow a "project default", that becomes a huge security risk, since anyone who clones the project and runs git-archive will execute aribtrary code on his account.

Besides, the original motivation for this were snapshots from gitweb. Gitweb frequently does not run with the uid of the project owner, so this becomes a security problem as well.

Maybe some kind of format-string in .git/config...

				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Ever try. Ever fail. No matter. // Try again. Fail again. Fail better.
		-- Samuel Beckett

I doubt executing a shell script is simple. :-D You'd possibly get different results on different platforms (dare I mention Windows?).

The template system I mentioned would be a kind of scripting language itself, but in this case we define its syntax and can guarantee consistency everywhere git runs. And since it would only have four types of tokens (@@VERSION@@, @@COMMITID@@, @@@@ and string literals) it could be fast and simple.

We could implement it as a checkout converter, preferably one that is only applied by git-archive. Then we'd rename git.spec.in to git.spec, assign the "specfile" attribute to it and let git-archive replace the string @@VERSION@@ with git-describe's output. git-checkout would not expand the special strings, so you can simply edit and version the file as you can do with git.spec.in now. Michael would have a file containing only @@COMMITID@@ to solve his original problem. Make sense?

Oh, yes, subprojects. git-archive currently exports them as empty directories. Using tar's append command you could simply build the project+subproject archive in the Makefile. That wouldn't work well with gitweb, though. Perhaps a --include-subproject=<path> option is needed?

René

[I'm quoting myself in full because I somehow sent my reply to everyone but Shawn. A patch can be found at the end.]

OK, so here's a first shot at the mentioned parser. It only understands @@COMMITID@@ and @@@@, but it's easily extendible. The internals of git-describe would need to be converted to library functions, preferably offering every piece of version info separately (see thread "[PATCH] Make sure an autogenerated version has at least four parts" for why).

Before doing that, we should determine if this is the way to, though.

René

 Documentation/gitattributes.txt |   19 ++++++-
 archive-tar.c                   |    5 ++-
 archive-zip.c                   |    5 ++-
 cache.h                         |    1 +
 convert.c                       |  112 +++++++++++++++++++++++++++++++++++++++
 5 files changed, 139 insertions(+), 3 deletions(-)

index d3ac9c7..84c414c 100644
@@ -72,7 +72,7 @@ EFFECTS
 -------
 
 Certain operations by git can be influenced by assigning
-particular attributes to a path.  Currently, three operations
+particular attributes to a path.  Currently, four operations
 are attributes-aware.
 
 Checking-out and checking-in

@@ -374,6 +374,23 @@ frotz	unspecified
 ----------------------------------------------------------------
 
 
+Creating an archive
+~~~~~~~~~~~~~~~~~~~
+
+
+`specfile`
+^^^^^^^^^^
+
+If the attribute `specfile` is set for a file then git will expand
+several placeholders when adding this file to an archive.  The
+expansion depends on the availability of a commit ID, i.e. if
+`git-archive` has been given a tree instead of a commit or a tag
+then no replacement will be done.
+
+`@@COMMITID@@`:: is replaced by the commit hash.
+`@@@@`:: is replaced by `@@`.
+
+
 GIT
 ---
 Part of the gitlink:git[7] suite

index 66fe3e3..eba24cb 100644
@@ -17,6 +17,7 @@ static unsigned long offset;
 static time_t archive_time;
 static int tar_umask = 002;
 static int verbose;
+static const unsigned char *commit_sha1;
 
 /* writes out the whole block, but only if it is full */
 static void write_if_needed(void)

@@ -285,7 +286,8 @@ static int write_tar_entry(const unsigned char *sha1,
 		buffer = NULL;
 		size = 0;
 	} else {
-		buffer = convert_sha1_file(path.buf, sha1, mode, &type, &size);
+		buffer = sha1_file_to_archive(path.buf, sha1, mode, &type,
+		                              &size, commit_sha1);
 		if (!buffer)
 			die("cannot read %s", sha1_to_hex(sha1));
 	}

@@ -304,6 +306,7 @@ int write_tar_archive(struct archiver_args *args)
 
 	archive_time = args->time;
 	verbose = args->verbose;
+	commit_sha1 = args->commit_sha1;
 
 	if (args->commit_sha1)
 		write_global_extended_header(args->commit_sha1);

index 444e162..93a5ab3 100644
@@ -12,6 +12,7 @@
 static int verbose;
 static int zip_date;
 static int zip_time;
+static const unsigned char *commit_sha1;
 
 static unsigned char *zip_dir;
 static unsigned int zip_dir_size;

@@ -195,7 +196,8 @@ static int write_zip_entry(const unsigned char *sha1,
 		if (S_ISREG(mode) && zlib_compression_level != 0)
 			method = 8;
 		result = 0;
-		buffer = convert_sha1_file(path, sha1, mode, &type, &size);
+		buffer = sha1_file_to_archive(path, sha1, mode, &type, &size,
+		                              commit_sha1);
 		if (!buffer)
 			die("cannot read %s", sha1_to_hex(sha1));
 		crc = crc32(crc, buffer, size);

@@ -316,6 +318,7 @@ int write_zip_archive(struct archiver_args *args)
 	zip_dir = xmalloc(ZIP_DIRECTORY_MIN_SIZE);
 	zip_dir_size = ZIP_DIRECTORY_MIN_SIZE;
 	verbose = args->verbose;
+	commit_sha1 = args->commit_sha1;
 
 	if (args->base && plen > 0 && args->base[plen - 1] == '/') {
 		char *base = xstrdup(args->base);

index cd875bc..0484904 100644
@@ -550,6 +550,7 @@ extern void trace_argv_printf(const char **argv, int count, const char *format,
 extern char *convert_to_git(const char *path, const char *src, unsigned long *sizep);
 extern char *convert_to_working_tree(const char *path, const char *src, unsigned long *sizep);
 extern void *convert_sha1_file(const char *path, const unsigned char *sha1, unsigned int mode, enum object_type *type, unsigned long *size);
+extern void *sha1_file_to_archive(const char *path, const unsigned char *sha1, unsigned int mode, enum object_type *type, unsigned long *size, const unsigned char *commit_sha1);
 
 /* match-trees.c */
 void shift_tree(const unsigned char *, const unsigned char *, unsigned char *, int);

Hmmm. I am torn.

It almost feels as if we'd better bite the bullet and do more insane things in ident substitution, instead of introducing this apparent syntax inconsistency between "$id$" and "@@COMMITID@@".

That is, we could (I am not seriously proposing to do this, as I expect this will lead to a lot of insanity at the end):

(1) introduce "const unsigned char commit_in_focus[20]", globally available to git suite, and clear it at the beginning of main();

(2) teach ident substitution to expand "$commit$" to

     sprintf("$commit: %40s $", sha1_to_hex(commit_in_focus[])),
     and unexpand "$commit: .* $".

(3) have git-archive set commit_in_focus[] before letting the convert_to_working_tree do its work.

(4) later, we _might_ teach a single tree read-tree to also set up commit_in_focus[], so that:

$ rm -f .git/index
$ git checkout -f HEAD

would expand "$commit$" in blobs.

This obviously have a lot of problems once we start adding the commit_in_focus[] to more random programs. Even two-tree read-tree case would behave in an unexpected way for an uninitiated person, if you do something like:

	$ git checkout master
        $ git checkout next

I am CC'ing Linus because he would literally hate me suggesting the above.

$Id$ (and $commit$) is reversible, @@COMMITID@@ is not. That means you can create a synthetic file byte for byte with @@COMMITID@@ (and its not yet implemented brethren), but you can't do that with $Id$ -- it's impossible to get rid of the dollar signs.

I'm not attached to any particular syntax. It all started with @@VERSION@@ from git.spec.in, which should not be implemented 1:1 anyway (we'd need to be able to use arbitrary separators between version parts to support different ways of ordering version numbers).

We could use $ to indicate reversible substitutions as before and @ (instead of @@) for one-way substitutions. I can't think of any other use than in archives, though. It sure would be very confusing to have such a conversion happen on checkout -- you'd need to use git-cat-file to see the real file contents.

Ugh. Requiring another global variable doesn't smell like good design.

By the way, we already have a similar, but very different syntax: the one format_commit_message in commit.c. It's a one-way conversion, too.
Maybe we should copy the relevant pieces like %H and %h from there..

Do we want git-archive specific one-way conversions that are capable of creating files like git.spec? Or is this just a shiny toy hypnotizing me? 8-)

René

Yes, and that's one of the big problems with the $Id$ syntax so commonly used by versioning systems. Most files you want to insert that automatic id into want a clean id string, not something that starts with $Id: and ends with $...

Since we are apparently supporting $Foo: ...$ to collapse back to $Foo$ reusing that syntax for git-archive is actually probably a bad idea. We should support the checkout filters in git-archive (as much as possible anyway) but what this thread has been going on is something quite different... so we probably want a different syntax. Which is why I'm also in favor of the @@COMMITID@@ syntax...

I agree. We already have a lot of globals. We need another one like we need a hole in the head. Especially a global like this one... ;-)

But aren't shiny toys fun? ;-)

Shawn.

Yes, and the Makefile does even more than that: it adds a version file, a spec file and another version file for git-gui.

The first two are probably useful for most projects that actually do versioned releases. We could have a simple parser that reads a template, replaces @@VERSION@@ with a git-describe output string and adds the result as a synthetic file to the archive. It's not exactly trivial -- e.g., how to specify git-describe options, template file and synthetic name, all in one command line parameter? -- but it's doable.

I'm not sure how the git-gui version file fits in. I guess it's just a special case and doesn't need git-archive support?

René

Silently ignoring conflicting command arguments is (almost always) a bug; either in the implementation or the design.

Currently the commit ID file is appended to the archive, so if there is a name conflict, we keep both files. unzip offers a choice between renaming, overwriting and keeping the first extracted version when the second one is about to be extracted. tar has a -k option: with -k you get the first version, without -k you get the second one.

To avoid confusion, we should disallow a name that already comes from the tree. Only I can't see an easy way to implement this. Perhaps a check using get_tree_entry is enough -- it doesn't take pathspecs into account, though. That means we would disallow all names in the tree, even if a pathspec excludes the chosen commit ID file name.

Before I roll my own path existence checker with pathspec support, is there something like that already implemented? I suspect it's hiding in the diff code, but I don't dare go near it. ;-)

As Angry said, erroring out is better. It's also easy to do.

 Documentation/git-archive.txt |    4 ++++
 archive-tar.c                 |    7 +++++++
 archive-zip.c                 |    7 +++++++
 archive.h                     |    1 +
 builtin-archive.c             |   32 ++++++++++++++++++++++++++++++++
 5 files changed, 51 insertions(+), 0 deletions(-)

index 721e035..7016d1e 100644
@@ -43,6 +43,10 @@ OPTIONS
 --prefix=<prefix>/::
 	Prepend <prefix>/ to each filename in the archive.
 
+--commit-id-file=<filename>::
+	Adds a file to the archive containing the commit ID.  This option
+	is can only be used if <tree-ish> references a commit or tag.
+
 <extra>::
 	This can be any options that the archiver backend understand.
 	See next section.

index 33e7657..555850a 100644
@@ -319,6 +319,13 @@ int write_tar_archive(struct archiver_args *args)
 	}
 	read_tree_recursive(args->tree, args->base, plen, 0,
 			    args->pathspec, write_tar_entry);
+	if (args->commit_sha1 && args->commit_sha1_file) {
+		unsigned char fake_sha1[20];
+		pretend_sha1_file(sha1_to_hex(args->commit_sha1), 40,
+		                  OBJ_BLOB, fake_sha1);
+		write_tar_entry(fake_sha1, args->base, plen,
+		                args->commit_sha1_file, 0100666, 0);
+	}
 	write_trailer();
 
 	return 0;

index 3cbf6bb..88c5dfa 100644
@@ -328,6 +328,13 @@ int write_zip_archive(struct archiver_args *args)
 	}
 	read_tree_recursive(args->tree, args->base, plen, 0,
 			    args->pathspec, write_zip_entry);
+	if (args->commit_sha1 && args->commit_sha1_file) {
+		unsigned char fake_sha1[20];
+		pretend_sha1_file(sha1_to_hex(args->commit_sha1), 40,
+		                  OBJ_BLOB, fake_sha1);
+		write_zip_entry(fake_sha1, args->base, plen,
+		                args->commit_sha1_file, 0100666, 0);
+	}
 	write_zip_trailer(args->commit_sha1);
 
 	free(zip_dir);

index 6838dc7..020f82f 100644
@@ -8,6 +8,7 @@ struct archiver_args {
 	const char *base;
 	struct tree *tree;
 	const unsigned char *commit_sha1;
+	const char *commit_sha1_file;
 	time_t time;
 	const char **pathspec;
 	unsigned int verbose : 1;

index 7f4e409..6bb0781 100644
@@ -151,6 +151,7 @@ int parse_archive_args(int argc, const char **argv, struct archiver *ar)
 	int extra_argc = 0;
 	const char *format = "tar";
 	const char *base = "";
+	const char *commit_sha1_file = NULL;
 	int verbose = 0;
 	int i;
 

@@ -174,6 +175,10 @@ int parse_archive_args(int argc, const char **argv, struct archiver *ar)
 			base = arg + 9;
 			continue;
 		}
+		if (!prefixcmp(arg, "--commit-id-file=")) {
+			commit_sha1_file = arg + 17;
+			continue;
+		}
 		if (!strcmp(arg, "--")) {
 			i++;
 			break;

@@ -192,6 +197,11 @@ int parse_archive_args(int argc, const char **argv, struct archiver *ar)
 		usage(archive_usage);
 	if (init_archiver(format, ar) < 0)
 		die("Unknown archive format '%s'", format);
+	if (commit_sha1_file) {
+		size_t namelen = strlen(commit_sha1_file);
+		if (namelen == 0 || commit_sha1_file[namelen - 1] == '/')
+			die("Invalid commit ID file name: %s", commit_sha1_file);
+	}
 
 	if (extra_argc) {
 		if (!ar->parse_extra)

@@ -201,6 +211,7 @@ int parse_archive_args(int argc, const char **argv, struct archiver *ar)
 	}
 	ar->args.verbose = verbose;
 	ar->args.base = base;
+	ar->args.commit_sha1_file = commit_sha1_file;
 
 	return i;
 }

@@ -236,6 +247,20 @@ static const char *extract_remote_arg(int *ac, const char **av)
 	return remote;
 }
 
+static int is_path_in_spec(struct tree *tree, const char **pathspec,
+                           const char *path)
+{
+	unsigned char sha1[20];
+	unsigned int mode;
+
+	if (get_tree_entry(tree->object.sha1, path, sha1, &mode))
+		return 0;
+	if (!pathspec)
+		return 1;
+	/* TODO: the actual pathspec check */
+	return 1;
+}
+
 int cmd_archive(int argc, const char **argv, const char *prefix)
 {
 	struct archiver ar;

@@ -257,5 +282,12 @@ int cmd_archive(int argc, const char **argv, const char *prefix)
 	parse_treeish_arg(argv, &ar.args, prefix);
 	parse_pathspec_arg(argv + 1, &ar.args);
 
+	if (ar.args.commit_sha1_file) {
+		if (is_path_in_spec(ar.args.tree, ar.args.pathspec, ar.args.commit_sha1_file))
+			die("Commit ID file name already exists in archive.");
+		if (!ar.args.commit_sha1)
+			die("Need a commit to use --commit-id-file, and not a tree.");
+	}
+
 	return ar.write_archive(&ar.args);
 }

Turns out a matcher for the kind of pathspecs used in git-archive is much easier to write than I thought. :-)

This is just a progress note and not for inclusion, yet -- Shawn has a good point suggesting git-describe output to be used instead of bare commit IDs.

 Documentation/git-archive.txt |    4 +++
 archive-tar.c                 |    7 ++++++
 archive-zip.c                 |    7 ++++++
 archive.h                     |    1 +
 builtin-archive.c             |   44 +++++++++++++++++++++++++++++++++++++++++
 5 files changed, 63 insertions(+), 0 deletions(-)

index 721e035..7016d1e 100644
@@ -43,6 +43,10 @@ OPTIONS
 --prefix=<prefix>/::
 	Prepend <prefix>/ to each filename in the archive.
 
+--commit-id-file=<filename>::
+	Adds a file to the archive containing the commit ID.  This option
+	is can only be used if <tree-ish> references a commit or tag.
+
 <extra>::
 	This can be any options that the archiver backend understand.
 	See next section.

index 33e7657..555850a 100644
@@ -319,6 +319,13 @@ int write_tar_archive(struct archiver_args *args)
 	}
 	read_tree_recursive(args->tree, args->base, plen, 0,
 			    args->pathspec, write_tar_entry);
+	if (args->commit_sha1 && args->commit_sha1_file) {
+		unsigned char fake_sha1[20];
+		pretend_sha1_file(sha1_to_hex(args->commit_sha1), 40,
+		                  OBJ_BLOB, fake_sha1);
+		write_tar_entry(fake_sha1, args->base, plen,
+		                args->commit_sha1_file, 0100666, 0);
+	}
 	write_trailer();
 
 	return 0;

index 3cbf6bb..88c5dfa 100644
@@ -328,6 +328,13 @@ int write_zip_archive(struct archiver_args *args)
 	}
 	read_tree_recursive(args->tree, args->base, plen, 0,
 			    args->pathspec, write_zip_entry);
+	if (args->commit_sha1 && args->commit_sha1_file) {
+		unsigned char fake_sha1[20];
+		pretend_sha1_file(sha1_to_hex(args->commit_sha1), 40,
+		                  OBJ_BLOB, fake_sha1);
+		write_zip_entry(fake_sha1, args->base, plen,
+		                args->commit_sha1_file, 0100666, 0);
+	}
 	write_zip_trailer(args->commit_sha1);
 
 	free(zip_dir);

index 6838dc7..020f82f 100644
@@ -8,6 +8,7 @@ struct archiver_args {
 	const char *base;
 	struct tree *tree;
 	const unsigned char *commit_sha1;
+	const char *commit_sha1_file;
 	time_t time;
 	const char **pathspec;
 	unsigned int verbose : 1;

$Id$ contains the blob id, not the commit id.

Gruesse,

Frank Lichtenheld <[email protected]>
www: http://www.djpig.de/

Oops. My bad. In that case, your solution (to add another commandline switch to git-archive) is definitely what Michael needs.

...Johan

Johan Herland, <[email protected]>
www.herland.net

Use git-get-tar-commit-id:

 $ gzip -cd snapshot.tar.gz | git-get-tar-commit-id

cheers,
Kristian

Hello,

not good enough. What he wants is:

- user loads tar tree down from gitweb via the 'snapshot' button.

- user extracts the tarball and types make

- The output of the produced binary contains a unique identifier.

Thomas