Skip to contentSkip to conversation

RFC: checkout/temporary branch switch restoring modification times

DISCUSSION Justin Frankel 3 participants 4 messages Jun 15, 2016

lore
Share & tools
lore
#0 lore raw

Hello all,

I often find myself wanting to switch temporarily to another branch, and then back to my original branch, but what I find is that my build tools such as MSVC or Xcode) notice the file modification times changed on the changed files, and then want to rebuild, even though the contents of the files are the same.

I've implemented a perl script (tested on OSX and msysGit), which I've named "git-cop" (so I can use via "git cop") to checkout, preserving file modification times.

The use case is:

git cop master
; build
git cop some-branch-that-affects-lots-of-files
; edit some things, commit
git cop master
; build (fast, nothing changed)

If you use "checkout" instead of "cop", the final build command might take longer, since the modification times on files may have changed.

To summarize the implementation of the script:

1) Get a list of what files will change
2) Read timestamps for changed files, updating .git/timestamp-list.txt which is effectively an associative array of timestamps, indexed by "blobhash:filename".
3) Call git checkout
4) Update file modification times with their new timestamps (read from .git/timestamp-list.txt, using newblobhash:filename).

This will get slower as that text file grows, but you can periodically remove .git/timestamp-list.txt (any time you're not in the middle of a temporary branch switch), and it will be regenerated.

As this is a request for comments, I'd be curious if anybody has thoughts on the best way to integrate this type of functionality into git, or indeed if anybody else thinks it is worth including (I do find it saves me a great deal of time, but it is easy enough to add to my git installs, too).

Additionally, is there a better place to store repository-local information, information which isn't terribly important and shouldn't be copied in case of clones, etc? I figured .git/timestamp-list.txt isn't the best place, but where would that be?

Cheers,

Justin

---------------- begin git-cop -----------------
#!/usr/bin/perl

my ($destbranch,$allargs) = ("", "");

foreach (@ARGV) {
	$allargs .= " $_";
	# last non-option parameter would be considered the new branch
	$destbranch = $_ if ($_ !~ /^-/);
}

$destbranch eq "" and die "Usage: git-cop [checkout options] <branch>\n";

my $modf = join('',git diff --name-only HEAD..$destbranch); $modf =~ s/ /\\ /g;
$modf =~ s/\n/ /g;

length($modf)>0 or exit(system("git checkout$allargs"));

chomp(my $pwd = git rev-parse --show-toplevel);
my $idxfile = "${pwd}/.git/timestamp-list.txt";
chdir($pwd);

my %times;

if (open(my $F, "<", $idxfile)) {
	# read cached timestamps
	while (<$F>) {
		chomp;
		my ($val, $nm) = split(/\t/, $_,2);
		$nm eq "" or $val eq "" or $times{$nm} = $val;
	}
	close($F);
}

my $idxfile_updcnt = 0;

foreach (`git ls-tree HEAD $modf`) {
	chomp;
	my ($perm, $type, $hash, $fn) = split(" ",$_,4);
	if ($type eq "blob") {
		# add/update timestamp in cache if needed
		$hash .= ":$fn";
		my $mt = int ((stat("${pwd}/$fn"))[9]);
		if ($mt && $mt != (int $times{$hash})) {
			$times{$hash} = $mt;
			$idxfile_updcnt++;
		}
	}
}
if ($idxfile_updcnt != 0) {
	print "Saved/updated timestamps for $idxfile_updcnt files\n";
	if (open(my $F, ">", $idxfile)) {
		print $F "$ts\t$nm\n" while (($nm,$ts) = each(%times));
		close($F);
	}
}

my $rv = system("git checkout$allargs");

!$rv or exit($rv);

# restore timestamps from cache
my $update_cnt = 0;
foreach (`git ls-tree HEAD $modf`) {
	chomp;
	my ($perm, $type, $hash, $fn) = split(" ",$_,4);
	if ($type eq "blob") {
		$hash .= ":$fn";
		my $tsreq = int $times{$hash};
		if ($tsreq &&
 		    $tsreq != (int ((stat("${pwd}/$fn"))[9]))) {
			utime(undef,$tsreq,"${pwd}/$fn");
			$update_cnt++;
		}
	}
}
$update_cnt < 1 or
	print "Updated file timestamps for $update_cnt files\n";

exit (0);

---------------- end git-cop -----------------

#1 lore raw

Hi,

Quoted 6 lines · Justin Frankel

git cop master
; build
git cop some-branch-that-affects-lots-of-files
; edit some things, commit
git cop master
; build (fast, nothing changed)

Interesting. I guess the intended use is that you only ever build on the master branch?

Have you ever tried the contrib/workdir/git-new-workdir script? I find it fits the use case well for me:

git clone $repo
cd repo
make
# oh, shoot! I need to try something out real quick. cd ..
git new-workdir repo repo2 origin/master
cd repo2
git am patch-to-test
make
# okay, back to what I was doing...
cd ../repo

Maybe it could be helpful for you, too?

Limitations:

- requires a file system with support for symbolic links (I think Pierre Habouzit and Junio discussed changing that);

- workdirs share refs. If you update master in one workdir and another workdir also has master checked out, the new changes will appear as staged changes.

- workdirs do not share HEAD. "git gc" from one workdir can completely trash another if it has a detached HEAD pointing to a commit that is not part of any local or remote branch.

#2 lore raw
Quoted 11 lines · Jonathan Nieder

Hi,

Quoted 6 lines · Justin Frankel

git cop master
; build
git cop some-branch-that-affects-lots-of-files
; edit some things, commit
git cop master
; build (fast, nothing changed)

Quoted 3 lines

Interesting. I guess the intended use is that you only ever build on the master branch?

The idea is that you're often building on a particular branch, but want to switch to another branch temporarily to either do a quick edit or to browse some code.

Quoted 23 lines

Have you ever tried the contrib/workdir/git-new-workdir script? I find it fits the use case well for me:

git clone $repo
cd repo
make
# oh, shoot! I need to try something out real quick. cd ..
git new-workdir repo repo2 origin/master
cd repo2
git am patch-to-test
make
# okay, back to what I was doing...
cd ../repo

Maybe it could be helpful for you, too?

Limitations:

- requires a file system with support for symbolic links (I think Pierre Habouzit and Junio discussed changing that);

Quoted 8 lines

- workdirs share refs. If you update master in one workdir and another workdir also has master checked out, the new changes will appear as staged changes.

- workdirs do not share HEAD. "git gc" from one workdir can completely trash another if it has a detached HEAD pointing to a commit that is not part of any local or remote branch.

Ahh, that would be great. Unfortunately my filesystem often doesn't support symlinks...

-Justin

#3 lore raw
In reply to Justin Frankel
Quoted 9 lines · Justin Frankel

I often find myself wanting to switch temporarily to another branch, and then back to my original branch, but what I find is that my build tools such as MSVC or Xcode) notice the file modification times changed on the changed files, and then want to rebuild, even though the contents of the files are the same.

I've implemented a perl script (tested on OSX and msysGit), which I've named "git-cop" (so I can use via "git cop") to checkout, preserving file modification times.

Maybe something like this is a worthwhile feature, although perhaps it would be better to give each file a timestamp corresponding to when it was last modified in Git? That would break in many cases, but a scheme like this inherently does.

Have you tried to use ccache? It should reduce your build times by a lot, so perhaps if you use it you won't need this anymore.