"git gc" doesn't seem to remove loose objects any more
I couldn't see a test for this, but perhaps I'm just missing it?
brs% git count-objects
161 objects, 1552 kilobytes
brs% git gc
Counting objects: 80621, done.
Compressing objects: 100% (22372/22372), done.
Writing objects: 100% (80621/80621), done.
Total 80621 (delta 57160), reused 80305 (delta 56884)
brs% git count-objects
207 objects, 2048 kilobytesAnd I see lots of directories under .git/objects which confirms things.
I don't think I've changed any relevant configuration.
This is with 8befc50c49e8a271fd3cd7fb34258fe88d1dfcad (also whatever version I used before, erm, probably de0db422782ddaf7754ac5b03fdc6dc5de1a9ae4), and possibly earlier versions---I've just started noticing now that the number of loose objects has started causing git gui to complain.
(Hmm, I note that git gui reports a larger number of loose objects than git count-objects. Ah, OK, it really is just an approximation, so no surprise.)
2008/12/15 Bruce Stephens <[email protected]>:
Quoted 24 lines
I couldn't see a test for this, but perhaps I'm just missing it?
brs% git count-objects 161 objects, 1552 kilobytes brs% git gc Counting objects: 80621, done. Compressing objects: 100% (22372/22372), done. Writing objects: 100% (80621/80621), done. Total 80621 (delta 57160), reused 80305 (delta 56884) brs% git count-objects 207 objects, 2048 kilobytesAnd I see lots of directories under .git/objects which confirms things.
I don't think I've changed any relevant configuration.
This is with
8befc50c49e8a271fd3cd7fb34258fe88d1dfcad(also whatever version I used before, erm, probablyde0db422782ddaf7754ac5b03fdc6dc5de1a9ae4), and possibly earlier versions---I've just started noticing now that the number of loose objects has started causing git gui to complain.
Quoted 7 lines
(Hmm, I note that git gui reports a larger number of loose objects than git count-objects. Ah, OK, it really is just an approximation, so no surprise.)
--
To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
IIRC git gc only removes loose objects older than two weeks, if you really want to remove them now, run git prune. But make sure no other git process can be active when you run it, or it could possibly step on something.
Signature · 1 line
Mikael Magnusson
Quoted 25 lines · Mikael Magnusson
2008/12/15 Bruce Stephens <[email protected]>:
Quoted 24 lines
I couldn't see a test for this, but perhaps I'm just missing it?
brs% git count-objects 161 objects, 1552 kilobytes brs% git gc Counting objects: 80621, done. Compressing objects: 100% (22372/22372), done. Writing objects: 100% (80621/80621), done. Total 80621 (delta 57160), reused 80305 (delta 56884) brs% git count-objects 207 objects, 2048 kilobytesAnd I see lots of directories under .git/objects which confirms things.
I don't think I've changed any relevant configuration.
This is with
8befc50c49e8a271fd3cd7fb34258fe88d1dfcad(also whatever version I used before, erm, probablyde0db422782ddaf7754ac5b03fdc6dc5de1a9ae4), and possibly earlier versions---I've just started noticing now that the number of loose objects has started causing git gui to complain.
Quoted 8 lines
Quoted 3 lines
(Hmm, I note that git gui reports a larger number of loose objects than git count-objects. Ah, OK, it really is just an approximation, so no surprise.)
IIRC git gc only removes loose objects older than two weeks, if you really want to remove them now, run git prune. But make sure no other git process can be active when you run it, or it could possibly step on something.
To clarify that a bit more: git gc keeps unreachable objects unpacked, so that git prune can drop them. And git gc invokes git prune so that only unreachable objects older than 2 weeks are dropped.
Björn
Quoted 3 lines · Björn Steinbrink
To clarify that a bit more: git gc keeps unreachable objects unpacked, so that git prune can drop them. And git gc invokes git prune so that only unreachable objects older than 2 weeks are dropped.
To be even more explicit, "git gc" will **unpack** objects that have become unreachable and were currently in packs. As a result, the amount of disk space used by a git repository can actually go **up** dramatically after a "git gc" operation, which could be surprising for someone who is running close to full on their filesystem, deletes a number of branches from a tracking repository, and then does a "git gc" may get a very unpleasant surprise.
A really good repository which shows this is linux-next, since it is constantly getting rewound, and old branches are reserved via a tag such as next-20081204. If you update the your local copy of the linux-next repository every day, you will accumulate a large number of these old branch tags. If you then delete a whole series of them, and run git-gc, the operation will take quite a while, and the number of blocks and inodes used will grow significantly. They will disappear after a "git prune", but when I do this housekeeping operation, I've often wished for a --yes-I-know-what-I-am-doing-and-it's-unsafe-but- just-drop-the-unreachable-objects-cause-this-is-just-a-tracking-repository option to "git gc".
- Ted
Quoted 6 lines · Theodore Tso
Quoted 3 lines · Bj?rn Steinbrink
To clarify that a bit more: git gc keeps unreachable objects unpacked, so that git prune can drop them. And git gc invokes git prune so that only unreachable objects older than 2 weeks are dropped.
Quoted 7 lines
To be even more explicit, "git gc" will **unpack** objects that have become unreachable and were currently in packs. As a result, the amount of disk space used by a git repository can actually go **up** dramatically after a "git gc" operation, which could be surprising for someone who is running close to full on their filesystem, deletes a number of branches from a tracking repository, and then does a "git gc" may get a very unpleasant surprise.
It can also cause things like the "please repack" warning in git gui to go off. This is especially unhelpful since they tend to tell you to go and do a gc to resolve the problem.
Quoted 16 lines · Mark Brown
Quoted 6 lines · Theodore Tso
Quoted 4 lines · Bj?rn Steinbrink
To clarify that a bit more: git gc keeps unreachable objects unpacked, so that git prune can drop them. And git gc invokes git prune so that only unreachable objects older than 2 weeks are dropped.
Quoted 8 lines
To be even more explicit, "git gc" will **unpack** objects that have become unreachable and were currently in packs. As a result, the amount of disk space used by a git repository can actually go **up** dramatically after a "git gc" operation, which could be surprising for someone who is running close to full on their filesystem, deletes a number of branches from a tracking repository, and then does a "git gc" may get a very unpleasant surprise.
Quoted 3 lines
It can also cause things like the "please repack" warning in git gui to go off. This is especially unhelpful since they tend to tell you to go and do a gc to resolve the problem.
Instead of exploding all unreachable objects into loose objects, does it make sense to repack them into a separate pack? AFAICS, that would solve both the disk usage problem and the git-gui-"please repack" problem. Also, it might make git-prune's job much easier, since unreachable objects are now located in a single pack only?
Have fun!
...Johan
Signature · 2 lines
Johan Herland, <[email protected]> www.herland.net
2008/12/15 Mark Brown <[email protected]>:
Quoted 15 lines · Theodore Tso
Quoted 6 lines · Bj?rn Steinbrink
Quoted 4 lines
To clarify that a bit more: git gc keeps unreachable objects unpacked, so that git prune can drop them. And git gc invokes git prune so that only unreachable objects older than 2 weeks are dropped.
Quoted 7 lines
To be even more explicit, "git gc" will **unpack** objects that have become unreachable and were currently in packs. As a result, the amount of disk space used by a git repository can actually go **up** dramatically after a "git gc" operation, which could be surprising for someone who is running close to full on their filesystem, deletes a number of branches from a tracking repository, and then does a "git gc" may get a very unpleasant surprise.
Quoted 3 lines
It can also cause things like the "please repack" warning in git gui to go off. This is especially unhelpful since they tend to tell you to go and do a gc to resolve the problem.
A thought that occurs to me is to add some sort of flag to git count-objects that prints the number of objects older than some interval in a separate field. That way git gui would give less (maybe no) false alarms.
Signature · 1 line
Mikael Magnusson
PATCH 1/1 objects to be pruned immediately don't have to be loosened
In reply to Theodore TsoWhen there is no grace period before pruning unreferenced objects, it is pointless to push those objects in their loose form just to delete them right away.
Also be more explicit about the possibility of using "now" in the gc.pruneexpire config variable (needed for the above behavior to happen).
---
Quoted 13 lines · Theodore Tso
Quoted 3 lines · Björn Steinbrink
To clarify that a bit more: git gc keeps unreachable objects unpacked, so that git prune can drop them. And git gc invokes git prune so that only unreachable objects older than 2 weeks are dropped.
To be even more explicit, "git gc" will **unpack** objects that have become unreachable and were currently in packs. As a result, the amount of disk space used by a git repository can actually go **up** dramatically after a "git gc" operation, which could be surprising for someone who is running close to full on their filesystem, deletes a number of branches from a tracking repository, and then does a "git gc" may get a very unpleasant surprise.
Quoted 11 lines
A really good repository which shows this is linux-next, since it is constantly getting rewound, and old branches are reserved via a tag such as next-20081204. If you update the your local copy of the linux-next repository every day, you will accumulate a large number of these old branch tags. If you then delete a whole series of them, and run git-gc, the operation will take quite a while, and the number of blocks and inodes used will grow significantly. They will disappear after a "git prune", but when I do this housekeeping operation, I've often wished for a --yes-I-know-what-I-am-doing-and-it's-unsafe-but- just-drop-the-unreachable-objects-cause-this-is-just-a-tracking-repository option to "git gc".
What about this?
Documentation/config.txt +3 -1
@@ -702,7 +702,9 @@ gc.packrefs:: gc.pruneexpire:: When 'git-gc' is run, it will call 'prune --expire 2.weeks.ago'. - Override the grace period with this config variable. + Override the grace period with this config variable. The value + "now" may be used to disable this grace period and always prune + unreachable objects immediately. gc.reflogexpire:: 'git-reflog expire' removes reflog entries older than
builtin-gc.c +6 -2
@@ -188,7 +188,9 @@ static int need_to_gc(void)* there is no need. */ if (too_many_packs ()) -append_option (argv_repack,"-A" ,MAX_ADD ); +append_option (argv_repack, + !strcmp (prune_expire,"now" ) ?"-a" :"-A" , +MAX_ADD );else if (!too_many_loose_objects ())return 0 ;
@@ -243,7 +245,9 @@ int cmd_gc(int argc, const char **argv, const char *prefix)"run \"git gc\" manually. See " "\"git help gc\" for more information.\n"); }else -append_option (argv_repack,"-A" ,MAX_ADD ); +append_option (argv_repack, + !strcmp (prune_expire,"now" ) ?"-a" :"-A" , +MAX_ADD );if (pack_refs &&run_command_v_opt (argv_pack_refs,RUN_GIT_CMD ))return error (FAILED_RUN , argv_pack_refs[0 ]);
Quoted 4 lines · Theodore Tso
I've
often wished for a --yes-I-know-what-I-am-doing-and-it's-unsafe-but- just-drop-the-unreachable-objects-cause-this-is-just-a-tracking-repository option to "git gc".
repack -a -d -l
Notice the lowercase 'a'.
git-gc calls repack with uppercase 'A' which is what causes the unreachable objects to be unpacked. Little 'a', is for people who know what they are doing, and want git to just drop unreachable objects.
-brandon
Quoted 14 lines · Theodore Tso
Quoted 3 lines · Björn Steinbrink
To clarify that a bit more: git gc keeps unreachable objects unpacked, so that git prune can drop them. And git gc invokes git prune so that only unreachable objects older than 2 weeks are dropped.
To be even more explicit, "git gc" will **unpack** objects that have become unreachable and were currently in packs. As a result, the amount of disk space used by a git repository can actually go **up** dramatically after a "git gc" operation, which could be surprising for someone who is running close to full on their filesystem, deletes a number of branches from a tracking repository, and then does a "git gc" may get a very unpleasant surprise.
Quoted 11 lines
A really good repository which shows this is linux-next, since it is constantly getting rewound, and old branches are reserved via a tag such as next-20081204. If you update the your local copy of the linux-next repository every day, you will accumulate a large number of these old branch tags. If you then delete a whole series of them, and run git-gc, the operation will take quite a while, and the number of blocks and inodes used will grow significantly. They will disappear after a "git prune", but when I do this housekeeping operation, I've often wished for a --yes-I-know-what-I-am-doing-and-it's-unsafe-but- just-drop-the-unreachable-objects-cause-this-is-just-a-tracking-repository option to "git gc".
There was an idea to have "git gc --prune" run "git prune" unconditionally, i.e. without grace period for dangling loose objects.
Signature · 3 lines
Jakub Narebski Poland ShadeHawk on #git
Quoted 4 lines · Jakub Narebski
There was an idea to have "git gc --prune" run "git prune" unconditionally, i.e. without grace period for dangling loose objects.
That doesn't help that much, since (temporarily) you still need all of the disk space for the exploded, unpacked objects. As Brandon Casey pointed out, the key is "git repack -a -d -l" vs "git repack -A -d -l". If there is going to be a git-gc option, it would need to change the options sent to git-repack. Or, I suppose the answer is to tell people who run into this problem use a plumbing command, manually. The question is how common is the use case of needing to gc a repository like linux-next, I suppose.
- Ted
"Mikael Magnusson" <[email protected]> writes:
[...]
Quoted 4 lines
IIRC git gc only removes loose objects older than two weeks, if you really want to remove them now, run git prune. But make sure no other git process can be active when you run it, or it could possibly step on something.
OK, that makes sense. Obviously I misunderstood this. That doesn't explain why the number of objects might increase after "git gc", but perhaps that's for a good reason too.
Surely "git gui"'s warning is unhelpful, then: it warns I have more than 2000 loose objects (in another checkout), offers to compress my database, and I end up with an unchanged repository (which it still complains about)? Is this warning just redundant now that we've got "git gc --auto"?