Skip to contentSkip to conversation

dangling commits

DISCUSSION Nick Williams 5 participants 15 messages Jun 15, 2016

lore
Share & tools
lore
#0 lore raw

Hi, after cloning the git repo with

cg-clone http://www.kernel.org/pub/scm/git/git.git git

and then doing

git-fsck-objects --full

I get the following

dangling commit 42db15448ea3c21ae458d5ea873157449042c07c dangling commit 4d04a4022e7f9f3ada3a64e2010ce65e1fcc5c64 dangling commit a773f5bda1835d739ee7209589e137ddd7199142 dangling commit ceb90a511add3b362f1384aa6ea35370d12db315

However if I do cg-clone git://git.kernel.org/pub/scm/git/git.git
there's no output from git-fsck --full

git version = 1.1.GIT
cogito version = cogito-0.17pre.GIT

did I do something wrong (again)?

#1 lore raw
Quoted 15 lines · Nick Williams

Hi, after cloning the git repo with

cg-clone http://www.kernel.org/pub/scm/git/git.git git

and then doing

git-fsck-objects --full

I get the following

dangling commit 42db15448ea3c21ae458d5ea873157449042c07c dangling commit 4d04a4022e7f9f3ada3a64e2010ce65e1fcc5c64 dangling commit a773f5bda1835d739ee7209589e137ddd7199142 dangling commit ceb90a511add3b362f1384aa6ea35370d12db315

Quoted 8 lines

However if I do cg-clone git://git.kernel.org/pub/scm/git/git.git
there's no output from git-fsck --full

git version = 1.1.GIT
cogito version = cogito-0.17pre.GIT

did I do something wrong (again)?

Nopes. One clones over http, so you'll get all objects in the object database. The other clones over the far more clever git protocol which calculates which objects you need. Obviously you don't need dangling commits (and their related blobs), so there will be no such items.

That there are on kernel.org at all is because Junio does rebases of the pu branch and then pushes them out, which means that the objects from the last rebase of that branch are left dangling.

Signature · 3 lines
Andreas Ericsson                   [email protected]
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231
#2 lore raw
Quoted 20 lines · Andreas Ericsson
Quoted 18 lines · Nick Williams

Hi, after cloning the git repo with

cg-clone http://www.kernel.org/pub/scm/git/git.git git

and then doing

git-fsck-objects --full

I get the following

dangling commit 42db15448ea3c21ae458d5ea873157449042c07c dangling commit 4d04a4022e7f9f3ada3a64e2010ce65e1fcc5c64 dangling commit a773f5bda1835d739ee7209589e137ddd7199142 dangling commit ceb90a511add3b362f1384aa6ea35370d12db315

However if I do cg-clone git://git.kernel.org/pub/scm/git/git.git
there's no output from git-fsck --full

Quoted 7 lines
Quoted 5 lines

git version = 1.1.GIT
cogito version = cogito-0.17pre.GIT

did I do something wrong (again)?

Nopes. One clones over http, so you'll get all objects in the object database. The other clones over the far more clever git protocol which calculates which objects you need. Obviously you don't need dangling commits (and their related blobs), so there will be no such items.

OK, that makes sense - thanks for the explanation.

Quoted 3 lines

That there are on kernel.org at all is because Junio does rebases of the pu branch and then pushes them out, which means that the objects from the last rebase of that branch are left dangling.

So, is there any advantage of using http? Seems like git:// makes more sense.

#3 lore raw
Quoted 7 lines · Nick Williams

Andreas Ericsson wrote:
..

Quoted 5 lines

Nopes. One clones over http, so you'll get all objects in the object database. The other clones over the far more clever git protocol which calculates which objects you need. Obviously you don't need dangling commits (and their related blobs), so there will be no such items.

Note that only because that is these dangling objects are packed in the past, and when fetching over http, packs are fetched as a whole.

Quoted 2 lines

So, is there any advantage of using http? Seems like git:// makes more sense.

As long as you can go native git:// protocol, I do not see much reason to use http:// commit walkers. OTOH, if you are firewalled and your sysadmins do not let you pass 9418/tcp outgoing, HTTP might be your only choice.

#4 lore raw
Quoted 4 lines · In message <[email protected]> you wrote:

Note that only because that is these dangling objects are packed in the past, and when fetching over http, packs are fetched as a whole.

Is ther eany way to clean up such a situation and really get rid of the dangling commits? I understand that I'd first need some way to "unpack" the packs, but how to do this?

Best regards,

Wolfgang Denk

Signature · 4 lines
Software Engineering:  Embedded and Realtime Systems,  Embedded Linux
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: [email protected]
Heavier than air flying machines are impossible.
                    -- Lord Kelvin, President, Royal Society, c. 1895
#5 lore raw
Quoted 3 lines · On Sunday January 15th 2006 Wolfgang Denk wrote:

Is ther eany way to clean up such a situation and really get rid of the dangling commits? I understand that I'd first need some way to "unpack" the packs, but how to do this?

Note that apart from the disk space they use up, dangling commits don't do any harm.

However you can easily get rid of them by using "git prune".

As far as I know although packs are used in transferring the commits to your local repository they are stored there as separate objects, so you certainly don't have to unpack things yourself for using "git prune". Git is quite smart, fast and safe on its own I find each time! It really is a wonderful tool by giving you every possibility to work with it without inflicting policy on you.

If wanted you can use "git repack -a -d" followed by "git prune-packed" to create a tight packed repository (all commits and blobs in one pack) but there is no specific need to.

Signature · 1 line
Marco Roeland
#6 lore raw
Quoted 3 lines · In message <[email protected]> you wrote:

Note that apart from the disk space they use up, dangling commits don't do any harm.

I like to have  my  repository  "clean"  so  there  are  no  warnings
normally  from  git-fsck-objects  -  if  you  get used to expect some

"harmless" messages you might easily miss a critical error.

Quoted 1 line

However you can easily get rid of them by using "git prune".

Tried this, didn't work.

Quoted 2 lines

As far as I know although packs are used in transferring the commits to your local repository they are stored there as separate objects, so you

Ummm... please have a look at the .git/objects/pack/ directory for example in your Linux repository.

Best regards,

Wolfgang Denk

Signature · 4 lines
Software Engineering:  Embedded and Realtime Systems,  Embedded Linux
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: [email protected]
It seems intuitively obvious to me, which  means  that  it  might  be
wrong.                                                 -- Chris Torek
#7 lore raw
Quoted 3 lines · On Monday January 16th 2006 Wolfgang Denk wrote:
I like to have  my  repository  "clean"  so  there  are  no  warnings
normally  from  git-fsck-objects  -  if  you  get used to expect some

"harmless" messages you might easily miss a critical error.

That's right, yes.

Quoted 3 lines
Quoted 1 line

However you can easily get rid of them by using "git prune".

Tried this, didn't work.

Ok, didn't know that it didn't prune directly from packs. I should have realised that it only can do so efficiently by repacking I suppose.

Quoted 2 lines

Ummm... please have a look at the .git/objects/pack/ directory for example in your Linux repository.

I did as a matter of fact, but I use "git" as protocol and also regularly "repack" the repository, when I'm not using the machine for a while, to make 'gitk' and 'qgit' work faster. It is less apparent then! Thanks very much for clearing up my knowledge. It only shows the power of git that even ignorant gits like myself still find it useful and productive. There's just enough rope to wiggle yourself out of precarious situations but fortunately most of the time not enough to hang yourself.

Signature · 1 line
Marco Roeland
#8 lore raw
In reply to Marco Roeland
Quoted 2 lines · Marco Roeland

As far as I know although packs are used in transferring the commits to your local repository they are stored there as separate objects,...

That is true only when you are using git native protocols (i.e. git:// and git over ssh). Some people pull over dumb transport (http -- some others still use rsync which is even dumber and has serious limitations), and when you need objects that are contained in a pack at the upstream, the packfile is downloaded as a whole, and it is left packed on your end.

Even when you use git native protocol, the objects the initial clone gives you are kept packed, so when I rewind and rebuild "pu" to make some of these objects orphaned, they will stay in the pack the initial clone gave you. Unpack+repack is needed to get rid of them.

As you said, they should not hurt much in practice, though.

#9 lore raw
Quoted 5 lines · On Monday January 16th Junio C Hamano wrote:

Even when you use git native protocol, the objects the initial clone gives you are kept packed, so when I rewind and rebuild "pu" to make some of these objects orphaned, they will stay in the pack the initial clone gave you. Unpack+repack is needed to get rid of them.

Thanks very much for explaining. It makes sense now.

Does it bring many advantages for you to keep rebasing "pu"? I started out following that branch long ago (well in git reckoning anyway) but got very scared each time I got a bunch of "errors" on that one. I even recloned a couple of times to get it "clean" again, until I understood from the mailing-list that the rebasing was the cause, not something I did. I since removed it from the "Pull" list, but understand that "+pu" should do the trick. I'll retry using it one of these days.

Signature · 1 line
Marco Roeland
#10 lore raw
Quoted 10 lines · Marco Roeland
Quoted 5 lines · On Monday January 16th Junio C Hamano wrote:

Even when you use git native protocol, the objects the initial clone gives you are kept packed, so when I rewind and rebuild "pu" to make some of these objects orphaned, they will stay in the pack the initial clone gave you. Unpack+repack is needed to get rid of them.

Quoted 3 lines

Thanks very much for explaining. It makes sense now.

Does it bring many advantages for you to keep rebasing "pu"?

Since "pu" = "proposed updates" it only makes sense to keep it on top of the current master, otherwise the effort required for anyone to test it in conjunction with the latest master branch would simply be too great.

Quoted 5 lines

I started
out following that branch long ago (well in git reckoning anyway) but got very scared each time I got a bunch of "errors" on that one. I since removed it from the "Pull" list, but understand that "+pu" should do the trick. I'll retry using it one of these days.

It does. I also remember seeing lots of errors on that one when I first started with git (around 0.99b), but that was fixed quite some time ago.

Signature · 3 lines
Andreas Ericsson                   [email protected]
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231
#11 lore raw
Quoted 3 lines · On Monday January 16th 2006 Andreas Ericsson wrote:

Since "pu" = "proposed updates" it only makes sense to keep it on top of the current master, otherwise the effort required for anyone to test it in conjunction with the latest master branch would simply be too great.

Certainly. And it probably is a good testbed for testing the rebasing routines as well.

But couldn't (in theory) the new "rebased" versions of blobs in the "pu" branch be first committed as the old not yet rebased version and then as the new version. Not the fact that the blobs in "pu" are constantly based on the latest master was the problem if I recollect, but the fact that blobs sometimes disappeared. In comparing this with Linus' recent explanation how git-bisect works in terms of light cones this might be understood as the inherent problems with tachyons I think...

Anyway this is now solved I understand. Thanks.

Quoted 5 lines
Quoted 2 lines

I since removed it from the "Pull" list, but understand that "+pu" should do the trick. I'll retry using it one of these days.

It does. I also remember seeing lots of errors on that one when I first started with git (around 0.99b), but that was fixed quite some time ago.

Ok, thanks very much for explaining.

Signature · 1 line
Marco Roeland
#12 lore raw
Quoted 9 lines · Marco Roeland
Quoted 1 line · On Monday January 16th 2006 Andreas Ericsson wrote:

Since "pu" = "proposed updates" it only makes sense to keep it on top of the current master, otherwise the effort required for anyone to test it in conjunction with the latest master branch would simply be too great.

But couldn't (in theory) the new "rebased" versions of blobs in the "pu" branch be first committed as the old not yet rebased version and then as the new version.

The blobs are immutable and never change for a rebase, unless the file(s) it applies to is changed in master as well. It's the commits that do because they get new parents.

Remember that the blob object is just the (deltified?) file that's the result of the commit operation. The commit object is an object in its own rights, holding author info and commit-time and such. Do

	$ git cat-file commit HEAD

and you'll see what a commit-object looks like.

Signature · 3 lines
Andreas Ericsson                   [email protected]
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231
#13 lore raw
Quoted 3 lines · On Monday January 16th 2006 Andreas Ericsson wrote:

The blobs are immutable and never change for a rebase, unless the file(s) it applies to is changed in master as well. It's the commits that do because they get new parents.

Ah, I need to rebase my mental picture of what a "rebase" is. ;-) And the fact that each commit _does_ have sort of a blob (well in my mind I called it so, the file under .git/objects, although its proper name is indeed "commit") in the repository doesn't make it any easier!

Current documentation about git-rebase(1) is technically correct of course then: "rebases local commits to the new head of the upstream tree" but rather sparse for the less initiated. In Dutch we have an expression for this, to "not be able to see the wood because of the trees", which is rather appropriate here. Perhaps we can introduce "liana" as an alternative for commit.

<Nice young men in clean white coats come in to take me away>

Seriously, yours and other peoples comments make the picture much clearer to me and help out enormously to me and hopefully other lurkers in working with git and more advanced SCM in general. Thanks,

Signature · 1 line
Marco Roeland
#14 lore raw
In reply to Wolfgang Denk
Quoted 3 lines · Wolfgang Denk

Is ther eany way to clean up such a situation and really get rid of the dangling commits? I understand that I'd first need some way to "unpack" the packs, but how to do this?

The easiest is to repack into a single big ball of wax:

	$ git repack -a -d

If you know the pack the stale object is in, you can move it out of objects/pack/ and repack only that one.

	$ mv .git/objects/packs/pack-$badone.{idx,pack} .
	$ git unpack-objects <pack-$badone.pack
        $ git repack

After you are done:

	$ git prune