Skip to contentSkip to conversation

better git-submodule status output

PATCH In review Sylvain Joyeux 6 participants 34 messages Jun 15, 2016 Documentation

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 better git-submodule status output

Oups.. sorry for the patch-as-attachment in the previous mail

This patch makes the output of git-submodule more useful to handle the management of a repository using multiple submodules. Namely, it displays informations about how the current checked out version relates to the registered one (i.e. direct parent, direct child, "just different", registered revision does not exist), and displays if the submodules contain uncommitted changes.

This (among other things) allows to do git-submodule update while knowing exactly what will happen.
--
Sylvain

Quoted 1 line

From 16553a9b210a956b0af961d55a9cf06f1b9b8114 Mon Sep 17 00:00:00 2001

From: Sylvain Joyeux <[email protected]>
Date: Tue, 1 Jul 2008 16:01:01 +0200
Subject: [PATCH] more information in git-submodule status output

This commit adds more information in the 'status' output of git-submodule. More specifically, it displays different flags if the submodule and the registered revision are direct parents (> and <, depending on which is the ancestor), if they are not direct parents (+) or if the registered revision cannot be found (i.e. if submodule update would fail, '!')

Finally, it shows if the submodule contains uncommitted changes (M flag)

---
 Documentation/git-submodule.txt |   23 ++++++++++++++++++-----
 git-submodule.sh                |   32 ++++++++++++++++++++++++++++----
 t/t7400-submodule-basic.sh      |   20 +++++++++++++-------
 3 files changed, 59 insertions(+), 16 deletions(-)
Documentation/git-submodule.txt +18 -5
index 441ae14..3820002 100644
@@ -33,10 +33,23 @@ status::
 	Show the status of the submodules. This will print the SHA-1 of the
 	currently checked out commit for each submodule, along with the
 	submodule path and the output of linkgit:git-describe[1] for the
-	SHA-1. Each SHA-1 will be prefixed with `-` if the submodule is not
-	initialized and `+` if the currently checked out submodule commit
-	does not match the SHA-1 found in the index of the containing
-	repository. This command is the default command for git-submodule.
+	SHA-1. Each SHA-1 will be prefixed with the following codes:
+	* `-` if the submodule is not initialized
+	* `<` if the current submodule HEAD is a direct ancestor of the
+	    revision registered in the containing repository.  I.e. a `git
+	    submodule update` would lead to a fast forward.
+	* `>` if the current submodule HEAD is a direct ancestor of the
+	    revision registered in the containing repository.
+	* `+` if the current submodule HEAD and the revision registered in
+	    the containing repository do not match, and there is no direct
+	    relationship between the two commits.
+	* `!` if the SHA-1 found in the index of the containing repository is
+	    not present in the the currently checked out submodule commit,
+	    nor be fetched from the associated remote.
+	
+	Additionally, a 'M' is displayed if the currently checked out
+	submodule contains uncommitted changes.
+	This command is the default command for git-submodule.
 
 init::
 	Initialize the submodules, i.e. register in .git/config each submodule
@@ -97,7 +110,7 @@ for details.
 
 AUTHOR
 ------
-Written by Lars Hjemli <[email protected]>
+Written by Lars Hjemli <[email protected]> and Sylvain Joyeux <[email protected]>
 
 GIT
 ---
git-submodule.sh +28 -4
index 3eb78cc..e2b91f6 100755
@@ -522,7 +522,11 @@ cmd_summary() {
 #
 # List all submodules, prefixed with:
 #  - submodule not initialized
-#  + different revision checked out
+#  > the checked out version is in advance with the current version (fast-forward)
+#  < the checked out version is late w.r.t the current version (fast-forward)
+#  + different revisions checked out, no fast-forward relationship
+#  ! the checked out version does not exist and cannot be fetched
+#  M submodule contains uncommitted changes
 #
 # If --cached was specified the revision in the index will be printed
 # instead of the currently checked out revision.
@@ -562,20 +566,40 @@ cmd_status()
 		url=$(git config submodule."$name".url)
 		if test -z "$url" || ! test -d "$path"/.git -o -f "$path"/.git
 		then
-			say "-$sha1 $path"
+			say "-  $sha1 $path"
 			continue;
 		fi
+                unclean=" "
+                if ! ( unset GIT_DIR; cd "$path" && git diff --quiet HEAD ); then
+                    unclean="M"
+                fi
 		set_name_rev "$path" "$sha1"
 		if git diff-files --quiet -- "$path"
 		then
-			say " $sha1 $path$revname"
+			say " $unclean $sha1 $path$revname"
 		else
+                        head=$(unset GIT_DIR; cd "$path" && git rev-parse --verify HEAD)
+                        common=$(unset GIT_DIR; cd "$path" && git merge-base HEAD $sha1)
+                        if test -z "$common"; then
+                            common=$(unset GIT_DIR; cd "$path" && git-fetch -q &&
+                                    git merge-base HEAD $sha1)
+                        fi
+                        if test "$common" = "$sha1"; then
+                            modflag=">"
+                        elif test "$common" = "$head"; then
+                            modflag="<"
+                        elif test "$common" = ""; then
+                            modflag="!"
+                        else
+                            modflag="+"
+                        fi
+
 			if test -z "$cached"
 			then
 				sha1=$(unset GIT_DIR; cd "$path" && git rev-parse --verify HEAD)
 				set_name_rev "$path" "$sha1"
 			fi
-			say "+$sha1 $path$revname"
+			say "$modflag$unclean $sha1 $path$revname"
 		fi
 	done
 }
t/t7400-submodule-basic.sh +13 -7
index 6c7b902..ffaa932 100755
@@ -65,7 +65,7 @@ test_expect_success 'status should only print one line' '
 '
 
 test_expect_success 'status should initially be "missing"' '
-	git-submodule status | grep "^-$rev1"
+	git-submodule status | grep "^-  $rev1"
 '
 
 test_expect_success 'init should register submodule url in .git/config' '
@@ -130,13 +130,19 @@ test_expect_success 'update should work when path is an empty dir' '
 '
 
 test_expect_success 'status should be "up-to-date" after update' '
-	git-submodule status | grep "^ $rev1"
+	git-submodule status | grep "^   $rev1"
 '
 
-test_expect_success 'status should be "modified" after submodule commit' '
+test_expect_success 'status should be "modified" after submodule change' '
 	cd init &&
 	echo b >b &&
-	git add b &&
+        git add b &&
+        cd .. &&
+	git-submodule status | grep "^ M $rev1"
+'
+
+test_expect_success 'status should be "in advance" after submodule commit' '
+	cd init &&
 	git-commit -m "submodule commit 2" &&
 	rev2=$(git rev-parse HEAD) &&
 	cd .. &&
@@ -145,11 +151,11 @@ test_expect_success 'status should be "modified" after submodule commit' '
 		echo "[OOPS] submodule git rev-parse returned nothing"
 		false
 	fi &&
-	git-submodule status | grep "^+$rev2"
+	git-submodule status | grep "^>  $rev2"
 '
 
 test_expect_success 'the --cached sha1 should be rev1' '
-	git-submodule --cached status | grep "^+$rev1"
+	git-submodule --cached status | grep "^>  $rev1"
 '
 
 test_expect_success 'git diff should report the SHA1 of the new submodule commit' '
@@ -171,7 +177,7 @@ test_expect_success 'update should checkout rev1' '
 '
 
 test_expect_success 'status should be "up-to-date" after update' '
-	git-submodule status | grep "^ $rev1"
+	git-submodule status | grep "^   $rev1"
 '
 
 test_expect_success 'checkout superproject with subproject already present' '
Signature · 1 line
1.5.6
#1 lore raw
Quoted 17 lines · Sylvain Joyeux

This patch makes the output of git-submodule more useful to handle the management of a repository using multiple submodules. Namely, it displays informations about how the current checked out version relates to the registered one (i.e. direct parent, direct child, "just different", registered revision does not exist), and displays if the submodules contain uncommitted changes.

This (among other things) allows to do git-submodule update while knowing exactly what will happen.
--
Sylvain

Quoted 1 line

From 16553a9b210a956b0af961d55a9cf06f1b9b8114 Mon Sep 17 00:00:00 2001

From: Sylvain Joyeux <[email protected]>
Date: Tue, 1 Jul 2008 16:01:01 +0200
Subject: [PATCH] more information in git-submodule status output

Quoted 8 lines

This commit adds more information in the 'status' output of git-submodule. More specifically, it displays different flags if the submodule and the registered revision are direct parents (> and <, depending on which is the ancestor), if they are not direct parents (+) or if the registered revision cannot be found (i.e. if submodule update would fail, '!')

Finally, it shows if the submodule contains uncommitted changes (M flag)

Which one is the commit message ;-)?

People who rely on working submodule support, do you have any feedback on this patch? I do not use submodule myself, so it is hard for me to judge how much value (if any) this patch is adding to the real world use of the status subcommand.

Documentation/git-submodule.txt for details. · line 113

Quoted 6 lines
Documentation/git-submodule.txt +1 -1
@@ -97,7 +110,7 @@ for details.
 
 AUTHOR
 ------
-Written by Lars Hjemli <[email protected]>
+Written by Lars Hjemli <[email protected]> and Sylvain Joyeux <[email protected]>

That is somehow inconsistent with what

	git-shortlog -s -n -e --no-merges git-submodule.sh 

tells me.

Honestly, I'd prefer (1) drop these "AUTHOR" lines, or (2) only list the primary author or two.

Quoted 7 lines
git-submodule.sh +0 -0
index 3eb78cc..e2b91f6 100755
@@ -562,20 +566,40 @@ cmd_status()
...
 		if git diff-files --quiet -- "$path"

git-submodule.sh cmd_status() · line 586

Quoted 10 lines
git-submodule.sh +7 -1
 		then
-			say " $sha1 $path$revname"
+			say " $unclean $sha1 $path$revname"
 		else
+                        head=$(unset GIT_DIR; cd "$path" && git rev-parse --verify HEAD)
+                        common=$(unset GIT_DIR; cd "$path" && git merge-base HEAD $sha1)
+                        if test -z "$common"; then
+                            common=$(unset GIT_DIR; cd "$path" && git-fetch -q &&
+                                    git merge-base HEAD $sha1)
+                        fi

This "fetch" feels very wrong. The user did not ask you to change the state of the repository, but this will silently change the remote tracking branches. The repository after all might be unreachable.

#2 lore raw
Quoted 7 lines · Junio C Hamano

Sylvain Joyeux <[email protected]> writes:

People who rely on working submodule support, do you have any feedback on this patch? I do not use submodule myself, so it is hard for me to judge how much value (if any) this patch is adding to the real world use of the status subcommand.

The new format is changed from the old in an incompatible way (anything that parses the old output will break), and the new output is very cryptic. I do not think that I would remember the various symbols in use here (+-<>!M). While the information may be useful, I would think a better way would be to append a more descriptive annotation (similar to that provided after a checkout), probably on a separate status line, definitely controlled by a "-v" option.

Quoted 4 lines

This "fetch" feels very wrong. The user did not ask you to change the state of the repository, but this will silently change the remote tracking branches. The repository after all might be unreachable.

Status is a local operation, it must not try to access the net (which for me will frequently fail). Status changing the remote branches is a definite no-no.

Mark

#3 lore raw
In reply to Junio C Hamano

Hi,

Quoted 26 lines · Junio C Hamano
Quoted 24 lines · Sylvain Joyeux

This patch makes the output of git-submodule more useful to handle the management of a repository using multiple submodules. Namely, it displays informations about how the current checked out version relates to the registered one (i.e. direct parent, direct child, "just different", registered revision does not exist), and displays if the submodules contain uncommitted changes.

This (among other things) allows to do git-submodule update while knowing exactly what will happen.
--
Sylvain

Quoted 1 line
Quoted 1 line

From 16553a9b210a956b0af961d55a9cf06f1b9b8114 Mon Sep 17 00:00:00 2001

From: Sylvain Joyeux <[email protected]>
Date: Tue, 1 Jul 2008 16:01:01 +0200
Subject: [PATCH] more information in git-submodule status output

This commit adds more information in the 'status' output of git-submodule. More specifically, it displays different flags if the submodule and the registered revision are direct parents (> and <, depending on which is the ancestor), if they are not direct parents (+) or if the registered revision cannot be found (i.e. if submodule update would fail, '!')

Quoted 3 lines
Quoted 1 line

Finally, it shows if the submodule contains uncommitted changes (M flag)

Which one is the commit message ;-)?

I think it is clear that Sylvain has not read
Documentation/SubmittingPatches yet.

Quoted 2 lines

People who rely on working submodule support, do you have any feedback on this patch?

Not yet. Will test/comment when the spurious "fetch" is fixed.

Ciao,
Dscho

#4 lore raw
Quoted 4 lines
Quoted 1 line

Which one is the commit message ;-)?

I think it is clear that Sylvain has not read
Documentation/SubmittingPatches yet.

Sylvain *has* read SubmittingPatches but has not understood every detail of this very long document.

If you could enlighten me on what is wrong ...

Quoted 4 lines
Quoted 2 lines

People who rely on working submodule support, do you have any feedback on this patch?

Not yet. Will test/comment when the spurious "fetch" is fixed.

I thought that the only thing that 'fetch' does is update FETCH_HEAD. My problem is that doing the fetch is the only way to know what is the status of the submodule w.r.t. the registered commit. If you have a better way to get that information, I'm all ears :P

--
Sylvain

#5 lore raw

Hi,

Quoted 7 lines · Sylvain Joyeux
Quoted 4 lines
Quoted 1 line

Which one is the commit message ;-)?

I think it is clear that Sylvain has not read
Documentation/SubmittingPatches yet.

Sylvain *has* read SubmittingPatches but has not understood every detail of this very long document.

Oh. But you should have already seen on this list that almost everybody puts the oneline as subject, the commit message as first part of the mail body, then a "--", optionally some personal stuff that should not go into the commit message, and then the diffstat and all the rest.

IOW the mail body should look like the output of format-patch, optionally some comments between "--" and the diffstat.

Write a patch for SubmittingPatches.

Quoted 8 lines
Quoted 6 lines

If you could enlighten me on what is wrong ...

Quoted 2 lines

People who rely on working submodule support, do you have any feedback on this patch?

Not yet. Will test/comment when the spurious "fetch" is fixed.

I thought that the only thing that 'fetch' does is update FETCH_HEAD.

Uhoh. 'fetch' fetches. Which means it does not only update something.

Quoted 3 lines

My problem is that doing the fetch is the only way to know what is the status of the submodule w.r.t. the registered commit. If you have a better way to get that information, I'm all ears :P

Are you sure that you understand the submodule thing enough to patch git-submodule?

There are three states that are interesting:

- the committed submodule state:

You get this in the superproject by "git ls-tree HEAD -- <dir>"

- the local HEAD of the submodule:

  (cd "<dir>" && git rev-parse --verify HEAD)

- the HEAD of the default branch of the upstream repository of the submodule:

  (cd "<dir>" && git ls-remote origin HEAD)

NOTE: this does not have to do _anything_ with the submodule: you can easily have two independent branches (see "html" and "next" in git.git), and the submodule is free to be pinned to whatever commit is available in the upstream repository.

It is even possible that it is pinned to a commit that is only reachable from a tag, not from a branch.

Hth,
Dscho

#6 lore raw
Quoted 4 lines

IOW the mail body should look like the output of format-patch, optionally some comments between "--" and the diffstat.

Write a patch for SubmittingPatches.

I shall do that :)

Quoted 5 lines
Quoted 3 lines
Quoted 1 line

Not yet. Will test/comment when the spurious "fetch" is fixed.

I thought that the only thing that 'fetch' does is update FETCH_HEAD.

Uhoh. 'fetch' fetches. Which means it does not only update something.

Yes, it fetches objects (admittedly, it updates something on the current partition), but AFAIK does not change the state of the repository. So I thought that it could be considered as a no-op from the repo point of view. The idea behind the 'fetch' thing was to get the latest objects from the repository, therefore being able to know the relationship between the currently committed submodule state

Quoted 13 lines

There are three states that are interesting:

- the committed submodule state:

You get this in the superproject by "git ls-tree HEAD -- <dir>"

- the local HEAD of the submodule:

  (cd "<dir>" && git rev-parse --verify HEAD)

- the HEAD of the default branch of the upstream repository of the submodule:

Quoted 8 lines

(cd "<dir>" && git ls-remote origin HEAD)
NOTE: this does not have to do _anything_ with the submodule: you can easily have two independent branches (see "html" and "next" in git.git), and the submodule is free to be pinned to whatever commit is available in the upstream repository.

It is even possible that it is pinned to a commit that is only reachable from a tag, not from a branch.

Then when you initialize your submodules it will be unable to fetch the commit object, isn't it ? Since git-submodule clones origin/HEAD, the submodule state committed in the repository must be reachable from origin/HEAD.

Or the third interesting state is not "the default branch of the upstream repository of the submodule", i.e. origin/HEAD, but the most-probably-interesting-branch, i.e. the branch configured in .git/config as being fetched for the current branch of the local submodule HEAD.

That is the point of this patch. It make clear, *for all registered submodules*, what is the current relationship between the current submodule state committed in the repository and the current submodule HEAD. The "fetch" is there to take into account the fact that the submodule state may not yet be available locally.

Best,
--
Sylvain

#7 lore raw
Quoted 6 lines · Sylvain Joyeux

Yes, it fetches objects (admittedly, it updates something on the current
partition), but AFAIK does not change the state of the repository. So I thought that it could be considered as a no-op from the repo point of view. The idea behind the 'fetch' thing was to get the latest objects from the repository, therefore being able to know the relationship between the currently committed submodule state

I haven't looked at your patch, so I don't know if you're calling fetch in such a way that it doesn't update any refs. Anyway, fetching is definitely not a no-op as far as the user is concerned. For example, if I'm on my laptop outside my company's LAN and I try to fetch, it might take 30+ seconds for fetch to time out waiting for our server. So it's not okay to fetch unless I specifically ask for a fetch.

The trick here, I think, is that there's only one time the submodule should be linked to a commit you don't actually have. It should only happen if your parent module (supermodule) has been updated, but you haven't recently done a fetch in the submodule. Thus, I'd say the best fix would be to find a way to have "git pull" or "git fetch" in the supermodule also do a fetch in the submodule. I think it's fair to expect "fetchy" operations like fetch and pull to perhaps do some extra fetches, but not fair to expect "status" to do extra work like that.

Also note that "your supermodule links to a commit that you haven't fetched yet" is a perfectly okay state to report to the user. I might use a "?" or something to indicate that.

Have fun,

Avery

#8 lore raw
Quoted 2 lines

I haven't looked at your patch, so I don't know if you're calling fetch in such a way that it doesn't update any refs.

Frankly, I don't know either. But since the question is more on a design level, it does not matter for now.

Quoted 6 lines

Thus, I'd say the
best fix would be to find a way to have "git pull" or "git fetch" in the supermodule also do a fetch in the submodule. I think it's fair to expect "fetchy" operations like fetch and pull to perhaps do some extra fetches, but not fair to expect "status" to do extra work like that.

I do agree on the principle, it is just that I don't know where to do it otherwise. Would the involved people be fine with a new "git submodule fetch" command ?

Quoted 3 lines

Also note that "your supermodule links to a commit that you haven't fetched yet" is a perfectly okay state to report to the user. I might use a "?" or something to indicate that.

For now, my patch shows '!' ... But I'm open to other suggestions.

Sylvain

#9 lore raw
In reply to Avery Pennarun

Hi,

Quoted 4 lines · Avery Pennarun

The trick here, I think, is that there's only one time the submodule should be linked to a commit you don't actually have. It should only happen if your parent module (supermodule) has been updated, but you haven't recently done a fetch in the submodule.

Noooooo!

If I am actively working on the submodule, the supermodule has _no business_ trying to wreck my state.

Hth,
Dscho

#10 lore raw
Quoted 8 lines · Johannes Schindelin
Quoted 2 lines · Avery Pennarun

Thus, I'd say the best fix would be to find a way to have "git pull" or "git fetch" in the supermodule also do a fetch in the submodule.

Noooooo!

If I am actively working on the submodule, the supermodule has _no business_ trying to wreck my state.

Hmm... how does doing a fetch wreck your state? I thought fetch was supposed to be a pretty harmless operation. We're not talking about doing a "git submodule update" automatically (which would be deadly, albeit only because "git submodule update" is so destructive at present).

All I'm suggesting is, when doing a "git fetch" in the supermodule, simply do a "cd submodule && git fetch" automatically. Probably should be optional, but seems like it could make sense for most uses, and avoids the (probably much more common) annoyance of fetching in the supermodule and taking my laptop on the road, only to find out that I don't actually have all the objects I need.

On the other hand, for my own workflow I'm shifting increasingly toward having all my "submodules" share a single repo anyway, in which case fetching in the supermodule *would* automatically fetch all my relevant submodule objects too.

Have fun,

Avery

#11 lore raw

Hi,

Quoted 11 lines · Avery Pennarun
Quoted 8 lines · Johannes Schindelin
Quoted 2 lines · Avery Pennarun

Thus, I'd say the best fix would be to find a way to have "git pull" or "git fetch" in the supermodule also do a fetch in the submodule.

Noooooo!

If I am actively working on the submodule, the supermodule has _no business_ trying to wreck my state.

Hmm... how does doing a fetch wreck your state?

It updates the tracking branches. And guess what I use the tracking branches for? Yes, to track other people's changes.

It would wreck my state.

Anyway, fetch is wrong, wrong, wrong, if all you want to do is see the _state_ of your submodule.

If that state is that the superproject has a commit that the submodule knows nothing about, then that is _exactly_ what needs to be reported.

Don't play cute games,
Dscho

#12 lore raw
Quoted 11 lines · Johannes Schindelin

Hi,

Quoted 7 lines · Avery Pennarun
Quoted 6 lines · Johannes Schindelin
Quoted 2 lines · Avery Pennarun

Thus, I'd say the best fix would be to find a way to have "git pull" or "git fetch" in the supermodule also do a fetch in the submodule.

Noooooo!

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

If I am actively working on the submodule, the supermodule has _no business_ trying to wreck my state.

Hmm... how does doing a fetch wreck your state?

It updates the tracking branches. And guess what I use the tracking branches for? Yes, to track other people's changes.

And ... I'll ask the same question again, then maybe you will be so nice as to give me the information that you do not want to share with the earthling that I am.

Is there no way to fetch only objects without updating no refs ?

Quoted 2 lines

Anyway, fetch is wrong, wrong, wrong, if all you want to do is see the _state_ of your submodule.

Agreed. Now, can we move on and discuss Avery's point which was to have a SEPARATE operation which fetches all submodules in one command ?

Quoted 1 line

Don't play cute games,

OK. Maybe you're god in the small world of git-submodule or whatever, but could you at least *try* to explain things to people which are trying to understand what is SOOOOOOOOOOOOOOO HOOORRRRRRRIIIIBBBBLLLLLLLLYYYYYYYYY BAD when the only information they have is:

THIS IS HORRIBLY BAD, GO AWAY

Sylvain

#13 lore raw
In reply to Johannes Schindelin
Quoted 8 lines · Johannes Schindelin
Quoted 1 line · Avery Pennarun

Hmm... how does doing a fetch wreck your state?

It updates the tracking branches. And guess what I use the tracking
branches for? Yes, to track other people's changes.

It would wreck my state.

Quoted 7 lines

Anyway, fetch is wrong, wrong, wrong, if all you want to do is see the _state_ of your submodule.

If that state is that the superproject has a commit that the submodule knows nothing about, then that is _exactly_ what needs to be reported.

Don't play cute games,

Hmm, somehow we're not understanding each other here.

I think we have already agreed that "git status" and/or "git submodule status" are obviously not good times to be fetching things. And a perfectly valid "submodule status" is that your submodule can't be updated because you're missing some objects and should fetch.

What I suggested was to recursively do "git fetch" in submodules at the time of doing fetch or pull operations on the supermodule, in order to help ensure that all necessary objects have been fetched. To reference an earlier email of yours, this would *improve* git's usability for people who try to use it on a plane, by reducing the number of cases where they have forgotten to fetch.

(1) Do you think fetching missing submodule objects when fetching the supermodule is a bad idea, or a good one? (2) On top of that, do you think updating the submodule's refs when I do "git fetch" in the supermodule is a good one (it seems you think it don't)? (3) And on top of that, if either of those is a good idea, should it be a config option? (4) And should that option default on or off?

In case anyone cares about my preferences, I think (1) makes sense,
(2) is probably too dangerous, (3) should be yes, or perhaps people should be taught to run "git submodule fetch" to fetch submodules and leave "git fetch" alone, and (4) if it's configurable, it should do it by default, because in 99% of situations I can imagine, fetching a supermodule and not fetching its referenced submodule commit will not be useful behaviour.

Thanks,

Avery

#14 lore raw

Hi,

Quoted 3 lines · Avery Pennarun

What I suggested was to recursively do "git fetch" in submodules at the time of doing fetch or pull operations on the supermodule, in order to help ensure that all necessary objects have been fetched.

IMO "git submodule update" is just good enough.

Ciao,
Dscho

#15 lore raw
In reply to Johannes Schindelin

Hi,

Quoted 11 lines · Johannes Schindelin
Quoted 4 lines · Avery Pennarun

The trick here, I think, is that there's only one time the submodule should be linked to a commit you don't actually have. It should only happen if your parent module (supermodule) has been updated, but you haven't recently done a fetch in the submodule.

Noooooo!

If I am actively working on the submodule, the supermodule has _no business_ trying to wreck my state.

Ooops. The part I should have quoted was this:

Quoted 2 lines

Thus, I'd say the best fix would be to find a way to have "git pull" or "git fetch" in the supermodule also do a fetch in the submodule.

Now you will understand my objection, hopefully,
Dscho

#16 lore raw
Quoted 4 lines
Quoted 2 lines

Thus, I'd say the best fix would be to find a way to have "git pull" or "git fetch" in the supermodule also do a fetch in the submodule.

If I am actively working on the submodule, the supermodule has _no business_ trying to wreck my state.

Is it possible to make 'fetch' only .. well .. fetch objects, without updating any refs ? In that case it would wreck no state as no state would be updated.

Signature · 6 lines
=======================================================================
Dr. Ing. Sylvain Joyeux                          [email protected]
Researcher
DFKI Robotik Lab -- Bremen                   http://www.dfki.de/robotik
Tel: 0049 421 218 64136
=======================================================================
#17 lore raw
Quoted 2 lines · Sylvain Joyeux

Is it possible to make 'fetch' only .. well .. fetch objects, without updating any refs?

Not very easily, and that is very deliberate. The objects fetched will become dangling if you allowed such a mode of operation, and will immediately become subject to gc.

As Avery said, when the user asks "what's the status" in the superproject, "the superproject binds a commit you do not have locally at these paths" is a perfectly valid answer, and as Dscho argues, it is not good to avoid giving that answer by running fetch behind the user's back.

And the thing is, if you do the "without updating refs" fetch, I think you would make things even worse. The superproject status will then may say that you have everything you need, but the commit is not anchored with any ref in the subproject repository and can be gc'ed at any time. The answer you gave to the "status" request cannot be trusted anymore.

By honestly saying "The superproject binds a commit you do not have locally at these paths", the user *can* choose *when* to go there and update, or have "git submodule update" command to that for him.

#18 lore raw
Quoted 6 lines · Junio C Hamano
Quoted 2 lines

Is it possible to make 'fetch' only .. well .. fetch objects, without updating any refs?

By honestly saying "The superproject binds a commit you do not have locally at these paths", the user *can* choose *when* to go there and update, or have "git submodule update" command to that for him.

Agreed. The problem is that git submodule update does not only fetches the new refs but *also* checks out the commit currently registered in the superproject. Therefore git-submodule status *and* git-submodule summary cannot tell you anything after a git-submodule update.

What I was missing is, given the current discussion, a git-submodule fetch alike to what Avery was proposing, which fetches all submodules in one command. Having to do the fetch/update manually for each submodule is really cumbersome -- I have around 30 modules in the superproject.

Sylvain

#19 lore raw

Hi,

Quoted 4 lines · Sylvain Joyeux

What I was missing is, given the current discussion, a git-submodule fetch alike to what Avery was proposing, which fetches all submodules in one command. Having to do the fetch/update manually for each submodule is really cumbersome -- I have around 30 modules in the superproject.

Did I not suggest "git submodule update" earlier? AFAIR this updates all submodules that have been initialized.

Hth,
Dscho

#20 lore raw
Quoted 7 lines
Quoted 4 lines

What I was missing is, given the current discussion, a git-submodule fetch alike to what Avery was proposing, which fetches all submodules in one command. Having to do the fetch/update manually for each submodule is really cumbersome -- I have around 30 modules in the superproject.

Did I not suggest "git submodule update" earlier? AFAIR this updates all submodules that have been initialized.

... and checkouts the commit registered in the superproject. Therefore neither git-submodule status nor git-submodule summary can give an idea of "what's new in submodules"

Sylvain

#21 lore raw

Hi,

Quoted 10 lines · Sylvain Joyeux

[somebody wrote]

Quoted 8 lines

[somebody else wrote]

Quoted 5 lines

What I was missing is, given the current discussion, a git-submodule fetch alike to what Avery was proposing, which fetches all submodules in one command. Having to do the fetch/update manually for each submodule is really cumbersome -- I have around 30 modules in the superproject.

Quoted 6 lines
Quoted 2 lines

Did I not suggest "git submodule update" earlier? AFAIR this updates all submodules that have been initialized.

... and checkouts the commit registered in the superproject. Therefore neither git-submodule status nor git-submodule summary can give an idea of "what's new in submodules"

Yeah, now I get what you are saying. So you want a "git submodule peek" or some such.

Ciao,
Dscho

#22 lore raw
Quoted 6 lines
Quoted 3 lines

... and checkouts the commit registered in the superproject. Therefore neither git-submodule status nor git-submodule summary can give an idea of "what's new in submodules"

Yeah, now I get what you are saying. So you want a "git submodule peek" or some such.

That is exactly the idea.

Sylvain

#23 lore raw
In reply to Junio C Hamano
Quoted 7 lines · Junio C Hamano
Quoted 2 lines · Sylvain Joyeux

Is it possible to make 'fetch' only .. well .. fetch objects, without updating any refs?

Not very easily, and that is very deliberate. The objects fetched will
become dangling if you allowed such a mode of operation, and will immediately become subject to gc.

Is this really desirable? I could imagine dropping an entry (or several entries, or whatever) into the reflog after each fetch, to prevent such fetched objects from being deleted right away.

Avery

#24 lore raw
Quoted 1 line · Avery Pennarun

Is this really desirable?

Yes.

After "git fetch $random_place master && git log ..FETCH_HEAD" to inspect what the other guy is up to, if you do not like what you see, you do want the objects that are only reachable from FETCH_HEAD to go away (note that it is not counted as the root of fsck traversal for this exact reason).

#25 lore raw
In reply to Sylvain Joyeux

Hi,

Quoted 8 lines · Sylvain Joyeux
Quoted 4 lines
Quoted 2 lines

Thus, I'd say the best fix would be to find a way to have "git pull" or "git fetch" in the supermodule also do a fetch in the submodule.

If I am actively working on the submodule, the supermodule has _no business_ trying to wreck my state.

Is it possible to make 'fetch' only .. well .. fetch objects, without updating any refs ? In that case it would wreck no state as no state would be updated.

You have no business sneaking a fetch into an operation that does not need one. Period.

If that is _still_ not enough to convince you: think about sitting in a plane, high above the clouds. Yes, we try to support that mode of operation.

Hthf,
Dscho

#26 lore raw
Quoted 11 lines · Johannes Schindelin
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

Thus, I'd say the best fix would be to find a way to have "git pull" or "git fetch" in the supermodule also do a fetch in the submodule.

If I am actively working on the submodule, the supermodule has _no business_ trying to wreck my state.

Is it possible to make 'fetch' only .. well .. fetch objects, without updating any refs ? In that case it would wreck no state as no state would be updated.

You have no business sneaking a fetch into an operation that does not need one. Period.

Well. Given that yourself were replying to having "git fetch" fetching submodules as well, maybe you could tell me how "git fetch" is an operation that does not need fetching[1].

Moreover, on the "need" point, I do think that git-submodule (or git in general) needs a way to fetch (or to git remote update) all submodules in one operation. Otherwise, managing submodules becomes hell very quickly.

Sylvain

[1] I know, it sounds like a smurf talking.

#27 lore raw
In reply to Sylvain Joyeux

Here is a (tentative) summary of the whole discussion:

* doing fetch in status is EVIL. Therefore, status should only report whatever information is available.
* nonetheless, having a "peek" mode seem to be accepted as a useful feature not only by me.
* changing the output format of git-submodules is not right either, because it would break existing tools which parses it at the moment.

Proposal
- remove fetch from status, and make the new output enabled when --verbose is set (can also be set in the config file I guess).

On the symbols side, I propose:

Quoted 1 line

submodule commit is a direct descendant of the commit in the

superproject
< submodule commit is a direct ancestor of the commit in the
superproject
+ no direct relation between submodule commit and commit in the superproject
? the commit in the superproject cannot be found in the submodule
(replaces the initial '!' in my patch)

A 'M' is appended if the submodule has uncommitted changes

- define a git-submodule 'fetch' subcommand which call fetch in each submodule and runs the verbose mode of git-status (can be disabled by a --quiet option).

Comments ? (I'm sure there are some ...)

Sylvain

#28 lore raw
Quoted 23 lines · Sylvain Joyeux

Here is a (tentative) summary of the whole discussion:

* doing fetch in status is EVIL. Therefore, status should only report whatever information is available.
* nonetheless, having a "peek" mode seem to be accepted as a useful feature not only by me.
* changing the output format of git-submodules is not right either, because it would break existing tools which parses it at the moment.

Proposal
- remove fetch from status, and make the new output enabled when --verbose is set (can also be set in the config file I guess).

On the symbols side, I propose:

Quoted 1 line

submodule commit is a direct descendant of the commit in the

superproject
< submodule commit is a direct ancestor of the commit in the
superproject
+ no direct relation between submodule commit and commit in the superproject
? the commit in the superproject cannot be found in the submodule
(replaces the initial '!' in my patch)

Quoted 8 lines

A 'M' is appended if the submodule has uncommitted changes

- define a git-submodule 'fetch' subcommand which call fetch in each submodule and runs the verbose mode of git-status (can be disabled by a --quiet option).

Comments ? (I'm sure there are some ...)

Re-use the existing symbolism from fetch status output. Using '+' to denote "no direct relation" in a tool that shows patches half the time is just plain horrible.

Signature · 3 lines
Andreas Ericsson                   [email protected]
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231
#29 lore raw
Quoted 30 lines · Andreas Ericsson
Quoted 29 lines · Sylvain Joyeux

Here is a (tentative) summary of the whole discussion:

* doing fetch in status is EVIL. Therefore, status should only report whatever information is available.
* nonetheless, having a "peek" mode seem to be accepted as a useful feature not only by me.
* changing the output format of git-submodules is not right either, because it would break existing tools which parses it at the moment.

Proposal
- remove fetch from status, and make the new output enabled when --verbose is set (can also be set in the config file I guess).

On the symbols side, I propose:

Quoted 1 line

submodule commit is a direct descendant of the commit in the

superproject
< submodule commit is a direct ancestor of the commit in the
superproject
+ no direct relation between submodule commit and commit in the superproject
? the commit in the superproject cannot be found in the submodule
(replaces the initial '!' in my patch)

A 'M' is appended if the submodule has uncommitted changes

- define a git-submodule 'fetch' subcommand which call fetch in each submodule and runs the verbose mode of git-status (can be disabled by a --quiet option).

Quoted 6 lines
Quoted 2 lines

Comments ? (I'm sure there are some ...)

Re-use the existing symbolism from fetch status output. Using '+' to denote "no direct relation" in a tool that shows patches half the time is just plain horrible.

Could you be more specific ? AFAIK, 'fetch' symbolism is based on files, the one I need is comparing commits.

And I don't like the '+' either. I chose it because it is the current symbol to show that there is a difference between superproject/submodule.

Signature · 6 lines
=======================================================================
Dr. Ing. Sylvain Joyeux                          [email protected]
Researcher
DFKI Robotik Lab -- Bremen                   http://www.dfki.de/robotik
Tel: 0049 421 218 64136
=======================================================================
#30 lore raw
In reply to Sylvain Joyeux

Hi,

Quoted 9 lines · Sylvain Joyeux

On the symbols side, I propose:

Quoted 1 line

submodule commit is a direct descendant of the commit in the

superproject
< submodule commit is a direct ancestor of the commit in the
superproject
+ no direct relation between submodule commit and commit in the superproject
? the commit in the superproject cannot be found in the submodule
(replaces the initial '!' in my patch)

I like the output of "git checkout" when your local branch is not agreeing with the tracked branch. Maybe it would be more user-friendly to have submodule use that type of message, instead of cryptic single letter?

On a related note, the long commit name has been a constant nuisance for me. A short commit name is perfectly enough, methinks.

Quoted 3 lines

- define a git-submodule 'fetch' subcommand which call fetch in each submodule and runs the verbose mode of git-status (can be disabled by a --quiet option).

I like it.

You probably can refactor the code for "update" to do that nicely.

Ciao,
Dscho

#31 lore raw
Quoted 10 lines
Quoted 9 lines

On the symbols side, I propose:

Quoted 1 line

submodule commit is a direct descendant of the commit in the

superproject
< submodule commit is a direct ancestor of the commit in the
superproject
+ no direct relation between submodule commit and commit in the superproject
? the commit in the superproject cannot be found in the submodule
(replaces the initial '!' in my patch)

Quoted 3 lines

I like the output of "git checkout" when your local branch is not agreeing with the tracked branch. Maybe it would be more user-friendly to have submodule use that type of message, instead of cryptic single letter?

Mmmm ... Yes that could be a great idea. I'll try it.

Quoted 2 lines

On a related note, the long commit name has been a constant nuisance for me. A short commit name is perfectly enough, methinks.

I also think so. The commit ID is completely useless for humans. Nonetheless, that would change the git-submodules output which (according to someone-that-I-dont-remember) would be a Bad Thing(tm).

Quoted 7 lines
Quoted 3 lines

- define a git-submodule 'fetch' subcommand which call fetch in each submodule and runs the verbose mode of git-status (can be disabled by a --quiet option).

I like it.

You probably can refactor the code for "update" to do that nicely.

OK

Sylvain

#32 lore raw

Hi,

Quoted 8 lines · Sylvain Joyeux

[Sylvain quoted somebody, but found it funny to let the reader guess]

Quoted 2 lines

On a related note, the long commit name has been a constant nuisance for me. A short commit name is perfectly enough, methinks.

I also think so. The commit ID is completely useless for humans. Nonetheless, that would change the git-submodules output which (according to someone-that-I-dont-remember) would be a Bad Thing(tm).

Is git-submodule supposed to be plumbing? I don't think so.

Ciao,
Dscho

#33 lore raw
In reply to Sylvain Joyeux
Quoted 2 lines · Sylvain Joyeux

* changing the output format of git-submodules is not right either, because it would break existing tools which parses it at the moment.

On other two points I do agree with people who objected, but I do not think this one is particularly bad. If 'git submodule status' is similar in spirit to 'git status', then it is more important to make it useful for human consumption than to keep the wording of the output set in stone.