Skip to contentSkip to conversation

submodule update continue

PATCH In review Fredrik Gustafsson 2 participants 6 messages Jun 15, 2016

lore
Share & tools
lore
#0 lore raw

v3 cover submodule update continue

This series adds support to submodule update to continue if some submodules fail to do so.

This is the third iteration of this patch series. The first can be found on:
http://thread.gmane.org/gmane.comp.version-control.git/171895
The second can be found on:
http://thread.gmane.org/gmane.comp.version-control.git/173629
Fredrik Gustafsson (2):
Add new function die_with_status submodule update: continue when a checkout fails

 git-sh-setup.sh             |   12 +++-
 git-submodule.sh            |   48 +++++++++++++--
 t/t7406-submodule-update.sh |  144 +++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 196 insertions(+), 8 deletions(-)
Signature · 1 line
1.7.6.rc0.14.g4fc55
#1 lore raw

PATCH v3 1/2 Add new function die_with_status

This function takes the exit code as a parameter.

This allows scripts to control the exit code when they need to die.

Make the old die-function to use die_with_status for simpler maintenance.

die no longer prints empty die messages, this is a changed behavior.

---
 git-sh-setup.sh |   12 ++++++++++--
 1 files changed, 10 insertions(+), 2 deletions(-)
git-sh-setup.sh +10 -2
index 94e26ed..1a91f6e 100644
@@ -40,8 +40,16 @@ git_broken_path_fix () {
 # @@BROKEN_PATH_FIX@@
 
 die() {
-	echo >&2 "$@"
-	exit 1
+	die_with_status 1 $@
+}
+die_with_status() {
+	stat=$1
+	shift
+	if test ${#@} -gt 0
+	then
+		echo >&2 "$@"
+	fi
+	exit $stat
 }
 
 GIT_QUIET=
Signature · 1 line
1.7.6.rc0.14.g4fc55
#2 lore raw

Re: [PATCH v3 1/2] Add new function die_with_status

Quoted 1 line · Fredrik Gustafsson

die no longer prints empty die messages, this is a changed behavior.

That is what we we usually call a regression, but I think the original is not correct to begin with. It should have used "$*" not "$@".

git-sh-setup.sh git_broken_path_fix () { · line 43

Quoted 11 lines
git-sh-setup.sh +1 -2
index 94e26ed..1a91f6e 100644
@@ -40,8 +40,16 @@ git_broken_path_fix () {
 # @@BROKEN_PATH_FIX@@
 
 die() {
-	echo >&2 "$@"
-	exit 1
+	die_with_status 1 $@

Make it a habit to always quote "$@" when relaying what was given to you, so that when somebody sends a parameter with IFS in it, you won't split it while passing it down. I.e.

	die () {
        	die_with_status 1 "$@"
	}

git-sh-setup.sh git_broken_path_fix () { · line 44

Quoted 1 line
git-sh-setup.sh +1 -0
+}

Blank line here after the closing brace and the beginning of next function.

git-sh-setup.sh git_broken_path_fix () { · line 51

Quoted 7 lines
git-sh-setup.sh +7 -0
+die_with_status() {
+	stat=$1
+	shift
+	if test ${#@} -gt 0
+	then
+		echo >&2 "$@"
+	fi

I think an unconditional

	echo >&2 "$*"

is good enough here.

Quoted 4 lines
git-sh-setup.sh +1 -0
+	exit $stat
 }
 
 GIT_QUIET=
#3 lore raw

PATCH v3 2/2 submodule update: continue when a checkout fails

In reply to Fredrik Gustafsson

"git submodule update" stops at the first error and gives control back to the user. Only after the user fixes the problematic submodule and runs "git submodule update" again, the second error is found. And the user needs to repeat until all the problems are found and fixed one by one. This is tedious.

Instead, the command can remember which submodules it had trouble with, continue updating the ones it can, and report which ones had errors at the end. The user can run "git submodule update", find all the ones that need minor fixing (e.g. working tree was dirty) to fix them in a single pass. Then another "git submodule update" can be run to update all.

Note that the problematic submodules are skipped only when they are to be integrated with a safer value of submodule.<name>.update option, namely "checkout". Fixing a failure in a submodule that uses "rebase" or "merge" may need an involved conflict resolution by the user, and leaving too many submodules in states that need resolution would not reduce the mental burden on the user.

---
 git-submodule.sh            |   48 +++++++++++++--
 t/t7406-submodule-update.sh |  144 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 186 insertions(+), 6 deletions(-)
git-submodule.sh +42 -6
index d189a24..69780a8 100755
@@ -444,7 +444,8 @@ cmd_update()
 	fi
 
 	cloned_modules=
-	module_list "$@" |
+	module_list "$@" | {
+	err=
 	while read mode sha1 stage path
 	do
 		if test "$stage" = U
@@ -525,17 +526,52 @@ cmd_update()
 				;;
 			esac
 
-			(clear_local_git_env; cd "$path" && $command "$sha1") ||
-			die "Unable to $action '$sha1' in submodule path '$path'"
-			say "Submodule path '$path': $msg '$sha1'"
+			if (clear_local_git_env; cd "$path" && $command "$sha1")
+			then
+				say "Submodule path '$path': $msg '$sha1'"
+			else
+				case $action in
+				rebase|merge)
+					die_with_status 2 "Unable to $action '$sha1' in submodule path '$path'"
+					;;
+				*)
+					err="${err} \"Failed to $action in submodule path '$path'\""
+					continue
+					;;
+				esac
+			fi
 		fi
 
 		if test -n "$recursive"
 		then
-			(clear_local_git_env; cd "$path" && eval cmd_update "$orig_flags") ||
-			die "Failed to recurse into submodule path '$path'"
+			(clear_local_git_env; cd "$path" && eval cmd_update "$orig_flags")
+			res=$?
+			if test $res -gt 0
+			then
+				if test $res -eq 1
+				then
+					err="${err} \"Failed to recurse into submodule path '$path'\""
+					continue
+				else
+					die_with_status $res "Failed to recurse into submodule path '$path'"
+				fi
+			fi
 		fi
 	done
+
+	if test -n "$err"
+	then
+		OIFS=$IFS
+		IFS=';'
+		for e in $err
+		do
+			echo $e
+		done
+		IFS=$OIFS
+		die
+	fi
+
+	}
 }
 
 set_name_rev () {
t/t7406-submodule-update.sh +144 -0

148 lines not on this page · raw message

Signature · 1 line
1.7.6.rc0.14.g4fc55
#4 lore raw

Re: [PATCH v3 2/2] submodule update: continue when a checkout fails

Quoted 7 lines · Fredrik Gustafsson
git-submodule.sh +6 -0
+				*)
+					err="${err} \"Failed to $action in submodule path '$path'\""
...
+					err="${err} \"Failed to recurse into submodule path '$path'\""
+					continue
+				else
+					die_with_status $res "Failed to recurse into submodule path '$path'"

git-submodule.sh cmd_update() · line 566

Quoted 10 lines
git-submodule.sh +8 -0
+				fi
+			fi
 		fi
 	done
+
+	if test -n "$err"
+	then
+		OIFS=$IFS
+		IFS=';'
+		for e in $err

Hmm, I do not see anybody concatenating things with semicolons in between... Am I reading a right patch?

Also I am puzzled what the double-quotes around each error message are doing. Are they meant to be shown to the end user?

git-submodule.sh cmd_update() · line 568

Quoted 2 lines
git-submodule.sh +2 -0
+		do
+			echo $e

Should this go to the standard error output?

git-submodule.sh cmd_update() · line 571

Quoted 3 lines
git-submodule.sh +3 -0
+		done
+		IFS=$OIFS
+		die

Do we want to give an extra empty line here with "die"?

#5 lore raw

PATCH v4 2/2 submodule update: continue when a checkout fails

"git submodule update" stops at the first error and gives control back to the user. Only after the user fixes the problematic submodule and runs "git submodule update" again, the second error is found. And the user needs to repeat until all the problems are found and fixed one by one. This is tedious.

Instead, the command can remember which submodules it had trouble with, continue updating the ones it can, and report which ones had errors at the end. The user can run "git submodule update", find all the ones that need minor fixing (e.g. working tree was dirty) to fix them in a single pass. Then another "git submodule update" can be run to update all.

Note that the problematic submodules are skipped only when they are to be integrated with a safer value of submodule.<name>.update option, namely "checkout". Fixing a failure in a submodule that uses "rebase" or "merge" may need an involved conflict resolution by the user, and leaving too many submodules in states that need resolution would not reduce the mental burden on the user.

---
I'm sorry. You were reading a mix between two solutions that slipped through. Here's a corrected patch.

I'll changed it to be printed to standard error output instead.

die will not print an empty line (as of the first patch in this series) but I changed this to "exit 1" instead to be more clear.

 git-submodule.sh            |   50 +++++++++++++--
 t/t7406-submodule-update.sh |  144 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 188 insertions(+), 6 deletions(-)
git-submodule.sh +44 -6
index d189a24..eb5eebc 100755
@@ -444,7 +444,8 @@ cmd_update()
 	fi
 
 	cloned_modules=
-	module_list "$@" |
+	module_list "$@" | {
+	err=
 	while read mode sha1 stage path
 	do
 		if test "$stage" = U
@@ -525,17 +526,54 @@ cmd_update()
 				;;
 			esac
 
-			(clear_local_git_env; cd "$path" && $command "$sha1") ||
-			die "Unable to $action '$sha1' in submodule path '$path'"
-			say "Submodule path '$path': $msg '$sha1'"
+			if (clear_local_git_env; cd "$path" && $command "$sha1")
+			then
+				say "Submodule path '$path': $msg '$sha1'"
+			else
+				case $action in
+				rebase|merge)
+					die_with_status 2 "Unable to $action '$sha1' in submodule path '$path'"
+					;;
+				*)
+					err="${err};Failed to $action in submodule path '$path'"
+					continue
+					;;
+				esac
+			fi
 		fi
 
 		if test -n "$recursive"
 		then
-			(clear_local_git_env; cd "$path" && eval cmd_update "$orig_flags") ||
-			die "Failed to recurse into submodule path '$path'"
+			(clear_local_git_env; cd "$path" && eval cmd_update "$orig_flags")
+			res=$?
+			if test $res -gt 0
+			then
+				if test $res -eq 1
+				then
+					err="${err};Failed to recurse into submodule path '$path'"
+					continue
+				else
+					die_with_status $res "Failed to recurse into submodule path '$path'"
+				fi
+			fi
 		fi
 	done
+
+	if test -n "$err"
+	then
+		OIFS=$IFS
+		IFS=';'
+		for e in $err
+		do
+			if test -n "$e"
+			then
+				echo >&2 "$e"
+			fi
+		done
+		IFS=$OIFS
+		exit 1
+	fi
+	}
 }
 
 set_name_rev () {
t/t7406-submodule-update.sh +144 -0

148 lines not on this page · raw message

Signature · 1 line
1.7.6.rc0.14.gc8898.dirty