Skip to contentSkip to conversation

unblock and unignore SIGPIPE

PATCH In review Patrick Reynolds 4 participants 6 messages Jun 15, 2016

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 unblock and unignore SIGPIPE

Blocked and ignored signals -- but not caught signals -- are inherited across exec. Some callers with sloppy signal-handling behavior can call git with SIGPIPE blocked or ignored, even non-deterministically. When SIGPIPE is blocked or ignored, several git commands can run indefinitely, ignoring EPIPE returns from write() calls, even when the process that called them has gone away. Our specific case involved a pipe of git diff-tree output to a script that reads a limited amount of diff data.

In an ideal world, git would never be called with SIGPIPE blocked or ignored. But in the real world, several real potential callers, including Perl, Apache, and Unicorn, sometimes spawn subprocesses with SIGPIPE ignored. It is easier and more productive to harden git against this mistake than to clean it up in every potential parent process.

---
1. Merged Junio's work from pu: moved restore_sigpipe_to_default into git.c and restyled the new tests.
2. Moved the new tests into t0005. This meant switching back to git diff as our data generator, as the sample repo in t0005 doesn't have any files for git ls-files to output.
3. Squashed.

 git.c              | 22 ++++++++++++++++++++++
 t/t0005-signals.sh | 22 ++++++++++++++++++++++
 2 files changed, 44 insertions(+)
git.c +22 -0
index 210f1ae..0f03d56 100644
@@ -593,6 +593,26 @@ static int run_argv(int *argcp, const char ***argv)
 	return done_alias;
 }
 
+/*
+ * Many parts of Git have subprograms communicate via pipe, expect the
+ * upstream of the pipe to die with SIGPIPE and the downstream process
+ * even knows to check and handle EPIPE correctly.  Some third-party
+ * programs that ignore or block SIGPIPE for their own reason forget
+ * to restore SIGPIPE handling to the default before spawning Git and
+ * break this carefully orchestrated machinery.
+ *
+ * Restore the way SIGPIPE is handled to default, which is what we
+ * expect.
+ */
+static void restore_sigpipe_to_default(void)
+{
+	sigset_t unblock;
+
+	sigemptyset(&unblock);
+	sigaddset(&unblock, SIGPIPE);
+	sigprocmask(SIG_UNBLOCK, &unblock, NULL);
+	signal(SIGPIPE, SIG_DFL);
+}
 
 int main(int argc, char **av)
 {
@@ -612,6 +632,8 @@ int main(int argc, char **av)
 	 */
 	sanitize_stdfds();
 
+	restore_sigpipe_to_default();
+
 	git_setup_gettext();
 
 	trace_command_performance(argv);
t/t0005-signals.sh +22 -0
index 981437b..638a355 100755
@@ -27,4 +27,26 @@ test_expect_success !MINGW 'signals are propagated using shell convention' '
 	test_expect_code 143 git sigterm
 '
 
+large_git () {
+	for i in $(test_seq 1 100)
+	do
+		git diff --cached --binary || return
+	done
+}
+
+test_expect_success 'create blob' '
+	test-genrandom foo 16384 >file &&
+	git add file
+'
+
+test_expect_success 'a constipated git dies with SIGPIPE' '
+	OUT=$( ((large_git; echo $? 1>&3) | :) 3>&1 )
+	test "$OUT" -eq 141
+'
+
+test_expect_success 'a constipated git dies with SIGPIPE even if parent ignores it' '
+	OUT=$( ((trap "" PIPE; large_git; echo $? 1>&3) | :) 3>&1 )
+	test "$OUT" -eq 141
+'
+
 test_done
Signature · 1 line
2.0.1
#1 lore raw

Am 18.09.2014 um 18:57 schrieb Patrick Reynolds:

Quoted 46 lines

Blocked and ignored signals -- but not caught signals -- are inherited across exec. Some callers with sloppy signal-handling behavior can call git with SIGPIPE blocked or ignored, even non-deterministically. When SIGPIPE is blocked or ignored, several git commands can run indefinitely, ignoring EPIPE returns from write() calls, even when the process that called them has gone away. Our specific case involved a pipe of git diff-tree output to a script that reads a limited amount of diff data.

In an ideal world, git would never be called with SIGPIPE blocked or ignored. But in the real world, several real potential callers, including Perl, Apache, and Unicorn, sometimes spawn subprocesses with SIGPIPE ignored. It is easier and more productive to harden git against this mistake than to clean it up in every potential parent process.

---
1. Merged Junio's work from pu: moved restore_sigpipe_to_default into git.c and restyled the new tests.
2. Moved the new tests into t0005. This meant switching back to git diff as our data generator, as the sample repo in t0005 doesn't have any files for git ls-files to output.
3. Squashed.

 git.c              | 22 ++++++++++++++++++++++
 t/t0005-signals.sh | 22 ++++++++++++++++++++++
 2 files changed, 44 insertions(+)
git.c +10 -0
index 210f1ae..0f03d56 100644
@@ -593,6 +593,26 @@ static int run_argv(int *argcp, const char ***argv)
 	return done_alias;
 }
 
+/*
+ * Many parts of Git have subprograms communicate via pipe, expect the
+ * upstream of the pipe to die with SIGPIPE and the downstream process
+ * even knows to check and handle EPIPE correctly.  Some third-party
+ * programs that ignore or block SIGPIPE for their own reason forget
+ * to restore SIGPIPE handling to the default before spawning Git and
+ * break this carefully orchestrated machinery.
+ *
+ * Restore the way SIGPIPE is handled to default, which is what we
+ * expect.

git.c static int run_argv(int *argcp, const char ***argv) · line 615

Quoted 10 lines
git.c +10 -0
+ */
+static void restore_sigpipe_to_default(void)
+{
+	sigset_t unblock;
+
+	sigemptyset(&unblock);
+	sigaddset(&unblock, SIGPIPE);
+	sigprocmask(SIG_UNBLOCK, &unblock, NULL);
+	signal(SIGPIPE, SIG_DFL);
+}

This does not build on MinGW due to missing sigaddset() and sigprocmask(). I've a patch that adds dummies for them (but I ran out of time to complete it for submission). But then the test cases ...

git.c static int run_argv(int *argcp, const char ***argv)

Quoted 9 lines
git.c +9 -0
+test_expect_success 'a constipated git dies with SIGPIPE' '
+	OUT=$( ((large_git; echo $? 1>&3) | :) 3>&1 )
+	test "$OUT" -eq 141
+'
+
+test_expect_success 'a constipated git dies with SIGPIPE even if parent ignores it' '
+	OUT=$( ((trap "" PIPE; large_git; echo $? 1>&3) | :) 3>&1 )
+	test "$OUT" -eq 141
+'

... fail always because we neither get SIGPIPE (we don't have it on Windows) nor do we see a write error (e.g. EPIPE) when writing to the pipe. Should I protect these tests with !MINGW or would it be an option to drop these tests alltogether?

-- Hannes

#2 lore raw
In reply to Johannes Sixt
Quoted 14 lines · Johannes Sixt

git.c static int run_argv(int *argcp, const char ***argv) · line 615

Quoted 9 lines
git.c +9 -0
+static void restore_sigpipe_to_default(void)
+{
+	sigset_t unblock;
+
+	sigemptyset(&unblock);
+	sigaddset(&unblock, SIGPIPE);
+	sigprocmask(SIG_UNBLOCK, &unblock, NULL);
+	signal(SIGPIPE, SIG_DFL);
+}

This does not build on MinGW due to missing sigaddset() and sigprocmask(). I've a patch that adds dummies for them (but I ran out of time to complete it for submission). But then the test cases ...

Quoted 14 lines

t/t0005-signals.sh test_expect_success !MINGW 'signals are propagated using shell convention' ' · line 50

Quoted 9 lines
t/t0005-signals.sh +9 -0
+test_expect_success 'a constipated git dies with SIGPIPE' '
+	OUT=$( ((large_git; echo $? 1>&3) | :) 3>&1 )
+	test "$OUT" -eq 141
+'
+
+test_expect_success 'a constipated git dies with SIGPIPE even if parent ignores it' '
+	OUT=$( ((trap "" PIPE; large_git; echo $? 1>&3) | :) 3>&1 )
+	test "$OUT" -eq 141
+'

... fail always because we neither get SIGPIPE (we don't have it on Windows) nor do we see a write error (e.g. EPIPE) when writing to the pipe. Should I protect these tests with !MINGW or would it be an option to drop these tests alltogether?

Let's do !MINGW for now, unless somebody can think of a reason why this change and tests are a bad idea (e.g. "we are not in the business of preventing users from shooting themselves; have the users bug those who wrote the software that spawns us with SIGPIPE ignored", to which I am sympathetic to some degree but not very much because I am also a practical person).

Thanks.

#3 lore raw

PATCH 1/1 mingw.h: add dummy functions for sigset_t operations

Windows does not have POSIX-like signals, and so we ignore all operations on the non-existent signal mask machinery.

Do not turn sigemptyset into a function, but leave it a macro that erases the code in the argument because it is used to set sa_mask of a struct sigaction, but our dummy in mingw.h does not have that member.

---
 compat/mingw.h     | 7 ++++++-
 t/t0005-signals.sh | 4 ++--
 2 files changed, 8 insertions(+), 3 deletions(-)
compat/mingw.h +6 -1
index 0b5f2fe..0e42653 100644
@@ -69,7 +69,6 @@ struct sigaction {
 	sig_handler_t sa_handler;
 	unsigned sa_flags;
 };
-#define sigemptyset(x) (void)0
 #define SA_RESTART 0
 
 struct itimerval {
@@ -116,6 +115,12 @@ static inline int fcntl(int fd, int cmd, ...)
 }
 /* bash cannot reliably detect negative return codes as failure */
 #define exit(code) exit((code) & 0xff)
+#define sigemptyset(x) (void)0
+static inline int sigaddset(sigset_t *set, int signum)
+{ return 0; }
+#define SIG_UNBLOCK 0
+static inline int sigprocmask(int how, const sigset_t *set, sigset_t *oldset)
+{ return 0; }
 
 /*
  * simple adaptors
t/t0005-signals.sh +2 -2
index 638a355..aeea50c 100755
@@ -39,12 +39,12 @@ test_expect_success 'create blob' '
 	git add file
 '
 
-test_expect_success 'a constipated git dies with SIGPIPE' '
+test_expect_success !MINGW 'a constipated git dies with SIGPIPE' '
 	OUT=$( ((large_git; echo $? 1>&3) | :) 3>&1 )
 	test "$OUT" -eq 141
 '
 
-test_expect_success 'a constipated git dies with SIGPIPE even if parent ignores it' '
+test_expect_success !MINGW 'a constipated git dies with SIGPIPE even if parent ignores it' '
 	OUT=$( ((trap "" PIPE; large_git; echo $? 1>&3) | :) 3>&1 )
 	test "$OUT" -eq 141
 '
Signature · 1 line
2.0.0.12.gbcf935e