Skip to contentSkip to conversation

fsmonitor: fix two bugs in watchman hook retry path

PATCH In review Paul Tarjan via GitGitGadget 4 participants 8 messages Jan 26 – Feb 28

lore
Share & tools
lore

Paul Tarjan via GitGitGadget

author
#0 lore raw

PATCH 1/1 fsmonitor: fix two bugs in watchman hook retry path

From: Claude <[email protected]>

The is_work_tree_watched() function in fsmonitor-watchman.sample has two bugs:

1. Wrong variable in error check: After calling watchman_clock(), the result is stored in $o, but the code checks $output->{error} instead of $o->{error}. This means errors from the clock command are silently ignored.

2. Double output violates protocol: When the retry path triggers (the directory wasn't initially watched), output_result() is called with the "/" flag, then launch_watchman() is called recursively which calls output_result() again. This outputs two clock tokens to stdout, but git's fsmonitor v2 protocol expects exactly one response.

Fix #1 by checking $o->{error} after watchman_clock().

Fix #2 by removing the recursive launch_watchman() call. The "/" "everything is dirty" flag already tells git to do a full scan, and git will call the hook again on the next invocation with a valid clock token.

Apply the same fixes to the test helper scripts in t/t7519/.

Signed-off-by Paul Tarjan <[email protected]>

---
fsmonitor-watchman: fix variable reference and remove redundant code

The is_work_tree_watched() function in fsmonitor-watchman.sample has two bugs:

1. Wrong variable in error check: After calling watchman_clock(), the result is stored in $o, but the code checks $output->{error} instead of $o->{error}. This means errors from the clock command are silently ignored.

2. Double output violates protocol: When the retry path triggers (the directory wasn't initially watched), output_result() is called with the "/" flag, then launch_watchman() is called recursively which calls output_result() again. This outputs two clock tokens to stdout, but git's fsmonitor v2 protocol expects exactly one response.

Fix #1 by checking $o->{error} after watchman_clock().

Fix #2 by removing the recursive launch_watchman() call. The "/" "everything is dirty" flag already tells git to do a full scan, and git will call the hook again on the next invocation with a valid clock token.

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-2180%2Fptarjan%2Fclaude%2Ffix-watchman-query-bug-sfbIw-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-2180/ptarjan/claude/fix-watchman-query-bug-sfbIw-v1
Pull-Request: https://github.com/git/git/pull/2180

 t/t7519/fsmonitor-watchman                | 1 -
 t/t7519/fsmonitor-watchman-v2             | 5 +----
 templates/hooks/fsmonitor-watchman.sample | 5 +----
 3 files changed, 2 insertions(+), 9 deletions(-)
t/t7519/fsmonitor-watchman +0 -1
index 264b9daf83..907bd1533c 100755
@@ -109,7 +109,6 @@ sub launch_watchman {
 		close $fh;
 
 		print "/\0";
-		eval { launch_watchman() };
 		exit 0;
 	}
t/t7519/fsmonitor-watchman-v2 +1 -4
index 14ed0aa42d..2c59120c3d 100755
@@ -141,15 +141,12 @@ sub is_work_tree_watched {
 		# Watchman query just to get it over with now so we won't pay
 		# the cost in git to look up each individual file.
 		my $o = watchman_clock();
-		$error = $output->{error};
+		$error = $o->{error};
 
 		die "Watchman: $error.\n" .
 		"Falling back to scanning...\n" if $error;
 
 		output_result($o->{clock}, ("/"));
-		$last_update_token = $o->{clock};
-
-		eval { launch_watchman() };
 		return 0;
 	}
templates/hooks/fsmonitor-watchman.sample +1 -4
index 23e856f5de..21c81b6804 100755
@@ -142,15 +142,12 @@ sub is_work_tree_watched {
 		# Watchman query just to get it over with now so we won't pay
 		# the cost in git to look up each individual file.
 		my $o = watchman_clock();
-		$error = $output->{error};
+		$error = $o->{error};
 
 		die "Watchman: $error.\n" .
 		"Falling back to scanning...\n" if $error;
 
 		output_result($o->{clock}, ("/"));
-		$last_update_token = $o->{clock};
-
-		eval { launch_watchman() };
 		return 0;
 	}

base-commit: 68cb7f9e92a5d8e9824f5b52ac3d0a9d8f653dbe

Signature · 1 line
gitgitgadget
#2 lore raw

Re: [PATCH] fsmonitor-watchman: fix variable reference and remove redundant code

Friendly ping on this patch. Happy to address any feedback.

Thanks,
Paul

#3 lore raw
In reply to Paul Tarjan via GitGitGadget
Quoted 116 lines · Paul Tarjan via GitGitGadget

From: Claude <[email protected]>

The is_work_tree_watched() function in fsmonitor-watchman.sample has two bugs:

1. Wrong variable in error check: After calling watchman_clock(), the result is stored in $o, but the code checks $output->{error} instead of $o->{error}. This means errors from the clock command are silently ignored.

2. Double output violates protocol: When the retry path triggers (the directory wasn't initially watched), output_result() is called with the "/" flag, then launch_watchman() is called recursively which calls output_result() again. This outputs two clock tokens to stdout, but git's fsmonitor v2 protocol expects exactly one response.

Fix #1 by checking $o->{error} after watchman_clock().

Fix #2 by removing the recursive launch_watchman() call. The "/" "everything is dirty" flag already tells git to do a full scan, and git will call the hook again on the next invocation with a valid clock token.

Apply the same fixes to the test helper scripts in t/t7519/.

Signed-off-by Paul Tarjan <[email protected]>

---
fsmonitor-watchman: fix variable reference and remove redundant code

The is_work_tree_watched() function in fsmonitor-watchman.sample has two bugs:

1. Wrong variable in error check: After calling watchman_clock(), the result is stored in $o, but the code checks $output->{error} instead of $o->{error}. This means errors from the clock command are silently ignored.

2. Double output violates protocol: When the retry path triggers (the directory wasn't initially watched), output_result() is called with the "/" flag, then launch_watchman() is called recursively which calls output_result() again. This outputs two clock tokens to stdout, but git's fsmonitor v2 protocol expects exactly one response.

Fix #1 by checking $o->{error} after watchman_clock().

Fix #2 by removing the recursive launch_watchman() call. The "/" "everything is dirty" flag already tells git to do a full scan, and git will call the hook again on the next invocation with a valid clock token.

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-2180%2Fptarjan%2Fclaude%2Ffix-watchman-query-bug-sfbIw-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-2180/ptarjan/claude/fix-watchman-query-bug-sfbIw-v1
Pull-Request: https://github.com/git/git/pull/2180

 t/t7519/fsmonitor-watchman                | 1 -
 t/t7519/fsmonitor-watchman-v2             | 5 +----
 templates/hooks/fsmonitor-watchman.sample | 5 +----
 3 files changed, 2 insertions(+), 9 deletions(-)
t/t7519/fsmonitor-watchman +0 -1
index 264b9daf83..907bd1533c 100755
@@ -109,7 +109,6 @@ sub launch_watchman {
                close $fh;
 
                print "/\0";
-               eval { launch_watchman() };
                exit 0;
        }
t/t7519/fsmonitor-watchman-v2 +1 -4
index 14ed0aa42d..2c59120c3d 100755
@@ -141,15 +141,12 @@ sub is_work_tree_watched {
                # Watchman query just to get it over with now so we won't pay
                # the cost in git to look up each individual file.
                my $o = watchman_clock();
-               $error = $output->{error};
+               $error = $o->{error};
 
                die "Watchman: $error.\n" .
                "Falling back to scanning...\n" if $error;
 
                output_result($o->{clock}, ("/"));
-               $last_update_token = $o->{clock};
-
-               eval { launch_watchman() };
                return 0;
        }
templates/hooks/fsmonitor-watchman.sample +1 -4
index 23e856f5de..21c81b6804 100755
@@ -142,15 +142,12 @@ sub is_work_tree_watched {
                # Watchman query just to get it over with now so we won't pay
                # the cost in git to look up each individual file.
                my $o = watchman_clock();
-               $error = $output->{error};
+               $error = $o->{error};
 
                die "Watchman: $error.\n" .
                "Falling back to scanning...\n" if $error;
 
                output_result($o->{clock}, ("/"));
-               $last_update_token = $o->{clock};
-
-               eval { launch_watchman() };
                return 0;
        }
Quoted 4 lines

base-commit: 68cb7f9e92a5d8e9824f5b52ac3d0a9d8f653dbe --
gitgitgadget

Although I don't have much experience with the Watchman backend, the fixes look correct to me.

One suggestion: it seems we can make the code even cleaner by removing $retry and its associated logic, as they appear to be no longer necessary after these changes.

--
Koji Nakamaru

#6 lore raw

Re: [PATCH] fsmonitor-watchman: fix variable reference and remove redundant code

Quoted 2 lines · brian m. carlson

This looks like it was written by an LLM. Is that the case, and if so, how does it line up with https://git-scm.com/docs/SubmittingPatches#ai?

I found these bugs during code review at my company where we use the watchman hook. The $output vs $o mixup and the double output_result() from the recursive call were both causing real issues. I used Claude to help with the mechanics of formatting and submitting the patch, and it ended up as the commit author, which was a mistake on my part. I'll fix that in v2 with myself as the author.

Thanks for flagging it,
Paul

#4 lore raw

Re: [PATCH] fsmonitor-watchman: fix variable reference and remove redundant code

In reply to Koji Nakamaru
Quoted 6 lines · Koji Nakamaru

Although I don't have much experience with the Watchman backend, the fixes look correct to me.

One suggestion: it seems we can make the code even cleaner by removing $retry and its associated logic, as they appear to be no longer necessary after these changes.

Good point. $retry only existed to prevent the infinite recursion from the recursive launch_watchman() call, so now that we've removed that call it's dead code. Cleaned it up in v2.

Thanks for the review,
Paul

author
#7 lore raw

PATCH v2 1/1 fsmonitor-watchman: fix variable reference and remove redundant code

In reply to Paul Tarjan via GitGitGadget

From: Paul Tarjan <[email protected]>

The is_work_tree_watched() function in fsmonitor-watchman.sample has two bugs:

1. Wrong variable in error check: After calling watchman_clock(), the result is stored in $o, but the code checks $output->{error} instead of $o->{error}. This means errors from the clock command are silently ignored.

2. Double output violates protocol: When the retry path triggers (the directory wasn't initially watched), output_result() is called with the "/" flag, then launch_watchman() is called recursively which calls output_result() again. This outputs two clock tokens to stdout, but git's fsmonitor v2 protocol expects exactly one response.

Fix #1 by checking $o->{error} after watchman_clock().

Fix #2 by removing the recursive launch_watchman() call. The "/" "everything is dirty" flag already tells git to do a full scan, and git will call the hook again on the next invocation with a valid clock token.

With the recursive call removed, the $retry guard is no longer needed since it only existed to prevent infinite recursion. Remove it.

Apply the same fixes to the test helper scripts in t/t7519/.

Signed-off-by Paul Tarjan <[email protected]>

---
fsmonitor-watchman: fix variable reference and remove redundant code

fsmonitor-watchman: fix variable reference and remove redundant code

The is_work_tree_watched() function in fsmonitor-watchman.sample has two bugs:

1. Wrong variable in error check: After calling watchman_clock(), the result is stored in $o, but the code checks $output->{error} instead of $o->{error}. This means errors from the clock command are silently ignored.

2. Double output violates protocol: When the retry path triggers (the directory wasn't initially watched), output_result() is called with the "/" flag, then launch_watchman() is called recursively which calls output_result() again. This outputs two clock tokens to stdout, but git's fsmonitor v2 protocol expects exactly one response.

Fix #1 by checking $o->{error} after watchman_clock().

Fix #2 by removing the recursive launch_watchman() call. The "/" "everything is dirty" flag already tells git to do a full scan, and git will call the hook again on the next invocation with a valid clock token.

Apply the same fixes to the test helper scripts in t/t7519/.

Changes since v1:

* Removed $retry variable and associated logic, which only existed to prevent infinite recursion from the now-removed recursive launch_watchman() call
* Fixed commit authorship

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-2180%2Fptarjan%2Fclaude%2Ffix-watchman-query-bug-sfbIw-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-2180/ptarjan/claude/fix-watchman-query-bug-sfbIw-v2
Pull-Request: https://github.com/git/git/pull/2180

Range-diff vs v1:

1: 116d26287f ! 1: b9e00b8ab5 fsmonitor: fix two bugs in watchman hook retry path
@@
## Metadata ##
-Author: Claude <[email protected]>
+Author: Paul Tarjan <[email protected]>

## Commit message ##
- fsmonitor: fix two bugs in watchman hook retry path
+ fsmonitor-watchman: fix variable reference and remove redundant code

The is_work_tree_watched() function in fsmonitor-watchman.sample has two bugs:
@@ Commit message
git will call the hook again on the next invocation with a valid clock token.

+ With the recursive call removed, the $retry guard is no longer needed
+ since it only existed to prevent infinite recursion. Remove it.
+
Apply the same fixes to the test helper scripts in t/t7519/.

Signed-off-by Paul Tarjan <[email protected]>

## t/t7519/fsmonitor-watchman ##
+@@ t/t7519/fsmonitor-watchman: if ($^O =~ 'msys' || $^O =~ 'cygwin') {
+ $git_work_tree = Cwd::cwd();
+ }
+
+-my $retry = 1;
+-
+ launch_watchman();
+
+ sub launch_watchman {
+@@ t/t7519/fsmonitor-watchman: sub launch_watchman {
+
+ my $o = $json_pkg->new->utf8->decode($response);
+
+- if ($retry > 0 and $o->{error} and $o->{error} =~ m/unable to resolve root .* directory (.*) is not watched/) {
++ if ($o->{error} and $o->{error} =~ m/unable to resolve root .* directory (.*) is not watched/) {
+ print STDERR "Adding '$git_work_tree' to watchman's watch list.\n";
+- $retry--;
+ qx/watchman watch "$git_work_tree"/;
+ die "Failed to make watchman watch '$git_work_tree'.\n" .
+ "Falling back to scanning...\n" if $? != 0;
@@ t/t7519/fsmonitor-watchman: sub launch_watchman {
close $fh;

@@ t/t7519/fsmonitor-watchman: sub launch_watchman {

## t/t7519/fsmonitor-watchman-v2 ##
+@@ t/t7519/fsmonitor-watchman-v2: if ($version ne 2) {
+
+ my $git_work_tree = get_working_dir();
+
+-my $retry = 1;
+-
+ my $json_pkg;
+ eval {
+ require JSON::XS;
+@@ t/t7519/fsmonitor-watchman-v2: sub watchman_query {
+ sub is_work_tree_watched {
+ my ($output) = @_;
+ my $error = $output->{error};
+- if ($retry > 0 and $error and $error =~ m/unable to resolve root .* directory (.*) is not watched/) {
+- $retry--;
++ if ($error and $error =~ m/unable to resolve root .* directory (.*) is not watched/) {
+ my $response = qx/watchman watch "$git_work_tree"/;
+ die "Failed to make watchman watch '$git_work_tree'.\n" .
+ "Falling back to scanning...\n" if $? != 0;
@@ t/t7519/fsmonitor-watchman-v2: sub is_work_tree_watched {
# Watchman query just to get it over with now so we won't pay # the cost in git to look up each individual file.
@@ t/t7519/fsmonitor-watchman-v2: sub is_work_tree_watched {

## templates/hooks/fsmonitor-watchman.sample ##
+@@ templates/hooks/fsmonitor-watchman.sample: if ($version ne 2) { +
+ my $git_work_tree = get_working_dir();
+
+-my $retry = 1;
+-
+ my $json_pkg;
+ eval {
+ require JSON::XS;
+@@ templates/hooks/fsmonitor-watchman.sample: sub watchman_query {
+ sub is_work_tree_watched {
+ my ($output) = @_;
+ my $error = $output->{error};
+- if ($retry > 0 and $error and $error =~ m/unable to resolve root .* directory (.*) is not watched/) {
+- $retry--;
++ if ($error and $error =~ m/unable to resolve root .* directory (.*) is not watched/) {
+ my $response = qx/watchman watch "$git_work_tree"/;
+ die "Failed to make watchman watch '$git_work_tree'.\n" .
+ "Falling back to scanning...\n" if $? != 0;
@@ templates/hooks/fsmonitor-watchman.sample: sub is_work_tree_watched {
# Watchman query just to get it over with now so we won't pay # the cost in git to look up each individual file.

 t/t7519/fsmonitor-watchman                |  6 +-----
 t/t7519/fsmonitor-watchman-v2             | 10 ++--------
 templates/hooks/fsmonitor-watchman.sample | 10 ++--------
 3 files changed, 5 insertions(+), 21 deletions(-)
t/t7519/fsmonitor-watchman +1 -5
index 264b9daf83..bcc055c1e0 100755
@@ -38,8 +38,6 @@ if ($^O =~ 'msys' || $^O =~ 'cygwin') {
 	$git_work_tree = Cwd::cwd();
 }
 
-my $retry = 1;
-
 launch_watchman();
 
 sub launch_watchman {

@@ -92,9 +90,8 @@ sub launch_watchman {
 
 	my $o = $json_pkg->new->utf8->decode($response);
 
-	if ($retry > 0 and $o->{error} and $o->{error} =~ m/unable to resolve root .* directory (.*) is not watched/) {
+	if ($o->{error} and $o->{error} =~ m/unable to resolve root .* directory (.*) is not watched/) {
 		print STDERR "Adding '$git_work_tree' to watchman's watch list.\n";
-		$retry--;
 		qx/watchman watch "$git_work_tree"/;
 		die "Failed to make watchman watch '$git_work_tree'.\n" .
 		    "Falling back to scanning...\n" if $? != 0;

@@ -109,7 +106,6 @@ sub launch_watchman {
 		close $fh;
 
 		print "/\0";
-		eval { launch_watchman() };
 		exit 0;
 	}
t/t7519/fsmonitor-watchman-v2 +2 -8
index 14ed0aa42d..368604c278 100755
@@ -29,8 +29,6 @@ if ($version ne 2) {
 
 my $git_work_tree = get_working_dir();
 
-my $retry = 1;
-
 my $json_pkg;
 eval {
 	require JSON::XS;

@@ -122,8 +120,7 @@ sub watchman_query {
 sub is_work_tree_watched {
 	my ($output) = @_;
 	my $error = $output->{error};
-	if ($retry > 0 and $error and $error =~ m/unable to resolve root .* directory (.*) is not watched/) {
-		$retry--;
+	if ($error and $error =~ m/unable to resolve root .* directory (.*) is not watched/) {
 		my $response = qx/watchman watch "$git_work_tree"/;
 		die "Failed to make watchman watch '$git_work_tree'.\n" .
 		    "Falling back to scanning...\n" if $? != 0;

@@ -141,15 +138,12 @@ sub is_work_tree_watched {
 		# Watchman query just to get it over with now so we won't pay
 		# the cost in git to look up each individual file.
 		my $o = watchman_clock();
-		$error = $output->{error};
+		$error = $o->{error};
 
 		die "Watchman: $error.\n" .
 		"Falling back to scanning...\n" if $error;
 
 		output_result($o->{clock}, ("/"));
-		$last_update_token = $o->{clock};
-
-		eval { launch_watchman() };
 		return 0;
 	}
templates/hooks/fsmonitor-watchman.sample +2 -8
index 23e856f5de..429e0a51c1 100755
@@ -29,8 +29,6 @@ if ($version ne 2) {
 
 my $git_work_tree = get_working_dir();
 
-my $retry = 1;
-
 my $json_pkg;
 eval {
 	require JSON::XS;

@@ -123,8 +121,7 @@ sub watchman_query {
 sub is_work_tree_watched {
 	my ($output) = @_;
 	my $error = $output->{error};
-	if ($retry > 0 and $error and $error =~ m/unable to resolve root .* directory (.*) is not watched/) {
-		$retry--;
+	if ($error and $error =~ m/unable to resolve root .* directory (.*) is not watched/) {
 		my $response = qx/watchman watch "$git_work_tree"/;
 		die "Failed to make watchman watch '$git_work_tree'.\n" .
 		    "Falling back to scanning...\n" if $? != 0;

@@ -142,15 +139,12 @@ sub is_work_tree_watched {
 		# Watchman query just to get it over with now so we won't pay
 		# the cost in git to look up each individual file.
 		my $o = watchman_clock();
-		$error = $output->{error};
+		$error = $o->{error};
 
 		die "Watchman: $error.\n" .
 		"Falling back to scanning...\n" if $error;
 
 		output_result($o->{clock}, ("/"));
-		$last_update_token = $o->{clock};
-
-		eval { launch_watchman() };
 		return 0;
 	}

base-commit: 68cb7f9e92a5d8e9824f5b52ac3d0a9d8f653dbe

Signature · 1 line
gitgitgadget