fsmonitor: only enable it in non-bare repositories

PATCH Superseded Johannes Schindelin via GitGitGadget 4 participants 7 messages Apr 29, 2021 – May 11, 2021

lore

Share & tools

lore

Johannes Schindelin via GitGitGadget

author

2021-04-29 07:46 UTC #0 lore raw

PATCH 1/1 fsmonitor: only enable it in non-bare repositories

From: Johannes Schindelin <[email protected]>

The entire point of the FSMonitor is to monitor the worktree changes in a more efficient manner than lstat()ing all worktree files every time we refresh the index.

But if there is no worktree, FSMonitor has nothing to monitor.

So let's ignore if an FSMonitor is configured (e.g. in ~/.gitconfig) and we're running in a repository without worktree.

Signed-off-by Johannes Schindelin <[email protected]>

---
fsmonitor: only enable it in non-bare repositories

Since I released Git for Windows v2.31.0, with brief interruption of two weeks, I enabled the built-in FSMonitor via my user config, and today was the first time I did anything in a bare repository. I was somewhat surprised that FSMonitor gave me trouble there, as the FSMonitor does not even make sense there...

This patch applies on top of jh/rfc-builtin-fsmonitor (not because it fixes a problem in the built-in FSMonitor, the bug existed for a long, long time before those patches, but because it would otherwise cause merge conflicts with that patch series).

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-942%2Fdscho%2Fbare-repositories-need-no-fsmonitor-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-942/dscho/bare-repositories-need-no-fsmonitor-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/942

 config.c | 6 ++++++
 1 file changed, 6 insertions(+)

config.c +6 -0

index 53e7dedc60de..fc5e744d81ca 100644
@@ -2515,6 +2515,12 @@ int git_config_get_max_percent_split_change(void)
 
 int repo_config_get_fsmonitor(struct repository *r)
 {
+	if (!r->worktree) {
+		/* FSMonitor makes no sense in bare repositories */
+		core_fsmonitor = 0;
+		return 1;
+	}
+
 	if (r->settings.use_builtin_fsmonitor > 0) {
 		core_fsmonitor = "(built-in daemon)";
 		return 1;

base-commit: 14d50074ff19e68e7a8d718b22d138882087bbc9

Signature · 1 line

gitgitgadget

Junio C Hamano

2021-04-30 13:55 UTC #1 lore raw

"Johannes Schindelin via GitGitGadget" <[email protected]>

Quoted 38 lines · writes:

From: Johannes Schindelin <[email protected]>
The entire point of the FSMonitor is to monitor the worktree changes in a more efficient manner than lstat()ing all worktree files every time we refresh the index.
But if there is no worktree, FSMonitor has nothing to monitor.
So let's ignore if an FSMonitor is configured (e.g. in ~/.gitconfig) and we're running in a repository without worktree.
Signed-off-by Johannes Schindelin <[email protected]>
---
fsmonitor: only enable it in non-bare repositories
Since I released Git for Windows v2.31.0, with brief interruption of two weeks, I enabled the built-in FSMonitor via my user config, and today was the first time I did anything in a bare repository. I was somewhat surprised that FSMonitor gave me trouble there, as the FSMonitor does not even make sense there...
This patch applies on top of jh/rfc-builtin-fsmonitor (not because it fixes a problem in the built-in FSMonitor, the bug existed for a long, long time before those patches, but because it would otherwise cause merge conflicts with that patch series).
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-942%2Fdscho%2Fbare-repositories-need-no-fsmonitor-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-942/dscho/bare-repositories-need-no-fsmonitor-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/942
 config.c | 6 ++++++
 1 file changed, 6 insertions(+)
config.c +0 -0
index 53e7dedc60de..fc5e744d81ca 100644

config.c int git_config_get_max_percent_split_change(void) · line 2520

Quoted 6 lines

config.c +3 -0

@@ -2515,6 +2515,12 @@ int git_config_get_max_percent_split_change(void)
 
 int repo_config_get_fsmonitor(struct repository *r)
 {
+	if (!r->worktree) {
+		/* FSMonitor makes no sense in bare repositories */
+		core_fsmonitor = 0;

Use NULL instead of integer 0 to mollify SP.

Quoted 8 lines

config.c +3 -0

+		return 1;
+	}
+
 	if (r->settings.use_builtin_fsmonitor > 0) {
 		core_fsmonitor = "(built-in daemon)";
 		return 1;

base-commit: 14d50074ff19e68e7a8d718b22d138882087bbc9

Johannes Schindelin via GitGitGadget

author

2021-05-03 09:11 UTC #2 lore raw

PATCH v2 1/1 fsmonitor: only enable it in non-bare repositories

In reply to Johannes Schindelin via GitGitGadget

From: Johannes Schindelin <[email protected]>

The entire point of the FSMonitor is to monitor the worktree changes in a more efficient manner than lstat()ing all worktree files every time we refresh the index.

But if there is no worktree, FSMonitor has nothing to monitor.

So let's ignore if an FSMonitor is configured (e.g. in ~/.gitconfig) and we're running in a repository without worktree.

Signed-off-by Johannes Schindelin <[email protected]>

---
fsmonitor: only enable it in non-bare repositories

For quite a while now, I run with the built-in FSMonitor via my user config. Happily, the only issue I ran into was that FSMonitor tried to run in a bare repository the other day. But an FSMonitor makes only sense if we have a worktree. So let's disable it automatically in bare repositories.

This patch applies near the top of jh/rfc-builtin-fsmonitor. I would like to keep it as a separate topic because the built-in FSMonitor did not introduce this bug. This bug has been in Git's FSMonitor feature for a long, long time.

Changes since v1:

* Using NULL instead of 0 (d'oh!)

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-942%2Fdscho%2Fbare-repositories-need-no-fsmonitor-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-942/dscho/bare-repositories-need-no-fsmonitor-v2
Pull-Request: https://github.com/gitgitgadget/git/pull/942

Range-diff vs v1:

1: 3a93be538d33 ! 1: 95333bbf45d5 fsmonitor: only enable it in non-bare repositories

     @@ config.c: int git_config_get_max_percent_split_change(void)
       {
      +	if (!r->worktree) {
      +		/* FSMonitor makes no sense in bare repositories */
     -+		core_fsmonitor = 0;
     ++		core_fsmonitor = NULL;
      +		return 1;
      +	}
      +

 config.c | 6 ++++++
 1 file changed, 6 insertions(+)

config.c +6 -0

index 53e7dedc60de..90ce4c23d3dc 100644
@@ -2515,6 +2515,12 @@ int git_config_get_max_percent_split_change(void)
 
 int repo_config_get_fsmonitor(struct repository *r)
 {
+	if (!r->worktree) {
+		/* FSMonitor makes no sense in bare repositories */
+		core_fsmonitor = NULL;
+		return 1;
+	}
+
 	if (r->settings.use_builtin_fsmonitor > 0) {
 		core_fsmonitor = "(built-in daemon)";
 		return 1;

base-commit: 14d50074ff19e68e7a8d718b22d138882087bbc9

Signature · 1 line

gitgitgadget

Ævar Arnfjörð Bjarmason

2021-05-03 13:59 UTC #3 lore raw

In reply to Johannes Schindelin via GitGitGadget

Quoted 51 lines · Johannes Schindelin via GitGitGadget

From: Johannes Schindelin <[email protected]>
The entire point of the FSMonitor is to monitor the worktree changes in a more efficient manner than lstat()ing all worktree files every time we refresh the index.
But if there is no worktree, FSMonitor has nothing to monitor.
So let's ignore if an FSMonitor is configured (e.g. in ~/.gitconfig) and we're running in a repository without worktree.
Signed-off-by Johannes Schindelin <[email protected]>
---
fsmonitor: only enable it in non-bare repositories
Since I released Git for Windows v2.31.0, with brief interruption of two weeks, I enabled the built-in FSMonitor via my user config, and today was the first time I did anything in a bare repository. I was somewhat surprised that FSMonitor gave me trouble there, as the FSMonitor does not even make sense there...
This patch applies on top of jh/rfc-builtin-fsmonitor (not because it fixes a problem in the built-in FSMonitor, the bug existed for a long, long time before those patches, but because it would otherwise cause merge conflicts with that patch series).
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-942%2Fdscho%2Fbare-repositories-need-no-fsmonitor-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-942/dscho/bare-repositories-need-no-fsmonitor-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/942
 config.c | 6 ++++++
 1 file changed, 6 insertions(+)
config.c +6 -0
index 53e7dedc60de..fc5e744d81ca 100644
@@ -2515,6 +2515,12 @@ int git_config_get_max_percent_split_change(void)
 
 int repo_config_get_fsmonitor(struct repository *r)
 {
+	if (!r->worktree) {
+		/* FSMonitor makes no sense in bare repositories */
+		core_fsmonitor = 0;
+		return 1;
+	}
+
 	if (r->settings.use_builtin_fsmonitor > 0) {
 		core_fsmonitor = "(built-in daemon)";
 		return 1;

Quoted 1 line

base-commit: 14d50074ff19e68e7a8d718b22d138882087bbc9

This is surely a correct fix for now, but wouldn't it in the future also be useful to run it in bare repositories e.g. to be able cache lookups for non-existing loose objects?

Jeff Hostetler

2021-05-03 17:27 UTC #4 lore raw

Quoted 16 lines · Ævar Arnfjörð Bjarmason

Quoted 13 lines · Johannes Schindelin via GitGitGadget
From: Johannes Schindelin <[email protected]>
The entire point of the FSMonitor is to monitor the worktree changes in a more efficient manner than lstat()ing all worktree files every time we refresh the index.
But if there is no worktree, FSMonitor has nothing to monitor.
So let's ignore if an FSMonitor is configured (e.g. in ~/.gitconfig) and we're running in a repository without worktree.
Signed-off-by Johannes Schindelin <[email protected]>
---

...

Quoted 7 lines

Quoted 2 lines
base-commit: 14d50074ff19e68e7a8d718b22d138882087bbc9
This is surely a correct fix for now, but wouldn't it in the future also be useful to run it in bare repositories e.g. to be able cache lookups for non-existing loose objects?

No, the FSMonitor feature only expects data for paths within the working directory. (And is independent of whether the FS change data is provided by my fsmonitor--daemon or provided by a hook-based provider, such as Watchman.) The FSMonitor feature uses that data to shortcut scans of the working directory.

There is no interaction with the contents of the .git/objects directory and I'm not sure how that would work.

Jeff

Ævar Arnfjörð Bjarmason

2021-05-04 09:50 UTC #5 lore raw

Quoted 25 lines · Jeff Hostetler

Quoted 15 lines · Ævar Arnfjörð Bjarmason
Quoted 13 lines · Johannes Schindelin via GitGitGadget
From: Johannes Schindelin <[email protected]>
The entire point of the FSMonitor is to monitor the worktree changes in a more efficient manner than lstat()ing all worktree files every time we refresh the index.
But if there is no worktree, FSMonitor has nothing to monitor.
So let's ignore if an FSMonitor is configured (e.g. in ~/.gitconfig) and we're running in a repository without worktree.
Signed-off-by Johannes Schindelin <[email protected]>
---
...
Quoted 7 lines
Quoted 2 lines
base-commit: 14d50074ff19e68e7a8d718b22d138882087bbc9
This is surely a correct fix for now, but wouldn't it in the future also
be useful to run it in bare repositories e.g. to be able cache lookups for non-existing loose objects?

Quoted 5 lines

No, the FSMonitor feature only expects data for paths within the working directory. (And is independent of whether the FS change data is provided by my fsmonitor--daemon or provided by a hook-based provider, such as Watchman.) The FSMonitor feature uses that data to shortcut scans of the working directory.

Indeed, hence "in the future". I'm not suggesting that it'll do anything useful by watching anything in the .git directory now, but that it might be an interesting thing to explore.

Quoted 2 lines

There is no interaction with the contents of the .git/objects directory and I'm not sure how that would work.

We'd watch .git/objects and .git/objects/{aa..ff}, then when about to check for a loose object we'd avoid hitting the FS.

I don't know how useful that is post-61c7711cfea (sha1-file: use loose object cache for quick existence check, 2018-11-12), but e.g. on NFS this sort of thing still mattered. I had a "bigger hammer" approach with
[1] that ran (and still does, I believe) on a big corporate installation.

More generally, if you strace .git access during repo operations you'll find we're doing all sorts of existence checks etc. all the time. Loose objects, refs, seeing what packs there are (better with the MIDX, but do we still fall back?) etc. If we had up-to-date inotify/fsmonitor info we could ask the daemon about it.

1. https://lore.kernel.org/git/[email protected]/

Jeff Hostetler

2021-05-11 19:24 UTC #6 lore raw

Quoted 49 lines · Ævar Arnfjörð Bjarmason

Quoted 25 lines · Jeff Hostetler
Quoted 15 lines · Ævar Arnfjörð Bjarmason
Quoted 13 lines · Johannes Schindelin via GitGitGadget
From: Johannes Schindelin <[email protected]>
The entire point of the FSMonitor is to monitor the worktree changes in a more efficient manner than lstat()ing all worktree files every time we refresh the index.
But if there is no worktree, FSMonitor has nothing to monitor.
So let's ignore if an FSMonitor is configured (e.g. in ~/.gitconfig) and we're running in a repository without worktree.
Signed-off-by Johannes Schindelin <[email protected]>
---
...
Quoted 7 lines
Quoted 2 lines
base-commit: 14d50074ff19e68e7a8d718b22d138882087bbc9
This is surely a correct fix for now, but wouldn't it in the future also
be useful to run it in bare repositories e.g. to be able cache lookups for non-existing loose objects?
Quoted 5 lines
No, the FSMonitor feature only expects data for paths within the working directory. (And is independent of whether the FS change data is provided by my fsmonitor--daemon or provided by a hook-based provider, such as Watchman.) The FSMonitor feature uses that data to shortcut scans of the working directory.
Indeed, hence "in the future". I'm not suggesting that it'll do anything useful by watching anything in the .git directory now, but that it might be an interesting thing to explore.
Quoted 2 lines
There is no interaction with the contents of the .git/objects directory and I'm not sure how that would work.
We'd watch .git/objects and .git/objects/{aa..ff}, then when about to check for a loose object we'd avoid hitting the FS.
I don't know how useful that is post-61c7711cfea (sha1-file: use loose object cache for quick existence check, 2018-11-12), but e.g. on NFS this sort of thing still mattered. I had a "bigger hammer" approach with
[1] that ran (and still does, I believe) on a big corporate installation.

Quoted 8 lines

More generally, if you strace .git access during repo operations you'll find we're doing all sorts of existence checks etc. all the time. Loose objects, refs, seeing what packs there are (better with the MIDX, but do we still fall back?) etc. If we had up-to-date inotify/fsmonitor info we could ask the daemon about it.
1. https://lore.kernel.org/git/[email protected]/

Interesting.

That's certainly something to look into later. I know there is code in the object lookup code to rescan/reload the packfiles or loose objects (under the assumption that another process just created a new packfile (and after our process loaded the packed-git list)). Such a daemon might be helpful to improve something like that.

But I can't think about any of that right now. I'd like to finish the current fsmonitor--daemon patch series and let it settle down before starting to think about an orthogonal use case such as this.

Thanks
Jeff