Fast git status via a file system watcher

t/t7519-status-fsmonitor.sh new file +1 -0

new file mode 100755
index 0000000000..395db46d55
@@ -0,0 +1,158 @@
+#!/bin/sh

t/t7519-status-fsmonitor.sh new file +4 -0

+dirty_repo () {
+       : >untracked
+       : >dir1/untracked
+       : >dir2/untracked

t/t7519-status-fsmonitor.sh new file +11 -0

+# The test query-fsmonitor hook proc will output a marker file we can use to
+# ensure the hook was actually used to generate the correct results.
+
+test_expect_success 'setup' '
+       mkdir -p .git/hooks &&
+       write_script .git/hooks/query-fsmonitor<<-\EOF &&
+       if [ $1 -ne 1 ]
+       then
+               echo -e "Unsupported query-fsmonitor hook version.\n" >&2
+               exit 1;
+       fi

t/t7519-status-fsmonitor.sh new file +11 -0

+       EOF
+       : >tracked &&
+       : >modified &&
+       mkdir dir1 &&
+       : >dir1/tracked &&
+       : >dir1/modified &&
+       mkdir dir2 &&
+       : >dir2/tracked &&
+       : >dir2/modified &&
+       git add . &&
+       test_tick &&

t/t7519-status-fsmonitor.sh new file +19 -0

+       clean_repo &&
+       git status &&
+       test_path_is_missing marker &&
+       dirty_repo &&
+       git status >output &&
+       test_path_is_file marker &&
+       test_i18ncmp expect output
+'
+
+# Ensure commands that call refresh_index() to move the index back in time
+# properly invalidate the fsmonitor cache
+
+test_expect_success 'refresh_index() invalidates fsmonitor cache' '
+       git config core.fsmonitor true &&
+       git config core.untrackedcache true &&
+       clean_repo &&
+       git status &&
+       test_path_is_missing marker &&
+       dirty_repo &&

t/t7519-status-fsmonitor.sh new file +7 -0

+       git config core.fsmonitor true &&
+       git config core.untrackedcache true &&
+       write_script .git/hooks/query-fsmonitor<<-\EOF &&
+       :>marker
+       EOF
+       clean_repo &&
+       git status &&

t/t7519-status-fsmonitor.sh new file +9 -1

+       printf "dir1/modified\0"
+       EOF
+       git status >output &&
+       test_path_is_file marker &&
+       test_i18ngrep "Changes not staged for commit:" output &&
+       test_i18ngrep "Untracked files:" output
+'
+
+test_done
--

2.13.0.windows.1.9.gc201c67b71

On Thu, May 25, 2017 at 8:36 PM, Ben Peart <[email protected]> wrote:

[...]

Quoted 7 lines

t/t7519-status-fsmonitor.sh new file +1 -0

new file mode 100755
index 0000000000..395db46d55
@@ -0,0 +1,158 @@
+#!/bin/sh

t/t7519-status-fsmonitor.sh line 11

Quoted 10 lines

t/t7519-status-fsmonitor.sh new file +10 -0

+
+test_description='git status with file system watcher'
+
+. ./test-lib.sh
+
+clean_repo () {
+       git reset --hard HEAD
+       git clean -fd
+       rm -f marker
+}

Maybe link all the commands in the function with "&&".

Quoted 4 lines

t/t7519-status-fsmonitor.sh new file +4 -0

+dirty_repo () {
+       : >untracked
+       : >dir1/untracked
+       : >dir2/untracked

t/t7519-status-fsmonitor.sh line 26

Quoted 10 lines

t/t7519-status-fsmonitor.sh new file +10 -0

+       echo 1 >modified
+       echo 2 >dir1/modified
+       echo 3 >dir2/modified
+       echo 4 >new
+       echo 5 >dir1/new
+       echo 6 >dir2/new
+       git add new
+       git add dir1/new
+       git add dir2/new
+}

Idem.

Quoted 11 lines

t/t7519-status-fsmonitor.sh new file +11 -0

+# The test query-fsmonitor hook proc will output a marker file we can use to
+# ensure the hook was actually used to generate the correct results.
+
+test_expect_success 'setup' '
+       mkdir -p .git/hooks &&
+       write_script .git/hooks/query-fsmonitor<<-\EOF &&
+       if [ $1 -ne 1 ]
+       then
+               echo -e "Unsupported query-fsmonitor hook version.\n" >&2
+               exit 1;
+       fi

t/t7519-status-fsmonitor.sh line 48

Quoted 10 lines

t/t7519-status-fsmonitor.sh new file +10 -0

+       : >marker
+       printf "untracked\0"
+       printf "dir1/untracked\0"
+       printf "dir2/untracked\0"
+       printf "modified\0"
+       printf "dir1/modified\0"
+       printf "dir2/modified\0"
+       printf "new\0""
+       printf "dir1/new\0"
+       printf "dir2/new\0"

Maybe something like the following to save a few lines and remove some redundancies:

        printf "%s\0" untracked dir1/untracked dir2/untracked \
                              modified dir1/modified dir2/modified \
                              new dir1/new dir2/new

or perhaps:

for f in untracked modified new
do
printf "%s\0" "$f" "dir1/$f" "dir2/$f"
done

Quoted 15 lines

t/t7519-status-fsmonitor.sh new file +15 -0

+       EOF
+       : >tracked &&
+       : >modified &&
+       mkdir dir1 &&
+       : >dir1/tracked &&
+       : >dir1/modified &&
+       mkdir dir2 &&
+       : >dir2/tracked &&
+       : >dir2/modified &&
+       git add . &&
+       test_tick &&
+       git commit -m initial &&
+       dirty_repo
+'
+

t/t7519-status-fsmonitor.sh line 69

Quoted 6 lines

t/t7519-status-fsmonitor.sh new file +6 -0

+cat >.gitignore <<\EOF
+.gitignore
+expect*
+output*
+marker*
+EOF

This could be part of the previous setup test.

t/t7519-status-fsmonitor.sh line 76

Quoted 6 lines

t/t7519-status-fsmonitor.sh new file +6 -0

+# Status is well tested elsewhere so we'll just ensure that the results are
+# the same when using core.fsmonitor. First call after turning on the option
+# does a complete scan so need to do two calls to ensure we test the new
+# codepath.
+
+test_expect_success 'status with core.untrackedcache true' '

If this test is using untracked cache, it should perhaps first check that untracked cache can be used on the current file system. t7063-status-untracked-cache.sh does that with the following:

test_lazy_prereq UNTRACKED_CACHE '
{ git update-index --test-untracked-cache; ret=$?; } && test $ret -ne 1
'

if ! test_have_prereq UNTRACKED_CACHE; then
skip_all='This system does not support untracked cache' test_done
fi

t/t7519-status-fsmonitor.sh line 79

Quoted 3 lines

t/t7519-status-fsmonitor.sh new file +3 -0

+       git config core.fsmonitor true  &&
+       git config core.untrackedcache true &&
+       git -c core.fsmonitor=false -c core.untrackedcache=true status >expect &&

I don't understand why there is " -c core.untrackedcache=true" in the above command as you already set core.untrackedcache to true on the previous line.

Quoted 4 lines

t/t7519-status-fsmonitor.sh new file +4 -0

+       clean_repo &&
+       git status &&
+       test_path_is_missing marker &&
+       dirty_repo &&

t/t7519-status-fsmonitor.sh line 89

Quoted 6 lines

t/t7519-status-fsmonitor.sh new file +6 -0

+       git status >output &&
+       test_path_is_file marker &&
+       test_i18ncmp expect output
+'
+
+

Spurious new line.

t/t7519-status-fsmonitor.sh line 93

Quoted 4 lines

t/t7519-status-fsmonitor.sh new file +4 -0

+test_expect_success 'status with core.untrackedcache false' '
+       git config core.fsmonitor true &&
+       git config core.untrackedcache false &&
+       git -c core.fsmonitor=false -c core.untrackedcache=false status >expect &&

Again core.untrackedcache is already set to false on the previous line.

Quoted 26 lines

t/t7519-status-fsmonitor.sh new file +26 -0

+       clean_repo &&
+       git status &&
+       test_path_is_missing marker &&
+       dirty_repo &&
+       git status >output &&
+       test_path_is_file marker &&
+       test_i18ncmp expect output
+'
+
+# Ensure commands that call refresh_index() to move the index back in time
+# properly invalidate the fsmonitor cache
+
+test_expect_success 'refresh_index() invalidates fsmonitor cache' '
+       git config core.fsmonitor true &&
+       git config core.untrackedcache true &&
+       clean_repo &&
+       git status &&
+       test_path_is_missing marker &&
+       dirty_repo &&
+       write_script .git/hooks/query-fsmonitor<<-\EOF &&
+       :>marker
+       EOF
+       git add . &&
+       git commit -m "to reset" &&
+       git status &&
+       test_path_is_file marker &&

t/t7519-status-fsmonitor.sh line 122

Quoted 3 lines

t/t7519-status-fsmonitor.sh new file +3 -0

+       git reset HEAD~1 &&
+       git status >output &&
+       test_path_is_file marker &&

You already checked that "marker" exists 3 lines above, and as far as I can see nothing could remove this file since the previous test, as the hook can only create it.
So I wonder if something is missing or if this test is redundant.

t/t7519-status-fsmonitor.sh line 132

Quoted 10 lines

t/t7519-status-fsmonitor.sh new file +10 -0

+       git -c core.fsmonitor=false status >expect &&
+       test_i18ncmp expect output
+'
+
+# Now make sure it's actually skipping the check for modified and untracked
+# files unless it is told about them.  Note, after "git reset --hard HEAD" no
+# extensions exist other than 'TREE' so do a "git status" to get the extension
+# written before testing the results.
+
+test_expect_success 'status doesnt detect unreported modifications' '

Maybe:

test_expect_success "status doesn't detect unreported modifications" '

Quoted 8 lines

t/t7519-status-fsmonitor.sh new file +7 -0

+       git config core.fsmonitor true &&
+       git config core.untrackedcache true &&
+       write_script .git/hooks/query-fsmonitor<<-\EOF &&
+       :>marker
+       EOF
+       clean_repo &&
+       git status &&

t/t7519-status-fsmonitor.sh line 149

Quoted 10 lines

t/t7519-status-fsmonitor.sh new file +10 -0

+       test_path_is_missing marker &&
+       : >untracked &&
+       echo 2 >dir1/modified &&
+       git status >output &&
+       test_path_is_file marker &&
+       test_i18ngrep ! "Changes not staged for commit:" output &&
+       test_i18ngrep ! "Untracked files:" output &&
+       write_script .git/hooks/query-fsmonitor<<-\EOF &&
+       :>marker
+       printf "untracked%s\0"

Why is there a %s in the above?

Quoted 13 lines

t/t7519-status-fsmonitor.sh new file +9 -1

+       printf "dir1/modified\0"
+       EOF
+       git status >output &&
+       test_path_is_file marker &&
+       test_i18ngrep "Changes not staged for commit:" output &&
+       test_i18ngrep "Untracked files:" output
+'
+
+test_done
--

2.13.0.windows.1.9.gc201c67b71

I did a quick search through the existing test scripts and the majority do not link commands together with && when they are in a sub function like this. I find not having them linked together is easier to write, maintain and is more readable.

Quoted 4 lines · Ben Peart

I did a quick search through the existing test scripts and the majority do not link commands together with && when they are in a sub function like this. I find not having them linked together is easier to write, maintain and is more readable.

I had an impression that it is true only when the series of commands are not about Git. When testing git commands, we should expect any of them to be broken (by somebody else ;-) and prepare to notice it.

Quoted 20 lines · Christian Couder

t/t7519-status-fsmonitor.sh line 48

Quoted 9 lines · Ben Peart

t/t7519-status-fsmonitor.sh new file +9 -0

+       printf "untracked\0"
+       printf "dir1/untracked\0"
+       printf "dir2/untracked\0"
+       printf "modified\0"
+       printf "dir1/modified\0"
+       printf "dir2/modified\0"
+       printf "new\0""
+       printf "dir1/new\0"
+       printf "dir2/new\0"

Maybe something like the following to save a few lines and remove some redundancies:

        printf "%s\0" untracked dir1/untracked dir2/untracked \
                              modified dir1/modified dir2/modified \
                              new dir1/new dir2/new

Quoted 6 lines

or perhaps:

for f in untracked modified new
do
printf "%s\0" "$f" "dir1/$f" "dir2/$f"
done

That is a clever solution that certainly is fewer lines of code. However, I have to read the loop and think through the logic to figure out what it is doing vs the existing implementation where what it is doing is apparent from just glancing at the code. I was also trying to maintain consistency with the other status test code in t7508-status.sh

Quoted 15 lines

Quoted 15 lines

t/t7519-status-fsmonitor.sh new file +15 -0

+       EOF
+       : >tracked &&
+       : >modified &&
+       mkdir dir1 &&
+       : >dir1/tracked &&
+       : >dir1/modified &&
+       mkdir dir2 &&
+       : >dir2/tracked &&
+       : >dir2/modified &&
+       git add . &&
+       test_tick &&
+       git commit -m initial &&
+       dirty_repo
+'
+

Quoted 8 lines

t/t7519-status-fsmonitor.sh line 69

Quoted 6 lines

t/t7519-status-fsmonitor.sh new file +6 -0

+cat >.gitignore <<\EOF
+.gitignore
+expect*
+output*
+marker*
+EOF

This could be part of the previous setup test.

I had followed the pattern in t7508-status.sh with this but I can move it in if that is the preferred model.

Quoted 5 lines

Quoted 4 lines

t/t7519-status-fsmonitor.sh new file +3 -0

+       git config core.fsmonitor true  &&
+       git config core.untrackedcache true &&
+       git -c core.fsmonitor=false -c core.untrackedcache=true status

Quoted 1 line

expect &&

Quoted 3 lines

I don't understand why there is " -c core.untrackedcache=true" in the above command as you already set core.untrackedcache to true on the previous line.

Defensive programming. :) The global setting was to ensure it was set when the test sub-functions clean and dirty were called and the command line settings were used to make it explicit what was being tested. I can remove them if it is causing confusion.

Quoted 6 lines

Quoted 6 lines

t/t7519-status-fsmonitor.sh new file +6 -0

+test_expect_success 'refresh_index() invalidates fsmonitor cache' '
+       git config core.fsmonitor true &&
+       git config core.untrackedcache true &&
+       clean_repo &&
+       git status &&
+       test_path_is_missing marker &&

Quoted 8 lines

t/t7519-status-fsmonitor.sh line 119

Quoted 8 lines

t/t7519-status-fsmonitor.sh new file +8 -0

+       dirty_repo &&
+       write_script .git/hooks/query-fsmonitor<<-\EOF &&
+       :>marker
+       EOF
+       git add . &&
+       git commit -m "to reset" &&
+       git status &&
+       test_path_is_file marker &&

Quoted 4 lines

Quoted 3 lines

t/t7519-status-fsmonitor.sh new file +3 -0

+       git reset HEAD~1 &&
+       git status >output &&
+       test_path_is_file marker &&

Quoted 4 lines

You already checked that "marker" exists 3 lines above, and as far as I can see nothing could remove this file since the previous test, as the hook can only create it.
So I wonder if something is missing or if this test is redundant.

Testing it each time ensures it is being created when it is supposed to be (ie when the test believes it is using the query-fsmonitor hook) and that it isn't when it isn't supposed to be (ie when the hook should not be called).

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

t/t7519-status-fsmonitor.sh new file +6 -0

+test_expect_success 'refresh_index() invalidates fsmonitor cache' '
+       git config core.fsmonitor true &&
+       git config core.untrackedcache true &&
+       clean_repo &&
+       git status &&
+       test_path_is_missing marker &&

Quoted 8 lines

Quoted 8 lines

t/t7519-status-fsmonitor.sh line 119

Quoted 8 lines

t/t7519-status-fsmonitor.sh new file +8 -0

+       dirty_repo &&
+       write_script .git/hooks/query-fsmonitor<<-\EOF &&
+       :>marker
+       EOF
+       git add . &&
+       git commit -m "to reset" &&
+       git status &&
+       test_path_is_file marker &&

Ok so "marker" is there now.

Quoted 9 lines

Quoted 8 lines

t/t7519-status-fsmonitor.sh line 122

Quoted 3 lines

t/t7519-status-fsmonitor.sh new file +3 -0

+       git reset HEAD~1 &&
+       git status >output &&
+       test_path_is_file marker &&

You already checked that "marker" exists 3 lines above, and as far as I can see nothing could remove this file since the previous test, as the hook can only create it.
So I wonder if something is missing or if this test is redundant.

Quoted 3 lines

Testing it each time ensures it is being created when it is supposed to be (ie when the test believes it is using the query-fsmonitor hook) and that it isn't when it isn't supposed to be (ie when the hook should not be called).

I would agree with that if the "marker" file was removed after the previous "test_path_is_file marker", but I don't see any "clean_repo" or "rm marker" call that removes it.

 templates/hooks--query-fsmonitor.sample | 37 +++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)
 create mode 100644 templates/hooks--query-fsmonitor.sample