[PATCH] fsmonitor: implement filesystem change listener for Linux (Page 1 of 2)

Paul Tarjan via GitGitGadget

author

2025-12-30 08:14 UTC #0 lore raw

PATCH 1/1 fsmonitor: implement filesystem change listener for Linux

Implement fsmonitor for Linux using the inotify API, bringing it to feature parity with existing Windows and macOS implementations.

The Linux implementation uses inotify to monitor filesystem events. Unlike macOS's FSEvents which can watch a single root directory, inotify requires registering watches on every directory of interest. The implementation carefully handles directory renames and moves using inotify's cookie mechanism to track IN_MOVED_FROM/IN_MOVED_TO event pairs.

Key implementation details:
- Uses inotify_init1(O_NONBLOCK) for non-blocking event monitoring
- Maintains bidirectional hashmaps between watch descriptors and paths for efficient event processing
- Handles directory creation, deletion, and renames dynamically
- Detects remote filesystems (NFS, CIFS, SMB, etc.) via statfs()
- Falls back to $HOME/.git-fsmonitor-* for socket when .git is remote
- Creates batches lazily (only for actual file events, not cookies) to avoid spurious sequence number increments

Build configuration:
- Enabled via FSMONITOR_DAEMON_BACKEND=linux and FSMONITOR_OS_SETTINGS=linux
- Requires NO_PTHREADS and NO_UNIX_SOCKETS to be unset
- Adds HAVE_LINUX_MAGIC_H for filesystem type detection

Documentation updated to note that fsmonitor.socketDir is now supported on both Mac OS and Linux, and adds a section about inotify watch limits.

Testing performed:
- Build succeeds with standard flags and SANITIZE=address
- All t7527-builtin-fsmonitor.sh tests pass on local filesystems
- Remote filesystem detection correctly rejects network mounts

Issues addressed from PR #1352 (git/git) review comments:
- GPLv3 ME_REMOTE macro: Rewrote remote filesystem detection from scratch using statfs() and linux/magic.h constants (no GPLv3 code)
- Memory leak on inotify_init1 failure: Added FREE_AND_NULL cleanup
- Unsafe hashmap iteration in dtor: Collect entries first, then modify
- Missing null checks in stop_async: Added proper guard conditions
- dirname() modifying argument: Create copy with xstrdup() first
- Non-portable f_fsid.__val: Use memcmp() for fsid comparison
- Missing worktree null check: Added BUG() for null worktree
- Header updates: Use git-compat-util.h, hash_to_hex_algop()
- Code style: Use xstrdup() not xmemdupz(), proper pointer style

Issues addressed from PR #1667 (git/git) review comments:
- EINTR handling: read() now handles both EAGAIN and EINTR
- Trailing pipe in log_mask_set: Added strbuf_strip_suffix()
- Unchecked add_watch return: Now logs failure in rename_dir()
- String building: Consolidated strbuf operations with strbuf_addf()
- Translation markers: Added _() to all error_errno() messages

Based on work from https://github.com/git/git/pull/1352 by Eric DeCosta, and https://github.com/git/git/pull/1667 by Marziyeh Esipreh, updated to work with the current codebase and address all review feedback.

Signed-off-by Paul Tarjan <[email protected]>

---
fsmonitor: implement filesystem change listener for Linux

Implement fsmonitor for Linux using the inotify API, bringing it to feature parity with existing Windows and macOS implementations.

The Linux implementation uses inotify to monitor filesystem events. Unlike macOS's FSEvents which can watch a single root directory, inotify requires registering watches on every directory of interest. The implementation carefully handles directory renames and moves using inotify's cookie mechanism to track IN_MOVED_FROM/IN_MOVED_TO event pairs.

Key implementation details:

* Uses inotify_init1(O_NONBLOCK) for non-blocking event monitoring
* Maintains bidirectional hashmaps between watch descriptors and paths for efficient event processing
* Handles directory creation, deletion, and renames dynamically
* Detects remote filesystems (NFS, CIFS, SMB, etc.) via statfs()
* Falls back to $HOME/.git-fsmonitor-* for socket when .git is remote

Build configuration:

* Enabled via FSMONITOR_DAEMON_BACKEND=linux and FSMONITOR_OS_SETTINGS=linux
* Requires NO_PTHREADS and NO_UNIX_SOCKETS to be unset
* Adds HAVE_LINUX_MAGIC_H for filesystem type detection

Documentation updated to note that fsmonitor.socketDir is now supported on both Mac OS and Linux, and adds a section about inotify watch limits.

Testing performed:

* Build succeeds with standard flags (NO_CURL, NO_GETTEXT, NO_TCLTK)
* Build succeeds with SANITIZE=address (AddressSanitizer)
* General tests pass (t0001-init, t3200-branch, etc.)
* fsmonitor tests require local filesystem (v9fs test environment is correctly detected as remote and rejected)
* Manual testing confirms daemon starts and monitors filesystem when configured with fsmonitor.allowremote=true and fsmonitor.socketdir pointing to a local filesystem (tmpfs)

Issues addressed from PR #1352 (git/git) review comments:

* GPLv3 ME_REMOTE macro: Rewrote remote filesystem detection from scratch using statfs() and linux/magic.h constants (no GPLv3 code)
* Memory leak on inotify_init1 failure: Added FREE_AND_NULL cleanup
* Unsafe hashmap iteration in dtor: Collect entries first, then modify
* Missing null checks in stop_async: Added proper guard conditions
* dirname() modifying argument: Create copy with xstrdup() first
* Non-portable f_fsid.__val: Use memcmp() for fsid comparison
* Missing worktree null check: Added BUG() for null worktree
* Header updates: Use git-compat-util.h, hash_to_hex_algop()
* Code style: Use xstrdup() not xmemdupz(), proper pointer style

Issues addressed from PR #1667 (git/git) review comments:

* EINTR handling: read() now handles both EAGAIN and EINTR
* Trailing pipe in log_mask_set: Added strbuf_strip_suffix()
* Unchecked add_watch return: Now logs failure in rename_dir()
* String building: Consolidated strbuf operations with strbuf_addf()
* Translation markers: Added _() to all error_errno() messages

Based on work from https://github.com/git/git/pull/1352 by Eric DeCosta, and https://github.com/git/git/pull/1667 by Marziyeh Esipreh, updated to work with the current codebase and address all review feedback.

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-2147%2Fptarjan%2Fclaude%2Fupdate-pr-1352-current-85Gk8-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-2147/ptarjan/claude/update-pr-1352-current-85Gk8-v1
Pull-Request: https://github.com/git/git/pull/2147

 Documentation/config/fsmonitor--daemon.adoc |   4 +-
 Documentation/git-fsmonitor--daemon.adoc    |  28 +-
 compat/fsmonitor/fsm-health-linux.c         |  33 +
 compat/fsmonitor/fsm-ipc-linux.c            |  61 ++
 compat/fsmonitor/fsm-listen-linux.c         | 738 ++++++++++++++++++++
 compat/fsmonitor/fsm-path-utils-linux.c     | 224 ++++++
 compat/fsmonitor/fsm-settings-linux.c       |  71 ++
 config.mak.uname                            |  10 +
 contrib/buildsystems/CMakeLists.txt         |  10 +
 9 files changed, 1173 insertions(+), 6 deletions(-)
 create mode 100644 compat/fsmonitor/fsm-health-linux.c
 create mode 100644 compat/fsmonitor/fsm-ipc-linux.c
 create mode 100644 compat/fsmonitor/fsm-listen-linux.c
 create mode 100644 compat/fsmonitor/fsm-path-utils-linux.c
 create mode 100644 compat/fsmonitor/fsm-settings-linux.c

Documentation/config/fsmonitor--daemon.adoc +2 -2

index 671f9b9462..6f8386e291 100644
@@ -4,8 +4,8 @@ fsmonitor.allowRemote::
     behavior.  Only respected when `core.fsmonitor` is set to `true`.
 
 fsmonitor.socketDir::
-    This Mac OS-specific option, if set, specifies the directory in
+    This Mac OS and Linux-specific option, if set, specifies the directory in
     which to create the Unix domain socket used for communication
     between the fsmonitor daemon and various Git commands. The directory must
-    reside on a native Mac OS filesystem.  Only respected when `core.fsmonitor`
+    reside on a native filesystem.  Only respected when `core.fsmonitor`
     is set to `true`.

Documentation/git-fsmonitor--daemon.adoc +24 -4

index 8fe5241b08..12fa866a64 100644
@@ -76,9 +76,9 @@ repositories; this may be overridden by setting `fsmonitor.allowRemote` to
 correctly with all network-mounted repositories, so such use is considered
 experimental.
 
-On Mac OS, the inter-process communication (IPC) between various Git
+On Mac OS and Linux, the inter-process communication (IPC) between various Git
 commands and the fsmonitor daemon is done via a Unix domain socket (UDS) -- a
-special type of file -- which is supported by native Mac OS filesystems,
+special type of file -- which is supported by native Mac OS and Linux filesystems,
 but not on network-mounted filesystems, NTFS, or FAT32.  Other filesystems
 may or may not have the needed support; the fsmonitor daemon is not guaranteed
 to work with these filesystems and such use is considered experimental.

@@ -87,13 +87,33 @@ By default, the socket is created in the `.git` directory.  However, if the
 `.git` directory is on a network-mounted filesystem, it will instead be
 created at `$HOME/.git-fsmonitor-*` unless `$HOME` itself is on a
 network-mounted filesystem, in which case you must set the configuration
-variable `fsmonitor.socketDir` to the path of a directory on a Mac OS native
+variable `fsmonitor.socketDir` to the path of a directory on a native
 filesystem in which to create the socket file.
 
 If none of the above directories (`.git`, `$HOME`, or `fsmonitor.socketDir`)
-is on a native Mac OS file filesystem the fsmonitor daemon will report an
+is on a native filesystem the fsmonitor daemon will report an
 error that will cause the daemon and the currently running command to exit.
 
+LINUX CAVEATS
+~~~~~~~~~~~~~
+
+On Linux, the fsmonitor daemon uses inotify to monitor filesystem events.
+The inotify system has per-user limits on the number of watches that can
+be created.  The default limit is typically 8192 watches per user.
+
+For large repositories with many directories, you may need to increase
+this limit.  Check the current limit with:
+
+    cat /proc/sys/fs/inotify/max_user_watches
+
+To temporarily increase the limit:
+
+    sudo sysctl fs.inotify.max_user_watches=65536
+
+To make the change permanent, add to `/etc/sysctl.conf`:
+
+    fs.inotify.max_user_watches=65536
+
 CONFIGURATION
 -------------

compat/fsmonitor/fsm-health-linux.c new file +33 -0

new file mode 100644
index 0000000000..43d67c4b8b
@@ -0,0 +1,33 @@
+#include "git-compat-util.h"
+#include "config.h"
+#include "fsmonitor-ll.h"
+#include "fsm-health.h"
+#include "fsmonitor--daemon.h"
+
+/*
+ * The Linux fsmonitor implementation uses inotify which has its own
+ * mechanisms for detecting filesystem unmount and other events that
+ * would require the daemon to shutdown.  Therefore, we don't need
+ * a separate health thread like Windows does.
+ *
+ * These stub functions satisfy the interface requirements.
+ */
+
+int fsm_health__ctor(struct fsmonitor_daemon_state *state UNUSED)
+{
+	return 0;
+}
+
+void fsm_health__dtor(struct fsmonitor_daemon_state *state UNUSED)
+{
+	return;
+}
+
+void fsm_health__loop(struct fsmonitor_daemon_state *state UNUSED)
+{
+	return;
+}
+
+void fsm_health__stop_async(struct fsmonitor_daemon_state *state UNUSED)
+{
+}

compat/fsmonitor/fsm-ipc-linux.c new file +61 -0

61 lines not on this page · raw message

compat/fsmonitor/fsm-listen-linux.c new file +738 -0

738 lines not on this page · raw message

compat/fsmonitor/fsm-path-utils-linux.c new file +224 -0

224 lines not on this page · raw message

compat/fsmonitor/fsm-settings-linux.c new file +71 -0

71 lines not on this page · raw message

config.mak.uname +10 -0

index 1691c6ae6e..a0fd0752a3 100644
@@ -68,6 +68,16 @@ ifeq ($(uname_S),Linux)
 		BASIC_CFLAGS += -std=c99
         endif
 	LINK_FUZZ_PROGRAMS = YesPlease
+
+	# The builtin FSMonitor on Linux builds upon Simple-IPC.  Both require
+	# Unix domain sockets and PThreads.
+        ifndef NO_PTHREADS
+        ifndef NO_UNIX_SOCKETS
+	FSMONITOR_DAEMON_BACKEND = linux
+	FSMONITOR_OS_SETTINGS = linux
+	BASIC_CFLAGS += -DHAVE_LINUX_MAGIC_H
+        endif
+        endif
 endif
 ifeq ($(uname_S),GNU/kFreeBSD)
 	HAVE_ALLOCA_H = YesPlease

contrib/buildsystems/CMakeLists.txt +10 -0

index 28877feb9d..141b05acb0 100644
@@ -308,6 +308,16 @@ if(SUPPORTS_SIMPLE_IPC)
 
 		add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
 		list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-darwin.c)
+	elseif(CMAKE_SYSTEM_NAME STREQUAL "Linux")
+		add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND)
+		add_compile_definitions(HAVE_LINUX_MAGIC_H)
+		list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-linux.c)
+		list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-linux.c)
+		list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-linux.c)
+		list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-linux.c)
+
+		add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
+		list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-linux.c)
 	endif()
 endif()

base-commit: 7c7698a654a7a0031f65b0ab0c1c4e438e95df60

Signature · 1 line

gitgitgadget

Junio C Hamano

2025-12-30 11:38 UTC #1 lore raw

Exciting.

It seems to die with leaks when "make SANITIZE=leak test" is run, though.

Initialized empty Git repository in /home/gitster/w/git.git/t/trash directory.t7527-builtin-fsmonitor/test_implicit/.git/
fsmonitor-daemon is not watching '/home/gitster/w/git.git/t/trash directory.t7527-builtin-fsmonitor/test_implicit'
builtin:0.145521.20251230T113644.793433Z:0Q/Q {"event":"data","sid":"20251230T113644.762195Z-H3cfff1b1-P0002386f","thread":"main","time":"2025-12-30T11:36:44.813131Z","file":"fsmonitor-ipc.c","line":99,"t_abs":0.052581,"t_rel":0.048830,"nesting":2,"category":"fsm_client","key":"query/response-length","value":"45"} fsmonitor-daemon is watching '/home/gitster/w/git.git/t/trash directory.t7527-builtin-fsmonitor/test_implicit'
fsmonitor-daemon is not watching '/home/gitster/w/git.git/t/trash directory.t7527-builtin-fsmonitor/test_implicit'
fatal: fsmonitor--daemon is not running
not ok 2 - implicit daemon start
#
# test_when_finished "stop_daemon_delete_repo test_implicit" &&
#
# git init test_implicit &&
# test_must_fail git -C test_implicit fsmonitor--daemon status &&
#

#               # query will implicitly start the daemon.
#               #
#               # for test-script simplicity, we send a V1 timestamp rather than
#               # a V2 token.  either way, the daemon response to any query contains
#               # a new V2 token.  (the daemon may complain that we sent a V1 request,
#               # but this test case is only concerned with whether the daemon was
#               # implicitly started.)

#
# GIT_TRACE2_EVENT="$PWD/.git/trace" \
# test-tool -C test_implicit fsmonitor-client query --token 0 >actual &&
# nul_to_q <actual >actual.filtered &&
# grep "builtin:" actual.filtered &&
#

#               # confirm that a daemon was started in the background.
#               #
#               # since the mechanism for starting the background daemon is platform
#               # dependent, just confirm that the foreground command received a
#               # response from the daemon.
#
#               have_t2_data_event fsm_client query/response-length <.git/trace &&
#
#               git -C test_implicit fsmonitor--daemon status &&
#               git -C test_implicit fsmonitor--daemon stop &&
#               test_must_fail git -C test_implicit fsmonitor--daemon status

#
1..2

=================================================================
==git==145489==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 512 byte(s) in 1 object(s) allocated from:

    #0 0x56179e6ce042 in calloc (git+0x8c042) (BuildId: de5ce3c9d0b0c09380c910e6a9eb181e324abde6)
    #1 0x56179ea0aef4 in xcalloc wrapper.c:154:8
    #2 0x56179e8b17b7 in alloc_table hashmap.c:79:2
    #3 0x56179e8b174c in hashmap_init hashmap.c:168:2
    #4 0x56179e73e6fe in fsmonitor_run_daemon builtin/fsmonitor--daemon.c:1288:2
    #5 0x56179e73e141 in try_to_run_foreground_daemon builtin/fsmonitor--daemon.c:1448:11
    #6 0x56179e73dc44 in cmd_fsmonitor__daemon builtin/fsmonitor--daemon.c:1584:12
    #7 0x56179e6d2c8a in run_builtin git.c:506:11
    #8 0x56179e6d1910 in handle_builtin git.c:779:9
    #9 0x56179e6d2747 in run_argv git.c:862:4
    #10 0x56179e6d169b in cmd_main git.c:984:19
    #11 0x56179e7f7a7a in main common-main.c:9:11
    #12 0x7f091ea66ca7 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #13 0x7f091ea66d64 in __libc_start_main csu/../csu/libc-start.c:360:3
    #14 0x56179e69e280 in _start (git+0x5c280) (BuildId: de5ce3c9d0b0c09380c910e6a9eb181e324abde6)

Paul Tarjan via GitGitGadget

author

2025-12-30 12:08 UTC #2 lore raw

PATCH v2 1/1 fsmonitor: implement filesystem change listener for Linux

In reply to Paul Tarjan via GitGitGadget

From: Paul Tarjan <[email protected]>