fsmonitor: option to allow fsmonitor to run against network-mounted repos
Subsystems (2)
PATCH 1/1 fsmonitor: option to allow fsmonitor to run against network-mounted repos
From: Eric DeCosta <[email protected]>
Though perhaps not common, there are uses cases where users have large, network-mounted repos. Having the ability to run fsmonitor against network paths would benefit those users.
As a first step towards enabling fsmonitor to work against network-mounted repos, a configuration option, 'fsmonitor.allowRemote' was introduced for Windows. Setting this option to true will override the default behavior (erroring-out) when a network-mounted repo is detected by fsmonitor. In order for macOS to have parity with Windows, the same option is now introduced for macOS.
The the added wrinkle being that the Unix domain socket (UDS) file used for IPC cannot be created in a network location; instead the temporary directory is used.
---
fsmonitor: option to allow fsmonitor to run against network-mounted repos
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1326%2Fedecosta-mw%2Ffsmonitor_macos-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1326/edecosta-mw/fsmonitor_macos-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1326
compat/fsmonitor/fsm-settings-darwin.c | 77 ++++++++++++++++++++++---- fsmonitor-ipc.c | 47 +++++++++++++++- fsmonitor-ipc.h | 6 ++ 3 files changed, 117 insertions(+), 13 deletions(-)
compat/fsmonitor/fsm-settings-darwin.c +65 -12
@@ -2,10 +2,28 @@#include "config.h" #include "repository.h" #include "fsmonitor-settings.h" +#include "fsmonitor-ipc.h" #include "fsmonitor.h" #include <sys/param.h> #include <sys/mount.h> +/* +* Check if monitoring remote working directories is allowed. +* +* By default, monitoring remote working directories is +* disabled. Users may override this behavior in enviroments where +* they have proper support. +*/ +static int check_config_allowremote (struct repository *r) +{ +int allow; + +if (!repo_config_get_bool (r,"fsmonitor.allowremote" , &allow)) +return allow; + +return -1 ;/* fsmonitor.allowremote not set */ +} +/* * [1] Remote working directories are problematic for FSMonitor. *
@@ -27,24 +45,22 @@* In theory, the above issues need to be addressed whether we are * using the Hook or IPC API. * +* So (for now at least), mark remote working directories as +* incompatible by default. +* * For the builtin FSMonitor, we create the Unix domain socket for the -* IPC in the .git directory. If the working directory is remote, -* then the socket will be created on the remote file system. This -* can fail if the remote file system does not support UDS file types -* (e.g. smbfs to a Windows server) or if the remote kernel does not -* allow a non-local process to bind() the socket. (These problems -* could be fixed by moving the UDS out of the .git directory and to a -* well-known local directory on the client machine, but care should -* be taken to ensure that $HOME is actually local and not a managed -* file share.) +* IPC in the temporary directory. If the temporary directory is +* remote, then the socket will be created on the remote file system. +* This can fail if the remote file system does not support UDS file +* types (e.g. smbfs to a Windows server) or if the remote kernel does +* not allow a non-local process to bind() the socket. * -* So (for now at least), mark remote working directories as -* incompatible. +* Therefore remote UDS locations are marked as incompatible. * * * [2] FAT32 and NTFS working directories are problematic too. * -* The builtin FSMonitor uses a Unix domain socket in the .git +* The builtin FSMonitor uses a Unix domain socket in the temporary * directory for IPC. These Windows drive formats do not support * Unix domain sockets, so mark them as incompatible for the daemon. *
@@ -65,6 +81,39 @@ static enum fsmonitor_reason check_volume(struct repository *r)"statfs('%s') [type 0x%08x][flags 0x%08x] '%s'" , r->worktree, fs.f_type, fs.f_flags, fs.f_fstypename); +if (!(fs.f_flags &MNT_LOCAL )) { +switch (check_config_allowremote (r)) { +case 0 :/* config overrides and disables */ +return FSMONITOR_REASON_REMOTE ; +case 1 :/* config overrides and enables */ +return FSMONITOR_REASON_OK ; +default : +break ;/* config has no opinion */ + } + +return FSMONITOR_REASON_REMOTE ; + } + +return FSMONITOR_REASON_OK ; +} + +static enum fsmonitor_reason check_uds_volume (void ) +{ +struct statfs fs; +const char *path =fsmonitor_ipc__get_path (); + +if (statfs (path, &fs) ==-1 ) { +int saved_errno = errno; +trace_printf_key (&trace_fsmonitor,"statfs('%s') failed: %s" , + path,strerror (saved_errno)); + errno = saved_errno; +return FSMONITOR_REASON_ERROR ; + } + +trace_printf_key (&trace_fsmonitor, +"statfs('%s') [type 0x%08x][flags 0x%08x] '%s'" , + path, fs.f_type, fs.f_flags, fs.f_fstypename); +if (!(fs.f_flags &MNT_LOCAL ))return FSMONITOR_REASON_REMOTE ;
@@ -85,5 +134,9 @@ enum fsmonitor_reason fsm_os__incompatible(struct repository *r)if (reason !=FSMONITOR_REASON_OK )return reason; + reason =check_uds_volume (); +if (reason !=FSMONITOR_REASON_OK ) +return reason; +return FSMONITOR_REASON_OK ; }
fsmonitor-ipc.c +46 -1
59 lines not on this page · raw message
fsmonitor-ipc.h +6 -0
@@ -18,6 +18,12 @@ int fsmonitor_ipc__is_supported(void);*/ const char *fsmonitor_ipc__get_path (void ); +/* +* Returns the pathname to the file that contains the pathname to the +* IPC named pipe or Unix domain socket. +*/ +const char *fsmonitor_ipc__get_pathfile (void ); +/* * Try to determine whether there is a `git-fsmonitor--daemon` process * listening on the IPC pipe/socket.
base-commit: 9bf691b78cf906751e65d65ba0c6ffdcd9a5a12c
Signature · 1 line
gitgitgadget
Quoted 7 lines · Eric DeCosta via GitGitGadget
From: Eric DeCosta <[email protected]>
Though perhaps not common, there are uses cases where users have large, network-mounted repos. Having the ability to run fsmonitor against network paths would benefit those users.
As a first step towards enabling fsmonitor to work ...
Didn't we already see the first step and got it reviewed? I think I've already merged the last round to 'next'.
Puzzled...
Quoted 8 lines · Junio C Hamano
Quoted 6 lines · Eric DeCosta via GitGitGadget
From: Eric DeCosta <[email protected]>
Though perhaps not common, there are uses cases where users have large, network-mounted repos. Having the ability to run fsmonitor against network paths would benefit those users.
Quoted 6 lines
Quoted 1 line
As a first step towards enabling fsmonitor to work ...
Didn't we already see the first step and got it reviewed? I think I've already merged the last round to 'next'.
Puzzled...
Ah, OK, so this is not exactly the "first step", but builds on top of the previous one? At least the patch needs a better title to make it distinguishable from the other one.
Will queue to wait for others to review.
Thanks.
Hi Eric,
Quoted 9 lines · Eric DeCosta via GitGitGadget
From: Eric DeCosta <[email protected]>
Though perhaps not common, there are uses cases where users have large, network-mounted repos. Having the ability to run fsmonitor against network paths would benefit those users.
As a first step towards enabling fsmonitor to work against network-mounted repos, a configuration option, 'fsmonitor.allowRemote' was introduced for Windows.
If you start the commit message along the following lines, it might be easier/quicker to grok the context for the keen reader:
In 85dc0da6dcf (fsmonitor: option to allow fsmonitor to run against network-mounted repos, 2022-08-11), the Windows backend of the FSMonitor learned to allow running on network drives, via the fsmonitor.allowRemote config setting.
Quoted 8 lines
Setting this option to true will override the default behavior (erroring-out) when a network-mounted repo is detected by fsmonitor. In order for macOS to have parity with Windows, the same option is now introduced for macOS.
The the added wrinkle being that the Unix domain socket (UDS) file used for IPC cannot be created in a network location; instead the temporary directory is used.
Thank you very much for this note, after a cursory read I expected that part of the code to be a left-over from some "We know better than the user" type of automatic default, and this paragraph definitely helped me overcome that expectation.
Quoted 6 lines
Signed-off-by Eric DeCosta <[email protected]>---
fsmonitor: option to allow fsmonitor to run against network-mounted repos
Quoted 8 lines
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1326%2Fedecosta-mw%2Ffsmonitor_macos-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1326/edecosta-mw/fsmonitor_macos-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1326compat/fsmonitor/fsm-settings-darwin.c | 77 ++++++++++++++++++++++---- fsmonitor-ipc.c | 47 +++++++++++++++- fsmonitor-ipc.h | 6 ++ 3 files changed, 117 insertions(+), 13 deletions(-)
I am somewhat puzzled that this has no corresponding change to Documentation/.
And now I realize that this was the case also for the patch adding fsmonitor.allowRemote support for Windows.
Could I ask you to add a patch to document this config setting?
Quoted 35 lines
compat/fsmonitor/fsm-settings-darwin.c +18
-0@@ -2,10 +2,28 @@#include "config.h" #include "repository.h" #include "fsmonitor-settings.h" +#include "fsmonitor-ipc.h" #include "fsmonitor.h" #include <sys/param.h> #include <sys/mount.h> +/* +* Check if monitoring remote working directories is allowed. +* +* By default, monitoring remote working directories is +* disabled. Users may override this behavior in enviroments where +* they have proper support. +*/ +static int check_config_allowremote (struct repository *r) +{ +int allow; + +if (!repo_config_get_bool (r,"fsmonitor.allowremote" , &allow)) +return allow; + +return -1 ;/* fsmonitor.allowremote not set */ +} +/* * [1] Remote working directories are problematic for FSMonitor. *
compat/fsmonitor/fsm-settings-darwin.c line 50
Quoted 6 lines
compat/fsmonitor/fsm-settings-darwin.c +3
-0@@ -27,24 +45,22 @@* In theory, the above issues need to be addressed whether we are * using the Hook or IPC API. * +* So (for now at least), mark remote working directories as +* incompatible by default. +*
This was moved up, okay.
compat/fsmonitor/fsm-settings-darwin.c line 52
Quoted 11 lines
compat/fsmonitor/fsm-settings-darwin.c +1
-9* For the builtin FSMonitor, we create the Unix domain socket for the -* IPC in the .git directory. If the working directory is remote, -* then the socket will be created on the remote file system. This -* can fail if the remote file system does not support UDS file types -* (e.g. smbfs to a Windows server) or if the remote kernel does not -* allow a non-local process to bind() the socket. (These problems -* could be fixed by moving the UDS out of the .git directory and to a -* well-known local directory on the client machine, but care should -* be taken to ensure that $HOME is actually local and not a managed -* file share.) +* IPC in the temporary directory. If the temporary directory is
This is incorrect. It is still the .git directory in the common case, not a temporary directory.
compat/fsmonitor/fsm-settings-darwin.c line 61
Quoted 11 lines
compat/fsmonitor/fsm-settings-darwin.c +5
-2+* remote, then the socket will be created on the remote file system. +* This can fail if the remote file system does not support UDS file +* types (e.g. smbfs to a Windows server) or if the remote kernel does +* not allow a non-local process to bind() the socket. * -* So (for now at least), mark remote working directories as -* incompatible. +* Therefore remote UDS locations are marked as incompatible. * * * [2] FAT32 and NTFS working directories are problematic too.
Doesn't this patch address this, too? See below for more on that.
Quoted 12 lines
compat/fsmonitor/fsm-settings-darwin.c +3
-1* -* The builtin FSMonitor uses a Unix domain socket in the .git +* The builtin FSMonitor uses a Unix domain socket in the temporary * directory for IPC. These Windows drive formats do not support * Unix domain sockets, so mark them as incompatible for the daemon. * @@ -65,6 +81,39 @@ static enum fsmonitor_reason check_volume(struct repository *r)"statfs('%s') [type 0x%08x][flags 0x%08x] '%s'" , r->worktree, fs.f_type, fs.f_flags, fs.f_fstypename); +if (!(fs.f_flags &MNT_LOCAL )) { +switch (check_config_allowremote (r)) {
compat/fsmonitor/fsm-settings-darwin.c static enum fsmonitor_reason check_volume(struct repository *r) · line 95
Quoted 10 lines
compat/fsmonitor/fsm-settings-darwin.c +10
-0+case 0 :/* config overrides and disables */ +return FSMONITOR_REASON_REMOTE ; +case 1 :/* config overrides and enables */ +return FSMONITOR_REASON_OK ; +default : +break ;/* config has no opinion */ + } + +return FSMONITOR_REASON_REMOTE ; + }
This switch() statement sounds like a verbose way to say the same as:
return check_config_allowremote(r) == 1 ? FSMONITOR_REASON_OK : FSMONITOR_REASON_REMOTE;
compat/fsmonitor/fsm-settings-darwin.c static enum fsmonitor_reason check_volume(struct repository *r) · line 100
Quoted 5 lines
compat/fsmonitor/fsm-settings-darwin.c +5
-0+ +return FSMONITOR_REASON_OK ; +} + +static enum fsmonitor_reason check_uds_volume (void )
What's an UDS volume? Do you mean to say "Unix Domain Socket Volume"?
If so, it would be better to turn this into a function called filesystem_supports_unix_sockets() and to return an int, 1 for "yes", 0 for "no".
compat/fsmonitor/fsm-settings-darwin.c enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
Quoted 20 lines
compat/fsmonitor/fsm-settings-darwin.c +16
-0+{ +struct statfs fs; +const char *path =fsmonitor_ipc__get_path (); + +if (statfs (path, &fs) ==-1 ) { +int saved_errno = errno; +trace_printf_key (&trace_fsmonitor,"statfs('%s') failed: %s" , + path,strerror (saved_errno)); + errno = saved_errno; +return FSMONITOR_REASON_ERROR ; + } + +trace_printf_key (&trace_fsmonitor, +"statfs('%s') [type 0x%08x][flags 0x%08x] '%s'" , + path, fs.f_type, fs.f_flags, fs.f_fstypename); +if (!(fs.f_flags &MNT_LOCAL ))return FSMONITOR_REASON_REMOTE ;@@ -85,5 +134,9 @@ enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
It is unfortunate that the diff hunk stops here, and mails lack a button to increase the diff context. In this instance, the hidden part of the check_volume() function is quite interesting: it returns FSMONITOR_REASON_NOSOCKETS for msdos and ntfs file systems.
Which means that your patch changes behavior not only for remote file systems, but also for local ones without support for Unix sockets.
To heed the principle of separation of concerns, please do split out that part. I would recommend to make it the first patch to support msdos/ntfs file systems (by registering the Unix sockets in a temporary directory instead of the .git/ directory). The second patch can then introduce support for fsmonitor.allowRemote on macOS on top of the first patch.
Quoted 22 lines
compat/fsmonitor/fsm-settings-darwin.c +4
-0if (reason !=FSMONITOR_REASON_OK )return reason; + reason =check_uds_volume (); +if (reason !=FSMONITOR_REASON_OK ) +return reason; +return FSMONITOR_REASON_OK ; }fsmonitor-ipc.c +1
-0@@ -4,6 +4,7 @@#include "fsmonitor-ipc.h" #include "run-command.h" #include "strbuf.h" +#include "tempfile.h" #include "trace2.h" #ifndef HAVE_FSMONITOR_DAEMON_BACKEND
fsmonitor-ipc.c int fsmonitor_ipc__is_supported(void) · line 51
Quoted 5 lines
fsmonitor-ipc.c +1
-1@@ -47,7 +48,51 @@ int fsmonitor_ipc__is_supported(void)return 1 ; } -GIT_PATH_FUNC (fsmonitor_ipc__get_path,"fsmonitor--daemon.ipc" ) +GIT_PATH_FUNC (fsmonitor_ipc__get_pathfile,"fsmonitor--daemon.ipc" )
Why rename this? That's unnecessary chatter in the patch. Let's avoid such things in the future, it only costs reviewers time.
Quoted 34 lines
fsmonitor-ipc.c +34
-0+ +static char *gen_ipc_file (void ) +{ +char *retval =NULL ; +struct tempfile *ipc; + +const char *ipc_file =fsmonitor_ipc__get_pathfile (); +FILE *fp =fopen (ipc_file,"w" ); + +if (!fp) +die_errno ("error opening '%s'" , ipc_file); + ipc =mks_tempfile_t ("fsmonitor_ipc_XXXXXX" ); +strbuf_write (&ipc->filename, fp); +fclose (fp); + retval =strbuf_detach (&ipc->filename,NULL ); +strbuf_release (&ipc->filename); +return retval; +} + +const char *fsmonitor_ipc__get_path (void ) +{ +char *retval =NULL ; +struct strbuf sb =STRBUF_INIT ; + +const char *ipc_file =fsmonitor_ipc__get_pathfile (); +FILE *fp =fopen (ipc_file,"r" ); + +if (!fp) { +return gen_ipc_file (); + }else { +strbuf_read (&sb,fileno (fp),0 ); +fclose (fp); + fp =fopen (sb.buf,"r" ); +if (!fp) {/* generate new file */
fsmonitor-ipc.c int fsmonitor_ipc__is_supported(void) · line 95
Quoted 10 lines
fsmonitor-ipc.c +10
-0+if (unlink (ipc_file) <0 ) +die_errno ("could not remove '%s'" , ipc_file); +return gen_ipc_file (); + } +fclose (fp); + retval =strbuf_detach (&sb,NULL ); +strbuf_release (&sb); +return retval; + } +}
I am afraid I do not understand how this code can guarantee a fixed path for the Unix domain socket.
It _needs_ to be fixed so that a singleton daemon can run and listen on it, and an arbitrary number of Git clients can connect to it.
If it is not fixed, you will cause Git to quite possibly start a new FSMonitor daemon for every invocation that wants to connect to an FSMonitor daemon.
This means that the path of the Unix socket needs to have a 1:1 relationship to the path of the .git/ directory. If you install it in that directory, that invariant is naturally fulfilled. If you want to install it elsewhere, you will have to come up with a reliable way to guarantee that connection.
One option would be to install the Unix sockets in the home directory, under a name like .git-fsmonitor-<hash> where the <hash> is e.g. a SHA-1/SHA-256 of the canonicalized path of the .git/ directory.
Quoted 10 lines
fsmonitor-ipc.c +0
-0enum ipc_active_state fsmonitor_ipc__get_state (void ) {fsmonitor-ipc.h +0
-0@@ -18,6 +18,12 @@ int fsmonitor_ipc__is_supported(void);*/ const char *fsmonitor_ipc__get_path (void );
fsmonitor-ipc.h int fsmonitor_ipc__is_supported(void); · line 29
Quoted 10 lines
fsmonitor-ipc.h +6
-0+/* +* Returns the pathname to the file that contains the pathname to the +* IPC named pipe or Unix domain socket. +*/ +const char *fsmonitor_ipc__get_pathfile (void ); +/* * Try to determine whether there is a `git-fsmonitor--daemon` process * listening on the IPC pipe/socket.
Thank you for working on this, also on the Windows side. It definitely helps!
Ciao,
Dscho
Quoted 6 lines · Eric DeCosta via GitGitGadget
From: Eric DeCosta <[email protected]>
Though perhaps not common, there are uses cases where users have large, network-mounted repos. Having the ability to run fsmonitor against network paths would benefit those users.
Quoted 6 lines
As a first step towards enabling fsmonitor to work against network-mounted repos, a configuration option, 'fsmonitor.allowRemote' was introduced for Windows. Setting this option to true will override the default behavior (erroring-out) when a network-mounted repo is detected by fsmonitor. In order for macOS to have parity with Windows, the same option is now introduced for macOS.
We might also say that this config option only allows FSMonitor to TRY to consider using a network-mounted repo. And that this ability is considered experimental until sufficient testing can be completed and we can determine the combinations of
{ client os } x { server os } x { remote access } x { file system type } that are known to work or not work and we can update the defaults and the documentation accordingly.
For example, on a MacOS client, we expect the local "fseventsd" service to send us recursive events on all files and sub directories under the repo root. If the server is a Linux machine (which doesn't really do recursive events), does exporting the FS from the server over NFS or SMB (or whatever) cause the Linux host to send enough information to the client machine for fseventsd to synthesize the recursive event stream locally that FSMonitor expects. It might. It might not. That combination should be tested (along with a lot of other combinations).
But again, this patch is just about allowing the (informed?) user to try it and begin testing various combinations.
Quoted 4 lines
The the added wrinkle being that the Unix domain socket (UDS) file used for IPC cannot be created in a network location; instead the temporary directory is used.
This scares me a bit. I put the socket in the .git directory so that we are guaranteed that only one daemon will run on the repository and that all clients will know where to find that socket (if it exists).
It looks like you're creating the UDS using a tmp pathname and writing the pathname to the actual .git/fsmonitor--daemon.ipc FILE. This adds a layer of indirection and is prone to races.
The act of creating the actual socket is protected by code in unix-socket.c and unix-stream-server.c to try to safely create the socket and avoid stepping on another active daemon (who currently has the server-side of the socket open).
My code also detects dead sockets (where a previous daemon died and failed to delete the socket).
Additionally, allowing remote access means that the repo could be used by multiple client machines and/or by the server machine itself. Consider the example of two MacOS clients mounting the remote repo and both wanting to start FSMonitor. They would constantly fight to recreate a new local-tmp-based socket and update your pathname FILE and end up invalidating each other on each command.
Also, if someone overwrites your new pathname FILE, but doesn't tell the daemon, the daemon will be orphaned -- still running, but no one will ever connect to it because the FILE no longer points to it.
There was a suggestion later in this thread about using a SHA-1 or SHA-256 hash of the pathname to avoid the tmp XXXXXX pattern and just put the socket in $HOME (and omit the need for the new fsmonitor-daemon.ipc FILE completely). This might work, but we need to be careful because a user might have hardlinks or symlinks locally so there may be more than one unique path to the repo on the local system. (It is OK to have more than one daemon listening to a repo, just less efficient.)
As an interim step, you might try using my original socket code plus just the config.allowRemote=true change. And test it on a mounted repo where you've converted the .git directory to a .git file and moved contents of the .git directory to somewhere local. Then the UDS would be created in the local GITDIR instead of on the remote system. This won't help any of the sharing cases I described above, but will let you experiment with getting remote events.
Jeff
Quoted 2 lines
base-commit:
9bf691b78cf906751e65d65ba0c6ffdcd9a5a12c
Quoted 27 lines
-----Original Message-----
From: Jeff Hostetler <[email protected]>
Sent: Friday, August 19, 2022 12:50 PM
To: Eric DeCosta via GitGitGadget <[email protected]>; [email protected]
Cc: Eric DeCosta <[email protected]>
Subject: Re: [PATCH] fsmonitor: option to allow fsmonitor to run against network-mounted reposQuoted 6 lines · Eric DeCosta via GitGitGadget
From: Eric DeCosta <[email protected]>
Though perhaps not common, there are uses cases where users have large, network-mounted repos. Having the ability to run fsmonitor against network paths would benefit those users.
Quoted 6 lines
As a first step towards enabling fsmonitor to work against network-mounted repos, a configuration option, 'fsmonitor.allowRemote' was introduced for Windows. Setting this option to true will override the default behavior (erroring-out) when a network-mounted repo is detected by fsmonitor. In order for macOS to have parity with Windows, the same option is now introduced for macOS.
Quoted 8 lines
We might also say that this config option only allows FSMonitor to TRY to consider using a network-mounted repo. And that this ability is considered experimental until sufficient testing can be completed and we can determine the combinations of
{ client os } x { server os } x { remote access } x { file system type } that are known to work or not work and we can update the defaults and the documentation accordingly.
Yes, very experimental.
Quoted 9 lines
For example, on a MacOS client, we expect the local "fseventsd" service to send us recursive events on all files and sub directories under the repo root. If the server is a Linux machine (which doesn't really do recursive events), does exporting the FS from the server over NFS or SMB (or whatever) cause the Linux host to send enough information to the client machine for fseventsd to synthesize the recursive event stream locally that FSMonitor expects. It might. It might not. That combination should be tested (along with a lot of other combinations).
Quoted 3 lines
But again, this patch is just about allowing the (informed?) user to try it and begin testing various combinations.
Yes, the point is to allow users to try it out. Self-servingly, I have about 3K users who make heavy use of network-mounted sandboxes on the three major platforms; all connecting via NFS or SMB to Linux file servers. Hardly exhaustive, but the file system change notification APIs (inotify, FSEvents, and ReadDirectoryCHangesW) all seem to work correctly. Thus my motivation to work on this aspect of git :-)
Quoted 6 lines
Quoted 5 lines
The the added wrinkle being that the Unix domain socket (UDS) file used for IPC cannot be created in a network location; instead the temporary directory is used.
Quoted 9 lines
This scares me a bit. I put the socket in the .git directory so that we are guaranteed that only one daemon will run on the repository and that all clients will know where to find that socket (if it exists).
It looks like you're creating the UDS using a tmp pathname and writing the pathname to the actual .git/fsmonitor--daemon.ipc FILE. This adds a layer of indirection and is prone to races.
Good point.
Quoted 10 lines
The act of creating the actual socket is protected by code in
unix-socket.candunix-stream-server.cto try to safely create the socket and avoid stepping on another active daemon (who currently has the server-side of the socket open).My code also detects dead sockets (where a previous daemon died and failed to delete the socket).
Quoted 8 lines
Additionally, allowing remote access means that the repo could be used by multiple client machines and/or by the server machine itself. Consider the example of two MacOS clients mounting the remote repo and both wanting to start FSMonitor. They would constantly fight to recreate a new local-tmp-based socket and update your pathname FILE and end up invalidating each other on each command.
I see your point - they'd stomp on each other.
As far as multiple client machines mounting the remote repo, I have doubts that FSMonitor would even see changes made from another machine. Worth trying out and documenting as needed - might even be better off being considered as unsupported.
Quoted 5 lines
Also, if someone overwrites your new pathname FILE, but doesn't tell the daemon, the daemon will be orphaned -- still running, but no one will ever connect to it because the FILE no longer points to it.
True, thanks for pointing that out.
Quoted 10 lines
There was a suggestion later in this thread about using a SHA-1 or SHA-256 hash of the pathname to avoid the tmp XXXXXX pattern and just put the socket in $HOME (and omit the need for the new fsmonitor-daemon.ipc FILE completely). This might work, but we need to be careful because a user might have hardlinks or symlinks locally so there may be more than one unique path to the repo on the local system. (It is OK to have more than one daemon listening to a repo, just less efficient.)
Ah, I see.
Quoted 10 lines
As an interim step, you might try using my original socket code plus just the config.allowRemote=true change. And test it on a mounted repo where you've converted the .git directory to a .git file and moved contents of the .git directory to somewhere local. Then the UDS would be created in the local GITDIR instead of on the remote system. This won't help any of the sharing cases I described above, but will let you experiment with getting remote events.
Within the context of the environment that I have available to me (macOS over NFS to a Linux file server), FSEvents is working correctly. I can make changes at any arbitrary place inside of the repo and an event is generated.
It's looking like that the Unix domain socket (UDS) file should remain where it is unless fsmonitor.allowRemote is true.
If fsmonitor.allowRemote is true then the UDS file can be located in $HOME with the caveat that if there is more than one path to the repo (via hard or sym links) that things might not work as expected. I think that's OK given the experimental nature of the feature.
-Eric
Quoted 6 lines
Jeff
Quoted 2 lines
base-commit:
9bf691b78cf906751e65d65ba0c6ffdcd9a5a12c
Quoted 16 lines · Eric DeCosta
Quoted 16 lines
-----Original Message-----
From: Jeff Hostetler <[email protected]>
Sent: Friday, August 19, 2022 12:50 PM
To: Eric DeCosta via GitGitGadget <[email protected]>; [email protected]
Cc: Eric DeCosta <[email protected]>
Subject: Re: [PATCH] fsmonitor: option to allow fsmonitor to run against network-mounted reposQuoted 2 lines · Eric DeCosta via GitGitGadget
From: Eric DeCosta <[email protected]>
[...]
Quoted 6 lines
Yes, the point is to allow users to try it out. Self-servingly, I have about 3K users who make heavy use of network-mounted sandboxes on the three major platforms; all connecting via NFS or SMB to Linux file servers. Hardly exhaustive, but the file system change notification APIs (inotify, FSEvents, and ReadDirectoryCHangesW) all seem to work correctly. Thus my motivation to work on this aspect of git :-)
Perfect. I have to be careful here, I don't want to discourage experimentation and testing -- and if you have access to a pool of users who can beat on it, then great let's try it.
At the time, I didn't have such a "captive audience"... :-)
And I just wanted to be sure that everyone understood that some of these combinations have never been tried....
[...]
Quoted 4 lines
As far as multiple client machines mounting the remote repo, I have doubts that FSMonitor would even see changes made from another machine. Worth trying out and documenting as needed - might even be better off being considered as unsupported.
Yeah, that's another dimension. (1) is whether we miss events that we generated locally. (2) is whether events get relayed to us from other clients.
Quoted 26 lines
Quoted 9 lines
There was a suggestion later in this thread about using a SHA-1 or SHA-256 hash of the pathname to avoid the tmp XXXXXX pattern and just put the socket in $HOME (and omit the need for the new fsmonitor-daemon.ipc FILE completely). This might work, but we need to be careful because a user might have hardlinks or symlinks locally so there may be more than one unique path to the repo on the local system. (It is OK to have more than one daemon listening to a repo, just less efficient.)
Ah, I see.
Quoted 10 lines
As an interim step, you might try using my original socket code plus just the config.allowRemote=true change. And test it on a mounted repo where you've converted the .git directory to a .git file and moved contents of the .git directory to somewhere local. Then the UDS would be created in the local GITDIR instead of on the remote system. This won't help any of the sharing cases I described above, but will let you experiment with getting remote events.
Within the context of the environment that I have available to me (macOS over NFS to a Linux file server), FSEvents is working correctly. I can make changes at any arbitrary place inside of the repo and an event is generated.
Quoted 8 lines
It's looking like that the Unix domain socket (UDS) file should remain where it is unless fsmonitor.allowRemote is true.
If fsmonitor.allowRemote is true then the UDS file can be located in $HOME with the caveat that if there is more than one path to the repo (via hard or sym links) that things might not work as expected. I think that's OK given the experimental nature of the feature.
Yeah, I'd experiment with moving the UDS to $HOME (assuming you don't remote mount home directories too) and see how it works for you. Later, after we have more experience with it, we can talk about just having it always be in $HOME.
Jeff
Quoted 1 line · Eric DeCosta via GitGitGadget
Though perhaps not common, there are uses cases where users have large,
s/uses cases/use cases/
Quoted 11 lines
network-mounted repos. Having the ability to run fsmonitor against network paths would benefit those users.
As a first step towards enabling fsmonitor to work against network-mounted repos, a configuration option, 'fsmonitor.allowRemote' was introduced for Windows. Setting this option to true will override the default behavior (erroring-out) when a network-mounted repo is detected by fsmonitor. In order for macOS to have parity with Windows, the same option is now introduced for macOS.
The the added wrinkle being that the Unix domain socket (UDS) file
s/The the/The/
Quoted 4 lines
used for IPC cannot be created in a network location; instead the temporary directory is used.
Signed-off-by Eric DeCosta <[email protected]>
Quoted 1 line · Eric DeCosta via GitGitGadget
From: Eric DeCosta <[email protected]>
Just some comments on the commit message, please see inline.
Quoted 4 lines
Though perhaps not common, there are uses cases where users have large, network-mounted repos. Having the ability to run fsmonitor against network paths would benefit those users.
I think that we can drop the "Though perhaps not common" - it doesn't add any information about the technical part of the patch. (And that is, what is important)
And I personally use network-mounted repos every day ;-)
Sone users have large, network-mounted repos. Having the ability to run fsmonitor against network paths would benefit those users.
Quoted 6 lines
As a first step towards enabling fsmonitor to work against network-mounted repos, a configuration option, 'fsmonitor.allowRemote' was introduced for Windows. Setting this option to true will override the default behavior (erroring-out) when a network-mounted repo is detected by fsmonitor.
[]
The same option is now introduced for macOS.
Quoted 2 lines
The the added wrinkle being that the Unix domain socket (UDS) file
^^^ ^^^
Quoted 2 lines
used for IPC cannot be created in a network location; instead the temporary directory is used.
Quoted 6 lines
As a first step towards enabling fsmonitor to work against network-mounted repos, a configuration option, 'fsmonitor.allowRemote' was introduced for Windows. Setting this option to true will override the default behavior (erroring-out) when a network-mounted repo is detected by fsmonitor. In order for macOS to have parity with Windows, the same option is now introduced for macOS.
With this merged in, recent CI runs for 'seen'
e.g. https://github.com/git/git/actions/runs/2892889122
seems to break macOS jobs, letting them hog CPU forever and exceed 6hr or whatever the limit is.
As an experiment I pushed out 'seen' but without this commit (not the entire topic is excluded, the Windows one is still included). As there is nothing specific to macOS between 'next' and 'seen', macOS jobs seem to pass, which is not very surprising.
https://github.com/git/git/actions/runs/2896207171
As the patch collected some review comments, I've already marked it in the "What's cooking" draft as expecting a reroll of that step; until that happens, let's keep it out of 'seen'.
Thanks.
Hi Junio,
Quoted 19 lines · Junio C Hamano
Quoted 6 lines
As a first step towards enabling fsmonitor to work against network-mounted repos, a configuration option, 'fsmonitor.allowRemote' was introduced for Windows. Setting this option to true will override the default behavior (erroring-out) when a network-mounted repo is detected by fsmonitor. In order for macOS to have parity with Windows, the same option is now introduced for macOS.
With this merged in, recent CI runs for 'seen'
e.g. https://github.com/git/git/actions/runs/2892889122
seems to break macOS jobs, letting them hog CPU forever and exceed 6hr or whatever the limit is.
As an experiment I pushed out 'seen' but without this commit (not the entire topic is excluded, the Windows one is still included). As there is nothing specific to macOS between 'next' and 'seen', macOS jobs seem to pass, which is not very surprising.
Quoted 5 lines
https://github.com/git/git/actions/runs/2896207171
As the patch collected some review comments, I've already marked it in the "What's cooking" draft as expecting a reroll of that step; until that happens, let's keep it out of 'seen'.
It makes sense to keep it out of seen, and at the same time I would like to encourage Eric to investigate what causes those time-outs.
When toggling timestamps (click on the wheel on the upper right) at
https://github.com/git/git/runs/7927812510?check_suite_focus=true#step:4:1774,
it can be seen that at close to 1am, t9903 finished, but then nothing happens until twenty past 6am.
I've downloaded the raw logs (also available via the wheel on the upper right) to find out which test timed out:
$ diff -u \ <(sed -n 's/.*\] \(t[0-9][^ ]*\).*/\1/p' <~/Downloads/17 | sort) \ <(git ls-tree upstream/seen:t | cut -c 54- | grep '^t[0-9].*-.*sh$')
--- /dev/fd/63 2022-08-22 14:56:05.510269527 +0200 +++ /dev/fd/62 2022-08-22 14:56:05.510269527 +0200 @@ -794,6 +794,7 @@ t7524-commit-summary.sh t7525-status-rename.sh t7526-commit-pathspec-file.sh +t7527-builtin-fsmonitor.sh t7528-signed-commit-ssh.sh t7600-merge.sh t7601-merge-pull-config.sh @@ -945,6 +946,7 @@ t9812-git-p4-wildcards.sh t9813-git-p4-preserve-users.sh t9814-git-p4-rename.sh +t9815-git-p4-submit-fail.sh t9816-git-p4-locked.sh t9817-git-p4-exclude.sh t9818-git-p4-block.sh @@ -964,5 +966,8 @@ t9832-unshelve.sh t9833-errors.sh t9834-git-p4-file-dir-bug.sh +t9835-git-p4-metadata-encoding-python2.sh +t9836-git-p4-metadata-encoding-python3.sh t9901-git-web--browse.sh +t9902-completion.sh t9903-bash-prompt.sh
I have no idea what's up with t98* and t9902, but I would bet that they were somehow "swallowed" by prove being terminated, and that the actual test script that times out is t7527.
Eric: To investigate, you will want to reproduce the problem on a macOS machine. If you have none available, you could create a temporary branch, heavily edit the CI definition, and push it to GitHub. And by heavy edits I mean something like this:
- Remove all non-macOS jobs from .github/workflows/main.yml (that means removing all but the regular job, removing all but at least one macos matrix entry, and removing the the needs: ci-config and corresponding if: line.
- Edit t/Makefile to define T = t7527-builtin-fsmonitor.sh instead of running all the tests.
- Edit .github/workflows/main.yml so that the step that causes the time-out has a chance of timing out much sooner (and the subsequent steps then have a chance to upload the relevant logs):
https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepstimeout-minutes
If this does not shed any light into the issue, please let me know, I have a couple more aces up my sleeve.
Ciao,
Dscho
Quoted 13 lines · Johannes Schindelin
Eric: To investigate, you will want to reproduce the problem on a macOS machine. If you have none available, you could create a temporary branch, heavily edit the CI definition, and push it to GitHub. And by heavy edits I mean something like this:
- Remove all non-macOS jobs from
.github/workflows/main.yml(that means removing all but theregularjob, removing all but at least onemacosmatrix entry, and removing the theneeds: ci-configand correspondingif:line.- Edit
t/Makefileto defineT = t7527-builtin-fsmonitor.shinstead of running all the tests.
Quoted 7 lines
- Edit
.github/workflows/main.ymlso that the step that causes the time-out has a chance of timing out much sooner (and the subsequent steps then have a chance to upload the relevant logs):
https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepstimeout-minutesIf this does not shed any light into the issue, please let me know, I have a couple more aces up my sleeve.
Thanks, both.
Quoted 86 lines · Johannes Schindelin
Hi Junio,
Quoted 19 lines · Junio C Hamano
Quoted 6 lines
As a first step towards enabling fsmonitor to work against network-mounted repos, a configuration option, 'fsmonitor.allowRemote' was introduced for Windows. Setting this option to true will override the default behavior (erroring-out) when a network-mounted repo is detected by fsmonitor. In order for macOS to have parity with Windows, the same option is now introduced for macOS.
With this merged in, recent CI runs for 'seen'
e.g. https://github.com/git/git/actions/runs/2892889122
seems to break macOS jobs, letting them hog CPU forever and exceed 6hr or whatever the limit is.
As an experiment I pushed out 'seen' but without this commit (not the entire topic is excluded, the Windows one is still included). As there is nothing specific to macOS between 'next' and 'seen', macOS jobs seem to pass, which is not very surprising.
Quoted 5 lines
https://github.com/git/git/actions/runs/2896207171
As the patch collected some review comments, I've already marked it in the "What's cooking" draft as expecting a reroll of that step; until that happens, let's keep it out of 'seen'.
It makes sense to keep it out of
seen, and at the same time I would like to encourage Eric to investigate what causes those time-outs.When toggling timestamps (click on the wheel on the upper right) at
https://github.com/git/git/runs/7927812510?check_suite_focus=true#step:4:1774,
it can be seen that at close to 1am, t9903 finished, but then nothing happens until twenty past 6am.I've downloaded the raw logs (also available via the wheel on the upper right) to find out which test timed out:
$ diff -u \ <(sed -n 's/.*\] \(t[0-9][^ ]*\).*/\1/p' <~/Downloads/17 | sort) \ <(git ls-tree upstream/seen:t | cut -c 54- | grep '^t[0-9].*-.*sh$')--- /dev/fd/63 2022-08-22 14:56:05.510269527 +0200 +++ /dev/fd/62 2022-08-22 14:56:05.510269527 +0200 @@ -794,6 +794,7 @@ t7524-commit-summary.sh t7525-status-rename.sh t7526-commit-pathspec-file.sh +t7527-builtin-fsmonitor.sh t7528-signed-commit-ssh.sh t7600-merge.sh t7601-merge-pull-config.sh @@ -945,6 +946,7 @@ t9812-git-p4-wildcards.sh t9813-git-p4-preserve-users.sh t9814-git-p4-rename.sh +t9815-git-p4-submit-fail.sh t9816-git-p4-locked.sh t9817-git-p4-exclude.sh t9818-git-p4-block.sh @@ -964,5 +966,8 @@ t9832-unshelve.sh t9833-errors.sh t9834-git-p4-file-dir-bug.sh +t9835-git-p4-metadata-encoding-python2.sh +t9836-git-p4-metadata-encoding-python3.sh t9901-git-web--browse.sh +t9902-completion.sh t9903-bash-prompt.shI have no idea what's up with t98* and t9902, but I would bet that they were somehow "swallowed" by
provebeing terminated, and that the actual test script that times out is t7527.Eric: To investigate, you will want to reproduce the problem on a macOS machine. If you have none available, you could create a temporary branch, heavily edit the CI definition, and push it to GitHub. And by heavy edits I mean something like this:
- Remove all non-macOS jobs from
.github/workflows/main.yml(that means removing all but theregularjob, removing all but at least onemacosmatrix entry, and removing the theneeds: ci-configand correspondingif:line.
Quoted 7 lines
- Edit
t/Makefileto defineT = t7527-builtin-fsmonitor.shinstead of running all the tests.- Edit
.github/workflows/main.ymlso that the step that causes the time-out has a chance of timing out much sooner (and the subsequent steps then have a chance to upload the relevant logs):
https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepstimeout-minutes
I would also set GIT_TRACE_FSMONITOR and GIT_TRACE2_PERF (on both daemon and client sides of the tests) and capture the logs and try to figure out what is happening.
I suspect that this testing should wait until you redo the patch to remove the tmp file stuff and just move the socket into $HOME as we talked about earlier.
Jeff
Quoted 9 lines
I would also set
GIT_TRACE_FSMONITORandGIT_TRACE2_PERF(on both daemon and client sides of the tests) and capture the logs and try to figure out what is happening.I suspect that this testing should wait until you redo the patch to remove the tmp file stuff and just move the socket into $HOME as we talked about earlier.
Jeff
All tests are passing now with the new patch. By default the socket is written into the original location (.git directory) unless 'fsmonitor.allowRemote' is true. Only then is $HOME used and if $HOME proves unsuitable the user can override that by setting 'fsmonitor.sockerDir' to some valid, local directory.
-Eric
v2 cover fsmonitor: option to allow fsmonitor to run against network-mounted repos
In reply to Eric DeCosta via GitGitGadgetcc: Johannes Schindelin [email protected] cc: Jeff Hostetler [email protected] cc: Eric Sunshine [email protected] cc: Torsten Bögershausen [email protected]
Eric DeCosta (2):
fsmonitor: option to allow fsmonitor to run against network-mounted
repos
fsmonitor: macOS: allow fsmonitor to run against network-mounted repos
edecosta (2):
Check working directory and Unix domain socket file for compatability Minor refactoring and simplification of Windows settings checks
compat/fsmonitor/fsm-settings-darwin.c | 72 +++++++++++++++++++------- compat/fsmonitor/fsm-settings-win32.c | 47 ++++++++++++++++- fsmonitor-ipc.c | 40 ++++++++++++-- fsmonitor-ipc.h | 6 +++ fsmonitor-settings.c | 67 +++++++++++++++++++++++- fsmonitor-settings.h | 4 ++ 6 files changed, 210 insertions(+), 26 deletions(-)
base-commit: 9bf691b78cf906751e65d65ba0c6ffdcd9a5a12c
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1326%2Fedecosta-mw%2Ffsmonitor_macos-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1326/edecosta-mw/fsmonitor_macos-v2
Pull-Request: https://github.com/gitgitgadget/git/pull/1326
Range-diff vs v1:
-: ----------- > 1: 40ce21e85c9 fsmonitor: option to allow fsmonitor to run against network-mounted repos
-: ----------- > 2: 46b4efd96cc fsmonitor: macOS: allow fsmonitor to run against network-mounted repos
1: 425df16ac0b ! 3: 9b128a98149 fsmonitor: option to allow fsmonitor to run against network-mounted repos
@@
## Metadata ##
-Author: Eric DeCosta <[email protected]>
+Author: edecosta <[email protected]>
## Commit message ##
- fsmonitor: option to allow fsmonitor to run against network-mounted repos
+ Check working directory and Unix domain socket file for compatability
- Though perhaps not common, there are uses cases where users have large,
- network-mounted repos. Having the ability to run fsmonitor against
- network paths would benefit those users.
+ Perform separate checks for the working directory and Unix domain socket
+ (UDS) file location. The working directory may be located on a
+ network-mounted file system if 'fsmonitor.allowRemote' is true. The UDS
+ file may never be located on a network-mounted file system; additionally
+ it may not be located on FAT32 or NTFS file systems.
- As a first step towards enabling fsmonitor to work against
- network-mounted repos, a configuration option, 'fsmonitor.allowRemote'
- was introduced for Windows. Setting this option to true will override
- the default behavior (erroring-out) when a network-mounted repo is
- detected by fsmonitor. In order for macOS to have parity with Windows,
- the same option is now introduced for macOS.
-
- The the added wrinkle being that the Unix domain socket (UDS) file
- used for IPC cannot be created in a network location; instead the
- temporary directory is used.
-
- Signed-off-by: Eric DeCosta <[email protected]>
+ Signed-off-by: edecosta <[email protected]>
## compat/fsmonitor/fsm-settings-darwin.c ##
@@
- #include "config.h"
#include "repository.h"
#include "fsmonitor-settings.h"
-+#include "fsmonitor-ipc.h"
#include "fsmonitor.h"
++#include "fsmonitor-ipc.h"
#include <sys/param.h>
#include <sys/mount.h>
-+/*
-+ * Check if monitoring remote working directories is allowed.
-+ *
-+ * By default, monitoring remote working directories is
-+ * disabled. Users may override this behavior in enviroments where
-+ * they have proper support.
-+ */
-+static int check_config_allowremote(struct repository *r)
-+{
-+ int allow;
-+
-+ if (!repo_config_get_bool(r, "fsmonitor.allowremote", &allow))
-+ return allow;
-+
-+ return -1; /* fsmonitor.allowremote not set */
-+}
-+
/*
- * [1] Remote working directories are problematic for FSMonitor.
+- * [1] Remote working directories are problematic for FSMonitor.
++ * Remote working directories are problematic for FSMonitor.
*
+ * The underlying file system on the server machine and/or the remote
+ * mount type (NFS, SAMBA, etc.) dictates whether notification events
@@
* In theory, the above issues need to be addressed whether we are
* using the Hook or IPC API.
*
-+ * So (for now at least), mark remote working directories as
-+ * incompatible by default.
-+ *
- * For the builtin FSMonitor, we create the Unix domain socket for the
+- * For the builtin FSMonitor, we create the Unix domain socket for the
- * IPC in the .git directory. If the working directory is remote,
- * then the socket will be created on the remote file system. This
- * can fail if the remote file system does not support UDS file types
@@ compat/fsmonitor/fsm-settings-darwin.c
- * well-known local directory on the client machine, but care should
- * be taken to ensure that $HOME is actually local and not a managed
- * file share.)
-+ * IPC in the temporary directory. If the temporary directory is
-+ * remote, then the socket will be created on the remote file system.
-+ * This can fail if the remote file system does not support UDS file
-+ * types (e.g. smbfs to a Windows server) or if the remote kernel does
-+ * not allow a non-local process to bind() the socket.
- *
-- * So (for now at least), mark remote working directories as
+- *
+ * So (for now at least), mark remote working directories as
- * incompatible.
-+ * Therefore remote UDS locations are marked as incompatible.
- *
- *
- * [2] FAT32 and NTFS working directories are problematic too.
- *
+- *
+- *
+- * [2] FAT32 and NTFS working directories are problematic too.
+- *
- * The builtin FSMonitor uses a Unix domain socket in the .git
-+ * The builtin FSMonitor uses a Unix domain socket in the temporary
- * directory for IPC. These Windows drive formats do not support
- * Unix domain sockets, so mark them as incompatible for the daemon.
+- * directory for IPC. These Windows drive formats do not support
+- * Unix domain sockets, so mark them as incompatible for the daemon.
++ * incompatible unless fsmonitor.allowRemote is true.
*
+ */
+ static enum fsmonitor_reason check_volume(struct repository *r)
@@ compat/fsmonitor/fsm-settings-darwin.c: static enum fsmonitor_reason check_volume(struct repository *r)
"statfs('%s') [type 0x%08x][flags 0x%08x] '%s'",
r->worktree, fs.f_type, fs.f_flags, fs.f_fstypename);
-+ if (!(fs.f_flags & MNT_LOCAL)) {
-+ switch (check_config_allowremote(r)) {
-+ case 0: /* config overrides and disables */
++ if (!(fs.f_flags & MNT_LOCAL)
++ && (fsm_settings__get_allow_remote(r) < 1))
+ return FSMONITOR_REASON_REMOTE;
-+ case 1: /* config overrides and enables */
-+ return FSMONITOR_REASON_OK;
-+ default:
-+ break; /* config has no opinion */
-+ }
-+
-+ return FSMONITOR_REASON_REMOTE;
-+ }
+
+ return FSMONITOR_REASON_OK;
+}
+
++/*
++ * For the builtin FSMonitor, we create the Unix domain socket (UDS)
++ * for the IPC in the .git directory by default or $HOME if
++ * fsmonitor.allowRemote is true. If the directory is remote,
++ * then the socket will be created on the remote file system. This
++ * can fail if the remote file system does not support UDS file types
++ * (e.g. smbfs to a Windows server) or if the remote kernel does not
++ * allow a non-local process to bind() the socket.
++ *
++ * Therefore remote UDS locations are marked as incompatible.
++ *
++ * FAT32 and NTFS working directories are problematic too.
++ *
++ * These Windows drive formats do not support Unix domain sockets, so
++ * mark them as incompatible for the location of the UDS file.
++ *
++ */
+static enum fsmonitor_reason check_uds_volume(void)
+{
+ struct statfs fs;
-+ const char *path = fsmonitor_ipc__get_path();
++ struct strbuf path = STRBUF_INIT;
++ const char *ipc_path = fsmonitor_ipc__get_path();
++ strbuf_add(&path, ipc_path, strlen(ipc_path));
+
-+ if (statfs(path, &fs) == -1) {
++ if (statfs(dirname(path.buf), &fs) == -1) {
+ int saved_errno = errno;
+ trace_printf_key(&trace_fsmonitor, "statfs('%s') failed: %s",
-+ path, strerror(saved_errno));
++ path.buf, strerror(saved_errno));
+ errno = saved_errno;
++ strbuf_release(&path);
+ return FSMONITOR_REASON_ERROR;
+ }
+
+ trace_printf_key(&trace_fsmonitor,
+ "statfs('%s') [type 0x%08x][flags 0x%08x] '%s'",
-+ path, fs.f_type, fs.f_flags, fs.f_fstypename);
++ path.buf, fs.f_type, fs.f_flags, fs.f_fstypename);
++ strbuf_release(&path);
+
if (!(fs.f_flags & MNT_LOCAL))
return FSMONITOR_REASON_REMOTE;
@@ compat/fsmonitor/fsm-settings-darwin.c: enum fsmonitor_reason fsm_os__incompatib
+
return FSMONITOR_REASON_OK;
}
-
- ## fsmonitor-ipc.c ##
-@@
- #include "fsmonitor-ipc.h"
- #include "run-command.h"
- #include "strbuf.h"
-+#include "tempfile.h"
- #include "trace2.h"
-
- #ifndef HAVE_FSMONITOR_DAEMON_BACKEND
-@@ fsmonitor-ipc.c: int fsmonitor_ipc__is_supported(void)
- return 1;
- }
-
--GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc")
-+GIT_PATH_FUNC(fsmonitor_ipc__get_pathfile, "fsmonitor--daemon.ipc")
-+
-+static char *gen_ipc_file(void)
-+{
-+ char *retval = NULL;
-+ struct tempfile *ipc;
-+
-+ const char *ipc_file = fsmonitor_ipc__get_pathfile();
-+ FILE *fp = fopen(ipc_file, "w");
-+
-+ if (!fp)
-+ die_errno("error opening '%s'", ipc_file);
-+ ipc = mks_tempfile_t("fsmonitor_ipc_XXXXXX");
-+ strbuf_write(&ipc->filename, fp);
-+ fclose(fp);
-+ retval = strbuf_detach(&ipc->filename, NULL);
-+ strbuf_release(&ipc->filename);
-+ return retval;
-+}
-+
-+const char *fsmonitor_ipc__get_path(void)
-+{
-+ char *retval = NULL;
-+ struct strbuf sb = STRBUF_INIT;
-+
-+ const char *ipc_file = fsmonitor_ipc__get_pathfile();
-+ FILE *fp = fopen(ipc_file, "r");
-+
-+ if (!fp) {
-+ return gen_ipc_file();
-+ } else {
-+ strbuf_read(&sb, fileno(fp), 0);
-+ fclose(fp);
-+ fp = fopen(sb.buf, "r");
-+ if (!fp) { /* generate new file */
-+ if (unlink(ipc_file) < 0)
-+ die_errno("could not remove '%s'", ipc_file);
-+ return gen_ipc_file();
-+ }
-+ fclose(fp);
-+ retval = strbuf_detach(&sb, NULL);
-+ strbuf_release(&sb);
-+ return retval;
-+ }
-+}
-
- enum ipc_active_state fsmonitor_ipc__get_state(void)
- {
-
- ## fsmonitor-ipc.h ##
-@@ fsmonitor-ipc.h: int fsmonitor_ipc__is_supported(void);
- */
- const char *fsmonitor_ipc__get_path(void);
-
-+/*
-+ * Returns the pathname to the file that contains the pathname to the
-+ * IPC named pipe or Unix domain socket.
-+ */
-+const char *fsmonitor_ipc__get_pathfile(void);
-+
- /*
- * Try to determine whether there is a `git-fsmonitor--daemon` process
- * listening on the IPC pipe/socket. -: ----------- > 4: 15c965801f8 Minor refactoring and simplification of Windows settings checks
Signature · 1 line
gitgitgadget
PATCH v2 1/4 fsmonitor: option to allow fsmonitor to run against network-mounted repos
From: Eric DeCosta <[email protected]>
Though perhaps not common, there are use cases where users have large, network-mounted repos. Having the ability to run fsmonitor against network paths would benefit those users.
Most modern Samba-based filers have the necessary support to enable fsmonitor on network-mounted repos. As a first step towards enabling fsmonitor to work against network-mounted repos, introduce a configuration option, 'fsmonitor.allowRemote'. Setting this option to true will override the default behavior (erroring-out) when a network-mounted repo is detected by fsmonitor.
--- compat/fsmonitor/fsm-settings-win32.c | 68 +++++++++++++++++++++++++++ 1 file changed, 68 insertions(+)
compat/fsmonitor/fsm-settings-win32.c +68 -0
@@ -24,6 +24,59 @@ static enum fsmonitor_reason check_vfs4git(struct repository *r)return FSMONITOR_REASON_OK ; } +/* +* Check if monitoring remote working directories is allowed. +* +* By default, monitoring remote working directories is +* disabled. Users may override this behavior in enviroments where +* they have proper support. +*/ +static int check_config_allowremote (struct repository *r) +{ +int allow; + +if (!repo_config_get_bool (r,"fsmonitor.allowremote" , &allow)) +return allow; + +return -1 ;/* fsmonitor.allowremote not set */ +} + +/* +* Check remote working directory protocol. +* +* Error if client machine cannot get remote protocol information. +*/ +static int check_remote_protocol (wchar_t *wpath) +{ +HANDLE h; +FILE_REMOTE_PROTOCOL_INFO proto_info; + + h =CreateFileW (wpath,GENERIC_READ ,FILE_SHARE_READ ,NULL ,OPEN_EXISTING , +FILE_FLAG_BACKUP_SEMANTICS ,NULL ); + +if (h ==INVALID_HANDLE_VALUE ) { +error (_ ("[GLE %ld] unable to open for read '%ls'" ), +GetLastError (), wpath); +return -1 ; + } + +if (!GetFileInformationByHandleEx (h, FileRemoteProtocolInfo, + &proto_info,sizeof (proto_info))) { +error (_ ("[GLE %ld] unable to get protocol information for '%ls'" ), +GetLastError (), wpath); +CloseHandle (h); +return -1 ; + } + +CloseHandle (h); + +trace_printf_key (&trace_fsmonitor, +"check_remote_protocol('%ls') remote protocol %#8.8lx" , + wpath, proto_info.Protocol); + +return 0 ; +} +/* * Remote working directories are problematic for FSMonitor. *
@@ -76,6 +129,7 @@ static enum fsmonitor_reason check_vfs4git(struct repository *r)*/ static enum fsmonitor_reason check_remote (struct repository *r) { +int ret;wchar_t wpath[MAX_PATH ];wchar_t wfullpath[MAX_PATH ];size_t wlen;
@@ -115,6 +169,20 @@ static enum fsmonitor_reason check_remote(struct repository *r)trace_printf_key (&trace_fsmonitor,"check_remote('%s') true" , r->worktree); + + ret =check_remote_protocol (wfullpath); +if (ret <0 ) +return FSMONITOR_REASON_ERROR ; + +switch (check_config_allowremote (r)) { +case 0 :/* config overrides and disables */ +return FSMONITOR_REASON_REMOTE ; +case 1 :/* config overrides and enables */ +return FSMONITOR_REASON_OK ; +default : +break ;/* config has no opinion */ + } +return FSMONITOR_REASON_REMOTE ; }
Signature · 1 line
gitgitgadget
PATCH v2 2/4 fsmonitor: macOS: allow fsmonitor to run against network-mounted repos
In reply to Eric DeCosta via GitGitGadgetFrom: Eric DeCosta <[email protected]>
Follow-on to the work done to allow Windows to work against network-mounted repos. Have macOS take advantage of the same configuration option, 'fsmonitor.allowRemote' that was introduced for Windows. Setting this option to true will override the default behavior (erroring-out) when a network-mounted repo is detected by fsmonitor.
The added wrinkle being that the Unix domain socket (UDS) file used for IPC cannot be created in a network location; instead $HOME is used if 'fsmonitor.allowRemote' is true.
If $HOME is in a network location, allow the user to override this via the 'fsmonitor.socketDir' configuration option. There the user can specify any local directory for the location of the UDS file.
--- fsmonitor-ipc.c | 40 ++++++++++++++++++++++++-- fsmonitor-ipc.h | 6 ++++ fsmonitor-settings.c | 67 ++++++++++++++++++++++++++++++++++++++++++-- fsmonitor-settings.h | 4 +++ 4 files changed, 112 insertions(+), 5 deletions(-)
fsmonitor-ipc.c +37 -3
@@ -1,7 +1,6 @@#include "cache.h" -#include "fsmonitor.h" -#include "simple-ipc.h" #include "fsmonitor-ipc.h" +#include "fsmonitor-settings.h" #include "run-command.h" #include "strbuf.h" #include "trace2.h"
@@ -47,7 +46,42 @@ int fsmonitor_ipc__is_supported(void)return 1 ; } -GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc") +GIT_PATH_FUNC(fsmonitor_ipc__get_default_path, "fsmonitor--daemon.ipc") + +const char *fsmonitor_ipc__get_path (void ) +{ +#ifdef WIN32 +return fsmonitor_ipc__get_default_path (); +#else +char *retval; +SHA_CTX sha1ctx; +const char *git_dir; +const char *sock_dir; +struct strbuf ipc_file =STRBUF_INIT ; +unsigned char hash[SHA_DIGEST_LENGTH ]; + +if (fsm_settings__get_allow_remote (the_repository) <1 ) +return fsmonitor_ipc__get_default_path (); + + git_dir =get_git_dir (); + sock_dir =fsm_settings__get_socket_dir (the_repository); + +SHA1_Init (&sha1ctx); +SHA1_Update (&sha1ctx, git_dir,strlen (git_dir)); +SHA1_Final (hash, &sha1ctx); + +if (sock_dir && *sock_dir) +strbuf_addf (&ipc_file,"%s/.git-fsmonitor-%s" , + sock_dir,hash_to_hex (hash)); +else +strbuf_addf (&ipc_file,"~/.git-fsmonitor-%s" ,hash_to_hex (hash)); + retval =interpolate_path (ipc_file.buf,1 ); +if (!retval) +die (_ ("Invalid path: %s" ), ipc_file.buf); +strbuf_release (&ipc_file); +return retval; +#endif +}enum ipc_active_state fsmonitor_ipc__get_state (void ) {
fsmonitor-ipc.h +6 -0
@@ -18,6 +18,12 @@ int fsmonitor_ipc__is_supported(void);*/ const char *fsmonitor_ipc__get_path (void ); +/* +* Returns the pathname to the default IPC named pipe or Unix domain +* socket. +*/ +const char *fsmonitor_ipc__get_default_path (void ); +/* * Try to determine whether there is a `git-fsmonitor--daemon` process * listening on the IPC pipe/socket.
fsmonitor-settings.c +65 -2
106 lines not on this page · raw message
fsmonitor-settings.h +4 -0
@@ -23,12 +23,16 @@ enum fsmonitor_reason { FSMONITOR_REASON_NOSOCKETS, /* NTFS,FAT32 do not support Unix sockets */ }; +void fsm_settings__set_allow_remote (struct repository *r); +void fsm_settings__set_socket_dir (struct repository *r);void fsm_settings__set_ipc (struct repository *r);void fsm_settings__set_hook (struct repository *r,const char *path);void fsm_settings__set_disabled (struct repository *r);void fsm_settings__set_incompatible (struct repository *r,enum fsmonitor_reason reason); +int fsm_settings__get_allow_remote (struct repository *r); +const char *fsm_settings__get_socket_dir (struct repository *r);enum fsmonitor_mode fsm_settings__get_mode (struct repository *r);const char *fsm_settings__get_hook_path (struct repository *r);
Signature · 1 line
gitgitgadget
PATCH v2 3/4 Check working directory and Unix domain socket file for compatability
In reply to Eric DeCosta via GitGitGadgetFrom: edecosta <[email protected]>
Perform separate checks for the working directory and Unix domain socket (UDS) file location. The working directory may be located on a network-mounted file system if 'fsmonitor.allowRemote' is true. The UDS file may never be located on a network-mounted file system; additionally it may not be located on FAT32 or NTFS file systems.
--- compat/fsmonitor/fsm-settings-darwin.c | 72 +++++++++++++++++++------- 1 file changed, 52 insertions(+), 20 deletions(-)
compat/fsmonitor/fsm-settings-darwin.c +52 -20
@@ -3,11 +3,12 @@#include "repository.h" #include "fsmonitor-settings.h" #include "fsmonitor.h" +#include "fsmonitor-ipc.h" #include <sys/param.h> #include <sys/mount.h> /* -* [1] Remote working directories are problematic for FSMonitor. +* Remote working directories are problematic for FSMonitor. * * The underlying file system on the server machine and/or the remote * mount type (NFS, SAMBA, etc.) dictates whether notification events
@@ -27,26 +28,8 @@* In theory, the above issues need to be addressed whether we are * using the Hook or IPC API. * -* For the builtin FSMonitor, we create the Unix domain socket for the -* IPC in the .git directory. If the working directory is remote, -* then the socket will be created on the remote file system. This -* can fail if the remote file system does not support UDS file types -* (e.g. smbfs to a Windows server) or if the remote kernel does not -* allow a non-local process to bind() the socket. (These problems -* could be fixed by moving the UDS out of the .git directory and to a -* well-known local directory on the client machine, but care should -* be taken to ensure that $HOME is actually local and not a managed -* file share.) -* * So (for now at least), mark remote working directories as -* incompatible. -* -* -* [2] FAT32 and NTFS working directories are problematic too. -* -* The builtin FSMonitor uses a Unix domain socket in the .git -* directory for IPC. These Windows drive formats do not support -* Unix domain sockets, so mark them as incompatible for the daemon. +* incompatible unless fsmonitor.allowRemote is true. * */ static enum fsmonitor_reason check_volume (struct repository *r)
@@ -65,6 +48,51 @@ static enum fsmonitor_reason check_volume(struct repository *r)"statfs('%s') [type 0x%08x][flags 0x%08x] '%s'" , r->worktree, fs.f_type, fs.f_flags, fs.f_fstypename); +if (!(fs.f_flags &MNT_LOCAL ) + && (fsm_settings__get_allow_remote (r) <1 )) +return FSMONITOR_REASON_REMOTE ; + +return FSMONITOR_REASON_OK ; +} + +/* +* For the builtin FSMonitor, we create the Unix domain socket (UDS) +* for the IPC in the .git directory by default or $HOME if +* fsmonitor.allowRemote is true. If the directory is remote, +* then the socket will be created on the remote file system. This +* can fail if the remote file system does not support UDS file types +* (e.g. smbfs to a Windows server) or if the remote kernel does not +* allow a non-local process to bind() the socket. +* +* Therefore remote UDS locations are marked as incompatible. +* +* FAT32 and NTFS working directories are problematic too. +* +* These Windows drive formats do not support Unix domain sockets, so +* mark them as incompatible for the location of the UDS file. +* +*/ +static enum fsmonitor_reason check_uds_volume (void ) +{ +struct statfs fs; +struct strbuf path =STRBUF_INIT ; +const char *ipc_path =fsmonitor_ipc__get_path (); +strbuf_add (&path, ipc_path,strlen (ipc_path)); + +if (statfs (dirname (path.buf), &fs) ==-1 ) { +int saved_errno = errno; +trace_printf_key (&trace_fsmonitor,"statfs('%s') failed: %s" , + path.buf,strerror (saved_errno)); + errno = saved_errno; +strbuf_release (&path); +return FSMONITOR_REASON_ERROR ; + } + +trace_printf_key (&trace_fsmonitor, +"statfs('%s') [type 0x%08x][flags 0x%08x] '%s'" , + path.buf, fs.f_type, fs.f_flags, fs.f_fstypename); +strbuf_release (&path); +if (!(fs.f_flags &MNT_LOCAL ))return FSMONITOR_REASON_REMOTE ;
@@ -85,5 +113,9 @@ enum fsmonitor_reason fsm_os__incompatible(struct repository *r)if (reason !=FSMONITOR_REASON_OK )return reason; + reason =check_uds_volume (); +if (reason !=FSMONITOR_REASON_OK ) +return reason; +return FSMONITOR_REASON_OK ; }
Signature · 1 line
gitgitgadget
PATCH v2 4/4 Minor refactoring and simplification of Windows settings checks
In reply to Eric DeCosta via GitGitGadgetFrom: edecosta <[email protected]>
Read the value of 'fsmonitor.allowRemote' from fsmonitor_settings via fsm_settings__get_allow_remote getter. Remove check_config_allowremote. Replace switch statement with a simpler if statement. Move the check for 'fsmonitor.allowRemote' above the remote protocol check.
--- compat/fsmonitor/fsm-settings-win32.c | 31 ++++----------------------- 1 file changed, 4 insertions(+), 27 deletions(-)
compat/fsmonitor/fsm-settings-win32.c +4 -27
@@ -24,23 +24,6 @@ static enum fsmonitor_reason check_vfs4git(struct repository *r)return FSMONITOR_REASON_OK ; } -/* -* Check if monitoring remote working directories is allowed. -* -* By default, monitoring remote working directories is -* disabled. Users may override this behavior in enviroments where -* they have proper support. -*/ -static int check_config_allowremote (struct repository *r) -{ -int allow; - -if (!repo_config_get_bool (r,"fsmonitor.allowremote" , &allow)) -return allow; - -return -1 ;/* fsmonitor.allowremote not set */ -} -/* * Check remote working directory protocol. *
@@ -170,20 +153,14 @@ static enum fsmonitor_reason check_remote(struct repository *r)"check_remote('%s') true" , r->worktree); +if (fsm_settings__get_allow_remote (r) <1 ) +return FSMONITOR_REASON_REMOTE ; + ret =check_remote_protocol (wfullpath);if (ret <0 )return FSMONITOR_REASON_ERROR ; -switch (check_config_allowremote (r)) { -case 0 :/* config overrides and disables */ -return FSMONITOR_REASON_REMOTE ; -case 1 :/* config overrides and enables */ -return FSMONITOR_REASON_OK ; -default : -break ;/* config has no opinion */ - } - -return FSMONITOR_REASON_REMOTE ; +return FSMONITOR_REASON_OK ; }return FSMONITOR_REASON_OK ;
Signature · 1 line
gitgitgadget
v3 cover fsmonitor: option to allow fsmonitor to run against network-mounted repos
In reply to Eric DeCosta via GitGitGadgetcc: Johannes Schindelin [email protected] cc: Jeff Hostetler [email protected] cc: Eric Sunshine [email protected] cc: Torsten Bögershausen [email protected]
Eric DeCosta (1):
fsmonitor: macOS: allow fsmonitor to run against network-mounted repos
edecosta (1):
Check working directory and Unix domain socket file for compatability
compat/fsmonitor/fsm-settings-darwin.c | 72 +++++++++++++++++++------- fsmonitor-ipc.c | 40 ++++++++++++-- fsmonitor-ipc.h | 6 +++ fsmonitor-settings.c | 67 +++++++++++++++++++++++- fsmonitor-settings.h | 4 ++ 5 files changed, 164 insertions(+), 25 deletions(-)
base-commit: 795ea8776befc95ea2becd8020c7a284677b4161
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1326%2Fedecosta-mw%2Ffsmonitor_macos-v3
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1326/edecosta-mw/fsmonitor_macos-v3
Pull-Request: https://github.com/gitgitgadget/git/pull/1326
Range-diff vs v2:
1: 40ce21e85c9 < -: ----------- fsmonitor: option to allow fsmonitor to run against network-mounted repos 2: 46b4efd96cc = 1: cd16d8bb3d6 fsmonitor: macOS: allow fsmonitor to run against network-mounted repos 3: 9b128a98149 = 2: f977d140afa Check working directory and Unix domain socket file for compatability 4: 15c965801f8 < -: ----------- Minor refactoring and simplification of Windows settings checks
Signature · 1 line
gitgitgadget
PATCH v3 1/2 fsmonitor: macOS: allow fsmonitor to run against network-mounted repos
From: Eric DeCosta <[email protected]>
Follow-on to the work done to allow Windows to work against network-mounted repos. Have macOS take advantage of the same configuration option, 'fsmonitor.allowRemote' that was introduced for Windows. Setting this option to true will override the default behavior (erroring-out) when a network-mounted repo is detected by fsmonitor.
The added wrinkle being that the Unix domain socket (UDS) file used for IPC cannot be created in a network location; instead $HOME is used if 'fsmonitor.allowRemote' is true.
If $HOME is in a network location, allow the user to override this via the 'fsmonitor.socketDir' configuration option. There the user can specify any local directory for the location of the UDS file.
--- fsmonitor-ipc.c | 40 ++++++++++++++++++++++++-- fsmonitor-ipc.h | 6 ++++ fsmonitor-settings.c | 67 ++++++++++++++++++++++++++++++++++++++++++-- fsmonitor-settings.h | 4 +++ 4 files changed, 112 insertions(+), 5 deletions(-)
fsmonitor-ipc.c +37 -3
@@ -1,7 +1,6 @@#include "cache.h" -#include "fsmonitor.h" -#include "simple-ipc.h" #include "fsmonitor-ipc.h" +#include "fsmonitor-settings.h" #include "run-command.h" #include "strbuf.h" #include "trace2.h"
@@ -47,7 +46,42 @@ int fsmonitor_ipc__is_supported(void)return 1 ; } -GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc") +GIT_PATH_FUNC(fsmonitor_ipc__get_default_path, "fsmonitor--daemon.ipc") + +const char *fsmonitor_ipc__get_path (void ) +{ +#ifdef WIN32 +return fsmonitor_ipc__get_default_path (); +#else +char *retval; +SHA_CTX sha1ctx; +const char *git_dir; +const char *sock_dir; +struct strbuf ipc_file =STRBUF_INIT ; +unsigned char hash[SHA_DIGEST_LENGTH ]; + +if (fsm_settings__get_allow_remote (the_repository) <1 ) +return fsmonitor_ipc__get_default_path (); + + git_dir =get_git_dir (); + sock_dir =fsm_settings__get_socket_dir (the_repository); + +SHA1_Init (&sha1ctx); +SHA1_Update (&sha1ctx, git_dir,strlen (git_dir)); +SHA1_Final (hash, &sha1ctx); + +if (sock_dir && *sock_dir) +strbuf_addf (&ipc_file,"%s/.git-fsmonitor-%s" , + sock_dir,hash_to_hex (hash)); +else +strbuf_addf (&ipc_file,"~/.git-fsmonitor-%s" ,hash_to_hex (hash)); + retval =interpolate_path (ipc_file.buf,1 ); +if (!retval) +die (_ ("Invalid path: %s" ), ipc_file.buf); +strbuf_release (&ipc_file); +return retval; +#endif +}enum ipc_active_state fsmonitor_ipc__get_state (void ) {
fsmonitor-ipc.h +6 -0
@@ -18,6 +18,12 @@ int fsmonitor_ipc__is_supported(void);*/ const char *fsmonitor_ipc__get_path (void ); +/* +* Returns the pathname to the default IPC named pipe or Unix domain +* socket. +*/ +const char *fsmonitor_ipc__get_default_path (void ); +/* * Try to determine whether there is a `git-fsmonitor--daemon` process * listening on the IPC pipe/socket.
fsmonitor-settings.c +65 -2
106 lines not on this page · raw message
fsmonitor-settings.h +4 -0
@@ -23,12 +23,16 @@ enum fsmonitor_reason { FSMONITOR_REASON_NOSOCKETS, /* NTFS,FAT32 do not support Unix sockets */ }; +void fsm_settings__set_allow_remote (struct repository *r); +void fsm_settings__set_socket_dir (struct repository *r);void fsm_settings__set_ipc (struct repository *r);void fsm_settings__set_hook (struct repository *r,const char *path);void fsm_settings__set_disabled (struct repository *r);void fsm_settings__set_incompatible (struct repository *r,enum fsmonitor_reason reason); +int fsm_settings__get_allow_remote (struct repository *r); +const char *fsm_settings__get_socket_dir (struct repository *r);enum fsmonitor_mode fsm_settings__get_mode (struct repository *r);const char *fsm_settings__get_hook_path (struct repository *r);
Signature · 1 line
gitgitgadget
PATCH v3 2/2 Check working directory and Unix domain socket file for compatability
In reply to Eric DeCosta via GitGitGadgetFrom: edecosta <[email protected]>
Perform separate checks for the working directory and Unix domain socket (UDS) file location. The working directory may be located on a network-mounted file system if 'fsmonitor.allowRemote' is true. The UDS file may never be located on a network-mounted file system; additionally it may not be located on FAT32 or NTFS file systems.
--- compat/fsmonitor/fsm-settings-darwin.c | 72 +++++++++++++++++++------- 1 file changed, 52 insertions(+), 20 deletions(-)
compat/fsmonitor/fsm-settings-darwin.c +52 -20
@@ -3,11 +3,12 @@#include "repository.h" #include "fsmonitor-settings.h" #include "fsmonitor.h" +#include "fsmonitor-ipc.h" #include <sys/param.h> #include <sys/mount.h> /* -* [1] Remote working directories are problematic for FSMonitor. +* Remote working directories are problematic for FSMonitor. * * The underlying file system on the server machine and/or the remote * mount type (NFS, SAMBA, etc.) dictates whether notification events
@@ -27,26 +28,8 @@* In theory, the above issues need to be addressed whether we are * using the Hook or IPC API. * -* For the builtin FSMonitor, we create the Unix domain socket for the -* IPC in the .git directory. If the working directory is remote, -* then the socket will be created on the remote file system. This -* can fail if the remote file system does not support UDS file types -* (e.g. smbfs to a Windows server) or if the remote kernel does not -* allow a non-local process to bind() the socket. (These problems -* could be fixed by moving the UDS out of the .git directory and to a -* well-known local directory on the client machine, but care should -* be taken to ensure that $HOME is actually local and not a managed -* file share.) -* * So (for now at least), mark remote working directories as -* incompatible. -* -* -* [2] FAT32 and NTFS working directories are problematic too. -* -* The builtin FSMonitor uses a Unix domain socket in the .git -* directory for IPC. These Windows drive formats do not support -* Unix domain sockets, so mark them as incompatible for the daemon. +* incompatible unless fsmonitor.allowRemote is true. * */ static enum fsmonitor_reason check_volume (struct repository *r)
@@ -65,6 +48,51 @@ static enum fsmonitor_reason check_volume(struct repository *r)"statfs('%s') [type 0x%08x][flags 0x%08x] '%s'" , r->worktree, fs.f_type, fs.f_flags, fs.f_fstypename); +if (!(fs.f_flags &MNT_LOCAL ) + && (fsm_settings__get_allow_remote (r) <1 )) +return FSMONITOR_REASON_REMOTE ; + +return FSMONITOR_REASON_OK ; +} + +/* +* For the builtin FSMonitor, we create the Unix domain socket (UDS) +* for the IPC in the .git directory by default or $HOME if +* fsmonitor.allowRemote is true. If the directory is remote, +* then the socket will be created on the remote file system. This +* can fail if the remote file system does not support UDS file types +* (e.g. smbfs to a Windows server) or if the remote kernel does not +* allow a non-local process to bind() the socket. +* +* Therefore remote UDS locations are marked as incompatible. +* +* FAT32 and NTFS working directories are problematic too. +* +* These Windows drive formats do not support Unix domain sockets, so +* mark them as incompatible for the location of the UDS file. +* +*/ +static enum fsmonitor_reason check_uds_volume (void ) +{ +struct statfs fs; +struct strbuf path =STRBUF_INIT ; +const char *ipc_path =fsmonitor_ipc__get_path (); +strbuf_add (&path, ipc_path,strlen (ipc_path)); + +if (statfs (dirname (path.buf), &fs) ==-1 ) { +int saved_errno = errno; +trace_printf_key (&trace_fsmonitor,"statfs('%s') failed: %s" , + path.buf,strerror (saved_errno)); + errno = saved_errno; +strbuf_release (&path); +return FSMONITOR_REASON_ERROR ; + } + +trace_printf_key (&trace_fsmonitor, +"statfs('%s') [type 0x%08x][flags 0x%08x] '%s'" , + path.buf, fs.f_type, fs.f_flags, fs.f_fstypename); +strbuf_release (&path); +if (!(fs.f_flags &MNT_LOCAL ))return FSMONITOR_REASON_REMOTE ;
@@ -85,5 +113,9 @@ enum fsmonitor_reason fsm_os__incompatible(struct repository *r)if (reason !=FSMONITOR_REASON_OK )return reason; + reason =check_uds_volume (); +if (reason !=FSMONITOR_REASON_OK ) +return reason; +return FSMONITOR_REASON_OK ; }
Signature · 1 line
gitgitgadget
Quoted 1 line · edecosta via GitGitGadget
From: edecosta <[email protected]>
Do you have some specific reason to avoid using
Eric DeCosta <[email protected]>
to identify yourself? Otherwise, we'd prefer to see it that way.
Quoted 1 line
Subject: Re: [PATCH v3 2/2] Check working directory and Unix domain socket file for compatability
This project is wider than just its fsmonitor part, so please make sure to give your change a title that will not be lost in the noise when it is shown together with other patfches in the output from "git shortlog --no-merges". I haven't spent more than 30 seconds so you may be able to summarize it better, but perhaps
fsmonitor: ensure filesystem and unix socket are usable by fsmonitor
or something? If I am reading the early part of the patch correctly, the value of this change primarily comes from the fact that it separates (1) filesystems that are inherently incompatible (e.g. cannot create sockets) and (2) filesystems that pretend to successfully create sockets but the sockets are unusable, so a title with "separately" somewhere might convey its essence more clearly. I dunno.
The patch itself looked sensible (I didn't look for typos in constants etc., though).
Thanks.
Quoted 9 lines · Eric DeCosta via GitGitGadget
cc: Johannes Schindelin [email protected] cc: Jeff Hostetler [email protected] cc: Eric Sunshine [email protected] cc: Torsten Bögershausen [email protected]
Eric DeCosta (1):
fsmonitor: macOS: allow fsmonitor to run against network-mounted reposedecosta (1):
Check working directory and Unix domain socket file for compatability
Please describe why v3 exists (iow, what was wrong in v2 that v3 improves) in the cover letter. Especially when you send out two revisions in less than 12 hours, it gets disorienting at the receiving end.
Thanks.
Quoted 19 lines
compat/fsmonitor/fsm-settings-darwin.c | 72 +++++++++++++++++++------- fsmonitor-ipc.c | 40 ++++++++++++-- fsmonitor-ipc.h | 6 +++ fsmonitor-settings.c | 67 +++++++++++++++++++++++- fsmonitor-settings.h | 4 ++ 5 files changed, 164 insertions(+), 25 deletions(-)base-commit:
795ea8776befc95ea2becd8020c7a284677b4161
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1326%2Fedecosta-mw%2Ffsmonitor_macos-v3
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1326/edecosta-mw/fsmonitor_macos-v3
Pull-Request: https://github.com/gitgitgadget/git/pull/1326Range-diff vs v2:
1: 40ce21e85c9 < -: ----------- fsmonitor: option to allow fsmonitor to run against network-mounted repos 2: 46b4efd96cc = 1: cd16d8bb3d6 fsmonitor: macOS: allow fsmonitor to run against network-mounted repos 3: 9b128a98149 = 2: f977d140afa Check working directory and Unix domain socket file for compatability 4: 15c965801f8 < -: ----------- Minor refactoring and simplification of Windows settings checks
v4 cover fsmonitor: option to allow fsmonitor to run against network-mounted repos
In reply to Eric DeCosta via GitGitGadgetcc: Johannes Schindelin [email protected] cc: Jeff Hostetler [email protected] cc: Eric Sunshine [email protected] cc: Torsten Bögershausen [email protected]
Allow fsmonitor to run against network-mounted repos on macOS.
There are four parts to this effort:
1. Introduce two new configuration options
fsmonitor.allowRemote - setting this to true overrides fsmonitor's default behavior of erroring out when enountering network file systems. Additionly, when true, the Unix domain socket (UDS) file used for IPC is located in $HOME rather than in the .git directory.
fsmonitor.socketDir - allows for the UDS file to be located anywhere the user chooses rather $HOME.
2. Using the values of above configuration options, locate the UDS file in the desired location with a unique name based on the SHA1 of the path of the .git folder.
3. Ensure that both the working directory (.git directory) and the UDS file location are compatible with fsmonitor
4. Normalize the paths returned by FSEvents to the real path for each affected file or directory
Eric DeCosta (4):
fsmonitor: add two new config options, allowRemote and socketDir
fsmonitor: generate unique Unix socket file name in the desired
location
fsmonitor: ensure filesystem and unix socket filesystem are compatible
fsmonitor: normalize FSEvents event paths to the real path
compat/fsmonitor/fsm-listen-darwin.c | 11 ++-- compat/fsmonitor/fsm-settings-darwin.c | 72 +++++++++++++++++++------- fsmonitor-ipc.c | 40 ++++++++++++-- fsmonitor-ipc.h | 6 +++ fsmonitor-settings.c | 67 +++++++++++++++++++++++- fsmonitor-settings.h | 4 ++ 6 files changed, 172 insertions(+), 28 deletions(-)
base-commit: 795ea8776befc95ea2becd8020c7a284677b4161
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1326%2Fedecosta-mw%2Ffsmonitor_macos-v4
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1326/edecosta-mw/fsmonitor_macos-v4
Pull-Request: https://github.com/gitgitgadget/git/pull/1326
Range-diff vs v3:
1: cd16d8bb3d6 ! 1: 836a791e6b7 fsmonitor: macOS: allow fsmonitor to run against network-mounted repos
@@ Metadata
Author: Eric DeCosta <[email protected]>
## Commit message ##
- fsmonitor: macOS: allow fsmonitor to run against network-mounted repos
+ fsmonitor: add two new config options, allowRemote and socketDir
- Follow-on to the work done to allow Windows to work against
- network-mounted repos. Have macOS take advantage of the same
- configuration option, 'fsmonitor.allowRemote' that was introduced for
- Windows. Setting this option to true will override the default behavior
- (erroring-out) when a network-mounted repo is detected by fsmonitor.
+ Introduce two new configuration options
- The added wrinkle being that the Unix domain socket (UDS) file used for
- IPC cannot be created in a network location; instead $HOME is used if
- 'fsmonitor.allowRemote' is true.
+ fsmonitor.allowRemote - setting this to true overrides fsmonitor's
+ default behavior of erroring out when enountering network file
+ systems. Additionly, when true, the Unix domain socket (UDS) file
+ used for IPC is located in $HOME rather than in the .git directory.
- If $HOME is in a network location, allow the user to override this via
- the 'fsmonitor.socketDir' configuration option. There the user can
- specify any local directory for the location of the UDS file.
+ fsmonitor.socketDir - allows for the UDS file to be located
+ anywhere the user chooses rather $HOME.
Signed-off-by: Eric DeCosta <[email protected]>
- ## fsmonitor-ipc.c ##
-@@
- #include "cache.h"
--#include "fsmonitor.h"
--#include "simple-ipc.h"
- #include "fsmonitor-ipc.h"
-+#include "fsmonitor-settings.h"
- #include "run-command.h"
- #include "strbuf.h"
- #include "trace2.h"
-@@ fsmonitor-ipc.c: int fsmonitor_ipc__is_supported(void)
- return 1;
- }
-
--GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc")
-+GIT_PATH_FUNC(fsmonitor_ipc__get_default_path, "fsmonitor--daemon.ipc")
-+
-+const char *fsmonitor_ipc__get_path(void)
-+{
-+#ifdef WIN32
-+ return fsmonitor_ipc__get_default_path();
-+#else
-+ char *retval;
-+ SHA_CTX sha1ctx;
-+ const char *git_dir;
-+ const char *sock_dir;
-+ struct strbuf ipc_file = STRBUF_INIT;
-+ unsigned char hash[SHA_DIGEST_LENGTH];
-+
-+ if (fsm_settings__get_allow_remote(the_repository) < 1)
-+ return fsmonitor_ipc__get_default_path();
-+
-+ git_dir = get_git_dir();
-+ sock_dir = fsm_settings__get_socket_dir(the_repository);
-+
-+ SHA1_Init(&sha1ctx);
-+ SHA1_Update(&sha1ctx, git_dir, strlen(git_dir));
-+ SHA1_Final(hash, &sha1ctx);
-+
-+ if (sock_dir && *sock_dir)
-+ strbuf_addf(&ipc_file, "%s/.git-fsmonitor-%s",
-+ sock_dir, hash_to_hex(hash));
-+ else
-+ strbuf_addf(&ipc_file, "~/.git-fsmonitor-%s", hash_to_hex(hash));
-+ retval = interpolate_path(ipc_file.buf, 1);
-+ if (!retval)
-+ die(_("Invalid path: %s"), ipc_file.buf);
-+ strbuf_release(&ipc_file);
-+ return retval;
-+#endif
-+}
-
- enum ipc_active_state fsmonitor_ipc__get_state(void)
- {
-
- ## fsmonitor-ipc.h ##
-@@ fsmonitor-ipc.h: int fsmonitor_ipc__is_supported(void);
- */
- const char *fsmonitor_ipc__get_path(void);
-
-+/*
-+ * Returns the pathname to the default IPC named pipe or Unix domain
-+ * socket.
-+ */
-+const char *fsmonitor_ipc__get_default_path(void);
-+
- /*
- * Try to determine whether there is a `git-fsmonitor--daemon` process
- * listening on the IPC pipe/socket.
-
## fsmonitor-settings.c ##
@@
struct fsmonitor_settings {
-: ----------- > 2: 2cb026a6317 fsmonitor: generate unique Unix socket file name in the desired location
2: f977d140afa ! 3: a3110f1e25a Check working directory and Unix domain socket file for compatability @@
## Metadata ##
-Author: edecosta <[email protected]> +Author: Eric DeCosta <[email protected]>
## Commit message ##
- Check working directory and Unix domain socket file for compatability
+ fsmonitor: ensure filesystem and unix socket filesystem are compatible
Perform separate checks for the working directory and Unix domain socket (UDS) file location. The working directory may be located on a
@@ Commit message
file may never be located on a network-mounted file system; additionally it may not be located on FAT32 or NTFS file systems.
- Signed-off-by: edecosta <[email protected]>
+ Signed-off-by: Eric DeCosta <[email protected]>
## compat/fsmonitor/fsm-settings-darwin.c ##
@@
-: ----------- > 4: 56cabf3be3b fsmonitor: normalize FSEvents event paths to the real path
Signature · 1 line
gitgitgadget
PATCH v4 1/4 fsmonitor: add two new config options, allowRemote and socketDir
From: Eric DeCosta <[email protected]>
Introduce two new configuration options
fsmonitor.allowRemote - setting this to true overrides fsmonitor's default behavior of erroring out when enountering network file systems. Additionly, when true, the Unix domain socket (UDS) file used for IPC is located in $HOME rather than in the .git directory.
fsmonitor.socketDir - allows for the UDS file to be located anywhere the user chooses rather $HOME.
--- fsmonitor-settings.c | 67 ++++++++++++++++++++++++++++++++++++++++++-- fsmonitor-settings.h | 4 +++ 2 files changed, 69 insertions(+), 2 deletions(-)
fsmonitor-settings.c +65 -2
@@ -10,7 +10,9 @@struct fsmonitor_settings {enum fsmonitor_mode mode;enum fsmonitor_reason reason; +int allow_remote;char *hook_path; +char *sock_dir; };static enum fsmonitor_reason check_for_incompatible (struct repository *r)
@@ -43,6 +45,7 @@ static struct fsmonitor_settings *alloc_settings(void)CALLOC_ARRAY (s,1 ); s->mode =FSMONITOR_MODE_DISABLED ; s->reason =FSMONITOR_REASON_UNTESTED ; + s->allow_remote =-1 ;return s; }
@@ -90,6 +93,26 @@ static void lookup_fsmonitor_settings(struct repository *r)fsm_settings__set_disabled (r); } +int fsm_settings__get_allow_remote (struct repository *r) +{ +if (!r) + r = the_repository; +if (!r->settings.fsmonitor) +lookup_fsmonitor_settings (r); + +return r->settings.fsmonitor->allow_remote; +} + +const char *fsm_settings__get_socket_dir (struct repository *r) +{ +if (!r) + r = the_repository; +if (!r->settings.fsmonitor) +lookup_fsmonitor_settings (r); + +return r->settings.fsmonitor->sock_dir; +} +enum fsmonitor_mode fsm_settings__get_mode (struct repository *r) {if (!r)
@@ -100,6 +123,7 @@ enum fsmonitor_mode fsm_settings__get_mode(struct repository *r)return r->settings.fsmonitor->mode; } +const char *fsm_settings__get_hook_path (struct repository *r) {if (!r)
@@ -110,9 +134,44 @@ const char *fsm_settings__get_hook_path(struct repository *r)return r->settings.fsmonitor->hook_path; } +void fsm_settings__set_allow_remote (struct repository *r) +{ +int allow; + +if (!r) + r = the_repository; +if (!r->settings.fsmonitor) + r->settings.fsmonitor =alloc_settings (); +if (!repo_config_get_bool (r,"fsmonitor.allowremote" , &allow)) + r->settings.fsmonitor->allow_remote = allow; + +return ; +} + +void fsm_settings__set_socket_dir (struct repository *r) +{ +const char *path; + +if (!r) + r = the_repository; +if (!r->settings.fsmonitor) + r->settings.fsmonitor =alloc_settings (); + +if (!repo_config_get_pathname (r,"fsmonitor.socketdir" , &path)) { +FREE_AND_NULL (r->settings.fsmonitor->sock_dir); + r->settings.fsmonitor->sock_dir =strdup (path); + } + +return ; +} +void fsm_settings__set_ipc (struct repository *r) { -enum fsmonitor_reason reason =check_for_incompatible (r); +enum fsmonitor_reason reason; + +fsm_settings__set_allow_remote (r); +fsm_settings__set_socket_dir (r); + reason =check_for_incompatible (r);if (reason !=FSMONITOR_REASON_OK ) {fsm_settings__set_incompatible (r, reason);
@@ -135,7 +194,11 @@ void fsm_settings__set_ipc(struct repository *r)void fsm_settings__set_hook (struct repository *r,const char *path) { -enum fsmonitor_reason reason =check_for_incompatible (r); +enum fsmonitor_reason reason; + +fsm_settings__set_allow_remote (r); +fsm_settings__set_socket_dir (r); + reason =check_for_incompatible (r);if (reason !=FSMONITOR_REASON_OK ) {fsm_settings__set_incompatible (r, reason);
fsmonitor-settings.h +4 -0
@@ -23,12 +23,16 @@ enum fsmonitor_reason { FSMONITOR_REASON_NOSOCKETS, /* NTFS,FAT32 do not support Unix sockets */ }; +void fsm_settings__set_allow_remote (struct repository *r); +void fsm_settings__set_socket_dir (struct repository *r);void fsm_settings__set_ipc (struct repository *r);void fsm_settings__set_hook (struct repository *r,const char *path);void fsm_settings__set_disabled (struct repository *r);void fsm_settings__set_incompatible (struct repository *r,enum fsmonitor_reason reason); +int fsm_settings__get_allow_remote (struct repository *r); +const char *fsm_settings__get_socket_dir (struct repository *r);enum fsmonitor_mode fsm_settings__get_mode (struct repository *r);const char *fsm_settings__get_hook_path (struct repository *r);
Signature · 1 line
gitgitgadget
Re: [PATCH v4 1/4] fsmonitor: add two new config options, allowRemote and socketDir
fsmonitor-settings.c line 16
Quoted 7 lines · Eric DeCosta via GitGitGadget
From: Eric DeCosta <[email protected]>
[...]fsmonitor-settings.c +2
-0enum fsmonitor_reason reason; +int allow_remote;char *hook_path; +char *sock_dir; };
Any reason we couldn't just add this to "struct repo_settings" and ...
Quoted 10 lines
fsmonitor-settings.c +10
-0+int fsm_settings__get_allow_remote (struct repository *r) +{ +if (!r) + r = the_repository; +if (!r->settings.fsmonitor) +lookup_fsmonitor_settings (r); + +return r->settings.fsmonitor->allow_remote; +} +
fsmonitor-settings.c static void lookup_fsmonitor_settings(struct repository *r) · line 115
Quoted 10 lines
fsmonitor-settings.c +10
-0+const char *fsm_settings__get_socket_dir (struct repository *r) +{ +if (!r) + r = the_repository; +if (!r->settings.fsmonitor) +lookup_fsmonitor_settings (r); + +return r->settings.fsmonitor->sock_dir; +} +
...instead of this whole ceremony...
fsmonitor-settings.c const char *fsm_settings__get_hook_path(struct repository *r) · line 149
Quoted 13 lines
fsmonitor-settings.c +13
-0+void fsm_settings__set_allow_remote (struct repository *r) +{ +int allow; + +if (!r) + r = the_repository; +if (!r->settings.fsmonitor) + r->settings.fsmonitor =alloc_settings (); +if (!repo_config_get_bool (r,"fsmonitor.allowremote" , &allow)) + r->settings.fsmonitor->allow_remote = allow; + +return ; +}
Just have a single repo_cfg_bool() line in prepare_repo_settings() instead?
(There are some reasons for the "lazy" behavior of fsmonitor-settings.c, but surely a simple boolean variable we can read on startup isn't it, and we already paid the cost to do so with the configset...)
fsmonitor-settings.c const char *fsm_settings__get_hook_path(struct repository *r) · line 161
Quoted 11 lines
fsmonitor-settings.c +11
-0+void fsm_settings__set_socket_dir (struct repository *r) +{ +const char *path; + +if (!r) + r = the_repository; +if (!r->settings.fsmonitor) + r->settings.fsmonitor =alloc_settings (); + +if (!repo_config_get_pathname (r,"fsmonitor.socketdir" , &path)) { +FREE_AND_NULL (r->settings.fsmonitor->sock_dir);
...maybe this socket dir stuff is the exception though.
fsmonitor-settings.c const char *fsm_settings__get_hook_path(struct repository *r) · line 162
Quoted 1 line
fsmonitor-settings.c +1
-0+ r->settings.fsmonitor->sock_dir =strdup (path);
Aren't you strdup()-ing an already strdup()'d string, and therefore leaking memory? Also this should be xstrdup(), surely?
fsmonitor-settings.c const char *fsm_settings__get_hook_path(struct repository *r) · line 174
Quoted 13 lines
fsmonitor-settings.c +10
-1+ } + +return ; +} +void fsm_settings__set_ipc (struct repository *r) { -enum fsmonitor_reason reason =check_for_incompatible (r); +enum fsmonitor_reason reason; + +fsm_settings__set_allow_remote (r); +fsm_settings__set_socket_dir (r); + reason =check_for_incompatible (r);
This seems rather backwards, as odd as this API itself is already, isn't the whole idea that after we call lookup_fsmonitor_settings() it will have set() anything that's mandatory?
But maybe I haven't grokked it ... :)
Re: [PATCH v4 1/4] fsmonitor: add two new config options, allowRemote and socketDir
In reply to Eric DeCosta via GitGitGadgetQuoted 11 lines · Eric DeCosta via GitGitGadget
From: Eric DeCosta <[email protected]>
Introduce two new configuration options
fsmonitor.allowRemote - setting this to true overrides fsmonitor's default behavior of erroring out when enountering network file systems. Additionly, when true, the Unix domain socket (UDS) file used for IPC is located in $HOME rather than in the .git directory.
fsmonitor.socketDir - allows for the UDS file to be located anywhere the user chooses rather $HOME.
Before describing "what they do", please justify why we need to add them. The usual way to do so is to start with some observation of the status quo, and describe the "gap" between what can be done with the current system and what the users would want to do. It might further be necessary to justify why "would want to do" is worthwhile to support. I suspect that you can do all of the above in a couple of paragraphs, and you'd succeed if the solution you chose would fall out as a natural consequence in the mind of readers who follow your line of thought by reading these introductory paragraphs. And then after the stage is set like so, the above description of what you chose to implement as a solution should come.
fsmonitor-settings.c line 13
Quoted 4 lines
struct
fsmonitor_settings{fsmonitor-settings.c +1
-0enum fsmonitor_mode mode;enum fsmonitor_reason reason; +int allow_remote;
I am debating myuself if a comment like
int allow_remote; /* -1: undecided, 0: unallowed, 1: allowed */
is necessary (I know it would help if we had one; I am just wondering if it is too obvious).
fsmonitor-settings.c static struct fsmonitor_settings *alloc_settings(void) · line 51
Quoted 13 lines
fsmonitor-settings.c +2
-0char *hook_path; +char *sock_dir; };static enum fsmonitor_reason check_for_incompatible (struct repository *r)@@ -43,6 +45,7 @@ static struct fsmonitor_settings *alloc_settings(void)CALLOC_ARRAY (s,1 ); s->mode =FSMONITOR_MODE_DISABLED ; s->reason =FSMONITOR_REASON_UNTESTED ; + s->allow_remote =-1 ;return s; }
OK. socket_dir is naturally NULL at the start.
fsmonitor-settings.c enum fsmonitor_mode fsm_settings__get_mode(struct repository *r) · line 129
Quoted 8 lines
fsmonitor-settings.c +1
-0@@ -100,6 +123,7 @@ enum fsmonitor_mode fsm_settings__get_mode(struct repository *r)return r->settings.fsmonitor->mode; } +const char *fsm_settings__get_hook_path (struct repository *r) {if (!r)
A noise hunk?
Quoted 4 lines
fsmonitor-settings.c +2
-0@@ -110,9 +134,44 @@ const char *fsm_settings__get_hook_path(struct repository *r) +void fsm_settings__set_socket_dir (struct repository *r) +{
fsmonitor-settings.c const char *fsm_settings__get_hook_path(struct repository *r) · line 162
Quoted 10 lines
fsmonitor-settings.c +10
-0+const char *path; + +if (!r) + r = the_repository; +if (!r->settings.fsmonitor) + r->settings.fsmonitor =alloc_settings (); + +if (!repo_config_get_pathname (r,"fsmonitor.socketdir" , &path)) { +FREE_AND_NULL (r->settings.fsmonitor->sock_dir); + r->settings.fsmonitor->sock_dir =strdup (path);
As we are overwriting it immediately, just calling free(), not FREE_AND_NULL(), is more appropriate, isn't it?
Quoted 18 lines
fsmonitor-settings.c +5
-1@@ -135,7 +194,11 @@ void fsm_settings__set_ipc(struct repository *r)void fsm_settings__set_hook (struct repository *r,const char *path) { -enum fsmonitor_reason reason =check_for_incompatible (r); +enum fsmonitor_reason reason; + +fsm_settings__set_allow_remote (r); +fsm_settings__set_socket_dir (r); + reason =check_for_incompatible (r);if (reason !=FSMONITOR_REASON_OK ) {fsm_settings__set_incompatible (r, reason);fsmonitor-settings.h +0
-0
fsmonitor-settings.h enum fsmonitor_reason { · line 27
Quoted 5 lines
fsmonitor-settings.h +2
-0@@ -23,12 +23,16 @@ enum fsmonitor_reason { FSMONITOR_REASON_NOSOCKETS, /* NTFS,FAT32 do not support Unix sockets */ }; +void fsm_settings__set_allow_remote (struct repository *r); +void fsm_settings__set_socket_dir (struct repository *r);
Do these two need to be extern?
I would imagine that implementations in compat/fsmonitor/* would just call fsm_settings__set_hook() or __set_ipc() and that causes these two to be called as part of the set-up sequence. Do they need to call these two directly?
If not, we probably should make them static. I suspect that some existing declarations in this header file fall into the same category and may need to become static for the same reason, which you can do as a preliminary clean-up patch, or post-clean-up after the dust settles. Regardless of which approach to take for existing ones, we do not want to make it worse by adding new externally visible names that do not have to be visible.
fsmonitor-settings.h enum fsmonitor_reason { · line 35
Quoted 8 lines
fsmonitor-settings.h +2
-0void fsm_settings__set_ipc (struct repository *r);void fsm_settings__set_hook (struct repository *r,const char *path);void fsm_settings__set_disabled (struct repository *r);void fsm_settings__set_incompatible (struct repository *r,enum fsmonitor_reason reason); +int fsm_settings__get_allow_remote (struct repository *r); +const char *fsm_settings__get_socket_dir (struct repository *r);
On the other hand, these may need to be query-able by the implementations.
Quoted 2 lines
fsmonitor-settings.h +0
-0enum fsmonitor_mode fsm_settings__get_mode (struct repository *r);const char *fsm_settings__get_hook_path (struct repository *r);
Re: [PATCH v4 1/4] fsmonitor: add two new config options, allowRemote and socketDir
Quoted 2 lines · Junio C Hamano
"Eric DeCosta via GitGitGadget" <[email protected]> writes:
[snip]
Quoted 8 lines
Quoted 8 lines
fsmonitor-settings.h +0
-0@@ -23,12 +23,16 @@ enum fsmonitor_reason { FSMONITOR_REASON_NOSOCKETS, /* NTFS,FAT32 do not support Unix sockets */ };
Quoted 5 lines
fsmonitor-settings.h enum fsmonitor_reason { · line 27
Quoted 2 lines
fsmonitor-settings.h +2
-0+void fsm_settings__set_allow_remote (struct repository *r); +void fsm_settings__set_socket_dir (struct repository *r);Do these two need to be extern?
On 'seen' after building:
...
$ ./static-check.pl >ssc $ diff nsc ssc 3a4
Quoted 1 line
bundle-uri.o -
for_all_bundles_in_list
15d15
< config.o - git_config_from_file_with_options
40a41,43
Quoted 3 lines
fsmonitor-settings.o -
fsm_settings__get_allow_remote
fsmonitor-settings.o -fsm_settings__get_socket_dir
fsmonitor-settings.o -fsm_settings__set_allow_remote
44a48
Quoted 1 line
fsmonitor-settings.o -
fsm_settings__set_socket_dir
$
.. so probably not. ;-)
ATB,
Ramsay Jones
Re: [PATCH v4 1/4] fsmonitor: add two new config options, allowRemote and socketDir
In reply to Eric DeCosta via GitGitGadgetQuoted 122 lines · Eric DeCosta via GitGitGadget
From: Eric DeCosta <[email protected]>
Introduce two new configuration options
fsmonitor.allowRemote - setting this to true overrides fsmonitor's default behavior of erroring out when enountering network file systems. Additionly, when true, the Unix domain socket (UDS) file used for IPC is located in $HOME rather than in the .git directory.
fsmonitor.socketDir - allows for the UDS file to be located anywhere the user chooses rather $HOME.
Signed-off-by Eric DeCosta <[email protected]>--- fsmonitor-settings.c | 67 ++++++++++++++++++++++++++++++++++++++++++-- fsmonitor-settings.h | 4 +++ 2 files changed, 69 insertions(+), 2 deletions(-)fsmonitor-settings.c +60
-1@@ -10,7 +10,9 @@struct fsmonitor_settings {enum fsmonitor_mode mode;enum fsmonitor_reason reason; +int allow_remote;char *hook_path; +char *sock_dir; };static enum fsmonitor_reason check_for_incompatible (struct repository *r)@@ -43,6 +45,7 @@ static struct fsmonitor_settings *alloc_settings(void)CALLOC_ARRAY (s,1 ); s->mode =FSMONITOR_MODE_DISABLED ; s->reason =FSMONITOR_REASON_UNTESTED ; + s->allow_remote =-1 ;return s; }@@ -90,6 +93,26 @@ static void lookup_fsmonitor_settings(struct repository *r)fsm_settings__set_disabled (r); } +int fsm_settings__get_allow_remote (struct repository *r) +{ +if (!r) + r = the_repository; +if (!r->settings.fsmonitor) +lookup_fsmonitor_settings (r); + +return r->settings.fsmonitor->allow_remote; +} + +const char *fsm_settings__get_socket_dir (struct repository *r) +{ +if (!r) + r = the_repository; +if (!r->settings.fsmonitor) +lookup_fsmonitor_settings (r); + +return r->settings.fsmonitor->sock_dir; +} +enum fsmonitor_mode fsm_settings__get_mode (struct repository *r) {if (!r)@@ -100,6 +123,7 @@ enum fsmonitor_mode fsm_settings__get_mode(struct repository *r)return r->settings.fsmonitor->mode; } +const char *fsm_settings__get_hook_path (struct repository *r) {if (!r)@@ -110,9 +134,44 @@ const char *fsm_settings__get_hook_path(struct repository *r)return r->settings.fsmonitor->hook_path; } +void fsm_settings__set_allow_remote (struct repository *r) +{ +int allow; + +if (!r) + r = the_repository; +if (!r->settings.fsmonitor) + r->settings.fsmonitor =alloc_settings (); +if (!repo_config_get_bool (r,"fsmonitor.allowremote" , &allow)) + r->settings.fsmonitor->allow_remote = allow; + +return ; +} + +void fsm_settings__set_socket_dir (struct repository *r) +{ +const char *path; + +if (!r) + r = the_repository; +if (!r->settings.fsmonitor) + r->settings.fsmonitor =alloc_settings (); + +if (!repo_config_get_pathname (r,"fsmonitor.socketdir" , &path)) { +FREE_AND_NULL (r->settings.fsmonitor->sock_dir); + r->settings.fsmonitor->sock_dir =strdup (path); + } + +return ; +} +void fsm_settings__set_ipc (struct repository *r) { -enum fsmonitor_reason reason =check_for_incompatible (r); +enum fsmonitor_reason reason; + +fsm_settings__set_allow_remote (r); +fsm_settings__set_socket_dir (r); + reason =check_for_incompatible (r);if (reason !=FSMONITOR_REASON_OK ) {fsm_settings__set_incompatible (r, reason);
fsmonitor-settings.c void fsm_settings__set_ipc(struct repository *r) · line 201
Quoted 9 lines
fsmonitor-settings.c +5
-1@@ -135,7 +194,11 @@ void fsm_settings__set_ipc(struct repository *r)void fsm_settings__set_hook (struct repository *r,const char *path) { -enum fsmonitor_reason reason =check_for_incompatible (r); +enum fsmonitor_reason reason; + +fsm_settings__set_allow_remote (r); +fsm_settings__set_socket_dir (r); + reason =check_for_incompatible (r);
When we use the hook interface (to talk to a third-party app like Watchman) we do not use the unix domain socket. (We talk to the hook child using a normal pipe.)
FWIW, I added code in my series to limit the use of hook-based fsmonitors to local filesystems for the paranoia-based reasons I've described before. It wasn't because of the UDS.
So we don't need to set the socket directory in this case.
Quoted 25 lines
fsmonitor-settings.c +0
-0if (reason !=FSMONITOR_REASON_OK ) {fsm_settings__set_incompatible (r, reason);fsmonitor-settings.h +4
-0@@ -23,12 +23,16 @@ enum fsmonitor_reason { FSMONITOR_REASON_NOSOCKETS, /* NTFS,FAT32 do not support Unix sockets */ }; +void fsm_settings__set_allow_remote (struct repository *r); +void fsm_settings__set_socket_dir (struct repository *r);void fsm_settings__set_ipc (struct repository *r);void fsm_settings__set_hook (struct repository *r,const char *path);void fsm_settings__set_disabled (struct repository *r);void fsm_settings__set_incompatible (struct repository *r,enum fsmonitor_reason reason); +int fsm_settings__get_allow_remote (struct repository *r); +const char *fsm_settings__get_socket_dir (struct repository *r);enum fsmonitor_mode fsm_settings__get_mode (struct repository *r);const char *fsm_settings__get_hook_path (struct repository *r);
Re: [PATCH v4 1/4] fsmonitor: add two new config options, allowRemote and socketDir
In reply to Eric DeCosta via GitGitGadgetQuoted 11 lines · Eric DeCosta via GitGitGadget
From: Eric DeCosta <[email protected]>
Introduce two new configuration options
fsmonitor.allowRemote - setting this to true overrides fsmonitor's default behavior of erroring out when enountering network file systems. Additionly, when true, the Unix domain socket (UDS) file used for IPC is located in $HOME rather than in the .git directory.
fsmonitor.socketDir - allows for the UDS file to be located anywhere the user chooses rather $HOME.
If we add these config settings, we should update the docs to describe them.
Jeff
Re: [PATCH v4 1/4] fsmonitor: add two new config options, allowRemote and socketDir
In reply to Eric DeCosta via GitGitGadgetQuoted 9 lines · Eric DeCosta via GitGitGadget
From: Eric DeCosta <[email protected]>
Introduce two new configuration options
fsmonitor.allowRemote - setting this to true overrides fsmonitor's default behavior of erroring out when enountering network file systems. Additionly, when true, the Unix domain socket (UDS) file used for IPC is located in $HOME rather than in the .git directory.
Quoted 4 lines
fsmonitor.socketDir - allows for the UDS file to be located anywhere the user chooses rather $HOME.
Signed-off-by Eric DeCosta <[email protected]>
I hate to say it, but I think I'd like to start over on this patch. I think it adds too much to the upper layers.
Could we:
1. Move the GIT_PATH_FUNC() down into compat/fsmonitor/fsm-settings-*.c so that we don't need the ifdef-ing because we only really need to change the path on MacOS.
Change it from this macro trick to be an actual function that computes the path and sticks it in a static buffer and returns it on future calls. (We call it from several different places.)
On MacOS have it always compute a $HOME or fsmonitor.socketDir based path.
(This is called early, so we don't know anything about the
r->settings.fsmonitor data yet (like whether we'll have an ipc or hook), so we don't know whether to worry about whether the socket-directory is local/remote yet.)
2. Get rid of the fsm_settings__get_allow_remote() and __get_socket_dir() functions in fsmonitor-settings.c.
Besides, having it at this layer says that "allow-remote" is an all-or-nothing flag (that the platform-independent layer doesn't know anything about). We may eventually find that we want the platform code to be smarter about that, such as allow remote from NFSv4 but not NFSv3. We don't want to litter the platform-independent code with that.
And it says that we always use a UDS. We might change that later on MacOS. And we don't need them at all on Windows.
3. Get rid of the fsm_settings__set_allow_remote() and __set_socket_dir(). The FSMonitor code only needs to access them once down in the fsm_os__incompatible() code, so we could just inline the relevant parts down there.
4. Leave the 'reason = check_for_incompatible()' as it was in fsm_setttings__set_ipc() and __set_hook().
But you might pass a boolean "is-ipc" or "is-hook" to check_for_incompatible() that it could pass down to fsm_os__incompatible(). That might save the need to the fsmonitor.socketDir stuff when they want to use Watchman.
5. In fsm-settings-darwin.c:fsm_os__incompatible() complain if the socket path is remote (when is-ipc is set).
This probably ought to be a new _REASON_ type for non-local UDS. And this is independent of whether the actual worktree is remote or whether remote worktrees are allowed.
6. In fsm-settings-darwin.c:check_volume()
All that should be necessary is to:
if (!(fs.f_flags & MNT_LOCAL))
+ {
+ // ... repo_config_get_bool(... "fsmonitor.allowremote" ...)
return FSMONITOR_REASON_REMOTE;+ }
Alternatively, for 5 & 6, we could pass a pathname to check_volume() so that fsm_os__incompatible() calls it twice: once for the worktree and once for the socketdir.
The ntfs and msdos restrictions were for UDS reasons, so you might want pass some flags to check_volume() too.
Sorry for the redesign and I hope this all makes sense, Jeff
Quoted 143 lines
--- fsmonitor-settings.c | 67 ++++++++++++++++++++++++++++++++++++++++++-- fsmonitor-settings.h | 4 +++ 2 files changed, 69 insertions(+), 2 deletions(-)fsmonitor-settings.c +65
-2@@ -10,7 +10,9 @@struct fsmonitor_settings {enum fsmonitor_mode mode;enum fsmonitor_reason reason; +int allow_remote;char *hook_path; +char *sock_dir; };static enum fsmonitor_reason check_for_incompatible (struct repository *r)@@ -43,6 +45,7 @@ static struct fsmonitor_settings *alloc_settings(void)CALLOC_ARRAY (s,1 ); s->mode =FSMONITOR_MODE_DISABLED ; s->reason =FSMONITOR_REASON_UNTESTED ; + s->allow_remote =-1 ;return s; }@@ -90,6 +93,26 @@ static void lookup_fsmonitor_settings(struct repository *r)fsm_settings__set_disabled (r); } +int fsm_settings__get_allow_remote (struct repository *r) +{ +if (!r) + r = the_repository; +if (!r->settings.fsmonitor) +lookup_fsmonitor_settings (r); + +return r->settings.fsmonitor->allow_remote; +} + +const char *fsm_settings__get_socket_dir (struct repository *r) +{ +if (!r) + r = the_repository; +if (!r->settings.fsmonitor) +lookup_fsmonitor_settings (r); + +return r->settings.fsmonitor->sock_dir; +} +enum fsmonitor_mode fsm_settings__get_mode (struct repository *r) {if (!r)@@ -100,6 +123,7 @@ enum fsmonitor_mode fsm_settings__get_mode(struct repository *r)return r->settings.fsmonitor->mode; } +const char *fsm_settings__get_hook_path (struct repository *r) {if (!r)@@ -110,9 +134,44 @@ const char *fsm_settings__get_hook_path(struct repository *r)return r->settings.fsmonitor->hook_path; } +void fsm_settings__set_allow_remote (struct repository *r) +{ +int allow; + +if (!r) + r = the_repository; +if (!r->settings.fsmonitor) + r->settings.fsmonitor =alloc_settings (); +if (!repo_config_get_bool (r,"fsmonitor.allowremote" , &allow)) + r->settings.fsmonitor->allow_remote = allow; + +return ; +} + +void fsm_settings__set_socket_dir (struct repository *r) +{ +const char *path; + +if (!r) + r = the_repository; +if (!r->settings.fsmonitor) + r->settings.fsmonitor =alloc_settings (); + +if (!repo_config_get_pathname (r,"fsmonitor.socketdir" , &path)) { +FREE_AND_NULL (r->settings.fsmonitor->sock_dir); + r->settings.fsmonitor->sock_dir =strdup (path); + } + +return ; +} +void fsm_settings__set_ipc (struct repository *r) { -enum fsmonitor_reason reason =check_for_incompatible (r); +enum fsmonitor_reason reason; + +fsm_settings__set_allow_remote (r); +fsm_settings__set_socket_dir (r); + reason =check_for_incompatible (r);if (reason !=FSMONITOR_REASON_OK ) {fsm_settings__set_incompatible (r, reason);@@ -135,7 +194,11 @@ void fsm_settings__set_ipc(struct repository *r)void fsm_settings__set_hook (struct repository *r,const char *path) { -enum fsmonitor_reason reason =check_for_incompatible (r); +enum fsmonitor_reason reason; + +fsm_settings__set_allow_remote (r); +fsm_settings__set_socket_dir (r); + reason =check_for_incompatible (r);if (reason !=FSMONITOR_REASON_OK ) {fsm_settings__set_incompatible (r, reason);fsmonitor-settings.h +4
-0@@ -23,12 +23,16 @@ enum fsmonitor_reason { FSMONITOR_REASON_NOSOCKETS, /* NTFS,FAT32 do not support Unix sockets */ }; +void fsm_settings__set_allow_remote (struct repository *r); +void fsm_settings__set_socket_dir (struct repository *r);void fsm_settings__set_ipc (struct repository *r);void fsm_settings__set_hook (struct repository *r,const char *path);void fsm_settings__set_disabled (struct repository *r);void fsm_settings__set_incompatible (struct repository *r,enum fsmonitor_reason reason); +int fsm_settings__get_allow_remote (struct repository *r); +const char *fsm_settings__get_socket_dir (struct repository *r);enum fsmonitor_mode fsm_settings__get_mode (struct repository *r);const char *fsm_settings__get_hook_path (struct repository *r);
RE: [PATCH v4 1/4] fsmonitor: add two new config options, allowRemote and socketDir
Quoted 60 lines
-----Original Message-----
From: Jeff Hostetler <[email protected]>
Sent: Thursday, September 1, 2022 5:22 PM
To: Eric DeCosta via GitGitGadget <[email protected]>; [email protected]
Cc: Eric DeCosta <[email protected]>
Subject: Re: [PATCH v4 1/4] fsmonitor: add two new config options, allowRemote and socketDirQuoted 9 lines · Eric DeCosta via GitGitGadget
From: Eric DeCosta <[email protected]>
Introduce two new configuration options
fsmonitor.allowRemote - setting this to true overrides fsmonitor's default behavior of erroring out when enountering network file systems. Additionly, when true, the Unix domain socket (UDS) file used for IPC is located in $HOME rather than in the .git directory.
Quoted 4 lines
fsmonitor.socketDir - allows for the UDS file to be located anywhere the user chooses rather $HOME.
Signed-off-by Eric DeCosta <[email protected]>I hate to say it, but I think I'd like to start over on this patch. I think it adds too much to the upper layers.
Could we:
1. Move the
GIT_PATH_FUNC()down intocompat/fsmonitor/fsm-settings-*.cso that we don't need the ifdef-ing because we only really need to change the path on MacOS.Change it from this macro trick to be an actual function that computes the path and sticks it in a static buffer and returns it on future calls. (We call it from several different places.)
On MacOS have it always compute a $HOME or fsmonitor.socketDir based path.
(This is called early, so we don't know anything about the
r->settings.fsmonitordata yet (like whether we'll have an ipc or hook), so we don't know whether to worry about whether the socket-directory is local/remote yet.)2. Get rid of the
fsm_settings__get_allow_remote()and__get_socket_dir()functions infsmonitor-settings.c.Besides, having it at this layer says that "allow-remote" is an all-or-nothing flag (that the platform-independent layer doesn't know anything about). We may eventually find that we want the platform code to be smarter about that, such as allow remote from NFSv4 but not NFSv3. We don't want to litter the platform-independent code with that.
Quoted 7 lines
And it says that we always use a UDS. We might change that later on MacOS. And we don't need them at all on Windows.
3. Get rid of the
fsm_settings__set_allow_remote()and__set_socket_dir(). The FSMonitor code only needs to access them once down in thefsm_os__incompatible()code, so we could just inline the relevant parts down there.
Got it.
Quoted 7 lines
4. Leave the 'reason =
check_for_incompatible()' as it was infsm_setttings__set_ipc()and__set_hook().But you might pass a boolean "is-ipc" or "is-hook" to
check_for_incompatible()that it could pass down tofsm_os__incompatible(). That might save the need to the fsmonitor.socketDir stuff when they want to use Watchman.
I see, avoid the UDS check altogether if "is-hook"
Quoted 6 lines
5. In fsm-settings-darwin.c:fsm_os__incompatible() complain if the socket path is remote (when is-ipc is set).
This probably ought to be a new _REASON_ type for non-local UDS. And this is independent of whether the actual worktree is remote or whether remote worktrees are allowed.
I thought that was what FSMONITOR_REASON_NOSOCKETS was for?
Quoted 11 lines
6. In fsm-settings-darwin.c:check_volume()
All that should be necessary is to:
if (!(fs.f_flags & MNT_LOCAL))fsmonitor-settings.c +3
-0+ { +// ... repo_config_get_bool(... "fsmonitor.allowremote" ...) return FSMONITOR_REASON_REMOTE ; + }
Quoted 9 lines
Alternatively, for 5 & 6, we could pass a pathname to
check_volume()so thatfsm_os__incompatible()calls it twice: once for the worktree and once for the socketdir.The ntfs and msdos restrictions were for UDS reasons, so you might want pass some flags to
check_volume()too.Sorry for the redesign and I hope this all makes sense, Jeff
I've taken a first pass at all of this, will work on it some more.
-Eric
Quoted 146 lines
Quoted 3 lines
--- fsmonitor-settings.c | 67++++++++++++++++++++++++++++++++++++++++++--
Quoted 17 lines
fsmonitor-settings.h | 4 +++ 2 files changed, 69 insertions(+), 2 deletions(-)fsmonitor-settings.c index renamed from fsmonitor-settings.c +0
-0
464424a1e92..a15eeecebf4100644fsmonitor-settings.c +2
-0@@ -10,7 +10,9 @@struct fsmonitor_settings {enum fsmonitor_mode mode;enum fsmonitor_reason reason; +int allow_remote;char *hook_path; +char *sock_dir; }; static enum fsmonitor_reason check_for_incompatible(structQuoted 1 line
repository *r) @@ -43,6 +45,7 @@ static struct
fsmonitor_settings
*alloc_settings(void)fsmonitor-settings.c static void lookup_fsmonitor_settings(struct
Quoted 8 lines
CALLOC_ARRAY(s, 1); s->mode = FSMONITOR_MODE_DISABLED; s->reason = FSMONITOR_REASON_UNTESTED;fsmonitor-settings.c +1
-0+ s->allow_remote =-1 ;return s; }@@ -90,6 +93,26 @@ static void lookup_fsmonitor_settings(structrepository *r)
fsmonitor-settings.c enum fsmonitor_mode
Quoted 24 lines
fsmonitor-settings.c +17
-0fsm_settings__set_disabled (r); } +int fsm_settings__get_allow_remote (struct repository *r) { +if (!r) + r = the_repository; +if (!r->settings.fsmonitor) +lookup_fsmonitor_settings (r); + +return r->settings.fsmonitor->allow_remote; +} + +const char *fsm_settings__get_socket_dir (struct repository *r) { +if (!r) + r = the_repository; +if (!r->settings.fsmonitor) +lookup_fsmonitor_settings (r); + +return r->settings.fsmonitor->sock_dir; } +enum fsmonitor_mode fsm_settings__get_mode (struct repository *r) {if (!r)@@ -100,6 +123,7 @@ enum fsmonitor_mode
fsm_settings__get_mode(struct repository *r)fsmonitor-settings.c const char *fsm_settings__get_hook_path(struct
Quoted 8 lines
fsmonitor-settings.c +1
-0return r->settings.fsmonitor->mode; } +const char *fsm_settings__get_hook_path (struct repository *r) {if (!r)@@ -110,9 +134,44 @@ const char *fsm_settings__get_hook_path(structrepository *r)
Quoted 33 lines
fsmonitor-settings.c +29
-0return r->settings.fsmonitor->hook_path; } +void fsm_settings__set_allow_remote (struct repository *r) { +int allow; + +if (!r) + r = the_repository; +if (!r->settings.fsmonitor) + r->settings.fsmonitor =alloc_settings (); +if (!repo_config_get_bool (r,"fsmonitor.allowremote" , &allow)) + r->settings.fsmonitor->allow_remote = allow; + +return ; +} + +void fsm_settings__set_socket_dir (struct repository *r) { +const char *path; + +if (!r) + r = the_repository; +if (!r->settings.fsmonitor) + r->settings.fsmonitor =alloc_settings (); + +if (!repo_config_get_pathname (r,"fsmonitor.socketdir" , &path)) { +FREE_AND_NULL (r->settings.fsmonitor->sock_dir); + r->settings.fsmonitor->sock_dir =strdup (path); + } + +return ; +} +void fsm_settings__set_ipc (struct repository *r)fsmonitor-settings.c const char *fsm_settings__get_hook_path(struct repository *r)
Quoted 10 lines
fsmonitor-settings.c +5
-1{ -enum fsmonitor_reason reason =check_for_incompatible (r); +enum fsmonitor_reason reason; + +fsm_settings__set_allow_remote (r); +fsm_settings__set_socket_dir (r); + reason =check_for_incompatible (r);if (reason !=FSMONITOR_REASON_OK ) { fsm_settings__set_incompatible(r, reason); @@ -135,7+194,11 @@
Quoted 18 lines
void
fsm_settings__set_ipc(struct repository *r)void fsm_settings__set_hook(struct repository *r, const char *path) {fsmonitor-settings.c +5
-1-enum fsmonitor_reason reason =check_for_incompatible (r); +enum fsmonitor_reason reason; + +fsm_settings__set_allow_remote (r); +fsm_settings__set_socket_dir (r); + reason =check_for_incompatible (r);if (reason !=FSMONITOR_REASON_OK ) { fsm_settings__set_incompatible(r, reason); diff --git
a/fsmonitor-settings.hb/fsmonitor-settings.hindexd9c2605197f..2de54c85e94100644fsmonitor-settings.h enum fsmonitor_reason { · line 23
Quoted 1 line
fsmonitor-settings.h +0
-0@@ -23,12 +23,16 @@ enum fsmonitor_reason { FSMONITOR_REASON_NOSOCKETS, /* NTFS,FAT32 do not supportUnix sockets */
Quoted 16 lines
fsmonitor-settings.h +4
-0}; +void fsm_settings__set_allow_remote (struct repository *r);void +fsm_settings__set_socket_dir (struct repository *r);void fsm_settings__set_ipc (struct repository *r);void fsm_settings__set_hook (struct repository *r,const char *path);void fsm_settings__set_disabled (struct repository *r);void fsm_settings__set_incompatible (struct repository *r,enum fsmonitor_reason reason); +int fsm_settings__get_allow_remote (struct repository *r);const char +*fsm_settings__get_socket_dir (struct repository *r);enum fsmonitor_mode fsm_settings__get_mode (struct repository *r);const char *fsm_settings__get_hook_path (struct repository *r);
Re: [PATCH v4 1/4] fsmonitor: add two new config options, allowRemote and socketDir
Quoted 17 lines · Eric DeCosta
Quoted 17 lines
-----Original Message-----
From: Jeff Hostetler <[email protected]>
Sent: Thursday, September 1, 2022 5:22 PM
To: Eric DeCosta via GitGitGadget <[email protected]>; [email protected]
Cc: Eric DeCosta <[email protected]>
Subject: Re: [PATCH v4 1/4] fsmonitor: add two new config options, allowRemote and socketDirQuoted 3 lines · Eric DeCosta via GitGitGadget
From: Eric DeCosta <[email protected]>
Introduce two new configuration options
[...]
Quoted 8 lines
Quoted 6 lines
5. In fsm-settings-darwin.c:fsm_os__incompatible() complain if the socket path is remote (when is-ipc is set).
This probably ought to be a new _REASON_ type for non-local UDS. And this is independent of whether the actual worktree is remote or whether remote worktrees are allowed.
I thought that was what
FSMONITOR_REASON_NOSOCKETSwas for?
Yeah, I was probably juggling too many things in my head when I was making that list. And I'm sure it'll fall our when you see what the code looks like. The _NOSOCKETS case should basically be just the places where you test the path of the socket-dir (if it is always outside of the .git dir). None of the tests on the actual workdir inspection should hit it anymore, right?
And we can save for a later day whether or not to test for NTFS and FAT32 (local or remote) in the Darwin code based on whether or not the kernel sends FSEvents for them.
Jeff
PATCH v4 2/4 fsmonitor: generate unique Unix socket file name in the desired location
In reply to Eric DeCosta via GitGitGadgetFrom: Eric DeCosta <[email protected]>
Based on the values of fsmonitor.allowRemote and fsmonitor.socketDir locate the Unix domain socket file in the desired location (either the .git directory, $HOME, or fsmonitor.socketDir). If the location is other than the .git directory, generate a unique file name based on the SHA1 has of the path to the .git directory.
--- fsmonitor-ipc.c | 40 +++++++++++++++++++++++++++++++++++++--- fsmonitor-ipc.h | 6 ++++++ 2 files changed, 43 insertions(+), 3 deletions(-)
fsmonitor-ipc.c +37 -3
@@ -1,7 +1,6 @@#include "cache.h" -#include "fsmonitor.h" -#include "simple-ipc.h" #include "fsmonitor-ipc.h" +#include "fsmonitor-settings.h" #include "run-command.h" #include "strbuf.h" #include "trace2.h"
@@ -47,7 +46,42 @@ int fsmonitor_ipc__is_supported(void)return 1 ; } -GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc") +GIT_PATH_FUNC(fsmonitor_ipc__get_default_path, "fsmonitor--daemon.ipc") + +const char *fsmonitor_ipc__get_path (void ) +{ +#ifdef WIN32 +return fsmonitor_ipc__get_default_path (); +#else +char *retval; +SHA_CTX sha1ctx; +const char *git_dir; +const char *sock_dir; +struct strbuf ipc_file =STRBUF_INIT ; +unsigned char hash[SHA_DIGEST_LENGTH ]; + +if (fsm_settings__get_allow_remote (the_repository) <1 ) +return fsmonitor_ipc__get_default_path (); + + git_dir =get_git_dir (); + sock_dir =fsm_settings__get_socket_dir (the_repository); + +SHA1_Init (&sha1ctx); +SHA1_Update (&sha1ctx, git_dir,strlen (git_dir)); +SHA1_Final (hash, &sha1ctx); + +if (sock_dir && *sock_dir) +strbuf_addf (&ipc_file,"%s/.git-fsmonitor-%s" , + sock_dir,hash_to_hex (hash)); +else +strbuf_addf (&ipc_file,"~/.git-fsmonitor-%s" ,hash_to_hex (hash)); + retval =interpolate_path (ipc_file.buf,1 ); +if (!retval) +die (_ ("Invalid path: %s" ), ipc_file.buf); +strbuf_release (&ipc_file); +return retval; +#endif +}enum ipc_active_state fsmonitor_ipc__get_state (void ) {
fsmonitor-ipc.h +6 -0
@@ -18,6 +18,12 @@ int fsmonitor_ipc__is_supported(void);*/ const char *fsmonitor_ipc__get_path (void ); +/* +* Returns the pathname to the default IPC named pipe or Unix domain +* socket. +*/ +const char *fsmonitor_ipc__get_default_path (void ); +/* * Try to determine whether there is a `git-fsmonitor--daemon` process * listening on the IPC pipe/socket.
Signature · 1 line
gitgitgadget
Re: [PATCH v4 2/4] fsmonitor: generate unique Unix socket file name in the desired location
Quoted 7 lines · Eric DeCosta via GitGitGadget
From: Eric DeCosta <[email protected]>
Based on the values of fsmonitor.allowRemote and fsmonitor.socketDir locate the Unix domain socket file in the desired location (either the .git directory, $HOME, or fsmonitor.socketDir). If the location is other than the .git directory, generate a unique file name based on the SHA1 has of the path to the .git directory.
Per:
fsmonitor-ipc.h- * Returns the pathname to the IPC named pipe or Unix domain socket fsmonitor-ipc.h- * where a git-fsmonitor--daemon process will listen. This is a fsmonitor-ipc.h- * per-worktree value.
fsmonitor-ipc.c int fsmonitor_ipc__is_supported(void) · line 76
Quoted 11 lines
fsmonitor-ipc.c +11
-0+ git_dir =get_git_dir (); + sock_dir =fsm_settings__get_socket_dir (the_repository); + +SHA1_Init (&sha1ctx); +SHA1_Update (&sha1ctx, git_dir,strlen (git_dir)); +SHA1_Final (hash, &sha1ctx); + +if (sock_dir && *sock_dir) +strbuf_addf (&ipc_file,"%s/.git-fsmonitor-%s" , + sock_dir,hash_to_hex (hash)); +else
But here we (from eyeballing this, maybe I've missed something):
1. Get the path to the git dir
2. SHA-1 hash that path, presumably to make it fixed size & get rid of slashes etc.
3. Make that the IPC filename
Per the "per worktree" can't we just check if:
* We have a .git/worktree/? If so derive the name from that.
* We don't? Then we just have one? Stick it in in there? I.e. isn't the hash_to_hex() here redundant?
...
fsmonitor-ipc.c int fsmonitor_ipc__is_supported(void) · line 77
Quoted 1 line
fsmonitor-ipc.c +1
-0+strbuf_addf (&ipc_file,"~/.git-fsmonitor-%s" ,hash_to_hex (hash));
... but not here, so is it just for this "else", but got carried over above?
I think if we're creating a new global cookie file, and presumably potentially a *lot* of them in the user's ~ we should at least prominently document that somewhere.
But more generally couldn't this?:
* Play nice with $HOME/.config/git/ etc, as is the usual convention these days on *nix
* We would have to the equivalent of a "mkdir -p", but if we stick it
in .config/git/fsmonitor-sockets or something we could have a nested path there that mirrors the path to the repo.
The latter can really help with debugging, you have this random .git-fsmonitor-XXXXX file, where the XXXX is totally mysterious, until you eventually find this code & discover it's a SHA-1 hash of some other path...
Re: [PATCH v4 2/4] fsmonitor: generate unique Unix socket file name in the desired location
In reply to Eric DeCosta via GitGitGadgetfsmonitor-ipc.c int fsmonitor_ipc__is_supported(void) · line 55
Quoted 5 lines · Eric DeCosta via GitGitGadget
fsmonitor-ipc.c +5
-0+const char *fsmonitor_ipc__get_path (void ) +{ +#ifdef WIN32 +return fsmonitor_ipc__get_default_path (); +#else
Hmph.
As compat/fsmonitor/ directory already sets up a good way to have platform-specific implementation of a common API function, I think this patch goes in a wrong direction by fighting it.
Shouldn't the rest of the function we see here be made to another implementation of fsmonitor_ipc__get_default_path() that is specific to platforms, i.e. in compat/fsmonitor/$somenewfile.c, that is used for !WIN32 case?
PATCH v4 3/4 fsmonitor: ensure filesystem and unix socket filesystem are compatible
In reply to Eric DeCosta via GitGitGadgetFrom: Eric DeCosta <[email protected]>
Perform separate checks for the working directory and Unix domain socket (UDS) file location. The working directory may be located on a network-mounted file system if 'fsmonitor.allowRemote' is true. The UDS file may never be located on a network-mounted file system; additionally it may not be located on FAT32 or NTFS file systems.
--- compat/fsmonitor/fsm-settings-darwin.c | 72 +++++++++++++++++++------- 1 file changed, 52 insertions(+), 20 deletions(-)
compat/fsmonitor/fsm-settings-darwin.c +52 -20
@@ -3,11 +3,12 @@#include "repository.h" #include "fsmonitor-settings.h" #include "fsmonitor.h" +#include "fsmonitor-ipc.h" #include <sys/param.h> #include <sys/mount.h> /* -* [1] Remote working directories are problematic for FSMonitor. +* Remote working directories are problematic for FSMonitor. * * The underlying file system on the server machine and/or the remote * mount type (NFS, SAMBA, etc.) dictates whether notification events
@@ -27,26 +28,8 @@* In theory, the above issues need to be addressed whether we are * using the Hook or IPC API. * -* For the builtin FSMonitor, we create the Unix domain socket for the -* IPC in the .git directory. If the working directory is remote, -* then the socket will be created on the remote file system. This -* can fail if the remote file system does not support UDS file types -* (e.g. smbfs to a Windows server) or if the remote kernel does not -* allow a non-local process to bind() the socket. (These problems -* could be fixed by moving the UDS out of the .git directory and to a -* well-known local directory on the client machine, but care should -* be taken to ensure that $HOME is actually local and not a managed -* file share.) -* * So (for now at least), mark remote working directories as -* incompatible. -* -* -* [2] FAT32 and NTFS working directories are problematic too. -* -* The builtin FSMonitor uses a Unix domain socket in the .git -* directory for IPC. These Windows drive formats do not support -* Unix domain sockets, so mark them as incompatible for the daemon. +* incompatible unless fsmonitor.allowRemote is true. * */ static enum fsmonitor_reason check_volume (struct repository *r)
@@ -65,6 +48,51 @@ static enum fsmonitor_reason check_volume(struct repository *r)"statfs('%s') [type 0x%08x][flags 0x%08x] '%s'" , r->worktree, fs.f_type, fs.f_flags, fs.f_fstypename); +if (!(fs.f_flags &MNT_LOCAL ) + && (fsm_settings__get_allow_remote (r) <1 )) +return FSMONITOR_REASON_REMOTE ; + +return FSMONITOR_REASON_OK ; +} + +/* +* For the builtin FSMonitor, we create the Unix domain socket (UDS) +* for the IPC in the .git directory by default or $HOME if +* fsmonitor.allowRemote is true. If the directory is remote, +* then the socket will be created on the remote file system. This +* can fail if the remote file system does not support UDS file types +* (e.g. smbfs to a Windows server) or if the remote kernel does not +* allow a non-local process to bind() the socket. +* +* Therefore remote UDS locations are marked as incompatible. +* +* FAT32 and NTFS working directories are problematic too. +* +* These Windows drive formats do not support Unix domain sockets, so +* mark them as incompatible for the location of the UDS file. +* +*/ +static enum fsmonitor_reason check_uds_volume (void ) +{ +struct statfs fs; +struct strbuf path =STRBUF_INIT ; +const char *ipc_path =fsmonitor_ipc__get_path (); +strbuf_add (&path, ipc_path,strlen (ipc_path)); + +if (statfs (dirname (path.buf), &fs) ==-1 ) { +int saved_errno = errno; +trace_printf_key (&trace_fsmonitor,"statfs('%s') failed: %s" , + path.buf,strerror (saved_errno)); + errno = saved_errno; +strbuf_release (&path); +return FSMONITOR_REASON_ERROR ; + } + +trace_printf_key (&trace_fsmonitor, +"statfs('%s') [type 0x%08x][flags 0x%08x] '%s'" , + path.buf, fs.f_type, fs.f_flags, fs.f_fstypename); +strbuf_release (&path); +if (!(fs.f_flags &MNT_LOCAL ))return FSMONITOR_REASON_REMOTE ;
@@ -85,5 +113,9 @@ enum fsmonitor_reason fsm_os__incompatible(struct repository *r)if (reason !=FSMONITOR_REASON_OK )return reason; + reason =check_uds_volume (); +if (reason !=FSMONITOR_REASON_OK ) +return reason; +return FSMONITOR_REASON_OK ; }
Signature · 1 line
gitgitgadget
PATCH v4 4/4 fsmonitor: normalize FSEvents event paths to the real path
In reply to Eric DeCosta via GitGitGadgetFrom: Eric DeCosta <[email protected]>
Consider the following network working directory that is mounted under /System/Volumes/Data:
/network/working/directory
The git working directory path is:
/System/Volumes/Data/network/working/directory
The paths reported by FSEvents always start with /network. fsmonitor expects paths to be under the working directory; therefore it fails to match /network/... and ignores the change.
Change things such that if fsmonitor.allowRemote is true that the paths reported via FSEevents are normalized to the real path.
--- compat/fsmonitor/fsm-listen-darwin.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)
compat/fsmonitor/fsm-listen-darwin.c +8 -3
@@ -26,6 +26,7 @@#include "fsmonitor.h" #include "fsm-listen.h" #include "fsmonitor--daemon.h" +#include "fsmonitor-settings.h" struct fsm_listen_data {
@@ -183,7 +184,6 @@ static void my_add_path(struct fsmonitor_batch *batch, const char *path)free (composed); } - static void fsevent_callback(ConstFSEventStreamRef streamRef, void *ctx, size_t num_of_events,
@@ -209,7 +209,12 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,/* * On Mac, we receive an array of absolute paths. */ - path_k = paths[k]; +if (fsm_settings__get_allow_remote (the_repository) >0 ) { +strbuf_reset (&tmp); +strbuf_realpath_forgiving (&tmp, paths[k],0 ); + path_k = tmp.buf; + }else + path_k = paths[k];/* * If you want to debug FSEvents, log them to GIT_TRACE_FSMONITOR.
@@ -237,6 +242,7 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,fsmonitor_force_resync (state);fsmonitor_batch__free_list (batch); + batch =NULL ;string_list_clear (&cookie_list,0 );/*
@@ -313,7 +319,6 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,case IS_WORKDIR_PATH :/* try to queue normal pathnames */ -if (trace_pass_fl (&trace_fsmonitor))log_flags_set (path_k, event_flags[k]);
Signature · 1 line
gitgitgadget
Re: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the real path
Quoted 36 lines · Eric DeCosta via GitGitGadget
From: Eric DeCosta <[email protected]>
Consider the following network working directory that is mounted under /System/Volumes/Data:
/network/working/directory
The git working directory path is:
/System/Volumes/Data/network/working/directory
The paths reported by FSEvents always start with /network. fsmonitor expects paths to be under the working directory; therefore it fails to match /network/... and ignores the change.
Change things such that if fsmonitor.allowRemote is true that the paths reported via FSEevents are normalized to the real path.
Signed-off-by Eric DeCosta <[email protected]>--- compat/fsmonitor/fsm-listen-darwin.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)compat/fsmonitor/fsm-listen-darwin.c +1
-0@@ -26,6 +26,7 @@#include "fsmonitor.h" #include "fsm-listen.h" #include "fsmonitor--daemon.h" +#include "fsmonitor-settings.h" struct fsm_listen_data {
compat/fsmonitor/fsm-listen-darwin.c static void my_add_path(struct fsmonitor_batch *batch, const char *path) · line 186
Quoted 4 lines
compat/fsmonitor/fsm-listen-darwin.c +0
-1@@ -183,7 +184,6 @@ static void my_add_path(struct fsmonitor_batch *batch, const char *path)free (composed); } -
Stray whitespace change, any one isn't much, but they add up (I saw another one earlier in this topic).
Quoted 4 lines
compat/fsmonitor/fsm-listen-darwin.c +0
-0static void fsevent_callback(ConstFSEventStreamRef streamRef, void *ctx, size_t num_of_events,
compat/fsmonitor/fsm-listen-darwin.c static void fsevent_callback(ConstFSEventStreamRef streamRef, · line 217
Quoted 10 lines
compat/fsmonitor/fsm-listen-darwin.c +6
-1@@ -209,7 +209,12 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,/* * On Mac, we receive an array of absolute paths. */ - path_k = paths[k]; +if (fsm_settings__get_allow_remote (the_repository) >0 ) { +strbuf_reset (&tmp); +strbuf_realpath_forgiving (&tmp, paths[k],0 ); + path_k = tmp.buf; + }else + path_k = paths[k];
Style: This else should have braces if any if/else arm does, see CodingGuidelines.
compat/fsmonitor/fsm-listen-darwin.c static void fsevent_callback(ConstFSEventStreamRef streamRef, · line 316
Quoted 6 lines
[...]
compat/fsmonitor/fsm-listen-darwin.c +0
-1@@ -313,7 +319,6 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,case IS_WORKDIR_PATH :/* try to queue normal pathnames */ -
More stray whitespace
Re: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the real path
In reply to Eric DeCosta via GitGitGadgetQuoted 7 lines · Eric DeCosta via GitGitGadget
From: Eric DeCosta <[email protected]>
Consider the following network working directory that is mounted under /System/Volumes/Data:
/network/working/directory
Quoted 7 lines
The git working directory path is:
/System/Volumes/Data/network/working/directory
The paths reported by FSEvents always start with /network. fsmonitor expects paths to be under the working directory; therefore it fails to match /network/... and ignores the change.
I'm not sure I understand what's going on here. Are you saying that FSEvents reports network mounted directories with a path relative to the mount-point rather than an absolute path?
In your example, is "/network/working/directory" a valid path on your system (that also happens to be the same directory as "/System/Volumes/...")
That is, do you have some aliasing going on where both paths are valid -- like a pair of hard-linked directories? Or, is there something special about a network mount point?
Did you have to "cd /System/Volumes/..." to get Git to have the absolute path be this? Or were you doing a "cd /network/..."? (Sorry to ask so many questions but I don't have a pair of systems to test any of this on right now.)
Quoted 30 lines
Change things such that if fsmonitor.allowRemote is true that the paths reported via FSEevents are normalized to the real path.
Signed-off-by Eric DeCosta <[email protected]>--- compat/fsmonitor/fsm-listen-darwin.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)compat/fsmonitor/fsm-listen-darwin.c +1
-1@@ -26,6 +26,7 @@#include "fsmonitor.h" #include "fsm-listen.h" #include "fsmonitor--daemon.h" +#include "fsmonitor-settings.h" struct fsm_listen_data {@@ -183,7 +184,6 @@ static void my_add_path(struct fsmonitor_batch *batch, const char *path)free (composed); } - static void fsevent_callback(ConstFSEventStreamRef streamRef, void *ctx, size_t num_of_events,
compat/fsmonitor/fsm-listen-darwin.c static void fsevent_callback(ConstFSEventStreamRef streamRef, · line 217
Quoted 10 lines
compat/fsmonitor/fsm-listen-darwin.c +6
-1@@ -209,7 +209,12 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,/* * On Mac, we receive an array of absolute paths. */ - path_k = paths[k]; +if (fsm_settings__get_allow_remote (the_repository) >0 ) { +strbuf_reset (&tmp); +strbuf_realpath_forgiving (&tmp, paths[k],0 ); + path_k = tmp.buf; + }else + path_k = paths[k];
This feels wrong.
It is not that fsmonitor.allowRemote is true, but whether or not this particular file system *IS* remote, right?
Quoted 20 lines
compat/fsmonitor/fsm-listen-darwin.c +1
-1/* * If you want to debug FSEvents, log them to GIT_TRACE_FSMONITOR. @@ -237,6 +242,7 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,fsmonitor_force_resync (state);fsmonitor_batch__free_list (batch); + batch =NULL ;string_list_clear (&cookie_list,0 );/* @@ -313,7 +319,6 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,case IS_WORKDIR_PATH :/* try to queue normal pathnames */ -if (trace_pass_fl (&trace_fsmonitor))log_flags_set (path_k, event_flags[k]);
RE: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the real path
Quoted 25 lines
-----Original Message-----
From: Jeff Hostetler <[email protected]>
Sent: Thursday, September 1, 2022 4:06 PM
To: Eric DeCosta via GitGitGadget <[email protected]>; [email protected]
Cc: Eric DeCosta <[email protected]>
Subject: Re: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the real pathQuoted 11 lines · Eric DeCosta via GitGitGadget
From: Eric DeCosta <[email protected]>
Consider the following network working directory that is mounted under /System/Volumes/Data:
/network/working/directory
The git working directory path is:
/System/Volumes/Data/network/working/directory
Quoted 8 lines
Quoted 3 lines
The paths reported by FSEvents always start with /network. fsmonitor expects paths to be under the working directory; therefore it fails to match /network/... and ignores the change.
I'm not sure I understand what's going on here.
Are you saying that FSEvents reports network mounted directories with a path relative to the mount-point rather than an absolute path?
Yes
Quoted 9 lines
In your example, is "/network/working/directory" a valid path on your system (that also happens to be the same directory as "/System/Volumes/...")
That is, do you have some aliasing going on where both paths are valid -- like a pair of hard-linked directories?
Or, is there something special about a network mount point?
Quoted 5 lines
Did you have to "cd /System/Volumes/..." to get Git to have the absolute path be this? Or were you doing a "cd /network/..."?
(Sorry to ask so many questions but I don't have a pair of systems to test any of this on right now.)
"/network/working/directory" is mounted under
"/System/Volumes/Data/network/working/directory"
There is also a symlink:
"/network" -> "/System/Volumes/Data/network"
No matter if I "cd /System/Volumes/Data/network/working/directory" or "cd /network/working/directory" the paths reported by FSEvents always start with "/network/working/directory"
If I do a similar symlink with local directories, I do not get this unexpected behavior.
I need to remove the symlink and try it that way, but I need to coordinate with the machine's owner.
Quoted 46 lines
compat/fsmonitor/fsm-listen-darwin.c static void my_add_path(struct fsmonitor_batch
Quoted 23 lines
Change things such that if fsmonitor.allowRemote is true that the paths reported via FSEevents are normalized to the real path.
Signed-off-by Eric DeCosta <[email protected]>--- compat/fsmonitor/fsm-listen-darwin.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)compat/fsmonitor/fsm-listen-darwin.c +1
-0@@ -26,6 +26,7 @@#include "fsmonitor.h" #include "fsm-listen.h" #include "fsmonitor--daemon.h" +#include "fsmonitor-settings.h" struct fsm_listen_data {@@ -183,7 +184,6 @@ static void my_add_path(struct fsmonitor_batch*batch, const char *path)
compat/fsmonitor/fsm-listen-darwin.c static void
Quoted 8 lines
compat/fsmonitor/fsm-listen-darwin.c +0
-1free (composed); } - static void fsevent_callback(ConstFSEventStreamRef streamRef, void *ctx, size_t num_of_events,@@ -209,7 +209,12 @@ static voidfsevent_callback(ConstFSEventStreamRef streamRef,
compat/fsmonitor/fsm-listen-darwin.c static void fsevent_callback(ConstFSEventStreamRef streamRef, · line 217
Quoted 10 lines
compat/fsmonitor/fsm-listen-darwin.c +6
-1/* * On Mac, we receive an array of absolute paths. */ - path_k = paths[k]; +if (fsm_settings__get_allow_remote (the_repository) >0 ) { +strbuf_reset (&tmp); +strbuf_realpath_forgiving (&tmp, paths[k],0 ); + path_k = tmp.buf; + }else + path_k = paths[k];This feels wrong.
Quoted 2 lines
It is not that fsmonitor.allowRemote is true, but whether or not this particular file system *IS* remote, right?
True. I suppose each path could be checked, or some bit could be set if the working directory is remote, perhaps along the lines of fsmonitor_ipc__get_path. Then the determination of the IPC path and whether it is remote would be in one place. fsm-settings-* would then just get that bit and check it against allowRemote.
Thoughts?
Quoted 25 lines
compat/fsmonitor/fsm-listen-darwin.c static void · line 219
Quoted 3 lines
compat/fsmonitor/fsm-listen-darwin.c +0
-0/* * If you want to debug FSEvents, log them to
GIT_TRACE_FSMONITOR.compat/fsmonitor/fsm-listen-darwin.c static void
Quoted 1 line
compat/fsmonitor/fsm-listen-darwin.c +0
-0@@ -237,6 +242,7 @@ static voidfsevent_callback(ConstFSEventStreamRef
compat/fsmonitor/fsm-listen-darwin.c static void
Quoted 9 lines
compat/fsmonitor/fsm-listen-darwin.c +1
-0streamRef,fsmonitor_force_resync (state);fsmonitor_batch__free_list (batch); + batch =NULL ;string_list_clear (&cookie_list,0 );/*
compat/fsmonitor/fsm-listen-darwin.c +0
-0@@ -313,7 +319,6 @@ static voidfsevent_callback(ConstFSEventStreamRef
Quoted 9 lines
compat/fsmonitor/fsm-listen-darwin.c +0
-1streamRef, case IS_WORKDIR_PATH:/* try to queue normal pathnames */ -if (trace_pass_fl (&trace_fsmonitor))log_flags_set (path_k, event_flags[k]);
Re: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the real path
Quoted 62 lines · Eric DeCosta
Quoted 25 lines
-----Original Message-----
From: Jeff Hostetler <[email protected]>
Sent: Thursday, September 1, 2022 4:06 PM
To: Eric DeCosta via GitGitGadget <[email protected]>; [email protected]
Cc: Eric DeCosta <[email protected]>
Subject: Re: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the real pathQuoted 11 lines · Eric DeCosta via GitGitGadget
From: Eric DeCosta <[email protected]>
Consider the following network working directory that is mounted under /System/Volumes/Data:
/network/working/directory
The git working directory path is:
/System/Volumes/Data/network/working/directory
Quoted 8 lines
Quoted 3 lines
The paths reported by FSEvents always start with /network. fsmonitor expects paths to be under the working directory; therefore it fails to match /network/... and ignores the change.
I'm not sure I understand what's going on here.
Are you saying that FSEvents reports network mounted directories with a path relative to the mount-point rather than an absolute path?Yes
Quoted 9 lines
In your example, is "/network/working/directory" a valid path on your system (that also happens to be the same directory as "/System/Volumes/...")
That is, do you have some aliasing going on where both paths are valid -- like a pair of hard-linked directories?
Or, is there something special about a network mount point?Quoted 5 lines
Did you have to "cd /System/Volumes/..." to get Git to have the absolute path be this? Or were you doing a "cd /network/..."?
(Sorry to ask so many questions but I don't have a pair of systems to test any of this on right now.)"/network/working/directory" is mounted under
"/System/Volumes/Data/network/working/directory"There is also a symlink:
"/network" -> "/System/Volumes/Data/network"
No matter if I "cd /System/Volumes/Data/network/working/directory" or "cd /network/working/directory" the paths reported by FSEvents always start with "/network/working/directory"
Quoted 5 lines
If I do a similar symlink with local directories, I do not get this unexpected behavior.
I need to remove the symlink and try it that way, but I need to coordinate with the machine's owner.
I think you have stumbled upon a recent MacOS feature called "firmlinks". I'm just reading up on it myself, so I shouldn't speculate here (yet), but maybe you too could do some reading on the subject.
This makes me wonder if the symlink is historical. The real magic is in the firmlinks. For example, if I do:
$ (cd / ; ls -l | grep Users) drwxr-xr-x 6 root admin 192 Apr 6 2020 Users
$ (cd /Users ; df .) Filesystem 512-blocks Used Available Capacity iused ifree %iused Mounted on /dev/disk1s1 976490576 608954344 338850488 65% 4181323 4878271557 0% /System/Volumes/Data
we can see that /Users is on /System/Volumes/Data (and there is a /System/Volumes/Data/Users directory with the same metadata), but it is not a symlink.
See [1] for more info.
[1] http://www.swiftforensics.com/2019/10/macos-1015-volumes-firmlink-magic.html
[...]
Quoted 18 lines
Quoted 15 lines
Review static void
Quoted 1 line
(quoted diff) +0
-0@@ -209,7 +209,12 @@ static voidfsevent_callback(ConstFSEventStreamRef streamRef,
compat/fsmonitor/fsm-listen-darwin.c static void fsevent_callback(ConstFSEventStreamRef streamRef, · line 217
Quoted 10 lines
compat/fsmonitor/fsm-listen-darwin.c +6
-1/* * On Mac, we receive an array of absolute paths. */ - path_k = paths[k]; +if (fsm_settings__get_allow_remote (the_repository) >0 ) { +strbuf_reset (&tmp); +strbuf_realpath_forgiving (&tmp, paths[k],0 ); + path_k = tmp.buf; + }else + path_k = paths[k];This feels wrong.
Quoted 2 lines
It is not that fsmonitor.allowRemote is true, but whether or not this particular file system *IS* remote, right?
Quoted 7 lines
True. I suppose each path could be checked, or some bit could be set if the working directory is remote, perhaps along the lines of
fsmonitor_ipc__get_path. Then the determination of the IPC path and whether it is remote would be in one place. fsm-settings-* would then just get that bit and check it against allowRemote.Thoughts?
I'll do some digging here. There ought to be a way to tell if a component directory in a pathname has a firmlink peer. (But I'm traveling for GitMerge, so I might not have time to look at this until afterwards.)
This would indicate that a "bi-directional wormhole" (their words) alias is available (and hopefully, a way to computer the other peer....)
I'm thinking that the "/network/..." path in the FSEvents is because FSEventD is using a particular peer spelling (that we weren't expecting).
If we can compute the spellings of the peers once when the daemon starts up (and maybe make a little dictionary), then we can do prefix tricks on the absolute path after the
path_k = paths[k];
step in fsevent_callback() to extract a relative path rather than an absolute path.
Then call fsmonitor_classify_path_relative() instead of _absolute().
Watch out though, there are several places that do
rel = path+k + ...state->path_worktree_watch.len...
that would need to be adjusted.
Hope this helps,
Jeff
RE: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the real path
Quoted 104 lines
-----Original Message-----
From: Jeff Hostetler <[email protected]>
Sent: Tuesday, September 6, 2022 1:14 PM
To: Eric DeCosta <[email protected]>; Eric DeCosta via GitGitGadget <[email protected]>; [email protected]
Subject: Re: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the real pathQuoted 63 lines · Eric DeCosta
Quoted 26 lines
-----Original Message-----
From: Jeff Hostetler <[email protected]>
Sent: Thursday, September 1, 2022 4:06 PM
To: Eric DeCosta via GitGitGadget <[email protected]>; [email protected]
Cc: Eric DeCosta <[email protected]>
Subject: Re: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the real pathQuoted 12 lines · Eric DeCosta via GitGitGadget
From: Eric DeCosta <[email protected]>
Consider the following network working directory that is mounted under
/System/Volumes/Data:/network/working/directory
The git working directory path is:
/System/Volumes/Data/network/working/directory
Quoted 8 lines
Quoted 3 lines
The paths reported by FSEvents always start with /network. fsmonitor expects paths to be under the working directory; therefore it fails to match /network/... and ignores the change.
I'm not sure I understand what's going on here.
Are you saying that FSEvents reports network mounted directories with a path relative to the mount-point rather than an absolute path?Yes
Quoted 9 lines
In your example, is "/network/working/directory" a valid path on your system (that also happens to be the same directory as "/System/Volumes/...")
That is, do you have some aliasing going on where both paths are valid -- like a pair of hard-linked directories?
Or, is there something special about a network mount point?Quoted 5 lines
Did you have to "cd /System/Volumes/..." to get Git to have the absolute path be this? Or were you doing a "cd /network/..."? (Sorry to ask so many questions but I don't have a pair of systems to test any of this on right now.)
"/network/working/directory" is mounted under
"/System/Volumes/Data/network/working/directory"There is also a symlink:
"/network" -> "/System/Volumes/Data/network"
No matter if I "cd /System/Volumes/Data/network/working/directory" or "cd /network/working/directory" the paths reported by FSEvents always start with "/network/working/directory"
Quoted 5 lines
If I do a similar symlink with local directories, I do not get this unexpected behavior.
I need to remove the symlink and try it that way, but I need to coordinate with the machine's owner.
I think you have stumbled upon a recent MacOS feature called "firmlinks". I'm just reading up on it myself, so I shouldn't speculate here (yet), but maybe you too could do some reading on the subject.
This makes me wonder if the symlink is historical. The real magic is in the firmlinks. For example, if I do:
$ (cd / ; ls -l | grep Users) drwxr-xr-x 6 root admin 192 Apr 6 2020 Users$ (cd /Users ; df .) Filesystem 512-blocks Used Available Capacity iused ifree %iused Mounted on /dev/disk1s1 976490576 608954344 338850488 65% 4181323 4878271557 0% /System/Volumes/Datawe can see that /Users is on /System/Volumes/Data (and there is a /System/Volumes/Data/Users directory with the same metadata), but it is not a symlink.
Quoted 9 lines
See [1] for more info.
[1]
http://www.swiftforensics.com/2019/10/macos-1015-volumes-firmlink-
magic.html <https://protect-
us.mimecast.com/s/QN4qCYEZB2h1p7jWcG1Cbw?domain=swiftforensics.co m>
Hmm, I don't see anything that suggests a firmlink is involved but this is new to me so maybe I just don't see it. There is an automount in addition to the symlink though.
So, dispensing with the abstract "/network" path, and getting to the details of my environment:
% mount | grep /mathworks
map auto.sfs.sol2.mathworks on /System/Volumes/Data/mathworks (autofs, automounted, nobrowse)
triggered map auto.sfs.sol2.devel on /System/Volumes/Data/mathworks/devel (autofs, automounted, nobrowse)
triggered map sfs.worldwide.US-Natick.sol2.devel.sbs on /System/Volumes/Data/mathworks/devel/sbs (autofs, automounted, nobrowse)
triggered map sfs.worldwide.US-Natick.devel.sbs.29 on /System/Volumes/Data/mathworks/devel/sbs/29 (autofs, automounted, nobrowse) batfs-sb29-nfs:/vmgr/sbs29/edecosta.Bbashprod1.j1928560.2 on /System/Volumes/Data/mathworks/devel/sbs/29/edecosta.Bbashprod1.j1928560.2 (nfs, nodev, automounted, noatime, nobrowse)
...
% ls -l / | grep mathworks
lrwxr-xr-x 1 root wheel 35 Aug 29 10:25 home@ -> /System/Volumes/Data/mathworks/home lrwxr-xr-x 1 root wheel 30 Aug 29 10:25 mathworks@ -> /System/Volumes/Data/mathworks
My worktree is: /System/Volumes/Data/mathworks/devel/sbs/29/edecosta.Bbashprod1.j1928560.2 or, equivalently via the symlink just: /mathworks/devel/sbs/29/edecosta.Bbashprod1.j1928560.2
I have sudo'ers access now; so I can try mounting things manually, mess about with symlinks and see what I get.
Quoted 46 lines
[...]
Quoted 18 lines
Quoted 13 lines
Review static void
Quoted 1 line
(quoted diff) +0
-0@@ -209,7 +209,12 @@ static voidfsevent_callback(ConstFSEventStreamRef streamRef,
compat/fsmonitor/fsm-listen-darwin.c static void fsevent_callback(ConstFSEventStreamRef streamRef, · line 217
Quoted 10 lines
compat/fsmonitor/fsm-listen-darwin.c +6
-1/* * On Mac, we receive an array of absolute paths. */ - path_k = paths[k]; +if (fsm_settings__get_allow_remote (the_repository) >0 ) { +strbuf_reset (&tmp); +strbuf_realpath_forgiving (&tmp, paths[k],0 ); + path_k = tmp.buf; + }else + path_k = paths[k];Quoted 4 lines
This feels wrong.
It is not that fsmonitor.allowRemote is true, but whether or not this particular file system *IS* remote, right?
Quoted 7 lines
True. I suppose each path could be checked, or some bit could be set if the working directory is remote, perhaps along the lines of
fsmonitor_ipc__get_path. Then the determination of the IPC path and whether it is remote would be in one place. fsm-settings-* would then just get that bit and check it against allowRemote.Thoughts?
I'll do some digging here. There ought to be a way to tell if a component directory in a pathname has a firmlink peer. (But I'm traveling for GitMerge, so I might not have time to look at this until afterwards.)
This would indicate that a "bi-directional wormhole" (their words) alias is available (and hopefully, a way to computer the other peer....)
I'm thinking that the "/network/..." path in the FSEvents is because FSEventD is using a particular peer spelling (that we weren't expecting).
If we can compute the spellings of the peers once when the daemon starts up (and maybe make a little dictionary), then we can do prefix tricks on the absolute path after the
path_k= paths[k];
step infsevent_callback()to extract a relative path rather than an absolute path.
Quoted 10 lines
Then call
fsmonitor_classify_path_relative()instead of_absolute().Watch out though, there are several places that do rel = path+k + ...state->path_worktree_watch.len... that would need to be adjusted.
Hope this helps,
Jeff
Thanks for the insights, I'll dig around and test things out some more.
-Eric
RE: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the real path
Quoted 209 lines
-----Original Message-----
From: Eric DeCosta
Sent: Tuesday, September 6, 2022 3:02 PM
To: Jeff Hostetler <[email protected]>; Eric DeCosta via GitGitGadget <[email protected]>; [email protected]
Subject: RE: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the real pathQuoted 102 lines
-----Original Message-----
From: Jeff Hostetler <[email protected]>
Sent: Tuesday, September 6, 2022 1:14 PM
To: Eric DeCosta <[email protected]>; Eric DeCosta via GitGitGadget <[email protected]>; [email protected]
Subject: Re: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the real pathQuoted 63 lines · Eric DeCosta
Quoted 26 lines
-----Original Message-----
From: Jeff Hostetler <[email protected]>
Sent: Thursday, September 1, 2022 4:06 PM
To: Eric DeCosta via GitGitGadget <[email protected]>; [email protected]
Cc: Eric DeCosta <[email protected]>
Subject: Re: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the real pathQuoted 12 lines · Eric DeCosta via GitGitGadget
From: Eric DeCosta <[email protected]>
Consider the following network working directory that is mounted under
/System/Volumes/Data:/network/working/directory
The git working directory path is:
/System/Volumes/Data/network/working/directory
Quoted 8 lines
Quoted 3 lines
The paths reported by FSEvents always start with /network. fsmonitor expects paths to be under the working directory; therefore it fails to match /network/... and ignores the change.
I'm not sure I understand what's going on here.
Are you saying that FSEvents reports network mounted directories with a path relative to the mount-point rather than an absolute path?Yes
Quoted 9 lines
In your example, is "/network/working/directory" a valid path on your system (that also happens to be the same directory as "/System/Volumes/...")
That is, do you have some aliasing going on where both paths are valid -- like a pair of hard-linked directories?
Or, is there something special about a network mount point?Quoted 5 lines
Did you have to "cd /System/Volumes/..." to get Git to have the absolute path be this? Or were you doing a "cd /network/..."? (Sorry to ask so many questions but I don't have a pair of systems to test any of this on right now.)
"/network/working/directory" is mounted under
"/System/Volumes/Data/network/working/directory"There is also a symlink:
"/network" -> "/System/Volumes/Data/network"
No matter if I "cd /System/Volumes/Data/network/working/directory" or "cd /network/working/directory" the paths reported by FSEvents always start with "/network/working/directory"
Quoted 5 lines
If I do a similar symlink with local directories, I do not get this unexpected behavior.
I need to remove the symlink and try it that way, but I need to coordinate with the machine's owner.
I think you have stumbled upon a recent MacOS feature called "firmlinks". I'm just reading up on it myself, so I shouldn't speculate here (yet), but maybe you too could do some reading on the subject.
This makes me wonder if the symlink is historical. The real magic is in the firmlinks. For example, if I do:
$ (cd / ; ls -l | grep Users) drwxr-xr-x 6 root admin 192 Apr 6 2020 Users$ (cd /Users ; df .) Filesystem 512-blocks Used Available Capacity iused ifree %iused Mounted on /dev/disk1s1 976490576 608954344 338850488 65% 4181323 4878271557 0% /System/Volumes/Datawe can see that /Users is on /System/Volumes/Data (and there is a /System/Volumes/Data/Users directory with the same metadata), but it is not a symlink.
Quoted 6 lines
See [1] for more info.
[1]
http://www.swiftforensics.com/2019/10/macos-1015-volumes-firmlink-
magic.html <https://protect-us.mimecast.com/s/QN4qCYEZB2h1p7jWcG1Cbw?domain=swiftforensics.co
Quoted 3 lines
m>
Hmm, I don't see anything that suggests a firmlink is involved but this is new to me so maybe I just don't see it. There is an automount in addition to the symlink though.
So, dispensing with the abstract "/network" path, and getting to the details of my environment:
% mount | grep /mathworks
map auto.sfs.sol2.mathworks on /System/Volumes/Data/mathworks (autofs, automounted, nobrowse) triggered map auto.sfs.sol2.devel on /System/Volumes/Data/mathworks/devel (autofs, automounted, nobrowse) triggered map sfs.worldwide.US-Natick.sol2.devel.sbs on /System/Volumes/Data/mathworks/devel/sbs (autofs, automounted, nobrowse) triggered map sfs.worldwide.US-Natick.devel.sbs.29 on /System/Volumes/Data/mathworks/devel/sbs/29 (autofs, automounted, nobrowse) batfs-sb29-nfs:/vmgr/sbs29/edecosta.Bbashprod1.j1928560.2 on /System/Volumes/Data/mathworks/devel/sbs/29/edecosta.Bbashprod1.j19 28560.2 (nfs, nodev, automounted, noatime, nobrowse) ...% ls -l / | grep mathworks
lrwxr-xr-x 1 root wheel 35 Aug 29 10:25 home@ ->
/System/Volumes/Data/mathworks/home
lrwxr-xr-x 1 root wheel 30 Aug 29 10:25 mathworks@ ->
/System/Volumes/Data/mathworksMy worktree is:
/System/Volumes/Data/mathworks/devel/sbs/29/edecosta.Bbashprod1.j19 28560.2
or, equivalently via the symlink just:
/mathworks/devel/sbs/29/edecosta.Bbashprod1.j1928560.2I have sudo'ers access now; so I can try mounting things manually, mess about with symlinks and see what I get.
Quoted 49 lines
[...]
Quoted 15 lines
Quoted 10 lines
Review static void
Quoted 1 line
(quoted diff) +0
-0@@ -209,7 +209,12 @@ static voidfsevent_callback(ConstFSEventStreamRef streamRef,
compat/fsmonitor/fsm-listen-darwin.c static void fsevent_callback(ConstFSEventStreamRef streamRef,
Quoted 7 lines
compat/fsmonitor/fsm-listen-darwin.c +3
-1/* * On Mac, we receive an array of absolute paths. */ - path_k = paths[k]; +if (fsm_settings__get_allow_remote (the_repository) >0 ) { +strbuf_reset (&tmp);strbuf_realpath_forgiving (&tmp, paths[k], +0 ); path_k = tmp.buf; }else path_k = paths[k];Quoted 4 lines
This feels wrong.
It is not that fsmonitor.allowRemote is true, but whether or not this particular file system *IS* remote, right?
Quoted 7 lines
True. I suppose each path could be checked, or some bit could be set if the working directory is remote, perhaps along the lines of
fsmonitor_ipc__get_path. Then the determination of the IPC path and whether it is remote would be in one place. fsm-settings-* would then just get that bit and check it against allowRemote.Thoughts?
I'll do some digging here. There ought to be a way to tell if a component directory in a pathname has a firmlink peer. (But I'm traveling for GitMerge, so I might not have time to look at this until afterwards.)
This would indicate that a "bi-directional wormhole" (their words) alias is available (and hopefully, a way to computer the other peer....)
I'm thinking that the "/network/..." path in the FSEvents is because FSEventD is using a particular peer spelling (that we weren't expecting).
If we can compute the spellings of the peers once when the daemon starts up (and maybe make a little dictionary), then we can do prefix tricks on the absolute path after the
path_k= paths[k]; step infsevent_callback()to extract a relative path rather than an absolute path.Then call
fsmonitor_classify_path_relative()instead of_absolute().Watch out though, there are several places that do rel = path+k + ...state->path_worktree_watch.len...
that would need to be adjusted.
Quoted 7 lines
Quoted 4 lines
Hope this helps,
JeffThanks for the insights, I'll dig around and test things out some more.
-Eric
This is informative:
https://developer.apple.com/forums/thread/120665
-Eric
RE: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the real path
Quoted 233 lines
-----Original Message-----
From: Eric DeCosta
Sent: Tuesday, September 6, 2022 3:34 PM
To: 'Jeff Hostetler' <[email protected]>; 'Eric DeCosta via GitGitGadget' <[email protected]>; '[email protected]' <[email protected]>
Subject: RE: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the real pathQuoted 92 lines
-----Original Message-----
From: Eric DeCosta
Sent: Tuesday, September 6, 2022 3:02 PM
To: Jeff Hostetler <[email protected]>; Eric DeCosta via GitGitGadget <[email protected]>; [email protected]
Subject: RE: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the real pathQuoted 82 lines
-----Original Message-----
From: Jeff Hostetler <[email protected]>
Sent: Tuesday, September 6, 2022 1:14 PM
To: Eric DeCosta <[email protected]>; Eric DeCosta via GitGitGadget <[email protected]>; [email protected]
Subject: Re: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the real pathQuoted 63 lines · Eric DeCosta
Quoted 26 lines
-----Original Message-----
From: Jeff Hostetler <[email protected]>
Sent: Thursday, September 1, 2022 4:06 PM
To: Eric DeCosta via GitGitGadget <[email protected]>; [email protected]
Cc: Eric DeCosta <[email protected]>
Subject: Re: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the real pathQuoted 12 lines · Eric DeCosta via GitGitGadget
From: Eric DeCosta <[email protected]>
Consider the following network working directory that is mounted under
/System/Volumes/Data:/network/working/directory
The git working directory path is:
/System/Volumes/Data/network/working/directory
Quoted 8 lines
Quoted 3 lines
The paths reported by FSEvents always start with /network. fsmonitor expects paths to be under the working directory; therefore it fails to match /network/... and ignores the change.
I'm not sure I understand what's going on here.
Are you saying that FSEvents reports network mounted directories with a path relative to the mount-point rather than an absolute path?Yes
Quoted 9 lines
In your example, is "/network/working/directory" a valid path on your system (that also happens to be the same directory as "/System/Volumes/...")
That is, do you have some aliasing going on where both paths are valid -- like a pair of hard-linked directories?
Or, is there something special about a network mount point?Quoted 5 lines
Did you have to "cd /System/Volumes/..." to get Git to have the absolute path be this? Or were you doing a "cd /network/..."? (Sorry to ask so many questions but I don't have a pair of systems to test any of this on right now.)
"/network/working/directory" is mounted under
"/System/Volumes/Data/network/working/directory"There is also a symlink:
"/network" -> "/System/Volumes/Data/network"
No matter if I "cd /System/Volumes/Data/network/working/directory" or "cd /network/working/directory" the paths reported by FSEvents always start with "/network/working/directory"
Quoted 5 lines
If I do a similar symlink with local directories, I do not get this unexpected behavior.
I need to remove the symlink and try it that way, but I need to coordinate with the machine's owner.
I think you have stumbled upon a recent MacOS feature called
"firmlinks".
Quoted 6 lines
Quoted 6 lines
I'm just reading up on it myself, so I shouldn't speculate here (yet), but maybe you too could do some reading on the subject.
This makes me wonder if the symlink is historical. The real magic is in the firmlinks. For example, if I do:
Quoted 7 lines
Quoted 7 lines
$ (cd / ; ls -l | grep Users) drwxr-xr-x 6 root admin 192 Apr 6 2020 Users$ (cd /Users ; df .) Filesystem 512-blocks Used Available Capacity iused ifree %iused Mounted on /dev/disk1s1 976490576 608954344 338850488 65% 4181323 48782715570%
Quoted 7 lines
Quoted 7 lines
/System/Volumes/Data
we can see that /Users is on /System/Volumes/Data (and there is a /System/Volumes/Data/Users directory with the same metadata), but it is not a symlink.
Quoted 7 lines
Quoted 6 lines
See [1] for more info.
[1]
http://www.swiftforensics.com/2019/10/macos-1015-volumes-firmlink-
magic.html <https://protect-us.mimecast.com/s/QN4qCYEZB2h1p7jWcG1Cbw?domain=swiftforensics.co
Quoted 8 lines
Quoted 3 lines
m>
Hmm, I don't see anything that suggests a firmlink is involved but this is new to me so maybe I just don't see it. There is an automount in addition to the symlink though.
Quoted 5 lines
So, dispensing with the abstract "/network" path, and getting to the details of my environment:
% mount | grep /mathworks
map auto.sfs.sol2.mathworks on /System/Volumes/Data/mathworks(autofs,
Quoted 9 lines
automounted, nobrowse) triggered map auto.sfs.sol2.devel on /System/Volumes/Data/mathworks/devel (autofs, automounted, nobrowse) triggered map sfs.worldwide.US-Natick.sol2.devel.sbs on /System/Volumes/Data/mathworks/devel/sbs (autofs, automounted, nobrowse) triggered map sfs.worldwide.US-Natick.devel.sbs.29 on /System/Volumes/Data/mathworks/devel/sbs/29 (autofs, automounted, nobrowse) batfs-sb29-nfs:/vmgr/sbs29/edecosta.Bbashprod1.j1928560.2 on
/System/Volumes/Data/mathworks/devel/sbs/29/edecosta.Bbashprod1.j19
Quoted 10 lines
28560.2 (nfs, nodev, automounted, noatime, nobrowse) ...
% ls -l / | grep mathworks
lrwxr-xr-x 1 root wheel 35 Aug 29 10:25 home@ ->
/System/Volumes/Data/mathworks/home
lrwxr-xr-x 1 root wheel 30 Aug 29 10:25 mathworks@ ->
/System/Volumes/Data/mathworksMy worktree is:
/System/Volumes/Data/mathworks/devel/sbs/29/edecosta.Bbashprod1.j19
Quoted 64 lines
28560.2
or, equivalently via the symlink just:
/mathworks/devel/sbs/29/edecosta.Bbashprod1.j1928560.2I have sudo'ers access now; so I can try mounting things manually, mess about with symlinks and see what I get.
Quoted 45 lines
[...]
Quoted 15 lines
Quoted 10 lines
Review static void
Quoted 1 line
(quoted diff) +0
-0@@ -209,7 +209,12 @@ static voidfsevent_callback(ConstFSEventStreamRef streamRef,
compat/fsmonitor/fsm-listen-darwin.c static void fsevent_callback(ConstFSEventStreamRef streamRef,
Quoted 7 lines
compat/fsmonitor/fsm-listen-darwin.c +3
-1/* * On Mac, we receive an array of absolute paths. */ - path_k = paths[k]; +if (fsm_settings__get_allow_remote (the_repository) >0 ) { +strbuf_reset (&tmp);strbuf_realpath_forgiving (&tmp, paths[k], +0 ); path_k = tmp.buf; }else path_k = paths[k];Quoted 4 lines
This feels wrong.
It is not that fsmonitor.allowRemote is true, but whether or not this particular file system *IS* remote, right?
Quoted 7 lines
True. I suppose each path could be checked, or some bit could be set if the working directory is remote, perhaps along the lines of
fsmonitor_ipc__get_path. Then the determination of the IPC path and whether it is remote would be in one place. fsm-settings-* would then just get that bit and check it against allowRemote.Thoughts?
I'll do some digging here. There ought to be a way to tell if a component directory in a pathname has a firmlink peer. (But I'm traveling for GitMerge, so I might not have time to look at this until
afterwards.)This would indicate that a "bi-directional wormhole" (their words) alias is available (and hopefully, a way to computer the other peer....)
I'm thinking that the "/network/..." path in the FSEvents is because FSEventD is using a particular peer spelling (that we weren't expecting).
If we can compute the spellings of the peers once when the daemon starts up (and maybe make a little dictionary), then we can do prefix tricks on the absolute path after the
path_k= paths[k]; step infsevent_callback()to extract a relative path rather than an absolute path.Quoted 10 lines
Then call
fsmonitor_classify_path_relative()instead of_absolute().Watch out though, there are several places that do rel = path+k + ...state->path_worktree_watch.len...
that would need to be adjusted.Hope this helps,
JeffThanks for the insights, I'll dig around and test things out some more.
Quoted 8 lines
Quoted 1 line
-Eric
This is informative:
https://developer.apple.com/forums/thread/120665
-Eric
Conversation with one of our macOS admins:
Q: Is /mathworks a symbolic link or some sort of firmlink or synthetic firmlink?
A: It’s a “synthetic” link, per apple. Config is managed in /etc/synthetic.conf
The auto mounter will also automatically create it based on the contents of /etc/auto_master even without the synthetic.conf file. We went back and forth with apple for a while on the “right” way to do it and now I’m not sure what we settled on, off the top of my head
I don’t know if they’re the same under the hood, but it seemed to be able to create root level links even though you can’t with ln anymore
I suspect apple wants users to use the synthetic.conf file in all cases...
----
It appears that the automounter is creating "synthetic firmlinks", a la https://derflounder.wordpress.com/2020/01/18/creating-root-level-directories-and-symbolic-links-on-macos-catalina/
When I look in /etc/auto_master I see several entries that appear as symlinks in the output of "ls -l /" but in reality they are synthetic firmlinks.
-Eric
Re: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the real path
In reply to Eric DeCostaQuoted 2 lines · Eric DeCosta
[...]
Quoted 8 lines
This is informative:
https://developer.apple.com/forums/thread/120665
-Eric
How strange........
Have you tried the:
fcntl(F_GETPATH)
vs fcntl(F_GETPATH_NOFIRMLINK)
vs realpath()
comparison suggested in the comments and does it return anything sensical? Such that we could record both spellings when the daemon starts up. I think we'd have to do it on an open fd on the worktree root directory.
Jeff
RE: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the real path
Quoted 39 lines
-----Original Message-----
From: Jeff Hostetler <[email protected]>
Sent: Wednesday, September 7, 2022 3:15 PM
To: Eric DeCosta <[email protected]>; Eric DeCosta via GitGitGadget <[email protected]>; [email protected]
Subject: Re: [PATCH v4 4/4] fsmonitor: normalize FSEvents event paths to the real pathQuoted 2 lines · Eric DeCosta
[...]
Quoted 5 lines
This is informative:
https://developer.apple.com/forums/thread/120665
<https://protect-us.mimecast.com/s/Vj1mCL9GlLUjmyJjfqLBqG?domain=devel
Quoted 5 lines
oper.apple.com>
-Eric
How strange........
Have you tried the:
fcntl(F_GETPATH)
vsfcntl(F_GETPATH_NOFIRMLINK)
vsrealpath()comparison suggested in the comments and does it return anything sensical? Such that we could record both spellings when the daemon starts up. I think we'd have to do it on an open fd on the worktree root directory.
Quoted 6 lines
Jeff
https://developer.apple.com/forums/thread/120665 <https://protect-
us.mimecast.com/s/Vj1mCL9GlLUjmyJjfqLBqG?domain=developer.apple.comQuoted 1 line
I'm not getting the same result:
Worktree is: '/System/Volumes/Data/mathworks/devel/sandbox/edecosta/git' F_GETPATH of worktree is '/System/Volumes/Data/mathworks/devel/sandbox/edecosta/git' F_GETPATH_NOFIRMLINK of worktree is '/System/Volumes/Data/mathworks/devel/sandbox/edecosta/git' realpath() of worktree is '/System/Volumes/Data/mathworks/devel/sandbox/edecosta/git'
Trying /mathworks/devel/sandbox/edecosta/gitF_GETPATH of /mathworks/devel/sandbox/edecosta/git is '/System/Volumes/Data/mathworks/devel/sandbox/edecosta/git' F_GETPATH_NOFIRMLINK of /mathworks/devel/sandbox/edecosta/git is '/System/Volumes/Data/mathworks/devel/sandbox/edecosta/git' realpath() of /mathworks/devel/sandbox/edecosta/git is '/System/Volumes/Data/mathworks/devel/sandbox/edecosta/git'
Either something has changed or the fact that "/mathworks" is a "synthetic firmlink" has something to do with it.
Maybe look for synthetic firmlinks in the root directory and use that information to figure out what aliases there might be to the worktree.
lrwxr-xr-x 1 root wheel 30 Sep 2 11:52 mathworks -> /System/Volumes/Data/mathworks
If what the link resolves to is a prefix of the worktree, then it is a firmlink to the worktree.
Something like that.
-Eric
v5 cover fsmonitor: option to allow fsmonitor to run against network-mounted repos
In reply to Eric DeCosta via GitGitGadgetFollow-on to the work done to allow Windows to work against network-mounted repos for macOS.
Have macOS take advantage of the same configuration option, 'fsmonitor.allowRemote' that was introduced for Windows. Setting this option to true will override the default behavior (erroring-out) when a network-mounted repo is detected by fsmonitor.
The added wrinkle being that the Unix domain socket (UDS) file used for IPC cannot be created in a network location; instead $HOME is used if the default location is on the network. The user may, optionally, set the 'fsmonitor.socketDir' configuration option to a valid, local directory if $HOME itself is on the network or is simply not the desired location for the UDS file.
An additional issue is that for mount points in the root directory, FSEvents does not report a path that matches the worktree directory due to the introduction of 'synthetic firmlinks'. fsmonitor must map the FSEvents paths to the worktree directory by interrogating the root filesystem for synthetic firmlinks and using that information to translate the path.
Eric DeCosta (4):
fsmonitor: refactor filesystem checks to common interface
fsmonitor: relocate socket file if .git directory is remote
fsmonitor: avoid socket location check if using hook
fsmonitor: deal with synthetic firmlinks on macOS
Makefile | 2 + builtin/fsmonitor--daemon.c | 8 ++ compat/fsmonitor/fsm-ipc-darwin.c | 46 ++++++++ compat/fsmonitor/fsm-ipc-win32.c | 4 + compat/fsmonitor/fsm-listen-darwin.c | 6 +- compat/fsmonitor/fsm-path-utils-darwin.c | 131 +++++++++++++++++++++++ compat/fsmonitor/fsm-path-utils-win32.c | 106 ++++++++++++++++++ compat/fsmonitor/fsm-settings-darwin.c | 70 ++++-------- compat/fsmonitor/fsm-settings-win32.c | 106 +----------------- contrib/buildsystems/CMakeLists.txt | 4 + fsmonitor--daemon.h | 6 ++ fsmonitor-ipc.c | 2 - fsmonitor-path-utils.h | 49 +++++++++ fsmonitor-settings.c | 58 +++++++++- fsmonitor-settings.h | 2 +- 15 files changed, 441 insertions(+), 159 deletions(-) create mode 100644 compat/fsmonitor/fsm-ipc-darwin.c create mode 100644 compat/fsmonitor/fsm-ipc-win32.c create mode 100644 compat/fsmonitor/fsm-path-utils-darwin.c create mode 100644 compat/fsmonitor/fsm-path-utils-win32.c create mode 100644 fsmonitor-path-utils.h
base-commit: be1a02a17ede4082a86dfbfee0f54f345e8b43ac
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1326%2Fedecosta-mw%2Ffsmonitor_macos-v5
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1326/edecosta-mw/fsmonitor_macos-v5
Pull-Request: https://github.com/gitgitgadget/git/pull/1326
Range-diff vs v4:
-: ----------- > 1: 2f647b53e4d fsmonitor: refactor filesystem checks to common interface
2: 2cb026a6317 ! 2: 28d08bcf808 fsmonitor: generate unique Unix socket file name in the desired location
@@ Metadata
Author: Eric DeCosta <[email protected]>
## Commit message ##
- fsmonitor: generate unique Unix socket file name in the desired location
+ fsmonitor: relocate socket file if .git directory is remote
- Based on the values of fsmonitor.allowRemote and fsmonitor.socketDir
- locate the Unix domain socket file in the desired location (either
- the .git directory, $HOME, or fsmonitor.socketDir). If the location
- is other than the .git directory, generate a unique file name based
- on the SHA1 has of the path to the .git directory.
+ If the .git directory is on a remote file system, create the socket
+ file in 'fsmonitor.socketDir' if it is defined, else create it in $HOME.
Signed-off-by: Eric DeCosta <[email protected]>
- ## fsmonitor-ipc.c ##
-@@
- #include "cache.h"
--#include "fsmonitor.h"
--#include "simple-ipc.h"
- #include "fsmonitor-ipc.h"
-+#include "fsmonitor-settings.h"
- #include "run-command.h"
- #include "strbuf.h"
- #include "trace2.h"
-@@ fsmonitor-ipc.c: int fsmonitor_ipc__is_supported(void)
- return 1;
- }
+ ## Makefile ##
+@@ Makefile: ifdef FSMONITOR_DAEMON_BACKEND
+ COMPAT_CFLAGS += -DHAVE_FSMONITOR_DAEMON_BACKEND
+ COMPAT_OBJS += compat/fsmonitor/fsm-listen-$(FSMONITOR_DAEMON_BACKEND).o
+ COMPAT_OBJS += compat/fsmonitor/fsm-health-$(FSMONITOR_DAEMON_BACKEND).o
++ COMPAT_OBJS += compat/fsmonitor/fsm-ipc-$(FSMONITOR_DAEMON_BACKEND).o
+ endif
--GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc")
-+GIT_PATH_FUNC(fsmonitor_ipc__get_default_path, "fsmonitor--daemon.ipc")
+ ifdef FSMONITOR_OS_SETTINGS
+
+ ## compat/fsmonitor/fsm-ipc-darwin.c (new) ##
+@@
++#include "cache.h"
++#include "config.h"
++#include "strbuf.h"
++#include "fsmonitor.h"
++#include "fsmonitor-ipc.h"
++#include "fsmonitor-path-utils.h"
++
++static GIT_PATH_FUNC(fsmonitor_ipc__get_default_path, "fsmonitor--daemon.ipc")
+
+const char *fsmonitor_ipc__get_path(void)
+{
-+#ifdef WIN32
-+ return fsmonitor_ipc__get_default_path();
-+#else
-+ char *retval;
++ static const char *ipc_path;
+ SHA_CTX sha1ctx;
-+ const char *git_dir;
-+ const char *sock_dir;
++ char *sock_dir;
+ struct strbuf ipc_file = STRBUF_INIT;
+ unsigned char hash[SHA_DIGEST_LENGTH];
+
-+ if (fsm_settings__get_allow_remote(the_repository) < 1)
-+ return fsmonitor_ipc__get_default_path();
++ if (ipc_path)
++ return ipc_path;
+
-+ git_dir = get_git_dir();
-+ sock_dir = fsm_settings__get_socket_dir(the_repository);
++ ipc_path = fsmonitor_ipc__get_default_path();
++
++ /* By default the socket file is created in the .git directory */
++ if (fsmonitor__is_fs_remote(ipc_path) < 1)
++ return ipc_path;
+
+ SHA1_Init(&sha1ctx);
-+ SHA1_Update(&sha1ctx, git_dir, strlen(git_dir));
++ SHA1_Update(&sha1ctx, the_repository->worktree, strlen(the_repository->worktree));
+ SHA1_Final(hash, &sha1ctx);
+
++ repo_config_get_string(the_repository, "fsmonitor.socketdir", &sock_dir);
++
++ /* Create the socket file in either socketDir or $HOME */
+ if (sock_dir && *sock_dir)
+ strbuf_addf(&ipc_file, "%s/.git-fsmonitor-%s",
+ sock_dir, hash_to_hex(hash));
+ else
+ strbuf_addf(&ipc_file, "~/.git-fsmonitor-%s", hash_to_hex(hash));
-+ retval = interpolate_path(ipc_file.buf, 1);
-+ if (!retval)
++
++ ipc_path = interpolate_path(ipc_file.buf, 1);
++ if (!ipc_path)
+ die(_("Invalid path: %s"), ipc_file.buf);
++
+ strbuf_release(&ipc_file);
-+ return retval;
-+#endif
++ return ipc_path;
+}
+
+ ## compat/fsmonitor/fsm-ipc-win32.c (new) ##
+@@
++#include "cache.h"
++#include "fsmonitor-ipc.h"
++
++GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc")
+
+ ## contrib/buildsystems/CMakeLists.txt ##
+@@ contrib/buildsystems/CMakeLists.txt: if(SUPPORTS_SIMPLE_IPC)
+ add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND)
+ list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c)
+ list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-win32.c)
++ list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-win32.c)
+ list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-win32.c)
- enum ipc_active_state fsmonitor_ipc__get_state(void)
- {
+ add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
+@@ contrib/buildsystems/CMakeLists.txt: if(SUPPORTS_SIMPLE_IPC)
+ add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND)
+ list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c)
+ list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c)
++ list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-darwin.c)
+ list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c)
+
+ add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
- ## fsmonitor-ipc.h ##
-@@ fsmonitor-ipc.h: int fsmonitor_ipc__is_supported(void);
- */
- const char *fsmonitor_ipc__get_path(void);
+ ## fsmonitor-ipc.c ##
+@@ fsmonitor-ipc.c: int fsmonitor_ipc__is_supported(void)
+ return 1;
+ }
-+/*
-+ * Returns the pathname to the default IPC named pipe or Unix domain
-+ * socket.
-+ */
-+const char *fsmonitor_ipc__get_default_path(void);
-+
- /*
- * Try to determine whether there is a `git-fsmonitor--daemon` process
- * listening on the IPC pipe/socket.
+-GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc")
+-
+ enum ipc_active_state fsmonitor_ipc__get_state(void)
+ {
+ return ipc_get_active_state(fsmonitor_ipc__get_path());
1: 836a791e6b7 ! 3: ff54b6e0bb5 fsmonitor: add two new config options, allowRemote and socketDir
@@ Metadata
Author: Eric DeCosta <[email protected]>
## Commit message ##
- fsmonitor: add two new config options, allowRemote and socketDir
+ fsmonitor: avoid socket location check if using hook
- Introduce two new configuration options
-
- fsmonitor.allowRemote - setting this to true overrides fsmonitor's
- default behavior of erroring out when enountering network file
- systems. Additionly, when true, the Unix domain socket (UDS) file
- used for IPC is located in $HOME rather than in the .git directory.
-
- fsmonitor.socketDir - allows for the UDS file to be located
- anywhere the user chooses rather $HOME.
+ If monitoring is done via fsmonitor hook rather than IPC there is no
+ need to check if the location of the Unix Domain socket (UDS) file is
+ on a remote filesystem.
Signed-off-by: Eric DeCosta <[email protected]>
- ## fsmonitor-settings.c ##
-@@
- struct fsmonitor_settings {
- enum fsmonitor_mode mode;
+ ## compat/fsmonitor/fsm-settings-darwin.c ##
+@@ compat/fsmonitor/fsm-settings-darwin.c: static enum fsmonitor_reason check_uds_volume(struct repository *r)
+ return FSMONITOR_REASON_OK;
+ }
+
+-enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
++enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc)
+ {
enum fsmonitor_reason reason;
-+ int allow_remote;
- char *hook_path;
-+ char *sock_dir;
- };
- static enum fsmonitor_reason check_for_incompatible(struct repository *r)
-@@ fsmonitor-settings.c: static struct fsmonitor_settings *alloc_settings(void)
- CALLOC_ARRAY(s, 1);
- s->mode = FSMONITOR_MODE_DISABLED;
- s->reason = FSMONITOR_REASON_UNTESTED;
-+ s->allow_remote = -1;
+- reason = check_uds_volume(r);
+- if (reason != FSMONITOR_REASON_OK)
+- return reason;
++ if (ipc) {
++ reason = check_uds_volume(r);
++ if (reason != FSMONITOR_REASON_OK)
++ return reason;
++ }
- return s;
+ return FSMONITOR_REASON_OK;
}
-@@ fsmonitor-settings.c: static void lookup_fsmonitor_settings(struct repository *r)
- fsm_settings__set_disabled(r);
+
+ ## compat/fsmonitor/fsm-settings-win32.c ##
+@@ compat/fsmonitor/fsm-settings-win32.c: static enum fsmonitor_reason check_vfs4git(struct repository *r)
+ return FSMONITOR_REASON_OK;
}
-+int fsm_settings__get_allow_remote(struct repository *r)
-+{
-+ if (!r)
-+ r = the_repository;
-+ if (!r->settings.fsmonitor)
-+ lookup_fsmonitor_settings(r);
-+
-+ return r->settings.fsmonitor->allow_remote;
-+}
-+
-+const char *fsm_settings__get_socket_dir(struct repository *r)
-+{
-+ if (!r)
-+ r = the_repository;
-+ if (!r->settings.fsmonitor)
-+ lookup_fsmonitor_settings(r);
-+
-+ return r->settings.fsmonitor->sock_dir;
-+}
-+
- enum fsmonitor_mode fsm_settings__get_mode(struct repository *r)
+-enum fsmonitor_reason fsm_os__incompatible(struct repository *r)
++enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc)
{
- if (!r)
-@@ fsmonitor-settings.c: enum fsmonitor_mode fsm_settings__get_mode(struct repository *r)
- return r->settings.fsmonitor->mode;
+ enum fsmonitor_reason reason;
+
+
+ ## fsmonitor-settings.c ##
+@@ fsmonitor-settings.c: static enum fsmonitor_reason check_remote(struct repository *r)
}
+ #endif
-+
- const char *fsm_settings__get_hook_path(struct repository *r)
+-static enum fsmonitor_reason check_for_incompatible(struct repository *r)
++static enum fsmonitor_reason check_for_incompatible(struct repository *r, int ipc)
{
- if (!r)
+ if (!r->worktree) {
+ /*
+@@ fsmonitor-settings.c: static enum fsmonitor_reason check_for_incompatible(struct repository *r)
+ reason = check_remote(r);
+ if (reason != FSMONITOR_REASON_OK)
+ return reason;
+- reason = fsm_os__incompatible(r);
++ reason = fsm_os__incompatible(r, ipc);
+ if (reason != FSMONITOR_REASON_OK)
+ return reason;
+ }
@@ fsmonitor-settings.c: const char *fsm_settings__get_hook_path(struct repository *r)
- return r->settings.fsmonitor->hook_path;
- }
-+void fsm_settings__set_allow_remote(struct repository *r)
-+{
-+ int allow;
-+
-+ if (!r)
-+ r = the_repository;
-+ if (!r->settings.fsmonitor)
-+ r->settings.fsmonitor = alloc_settings();
-+ if (!repo_config_get_bool(r, "fsmonitor.allowremote", &allow))
-+ r->settings.fsmonitor->allow_remote = allow;
-+
-+ return;
-+}
-+
-+void fsm_settings__set_socket_dir(struct repository *r)
-+{
-+ const char *path;
-+
-+ if (!r)
-+ r = the_repository;
-+ if (!r->settings.fsmonitor)
-+ r->settings.fsmonitor = alloc_settings();
-+
-+ if (!repo_config_get_pathname(r, "fsmonitor.socketdir", &path)) {
-+ FREE_AND_NULL(r->settings.fsmonitor->sock_dir);
-+ r->settings.fsmonitor->sock_dir = strdup(path);
-+ }
-+
-+ return;
-+}
-+
void fsm_settings__set_ipc(struct repository *r)
{
- enum fsmonitor_reason reason = check_for_incompatible(r);
-+ enum fsmonitor_reason reason;
-+
-+ fsm_settings__set_allow_remote(r);
-+ fsm_settings__set_socket_dir(r);
-+ reason = check_for_incompatible(r);
++ enum fsmonitor_reason reason = check_for_incompatible(r, 1);
if (reason != FSMONITOR_REASON_OK) {
fsm_settings__set_incompatible(r, reason);
@@ fsmonitor-settings.c: void fsm_settings__set_ipc(struct repository *r)
void fsm_settings__set_hook(struct repository *r, const char *path)
{
- enum fsmonitor_reason reason = check_for_incompatible(r);
-+ enum fsmonitor_reason reason;
-+
-+ fsm_settings__set_allow_remote(r);
-+ fsm_settings__set_socket_dir(r);
-+ reason = check_for_incompatible(r);
++ enum fsmonitor_reason reason = check_for_incompatible(r, 0);
if (reason != FSMONITOR_REASON_OK) {
fsm_settings__set_incompatible(r, reason);
## fsmonitor-settings.h ##
-@@ fsmonitor-settings.h: enum fsmonitor_reason {
- FSMONITOR_REASON_NOSOCKETS, /* NTFS,FAT32 do not support Unix sockets */
- };
-
-+void fsm_settings__set_allow_remote(struct repository *r);
-+void fsm_settings__set_socket_dir(struct repository *r);
- void fsm_settings__set_ipc(struct repository *r);
- void fsm_settings__set_hook(struct repository *r, const char *path);
- void fsm_settings__set_disabled(struct repository *r);
- void fsm_settings__set_incompatible(struct repository *r,
- enum fsmonitor_reason reason);
-
-+int fsm_settings__get_allow_remote(struct repository *r);
-+const char *fsm_settings__get_socket_dir(struct repository *r);
- enum fsmonitor_mode fsm_settings__get_mode(struct repository *r);
- const char *fsm_settings__get_hook_path(struct repository *r);
+@@ fsmonitor-settings.h: struct fsmonitor_settings;
+ * fsm_os__* routines should considered private to fsm_settings__
+ * routines.
+ */
+-enum fsmonitor_reason fsm_os__incompatible(struct repository *r);
++enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc);
+ #endif /* HAVE_FSMONITOR_OS_SETTINGS */
+ #endif /* FSMONITOR_SETTINGS_H */
3: a3110f1e25a < -: ----------- fsmonitor: ensure filesystem and unix socket filesystem are compatible
4: 56cabf3be3b ! 4: b7d6cf44695 fsmonitor: normalize FSEvents event paths to the real path
@@ Metadata
Author: Eric DeCosta <[email protected]>
## Commit message ##
- fsmonitor: normalize FSEvents event paths to the real path
+ fsmonitor: deal with synthetic firmlinks on macOS
- Consider the following network working directory that is mounted under
- /System/Volumes/Data:
+ Starting with macOS 10.15 (Catalina), Apple introduced a new feature
+ called 'firmlinks' in order to separate the boot volume into two
+ volumes, one read-only and one writable but still present them to the
+ user as a single volume. Along with this change, Apple removed the
+ ability to create symlinks in the root directory and replaced them with
+ 'synthetic firmlinks'. See 'man synthetic.conf'
- /network/working/directory
+ When FSEevents reports the path of changed files, if the path invloves
+ a synthetic firmlink, the path is reported from the point of the
+ synthetic firmlink and not the real path. For example:
- The git working directory path is:
+ Real path:
+ /System/Volumes/Data/network/working/directory/foo.txt
- /System/Volumes/Data/network/working/directory
+ Synthetic firmlink:
+ /network -> /System/Volumes/Data/network
- The paths reported by FSEvents always start with /network. fsmonitor
- expects paths to be under the working directory; therefore it
- fails to match /network/... and ignores the change.
+ FSEvents path:
+ /network/working/directory/foo.txt
- Change things such that if fsmonitor.allowRemote is true that the
- paths reported via FSEevents are normalized to the real path.
+ This causes the FSEvents path to not match against the worktree
+ directory.
+
+ There are several ways in which synthetic firmlinks can be created:
+ they can be defined in /etc/synthetic.conf, the automounter can create
+ them, and there may be other means. Simply reading /etc/synthetic.conf
+ is insufficient. No matter what process creates synthetic firmlinks,
+ they all get created in the root directory.
+
+ Therefore, in order to deal with synthetic firmlinks, the root directory
+ is scanned and the first possible synthetic firmink that, when resolved,
+ is a prefix of the worktree is used to map FSEvents paths to worktree
+ paths.
Signed-off-by: Eric DeCosta <[email protected]>
+ ## builtin/fsmonitor--daemon.c ##
+@@
+ #include "parse-options.h"
+ #include "fsmonitor.h"
+ #include "fsmonitor-ipc.h"
++#include "fsmonitor-path-utils.h"
+ #include "compat/fsmonitor/fsm-health.h"
+ #include "compat/fsmonitor/fsm-listen.h"
+ #include "fsmonitor--daemon.h"
+@@ builtin/fsmonitor--daemon.c: static int fsmonitor_run_daemon(void)
+ strbuf_addstr(&state.path_worktree_watch, absolute_path(get_git_work_tree()));
+ state.nr_paths_watching = 1;
+
++ strbuf_init(&state.alias.alias, 0);
++ strbuf_init(&state.alias.points_to, 0);
++ if (fsmonitor__get_alias(state.path_worktree_watch.buf, &state.alias)) {
++ err = error(_("could not get worktree alias"));
++ goto done;
++ }
++
+ /*
+ * We create and delete cookie files somewhere inside the .git
+ * directory to help us keep sync with the file system. If
+
## compat/fsmonitor/fsm-listen-darwin.c ##
@@
#include "fsmonitor.h"
#include "fsm-listen.h"
#include "fsmonitor--daemon.h"
-+#include "fsmonitor-settings.h"
++#include "fsmonitor-path-utils.h"
struct fsm_listen_data
{
-@@ compat/fsmonitor/fsm-listen-darwin.c: static void my_add_path(struct fsmonitor_batch *batch, const char *path)
- free(composed);
- }
-
--
- static void fsevent_callback(ConstFSEventStreamRef streamRef,
- void *ctx,
- size_t num_of_events,
@@ compat/fsmonitor/fsm-listen-darwin.c: static void fsevent_callback(ConstFSEventStreamRef streamRef,
/*
* On Mac, we receive an array of absolute paths.
*/
- path_k = paths[k];
-+ if (fsm_settings__get_allow_remote(the_repository) > 0) {
-+ strbuf_reset(&tmp);
-+ strbuf_realpath_forgiving(&tmp, paths[k], 0);
-+ path_k = tmp.buf;
-+ } else
++ path_k = fsmonitor__resolve_alias(paths[k], &state->alias);
++ if (!path_k)
+ path_k = paths[k];
/*
* If you want to debug FSEvents, log them to GIT_TRACE_FSMONITOR.
@@ compat/fsmonitor/fsm-listen-darwin.c: static void fsevent_callback(ConstFSEventStreamRef streamRef,
-
fsmonitor_force_resync(state);
fsmonitor_batch__free_list(batch);
-+ batch = NULL;
string_list_clear(&cookie_list, 0);
++ batch = NULL;
/*
-@@ compat/fsmonitor/fsm-listen-darwin.c: static void fsevent_callback(ConstFSEventStreamRef streamRef,
+ * We assume that any events that we received
+
+ ## compat/fsmonitor/fsm-path-utils-darwin.c ##
+@@
+ #include "fsmonitor.h"
+ #include "fsmonitor-path-utils.h"
++#include <dirent.h>
++#include <errno.h>
++#include <fcntl.h>
+ #include <sys/param.h>
+ #include <sys/mount.h>
+
+@@ compat/fsmonitor/fsm-path-utils-darwin.c: int fsmonitor__is_fs_remote(const char *path)
+ return -1;
+ return fs.is_remote;
+ }
++
++/*
++ * Scan the root directory for synthetic firmlinks that when resolved
++ * are a prefix of the path, stopping at the first one found.
++ *
++ * Some information about firmlinks and synthetic firmlinks:
++ * https://eclecticlight.co/2020/01/23/catalina-boot-volumes/
++ *
++ * macOS no longer allows symlinks in the root directory; any link found
++ * there is therefore a synthetic firmlink.
++ *
++ * If this function gets called often, will want to cache all the firmlink
++ * information, but for now there is only one caller of this function.
++ *
++ * If there is more than one alias for the path, that is another
++ * matter altogteher.
++ */
++int fsmonitor__get_alias(const char *path, struct alias_info *info)
++{
++ DIR * dir;
++ int read;
++ int retval;
++ struct dirent *de;
++ struct strbuf alias;
++ struct strbuf points_to;
++
++ retval = 0;
++ dir = opendir("/");
++ if (!dir)
++ return -1;
++
++ strbuf_init(&alias, 256);
++ strbuf_init(&points_to, MAXPATHLEN);
++
++ while ((de = readdir(dir)) != NULL) {
++ strbuf_reset(&alias);
++ strbuf_addch(&alias, '/');
++ strbuf_add(&alias, de->d_name, strlen(de->d_name));
++
++ read = readlink(alias.buf, points_to.buf, MAXPATHLEN);
++ if (read > 0) {
++ strbuf_setlen(&points_to, read);
++ if ((strncmp(points_to.buf, path, points_to.len) == 0)
++ && path[points_to.len] == '/') {
++ strbuf_addbuf(&info->alias, &alias);
++ strbuf_addbuf(&info->points_to, &points_to);
++ trace_printf_key(&trace_fsmonitor,
++ "Found alias for '%s' : '%s' -> '%s'",
++ path, info->alias.buf, info->points_to.buf);
++ retval = 0;
++ goto done;
++ }
++ } else if (errno != EINVAL) { /* Something other than not a link */
++ trace_printf_key(&trace_fsmonitor, "Error %s", strerror(errno));
++ retval = -1;
++ goto done;
++ }
++ }
++
++ done:
++ closedir(dir);
++ strbuf_release(&alias);
++ strbuf_release(&points_to);
++ return retval;
++}
++
++char *fsmonitor__resolve_alias(const char *path,
++ const struct alias_info *info)
++{
++ int len = info->alias.len;
++
++ if (!len)
++ return NULL;
++
++ if ((strncmp(info->alias.buf, path, len) == 0)
++ && path[len] == '/') {
++ struct strbuf tmp;
++ const char *remainder = path + len;
++ int rem_len = strlen(remainder);
++
++ strbuf_init(&tmp, info->points_to.len + rem_len);
++ strbuf_addbuf(&tmp, &info->points_to);
++ strbuf_add(&tmp, remainder, rem_len);
++ return strbuf_detach(&tmp, NULL);
++ }
++
++ return NULL;
++}
+
+ ## compat/fsmonitor/fsm-path-utils-win32.c ##
+@@ compat/fsmonitor/fsm-path-utils-win32.c: int fsmonitor__is_fs_remote(const char *path)
+ return -1;
+ return fs.is_remote;
+ }
++
++/*
++ * No-op for now.
++ */
++int fsmonitor__get_alias(const char *path, struct alias_info *info)
++{
++ return 0;
++}
++
++/*
++ * No-op for now.
++ */
++char *fsmonitor__resolve_alias(const char *path,
++ const struct alias_info *info)
++{
++ return NULL;
++}
+
+ ## fsmonitor--daemon.h ##
+@@
+ #include "run-command.h"
+ #include "simple-ipc.h"
+ #include "thread-utils.h"
++#include "fsmonitor-path-utils.h"
+
+ struct fsmonitor_batch;
+ struct fsmonitor_token_data;
+@@ fsmonitor--daemon.h: struct fsmonitor_daemon_state {
+
+ struct strbuf path_worktree_watch;
+ struct strbuf path_gitdir_watch;
++ struct alias_info alias;
+ int nr_paths_watching;
+
+ struct fsmonitor_token_data *current_token_data;
+@@ fsmonitor--daemon.h: struct fsmonitor_daemon_state {
+
+ struct ipc_server_data *ipc_server_data;
+ struct strbuf path_ipc;
++
+ };
+
+ /*
+@@ fsmonitor--daemon.h: void fsmonitor_publish(struct fsmonitor_daemon_state *state,
+ */
+ void fsmonitor_force_resync(struct fsmonitor_daemon_state *state);
+
++char *fsmonitor_resolve_alias(const char *path,
++ struct alias_info *alias);
++
+ #endif /* HAVE_FSMONITOR_DAEMON_BACKEND */
+ #endif /* FSMONITOR_DAEMON_H */
+
+ ## fsmonitor-path-utils.h ##
+@@
+ #ifndef FSM_PATH_UTILS_H
+ #define FSM_PATH_UTILS_H
- case IS_WORKDIR_PATH:
- /* try to queue normal pathnames */
--
- if (trace_pass_fl(&trace_fsmonitor))
- log_flags_set(path_k, event_flags[k]);
++#include "strbuf.h"
++
++struct alias_info
++{
++ struct strbuf alias;
++ struct strbuf points_to;
++};
++
+ struct fs_info {
+ int is_remote;
+ char *typename;
+@@ fsmonitor-path-utils.h: int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info);
+ */
+ int fsmonitor__is_fs_remote(const char *path);
++/*
++ * Get the alias in given path, if any.
++ *
++ * Sets alias to the first alias that matches any part of the path.
++ *
++ * Returns -1 on error, 0 otherwise.
++ */
++int fsmonitor__get_alias(const char *path, struct alias_info *info);
++
++/*
++ * Resolve the path against the given alias.
++ *
++ * Returns the resolved path if there is one, NULL otherwise.
++ */
++char *fsmonitor__resolve_alias(const char *path,
++ const struct alias_info *info);
++
++
+ #endifSignature · 1 line
gitgitgadget
PATCH v5 1/4 fsmonitor: refactor filesystem checks to common interface
From: Eric DeCosta <[email protected]>
Provide a common interface for getting basic filesystem information including filesystem type and whether the filesystem is remote.
Refactor existing code for getting basic filesystem info and detecting remote file systems to the new interface.
Refactor filesystem checks to leverage new interface. For macOS, error-out if the Unix Domain socket (UDS) file is on a remote filesystem.
--- Makefile | 1 + compat/fsmonitor/fsm-path-utils-darwin.c | 40 +++++++++ compat/fsmonitor/fsm-path-utils-win32.c | 89 +++++++++++++++++++ compat/fsmonitor/fsm-settings-darwin.c | 62 ++++---------- compat/fsmonitor/fsm-settings-win32.c | 104 +---------------------- contrib/buildsystems/CMakeLists.txt | 2 + fsmonitor-path-utils.h | 23 +++++ fsmonitor-settings.c | 50 +++++++++++ 8 files changed, 224 insertions(+), 147 deletions(-) create mode 100644 compat/fsmonitor/fsm-path-utils-darwin.c create mode 100644 compat/fsmonitor/fsm-path-utils-win32.c create mode 100644 fsmonitor-path-utils.h
Makefile +1 -0
@@ -2042,6 +2042,7 @@ endif ifdef FSMONITOR_OS_SETTINGS COMPAT_CFLAGS += -DHAVE_FSMONITOR_OS_SETTINGS COMPAT_OBJS += compat/fsmonitor/fsm-settings-$(FSMONITOR_OS_SETTINGS).o + COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_OS_SETTINGS).o endif ifeq ($(TCLTK_PATH),)
compat/fsmonitor/fsm-path-utils-darwin.c new file +40 -0
@@ -0,0 +1,40 @@ +#include "fsmonitor.h" +#include "fsmonitor-path-utils.h" +#include <sys/param.h> +#include <sys/mount.h> + +int fsmonitor__get_fs_info (const char *path,struct fs_info *fs_info) +{ +struct statfs fs; +if (statfs (path, &fs) ==-1 ) { +int saved_errno = errno; +trace_printf_key (&trace_fsmonitor,"statfs('%s') failed: %s" , + path,strerror (saved_errno)); + errno = saved_errno; +return -1 ; + } + +trace_printf_key (&trace_fsmonitor, +"statfs('%s') [type 0x%08x][flags 0x%08x] '%s'" , + path, fs.f_type, fs.f_flags, fs.f_fstypename); + +if (!(fs.f_flags &MNT_LOCAL )) + fs_info->is_remote =1 ; +else + fs_info->is_remote =0 ; + + fs_info->typename = fs.f_fstypename; + +trace_printf_key (&trace_fsmonitor, +"'%s' is_remote: %d" , + path, fs_info->is_remote); +return 0 ; +} + +int fsmonitor__is_fs_remote (const char *path) +{ +struct fs_info fs; +if (fsmonitor__get_fs_info (path, &fs)) +return -1 ; +return fs.is_remote; +}
compat/fsmonitor/fsm-path-utils-win32.c new file +89 -0
89 lines not on this page · raw message
compat/fsmonitor/fsm-settings-darwin.c +17 -45
96 lines not on this page · raw message
compat/fsmonitor/fsm-settings-win32.c +2 -102
122 lines not on this page · raw message
contrib/buildsystems/CMakeLists.txt +2 -0
@@ -308,6 +308,7 @@ if(SUPPORTS_SIMPLE_IPC) add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-win32.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-win32.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-win32.c)
@@ -315,6 +316,7 @@ if(SUPPORTS_SIMPLE_IPC) add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-darwin.c)
fsmonitor-path-utils.h new file +23 -0
@@ -0,0 +1,23 @@ +#ifndef FSM_PATH_UTILS_H +#define FSM_PATH_UTILS_H + +struct fs_info { +int is_remote; +char *typename; +}; + +/* +* Get some basic filesystem informtion for the given path +* +* Returns -1 on error, zero otherwise. +*/ +int fsmonitor__get_fs_info (const char *path,struct fs_info *fs_info); + +/* +* Determines if the filesystem that path resides on is remote. +* +* Returns -1 on error, 0 if not remote, 1 if remote. +*/ +int fsmonitor__is_fs_remote (const char *path); + +#endif
fsmonitor-settings.c +50 -0
68 lines not on this page · raw message
Signature · 1 line
gitgitgadget
PATCH v5 2/4 fsmonitor: relocate socket file if .git directory is remote
In reply to Eric DeCosta via GitGitGadgetFrom: Eric DeCosta <[email protected]>
If the .git directory is on a remote file system, create the socket file in 'fsmonitor.socketDir' if it is defined, else create it in $HOME.
--- Makefile | 1 + compat/fsmonitor/fsm-ipc-darwin.c | 46 +++++++++++++++++++++++++++++ compat/fsmonitor/fsm-ipc-win32.c | 4 +++ contrib/buildsystems/CMakeLists.txt | 2 ++ fsmonitor-ipc.c | 2 -- 5 files changed, 53 insertions(+), 2 deletions(-) create mode 100644 compat/fsmonitor/fsm-ipc-darwin.c create mode 100644 compat/fsmonitor/fsm-ipc-win32.c
Makefile +1 -0
@@ -2037,6 +2037,7 @@ ifdef FSMONITOR_DAEMON_BACKEND COMPAT_CFLAGS += -DHAVE_FSMONITOR_DAEMON_BACKEND COMPAT_OBJS += compat/fsmonitor/fsm-listen-$(FSMONITOR_DAEMON_BACKEND).o COMPAT_OBJS += compat/fsmonitor/fsm-health-$(FSMONITOR_DAEMON_BACKEND).o + COMPAT_OBJS += compat/fsmonitor/fsm-ipc-$(FSMONITOR_DAEMON_BACKEND).o endif ifdef FSMONITOR_OS_SETTINGS
compat/fsmonitor/fsm-ipc-darwin.c new file +46 -0
46 lines not on this page · raw message
compat/fsmonitor/fsm-ipc-win32.c new file +4 -0
@@ -0,0 +1,4 @@ +#include "cache.h" +#include "fsmonitor-ipc.h" + +GIT_PATH_FUNC (fsmonitor_ipc__get_path,"fsmonitor--daemon.ipc" )
contrib/buildsystems/CMakeLists.txt +2 -0
@@ -308,6 +308,7 @@ if(SUPPORTS_SIMPLE_IPC) add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-win32.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-win32.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-win32.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
@@ -316,6 +317,7 @@ if(SUPPORTS_SIMPLE_IPC) add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-darwin.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
fsmonitor-ipc.c +0 -2
@@ -47,8 +47,6 @@ int fsmonitor_ipc__is_supported(void)return 1 ; } -GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc") -enum ipc_active_state fsmonitor_ipc__get_state (void ) {return ipc_get_active_state (fsmonitor_ipc__get_path ());
Signature · 1 line
gitgitgadget
PATCH v5 3/4 fsmonitor: avoid socket location check if using hook
In reply to Eric DeCosta via GitGitGadgetFrom: Eric DeCosta <[email protected]>
If monitoring is done via fsmonitor hook rather than IPC there is no need to check if the location of the Unix Domain socket (UDS) file is on a remote filesystem.
--- compat/fsmonitor/fsm-settings-darwin.c | 10 ++++++---- compat/fsmonitor/fsm-settings-win32.c | 2 +- fsmonitor-settings.c | 8 ++++---- fsmonitor-settings.h | 2 +- 4 files changed, 12 insertions(+), 10 deletions(-)
compat/fsmonitor/fsm-settings-darwin.c +6 -4
@@ -49,13 +49,15 @@ static enum fsmonitor_reason check_uds_volume(struct repository *r)return FSMONITOR_REASON_OK ; } -enum fsmonitor_reason fsm_os__incompatible (struct repository *r) +enum fsmonitor_reason fsm_os__incompatible (struct repository *r,int ipc) {enum fsmonitor_reason reason; - reason =check_uds_volume (r); -if (reason !=FSMONITOR_REASON_OK ) -return reason; +if (ipc) { + reason =check_uds_volume (r); +if (reason !=FSMONITOR_REASON_OK ) +return reason; + }return FSMONITOR_REASON_OK ; }
compat/fsmonitor/fsm-settings-win32.c +1 -1
@@ -25,7 +25,7 @@ static enum fsmonitor_reason check_vfs4git(struct repository *r)return FSMONITOR_REASON_OK ; } -enum fsmonitor_reason fsm_os__incompatible (struct repository *r) +enum fsmonitor_reason fsm_os__incompatible (struct repository *r,int ipc) {enum fsmonitor_reason reason;
fsmonitor-settings.c +4 -4
@@ -60,7 +60,7 @@ static enum fsmonitor_reason check_remote(struct repository *r) }#endif -static enum fsmonitor_reason check_for_incompatible (struct repository *r) +static enum fsmonitor_reason check_for_incompatible (struct repository *r,int ipc) {if (!r->worktree) {/*
@@ -77,7 +77,7 @@ static enum fsmonitor_reason check_for_incompatible(struct repository *r) reason =check_remote (r);if (reason !=FSMONITOR_REASON_OK )return reason; - reason =fsm_os__incompatible (r); + reason =fsm_os__incompatible (r, ipc);if (reason !=FSMONITOR_REASON_OK )return reason; }
@@ -162,7 +162,7 @@ const char *fsm_settings__get_hook_path(struct repository *r)void fsm_settings__set_ipc (struct repository *r) { -enum fsmonitor_reason reason =check_for_incompatible (r); +enum fsmonitor_reason reason =check_for_incompatible (r,1 );if (reason !=FSMONITOR_REASON_OK ) {fsm_settings__set_incompatible (r, reason);
@@ -185,7 +185,7 @@ void fsm_settings__set_ipc(struct repository *r)void fsm_settings__set_hook (struct repository *r,const char *path) { -enum fsmonitor_reason reason =check_for_incompatible (r); +enum fsmonitor_reason reason =check_for_incompatible (r,0 );if (reason !=FSMONITOR_REASON_OK ) {fsm_settings__set_incompatible (r, reason);
fsmonitor-settings.h +1 -1
@@ -48,7 +48,7 @@ struct fsmonitor_settings;* fsm_os__* routines should considered private to fsm_settings__ * routines. */ -enum fsmonitor_reason fsm_os__incompatible (struct repository *r); +enum fsmonitor_reason fsm_os__incompatible (struct repository *r,int ipc);#endif /* HAVE_FSMONITOR_OS_SETTINGS */ #endif /* FSMONITOR_SETTINGS_H */
Signature · 1 line
gitgitgadget
PATCH v5 4/4 fsmonitor: deal with synthetic firmlinks on macOS
In reply to Eric DeCosta via GitGitGadgetFrom: Eric DeCosta <[email protected]>
Starting with macOS 10.15 (Catalina), Apple introduced a new feature called 'firmlinks' in order to separate the boot volume into two volumes, one read-only and one writable but still present them to the user as a single volume. Along with this change, Apple removed the ability to create symlinks in the root directory and replaced them with 'synthetic firmlinks'. See 'man synthetic.conf'
When FSEevents reports the path of changed files, if the path invloves a synthetic firmlink, the path is reported from the point of the synthetic firmlink and not the real path. For example:
Real path:
/System/Volumes/Data/network/working/directory/foo.txt
Synthetic firmlink:
/network -> /System/Volumes/Data/network
FSEvents path:
/network/working/directory/foo.txt
This causes the FSEvents path to not match against the worktree directory.
There are several ways in which synthetic firmlinks can be created:
they can be defined in /etc/synthetic.conf, the automounter can create them, and there may be other means. Simply reading /etc/synthetic.conf is insufficient. No matter what process creates synthetic firmlinks, they all get created in the root directory.
Therefore, in order to deal with synthetic firmlinks, the root directory is scanned and the first possible synthetic firmink that, when resolved, is a prefix of the worktree is used to map FSEvents paths to worktree paths.
--- builtin/fsmonitor--daemon.c | 8 +++ compat/fsmonitor/fsm-listen-darwin.c | 6 +- compat/fsmonitor/fsm-path-utils-darwin.c | 91 ++++++++++++++++++++++++ compat/fsmonitor/fsm-path-utils-win32.c | 17 +++++ fsmonitor--daemon.h | 6 ++ fsmonitor-path-utils.h | 26 +++++++ 6 files changed, 153 insertions(+), 1 deletion(-)
builtin/fsmonitor--daemon.c +8 -0
@@ -3,6 +3,7 @@#include "parse-options.h" #include "fsmonitor.h" #include "fsmonitor-ipc.h" +#include "fsmonitor-path-utils.h" #include "compat/fsmonitor/fsm-health.h" #include "compat/fsmonitor/fsm-listen.h" #include "fsmonitor--daemon.h"
@@ -1282,6 +1283,13 @@ static int fsmonitor_run_daemon(void)strbuf_addstr (&state.path_worktree_watch,absolute_path (get_git_work_tree ())); state.nr_paths_watching =1 ; +strbuf_init (&state.alias.alias,0 ); +strbuf_init (&state.alias.points_to,0 ); +if (fsmonitor__get_alias (state.path_worktree_watch.buf, &state.alias)) { + err =error (_ ("could not get worktree alias" )); + goto done; + } +/* * We create and delete cookie files somewhere inside the .git * directory to help us keep sync with the file system. If
compat/fsmonitor/fsm-listen-darwin.c +5 -1
@@ -26,6 +26,7 @@#include "fsmonitor.h" #include "fsm-listen.h" #include "fsmonitor--daemon.h" +#include "fsmonitor-path-utils.h" struct fsm_listen_data {
@@ -209,7 +210,9 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,/* * On Mac, we receive an array of absolute paths. */ - path_k = paths[k]; + path_k =fsmonitor__resolve_alias (paths[k], &state->alias); +if (!path_k) + path_k = paths[k];/* * If you want to debug FSEvents, log them to GIT_TRACE_FSMONITOR.
@@ -238,6 +241,7 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,fsmonitor_force_resync (state);fsmonitor_batch__free_list (batch);string_list_clear (&cookie_list,0 ); + batch =NULL ;/* * We assume that any events that we received
compat/fsmonitor/fsm-path-utils-darwin.c +91 -0
99 lines not on this page · raw message
compat/fsmonitor/fsm-path-utils-win32.c +17 -0
@@ -87,3 +87,20 @@ int fsmonitor__is_fs_remote(const char *path)return -1 ;return fs.is_remote; } + +/* +* No-op for now. +*/ +int fsmonitor__get_alias (const char *path,struct alias_info *info) +{ +return 0 ; +} + +/* +* No-op for now. +*/ +char *fsmonitor__resolve_alias (const char *path, +const struct alias_info *info) +{ +return NULL ; +}
fsmonitor--daemon.h +6 -0
@@ -8,6 +8,7 @@#include "run-command.h" #include "simple-ipc.h" #include "thread-utils.h" +#include "fsmonitor-path-utils.h" struct fsmonitor_batch ;struct fsmonitor_token_data ;
@@ -43,6 +44,7 @@ struct fsmonitor_daemon_state {struct strbuf path_worktree_watch;struct strbuf path_gitdir_watch; +struct alias_info alias;int nr_paths_watching;struct fsmonitor_token_data *current_token_data;
@@ -59,6 +61,7 @@ struct fsmonitor_daemon_state {struct ipc_server_data *ipc_server_data;struct strbuf path_ipc; + };/*
@@ -167,5 +170,8 @@ void fsmonitor_publish(struct fsmonitor_daemon_state *state,*/ void fsmonitor_force_resync (struct fsmonitor_daemon_state *state); +char *fsmonitor_resolve_alias (const char *path, +struct alias_info *alias); +#endif /* HAVE_FSMONITOR_DAEMON_BACKEND */ #endif /* FSMONITOR_DAEMON_H */
fsmonitor-path-utils.h +26 -0
@@ -1,6 +1,14 @@ #ifndef FSM_PATH_UTILS_H #define FSM_PATH_UTILS_H +#include "strbuf.h" + +struct alias_info +{ + struct strbuf alias; + struct strbuf points_to; +}; + struct fs_info { int is_remote; char *typename;
@@ -20,4 +28,22 @@ int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info);*/ int fsmonitor__is_fs_remote (const char *path); +/* +* Get the alias in given path, if any. +* +* Sets alias to the first alias that matches any part of the path. +* +* Returns -1 on error, 0 otherwise. +*/ +int fsmonitor__get_alias (const char *path,struct alias_info *info); + +/* +* Resolve the path against the given alias. +* +* Returns the resolved path if there is one, NULL otherwise. +*/ +char *fsmonitor__resolve_alias (const char *path, +const struct alias_info *info); + +#endif
Signature · 1 line
gitgitgadget
Re: [PATCH v5 4/4] fsmonitor: deal with synthetic firmlinks on macOS
Quoted 10 lines · Eric DeCosta via GitGitGadget
Starting with macOS 10.15 (Catalina), Apple introduced a new feature called 'firmlinks' in order to separate the boot volume into two volumes, one read-only and one writable but still present them to the user as a single volume. Along with this change, Apple removed the ability to create symlinks in the root directory and replaced them with 'synthetic firmlinks'. See 'man synthetic.conf'
When FSEevents reports the path of changed files, if the path invloves a synthetic firmlink, the path is reported from the point of the synthetic firmlink and not the real path. For example:
s/invloves/involves/
Quoted 33 lines
Real path:
/System/Volumes/Data/network/working/directory/foo.txtSynthetic firmlink:
/network -> /System/Volumes/Data/networkFSEvents path:
/network/working/directory/foo.txtThis causes the FSEvents path to not match against the worktree directory.
There are several ways in which synthetic firmlinks can be created:
they can be defined in/etc/synthetic.conf, the automounter can create them, and there may be other means. Simply reading/etc/synthetic.confis insufficient. No matter what process creates synthetic firmlinks, they all get created in the root directory.Therefore, in order to deal with synthetic firmlinks, the root directory is scanned and the first possible synthetic firmink that, when resolved, is a prefix of the worktree is used to map FSEvents paths to worktree paths.
Signed-off-by Eric DeCosta <[email protected]>---
compat/fsmonitor/fsm-path-utils-darwin.c +6
-0@@ -38,3 +41,91 @@ int fsmonitor__is_fs_remote(const char *path) +/* +* Scan the root directory for synthetic firmlinks that when resolved +* are a prefix of the path, stopping at the first one found. +* +* Some information about firmlinks and synthetic firmlinks: +* https://eclecticlight.co/2020/01/23/catalina-boot-volumes/
compat/fsmonitor/fsm-path-utils-darwin.c int fsmonitor__is_fs_remote(const char *path) · line 56
Quoted 10 lines
compat/fsmonitor/fsm-path-utils-darwin.c +10
-0+* +* macOS no longer allows symlinks in the root directory; any link found +* there is therefore a synthetic firmlink. +* +* If this function gets called often, will want to cache all the firmlink +* information, but for now there is only one caller of this function. +* +* If there is more than one alias for the path, that is another +* matter altogteher. +*/
s/altogteher/altogether/
Quoted 5 lines
fsmonitor-path-utils.h +3
-0@@ -1,6 +1,14 @@ +struct alias_info +{ +struct strbuf alias;
fsmonitor-path-utils.h line 13
Quoted 10 lines
fsmonitor-path-utils.h +10
-0+struct strbuf points_to; +}; +/* +* Get the alias in given path, if any. +* +* Sets alias to the first alias that matches any part of the path. +* +* Returns -1 on error, 0 otherwise. +*/ +int fsmonitor__get_alias (const char *path,struct alias_info *info);
I suppose it's somewhat clear here that the caller is responsible for releasing the strbufs in alias_info (though it's not clear why they need to be strbufs in the first place)...
fsmonitor-path-utils.h int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info); · line 46
Quoted 7 lines
fsmonitor-path-utils.h +7
-0+/* +* Resolve the path against the given alias. +* +* Returns the resolved path if there is one, NULL otherwise. +*/ +char *fsmonitor__resolve_alias (const char *path, +const struct alias_info *info);
But what about the return value from this function? It's char* which suggests a possible ownership transfer(?). Is the caller responsible for freeing it? If so, perhaps the documentation could state that.
Quoted 21 lines · Eric DeCosta via GitGitGadget
Follow-on to the work done to allow Windows to work against network-mounted repos for macOS.
Have macOS take advantage of the same configuration option, 'fsmonitor.allowRemote' that was introduced for Windows. Setting this option to true will override the default behavior (erroring-out) when a network-mounted repo is detected by fsmonitor.
The added wrinkle being that the Unix domain socket (UDS) file used for IPC cannot be created in a network location; instead $HOME is used if the default location is on the network. The user may, optionally, set the 'fsmonitor.socketDir' configuration option to a valid, local directory if $HOME itself is on the network or is simply not the desired location for the UDS file.
An additional issue is that for mount points in the root directory, FSEvents does not report a path that matches the worktree directory due to the introduction of 'synthetic firmlinks'. fsmonitor must map the FSEvents paths to the worktree directory by interrogating the root filesystem for synthetic firmlinks and using that information to translate the path.
Quoted 5 lines
Eric DeCosta (4):
fsmonitor: refactor filesystem checks to common interface
fsmonitor: relocate socket file if .git directory is remote
fsmonitor: avoid socket location check if using hook
fsmonitor: deal with synthetic firmlinks on macOS
It looks like this one is organized quite differently from the last iteration <[email protected]>
Unless the changes since the last iteration is so obvious [*] that range-diff will explain all of it, it would help the topic to make it easier for reviewers what has changed relative to the previous one in your own words here in the cover letter.
[*} Here, "obvious" does not mean "diff is small and can be read in 10 minutes". It is "diff shows the reason why these changes are made, so there is no need for additional explanation", e.g. typofixes, renaming of variables, making a helper function that was added as extern in the previous iteration to static, etc.
Thanks.
Quoted 724 lines
Makefile | 2 + builtin/fsmonitor--daemon.c | 8 ++ compat/fsmonitor/fsm-ipc-darwin.c | 46 ++++++++ compat/fsmonitor/fsm-ipc-win32.c | 4 + compat/fsmonitor/fsm-listen-darwin.c | 6 +- compat/fsmonitor/fsm-path-utils-darwin.c | 131 +++++++++++++++++++++++ compat/fsmonitor/fsm-path-utils-win32.c | 106 ++++++++++++++++++ compat/fsmonitor/fsm-settings-darwin.c | 70 ++++-------- compat/fsmonitor/fsm-settings-win32.c | 106 +----------------- contrib/buildsystems/CMakeLists.txt | 4 + fsmonitor--daemon.h | 6 ++ fsmonitor-ipc.c | 2 - fsmonitor-path-utils.h | 49 +++++++++ fsmonitor-settings.c | 58 +++++++++- fsmonitor-settings.h | 2 +- 15 files changed, 441 insertions(+), 159 deletions(-) create mode 100644 compat/fsmonitor/fsm-ipc-darwin.c create mode 100644 compat/fsmonitor/fsm-ipc-win32.c create mode 100644 compat/fsmonitor/fsm-path-utils-darwin.c create mode 100644 compat/fsmonitor/fsm-path-utils-win32.c create mode 100644 fsmonitor-path-utils.hbase-commit:
be1a02a17ede4082a86dfbfee0f54f345e8b43ac
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1326%2Fedecosta-mw%2Ffsmonitor_macos-v5
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1326/edecosta-mw/fsmonitor_macos-v5
Pull-Request: https://github.com/gitgitgadget/git/pull/1326Range-diff vs v4:
-: ----------- > 1: 2f647b53e4d fsmonitor: refactor filesystem checks to common interface 2: 2cb026a6317 ! 2: 28d08bcf808 fsmonitor: generate unique Unix socket file name in the desired location @@ Metadata Author: Eric DeCosta <[email protected]> ## Commit message ## - fsmonitor: generate unique Unix socket file name in the desired location + fsmonitor: relocate socket file if .git directory is remote - Based on the values of fsmonitor.allowRemote and fsmonitor.socketDir - locate the Unix domain socket file in the desired location (either - the .git directory, $HOME, or fsmonitor.socketDir). If the location - is other than the .git directory, generate a unique file name based - on the SHA1 has of the path to the .git directory. + If the .git directory is on a remote file system, create the socket + file in 'fsmonitor.socketDir' if it is defined, else create it in $HOME. Signed-off-by: Eric DeCosta <[email protected]> - ## fsmonitor-ipc.c ## -@@ - #include "cache.h" --#include "fsmonitor.h" --#include "simple-ipc.h" - #include "fsmonitor-ipc.h" -+#include "fsmonitor-settings.h" - #include "run-command.h" - #include "strbuf.h" - #include "trace2.h" -@@ fsmonitor-ipc.c: int fsmonitor_ipc__is_supported(void) - return 1; - } + ## Makefile ## +@@ Makefile: ifdef FSMONITOR_DAEMON_BACKEND + COMPAT_CFLAGS += -DHAVE_FSMONITOR_DAEMON_BACKEND + COMPAT_OBJS += compat/fsmonitor/fsm-listen-$(FSMONITOR_DAEMON_BACKEND).o + COMPAT_OBJS += compat/fsmonitor/fsm-health-$(FSMONITOR_DAEMON_BACKEND).o ++ COMPAT_OBJS += compat/fsmonitor/fsm-ipc-$(FSMONITOR_DAEMON_BACKEND).o + endif --GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc") -+GIT_PATH_FUNC(fsmonitor_ipc__get_default_path, "fsmonitor--daemon.ipc") + ifdef FSMONITOR_OS_SETTINGS + + ## compat/fsmonitor/fsm-ipc-darwin.c (new) ## +@@ ++#include "cache.h" ++#include "config.h" ++#include "strbuf.h" ++#include "fsmonitor.h" ++#include "fsmonitor-ipc.h" ++#include "fsmonitor-path-utils.h" ++ ++static GIT_PATH_FUNC(fsmonitor_ipc__get_default_path, "fsmonitor--daemon.ipc") + +const char *fsmonitor_ipc__get_path(void) +{ -+#ifdef WIN32 -+ return fsmonitor_ipc__get_default_path(); -+#else -+ char *retval; ++ static const char *ipc_path; + SHA_CTX sha1ctx; -+ const char *git_dir; -+ const char *sock_dir; ++ char *sock_dir; + struct strbuf ipc_file = STRBUF_INIT; + unsigned char hash[SHA_DIGEST_LENGTH]; + -+ if (fsm_settings__get_allow_remote(the_repository) < 1) -+ return fsmonitor_ipc__get_default_path(); ++ if (ipc_path) ++ return ipc_path; + -+ git_dir = get_git_dir(); -+ sock_dir = fsm_settings__get_socket_dir(the_repository); ++ ipc_path = fsmonitor_ipc__get_default_path(); ++ ++ /* By default the socket file is created in the .git directory */ ++ if (fsmonitor__is_fs_remote(ipc_path) < 1) ++ return ipc_path; + + SHA1_Init(&sha1ctx); -+ SHA1_Update(&sha1ctx, git_dir, strlen(git_dir)); ++ SHA1_Update(&sha1ctx, the_repository->worktree, strlen(the_repository->worktree)); + SHA1_Final(hash, &sha1ctx); + ++ repo_config_get_string(the_repository, "fsmonitor.socketdir", &sock_dir); ++ ++ /* Create the socket file in either socketDir or $HOME */ + if (sock_dir && *sock_dir) + strbuf_addf(&ipc_file, "%s/.git-fsmonitor-%s", + sock_dir, hash_to_hex(hash)); + else + strbuf_addf(&ipc_file, "~/.git-fsmonitor-%s", hash_to_hex(hash)); -+ retval = interpolate_path(ipc_file.buf, 1); -+ if (!retval) ++ ++ ipc_path = interpolate_path(ipc_file.buf, 1); ++ if (!ipc_path) + die(_("Invalid path: %s"), ipc_file.buf); ++ + strbuf_release(&ipc_file); -+ return retval; -+#endif ++ return ipc_path; +} + + ## compat/fsmonitor/fsm-ipc-win32.c (new) ## +@@ ++#include "cache.h" ++#include "fsmonitor-ipc.h" ++ ++GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc") + + ## contrib/buildsystems/CMakeLists.txt ## +@@ contrib/buildsystems/CMakeLists.txt: if(SUPPORTS_SIMPLE_IPC) + add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-win32.c) ++ list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-win32.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-win32.c) - enum ipc_active_state fsmonitor_ipc__get_state(void) - { + add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) +@@ contrib/buildsystems/CMakeLists.txt: if(SUPPORTS_SIMPLE_IPC) + add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c) ++ list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-darwin.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c) + + add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) - ## fsmonitor-ipc.h ## -@@ fsmonitor-ipc.h: int fsmonitor_ipc__is_supported(void); - */ - const char *fsmonitor_ipc__get_path(void); + ## fsmonitor-ipc.c ## +@@ fsmonitor-ipc.c: int fsmonitor_ipc__is_supported(void) + return 1; + } -+/* -+ * Returns the pathname to the default IPC named pipe or Unix domain -+ * socket. -+ */ -+const char *fsmonitor_ipc__get_default_path(void); -+ - /* - * Try to determine whether there is a `git-fsmonitor--daemon` process - * listening on the IPC pipe/socket. +-GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc") +- + enum ipc_active_state fsmonitor_ipc__get_state(void) + { + return ipc_get_active_state(fsmonitor_ipc__get_path()); 1: 836a791e6b7 ! 3: ff54b6e0bb5 fsmonitor: add two new config options, allowRemote and socketDir @@ Metadata Author: Eric DeCosta <[email protected]> ## Commit message ## - fsmonitor: add two new config options, allowRemote and socketDir + fsmonitor: avoid socket location check if using hook - Introduce two new configuration options - - fsmonitor.allowRemote - setting this to true overrides fsmonitor's - default behavior of erroring out when enountering network file - systems. Additionly, when true, the Unix domain socket (UDS) file - used for IPC is located in $HOME rather than in the .git directory. - - fsmonitor.socketDir - allows for the UDS file to be located - anywhere the user chooses rather $HOME. + If monitoring is done via fsmonitor hook rather than IPC there is no + need to check if the location of the Unix Domain socket (UDS) file is + on a remote filesystem. Signed-off-by: Eric DeCosta <[email protected]> - ## fsmonitor-settings.c ## -@@ - struct fsmonitor_settings { - enum fsmonitor_mode mode; + ## compat/fsmonitor/fsm-settings-darwin.c ## +@@ compat/fsmonitor/fsm-settings-darwin.c: static enum fsmonitor_reason check_uds_volume(struct repository *r) + return FSMONITOR_REASON_OK; + } + +-enum fsmonitor_reason fsm_os__incompatible(struct repository *r) ++enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc) + { enum fsmonitor_reason reason; -+ int allow_remote; - char *hook_path; -+ char *sock_dir; - }; - static enum fsmonitor_reason check_for_incompatible(struct repository *r) -@@ fsmonitor-settings.c: static struct fsmonitor_settings *alloc_settings(void) - CALLOC_ARRAY(s, 1); - s->mode = FSMONITOR_MODE_DISABLED; - s->reason = FSMONITOR_REASON_UNTESTED; -+ s->allow_remote = -1; +- reason = check_uds_volume(r); +- if (reason != FSMONITOR_REASON_OK) +- return reason; ++ if (ipc) { ++ reason = check_uds_volume(r); ++ if (reason != FSMONITOR_REASON_OK) ++ return reason; ++ } - return s; + return FSMONITOR_REASON_OK; } -@@ fsmonitor-settings.c: static void lookup_fsmonitor_settings(struct repository *r) - fsm_settings__set_disabled(r); + + ## compat/fsmonitor/fsm-settings-win32.c ## +@@ compat/fsmonitor/fsm-settings-win32.c: static enum fsmonitor_reason check_vfs4git(struct repository *r) + return FSMONITOR_REASON_OK; } -+int fsm_settings__get_allow_remote(struct repository *r) -+{ -+ if (!r) -+ r = the_repository; -+ if (!r->settings.fsmonitor) -+ lookup_fsmonitor_settings(r); -+ -+ return r->settings.fsmonitor->allow_remote; -+} -+ -+const char *fsm_settings__get_socket_dir(struct repository *r) -+{ -+ if (!r) -+ r = the_repository; -+ if (!r->settings.fsmonitor) -+ lookup_fsmonitor_settings(r); -+ -+ return r->settings.fsmonitor->sock_dir; -+} -+ - enum fsmonitor_mode fsm_settings__get_mode(struct repository *r) +-enum fsmonitor_reason fsm_os__incompatible(struct repository *r) ++enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc) { - if (!r) -@@ fsmonitor-settings.c: enum fsmonitor_mode fsm_settings__get_mode(struct repository *r) - return r->settings.fsmonitor->mode; + enum fsmonitor_reason reason; + + + ## fsmonitor-settings.c ## +@@ fsmonitor-settings.c: static enum fsmonitor_reason check_remote(struct repository *r) } + #endif -+ - const char *fsm_settings__get_hook_path(struct repository *r) +-static enum fsmonitor_reason check_for_incompatible(struct repository *r) ++static enum fsmonitor_reason check_for_incompatible(struct repository *r, int ipc) { - if (!r) + if (!r->worktree) { + /* +@@ fsmonitor-settings.c: static enum fsmonitor_reason check_for_incompatible(struct repository *r) + reason = check_remote(r); + if (reason != FSMONITOR_REASON_OK) + return reason; +- reason = fsm_os__incompatible(r); ++ reason = fsm_os__incompatible(r, ipc); + if (reason != FSMONITOR_REASON_OK) + return reason; + } @@ fsmonitor-settings.c: const char *fsm_settings__get_hook_path(struct repository *r) - return r->settings.fsmonitor->hook_path; - } -+void fsm_settings__set_allow_remote(struct repository *r) -+{ -+ int allow; -+ -+ if (!r) -+ r = the_repository; -+ if (!r->settings.fsmonitor) -+ r->settings.fsmonitor = alloc_settings(); -+ if (!repo_config_get_bool(r, "fsmonitor.allowremote", &allow)) -+ r->settings.fsmonitor->allow_remote = allow; -+ -+ return; -+} -+ -+void fsm_settings__set_socket_dir(struct repository *r) -+{ -+ const char *path; -+ -+ if (!r) -+ r = the_repository; -+ if (!r->settings.fsmonitor) -+ r->settings.fsmonitor = alloc_settings(); -+ -+ if (!repo_config_get_pathname(r, "fsmonitor.socketdir", &path)) { -+ FREE_AND_NULL(r->settings.fsmonitor->sock_dir); -+ r->settings.fsmonitor->sock_dir = strdup(path); -+ } -+ -+ return; -+} -+ void fsm_settings__set_ipc(struct repository *r) { - enum fsmonitor_reason reason = check_for_incompatible(r); -+ enum fsmonitor_reason reason; -+ -+ fsm_settings__set_allow_remote(r); -+ fsm_settings__set_socket_dir(r); -+ reason = check_for_incompatible(r); ++ enum fsmonitor_reason reason = check_for_incompatible(r, 1); if (reason != FSMONITOR_REASON_OK) { fsm_settings__set_incompatible(r, reason); @@ fsmonitor-settings.c: void fsm_settings__set_ipc(struct repository *r) void fsm_settings__set_hook(struct repository *r, const char *path) { - enum fsmonitor_reason reason = check_for_incompatible(r); -+ enum fsmonitor_reason reason; -+ -+ fsm_settings__set_allow_remote(r); -+ fsm_settings__set_socket_dir(r); -+ reason = check_for_incompatible(r); ++ enum fsmonitor_reason reason = check_for_incompatible(r, 0); if (reason != FSMONITOR_REASON_OK) { fsm_settings__set_incompatible(r, reason); ## fsmonitor-settings.h ## -@@ fsmonitor-settings.h: enum fsmonitor_reason { - FSMONITOR_REASON_NOSOCKETS, /* NTFS,FAT32 do not support Unix sockets */ - }; - -+void fsm_settings__set_allow_remote(struct repository *r); -+void fsm_settings__set_socket_dir(struct repository *r); - void fsm_settings__set_ipc(struct repository *r); - void fsm_settings__set_hook(struct repository *r, const char *path); - void fsm_settings__set_disabled(struct repository *r); - void fsm_settings__set_incompatible(struct repository *r, - enum fsmonitor_reason reason); - -+int fsm_settings__get_allow_remote(struct repository *r); -+const char *fsm_settings__get_socket_dir(struct repository *r); - enum fsmonitor_mode fsm_settings__get_mode(struct repository *r); - const char *fsm_settings__get_hook_path(struct repository *r); +@@ fsmonitor-settings.h: struct fsmonitor_settings; + * fsm_os__* routines should considered private to fsm_settings__ + * routines. + */ +-enum fsmonitor_reason fsm_os__incompatible(struct repository *r); ++enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc); + #endif /* HAVE_FSMONITOR_OS_SETTINGS */ + #endif /* FSMONITOR_SETTINGS_H */ 3: a3110f1e25a < -: ----------- fsmonitor: ensure filesystem and unix socket filesystem are compatible 4: 56cabf3be3b ! 4: b7d6cf44695 fsmonitor: normalize FSEvents event paths to the real path @@ Metadata Author: Eric DeCosta <[email protected]> ## Commit message ## - fsmonitor: normalize FSEvents event paths to the real path + fsmonitor: deal with synthetic firmlinks on macOS - Consider the following network working directory that is mounted under - /System/Volumes/Data: + Starting with macOS 10.15 (Catalina), Apple introduced a new feature + called 'firmlinks' in order to separate the boot volume into two + volumes, one read-only and one writable but still present them to the + user as a single volume. Along with this change, Apple removed the + ability to create symlinks in the root directory and replaced them with + 'synthetic firmlinks'. See 'man synthetic.conf' - /network/working/directory + When FSEevents reports the path of changed files, if the path invloves + a synthetic firmlink, the path is reported from the point of the + synthetic firmlink and not the real path. For example: - The git working directory path is: + Real path: + /System/Volumes/Data/network/working/directory/foo.txt - /System/Volumes/Data/network/working/directory + Synthetic firmlink: + /network -> /System/Volumes/Data/network - The paths reported by FSEvents always start with /network. fsmonitor - expects paths to be under the working directory; therefore it - fails to match /network/... and ignores the change. + FSEvents path: + /network/working/directory/foo.txt - Change things such that if fsmonitor.allowRemote is true that the - paths reported via FSEevents are normalized to the real path. + This causes the FSEvents path to not match against the worktree + directory. + + There are several ways in which synthetic firmlinks can be created: + they can be defined in /etc/synthetic.conf, the automounter can create + them, and there may be other means. Simply reading /etc/synthetic.conf + is insufficient. No matter what process creates synthetic firmlinks, + they all get created in the root directory. + + Therefore, in order to deal with synthetic firmlinks, the root directory + is scanned and the first possible synthetic firmink that, when resolved, + is a prefix of the worktree is used to map FSEvents paths to worktree + paths. Signed-off-by: Eric DeCosta <[email protected]> + ## builtin/fsmonitor--daemon.c ## +@@ + #include "parse-options.h" + #include "fsmonitor.h" + #include "fsmonitor-ipc.h" ++#include "fsmonitor-path-utils.h" + #include "compat/fsmonitor/fsm-health.h" + #include "compat/fsmonitor/fsm-listen.h" + #include "fsmonitor--daemon.h" +@@ builtin/fsmonitor--daemon.c: static int fsmonitor_run_daemon(void) + strbuf_addstr(&state.path_worktree_watch, absolute_path(get_git_work_tree())); + state.nr_paths_watching = 1; + ++ strbuf_init(&state.alias.alias, 0); ++ strbuf_init(&state.alias.points_to, 0); ++ if (fsmonitor__get_alias(state.path_worktree_watch.buf, &state.alias)) { ++ err = error(_("could not get worktree alias")); ++ goto done; ++ } ++ + /* + * We create and delete cookie files somewhere inside the .git + * directory to help us keep sync with the file system. If + ## compat/fsmonitor/fsm-listen-darwin.c ## @@ #include "fsmonitor.h" #include "fsm-listen.h" #include "fsmonitor--daemon.h" -+#include "fsmonitor-settings.h" ++#include "fsmonitor-path-utils.h" struct fsm_listen_data { -@@ compat/fsmonitor/fsm-listen-darwin.c: static void my_add_path(struct fsmonitor_batch *batch, const char *path) - free(composed); - } - -- - static void fsevent_callback(ConstFSEventStreamRef streamRef, - void *ctx, - size_t num_of_events, @@ compat/fsmonitor/fsm-listen-darwin.c: static void fsevent_callback(ConstFSEventStreamRef streamRef, /* * On Mac, we receive an array of absolute paths. */ - path_k = paths[k]; -+ if (fsm_settings__get_allow_remote(the_repository) > 0) { -+ strbuf_reset(&tmp); -+ strbuf_realpath_forgiving(&tmp, paths[k], 0); -+ path_k = tmp.buf; -+ } else ++ path_k = fsmonitor__resolve_alias(paths[k], &state->alias); ++ if (!path_k) + path_k = paths[k]; /* * If you want to debug FSEvents, log them to GIT_TRACE_FSMONITOR. @@ compat/fsmonitor/fsm-listen-darwin.c: static void fsevent_callback(ConstFSEventStreamRef streamRef, - fsmonitor_force_resync(state); fsmonitor_batch__free_list(batch); -+ batch = NULL; string_list_clear(&cookie_list, 0); ++ batch = NULL; /* -@@ compat/fsmonitor/fsm-listen-darwin.c: static void fsevent_callback(ConstFSEventStreamRef streamRef, + * We assume that any events that we received + + ## compat/fsmonitor/fsm-path-utils-darwin.c ## +@@ + #include "fsmonitor.h" + #include "fsmonitor-path-utils.h" ++#include <dirent.h> ++#include <errno.h> ++#include <fcntl.h> + #include <sys/param.h> + #include <sys/mount.h> + +@@ compat/fsmonitor/fsm-path-utils-darwin.c: int fsmonitor__is_fs_remote(const char *path) + return -1; + return fs.is_remote; + } ++ ++/* ++ * Scan the root directory for synthetic firmlinks that when resolved ++ * are a prefix of the path, stopping at the first one found. ++ * ++ * Some information about firmlinks and synthetic firmlinks: ++ * https://eclecticlight.co/2020/01/23/catalina-boot-volumes/ ++ * ++ * macOS no longer allows symlinks in the root directory; any link found ++ * there is therefore a synthetic firmlink. ++ * ++ * If this function gets called often, will want to cache all the firmlink ++ * information, but for now there is only one caller of this function. ++ * ++ * If there is more than one alias for the path, that is another ++ * matter altogteher. ++ */ ++int fsmonitor__get_alias(const char *path, struct alias_info *info) ++{ ++ DIR * dir; ++ int read; ++ int retval; ++ struct dirent *de; ++ struct strbuf alias; ++ struct strbuf points_to; ++ ++ retval = 0; ++ dir = opendir("/"); ++ if (!dir) ++ return -1; ++ ++ strbuf_init(&alias, 256); ++ strbuf_init(&points_to, MAXPATHLEN); ++ ++ while ((de = readdir(dir)) != NULL) { ++ strbuf_reset(&alias); ++ strbuf_addch(&alias, '/'); ++ strbuf_add(&alias, de->d_name, strlen(de->d_name)); ++ ++ read = readlink(alias.buf, points_to.buf, MAXPATHLEN); ++ if (read > 0) { ++ strbuf_setlen(&points_to, read); ++ if ((strncmp(points_to.buf, path, points_to.len) == 0) ++ && path[points_to.len] == '/') { ++ strbuf_addbuf(&info->alias, &alias); ++ strbuf_addbuf(&info->points_to, &points_to); ++ trace_printf_key(&trace_fsmonitor, ++ "Found alias for '%s' : '%s' -> '%s'", ++ path, info->alias.buf, info->points_to.buf); ++ retval = 0; ++ goto done; ++ } ++ } else if (errno != EINVAL) { /* Something other than not a link */ ++ trace_printf_key(&trace_fsmonitor, "Error %s", strerror(errno)); ++ retval = -1; ++ goto done; ++ } ++ } ++ ++ done: ++ closedir(dir); ++ strbuf_release(&alias); ++ strbuf_release(&points_to); ++ return retval; ++} ++ ++char *fsmonitor__resolve_alias(const char *path, ++ const struct alias_info *info) ++{ ++ int len = info->alias.len; ++ ++ if (!len) ++ return NULL; ++ ++ if ((strncmp(info->alias.buf, path, len) == 0) ++ && path[len] == '/') { ++ struct strbuf tmp; ++ const char *remainder = path + len; ++ int rem_len = strlen(remainder); ++ ++ strbuf_init(&tmp, info->points_to.len + rem_len); ++ strbuf_addbuf(&tmp, &info->points_to); ++ strbuf_add(&tmp, remainder, rem_len); ++ return strbuf_detach(&tmp, NULL); ++ } ++ ++ return NULL; ++} + + ## compat/fsmonitor/fsm-path-utils-win32.c ## +@@ compat/fsmonitor/fsm-path-utils-win32.c: int fsmonitor__is_fs_remote(const char *path) + return -1; + return fs.is_remote; + } ++ ++/* ++ * No-op for now. ++ */ ++int fsmonitor__get_alias(const char *path, struct alias_info *info) ++{ ++ return 0; ++} ++ ++/* ++ * No-op for now. ++ */ ++char *fsmonitor__resolve_alias(const char *path, ++ const struct alias_info *info) ++{ ++ return NULL; ++} + + ## fsmonitor--daemon.h ## +@@ + #include "run-command.h" + #include "simple-ipc.h" + #include "thread-utils.h" ++#include "fsmonitor-path-utils.h" + + struct fsmonitor_batch; + struct fsmonitor_token_data; +@@ fsmonitor--daemon.h: struct fsmonitor_daemon_state { + + struct strbuf path_worktree_watch; + struct strbuf path_gitdir_watch; ++ struct alias_info alias; + int nr_paths_watching; + + struct fsmonitor_token_data *current_token_data; +@@ fsmonitor--daemon.h: struct fsmonitor_daemon_state { + + struct ipc_server_data *ipc_server_data; + struct strbuf path_ipc; ++ + }; + + /* +@@ fsmonitor--daemon.h: void fsmonitor_publish(struct fsmonitor_daemon_state *state, + */ + void fsmonitor_force_resync(struct fsmonitor_daemon_state *state); + ++char *fsmonitor_resolve_alias(const char *path, ++ struct alias_info *alias); ++ + #endif /* HAVE_FSMONITOR_DAEMON_BACKEND */ + #endif /* FSMONITOR_DAEMON_H */ + + ## fsmonitor-path-utils.h ## +@@ + #ifndef FSM_PATH_UTILS_H + #define FSM_PATH_UTILS_H - case IS_WORKDIR_PATH: - /* try to queue normal pathnames */ -- - if (trace_pass_fl(&trace_fsmonitor)) - log_flags_set(path_k, event_flags[k]); ++#include "strbuf.h" ++ ++struct alias_info ++{ ++ struct strbuf alias; ++ struct strbuf points_to; ++}; ++ + struct fs_info { + int is_remote; + char *typename; +@@ fsmonitor-path-utils.h: int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info); + */ + int fsmonitor__is_fs_remote(const char *path); ++/* ++ * Get the alias in given path, if any. ++ * ++ * Sets alias to the first alias that matches any part of the path. ++ * ++ * Returns -1 on error, 0 otherwise. ++ */ ++int fsmonitor__get_alias(const char *path, struct alias_info *info); ++ ++/* ++ * Resolve the path against the given alias. ++ * ++ * Returns the resolved path if there is one, NULL otherwise. ++ */ ++char *fsmonitor__resolve_alias(const char *path, ++ const struct alias_info *info); ++ ++ + #endif
Quoted 8 lines · Junio C Hamano
Quoted 5 lines
Eric DeCosta (4):
fsmonitor: refactor filesystem checks to common interface
fsmonitor: relocate socket file if .git directory is remote
fsmonitor: avoid socket location check if using hook
fsmonitor: deal with synthetic firmlinks on macOSIt looks like this one is organized quite differently from the last iteration <[email protected]>
Given that we are very close to -rc period, I am very tempted to say that this follow-up effort can and should be separated out of the previous "windows only" one, and we should merge the single patch that has been in 'next' to 'master' without waiting for this "macOS, too" topic. What do people think?
Thanks.
Quoted 25 lines
-----Original Message-----
From: Junio C Hamano <[email protected]> On Behalf Of Junio C Hamano
Sent: Monday, September 12, 2022 3:38 PM
To: Eric DeCosta via GitGitGadget <[email protected]>
Cc: [email protected]; Jeff Hostetler <[email protected]>; Eric Sunshine <[email protected]>; Torsten Bögershausen <[email protected]>; Ævar Arnfjörð Bjarmason <[email protected]>; Ramsay Jones <[email protected]>; Johannes Schindelin <[email protected]>; Eric DeCosta <[email protected]>
Subject: Re: [PATCH v5 0/4] fsmonitor: option to allow fsmonitor to run against network-mounted reposQuoted 8 lines · Junio C Hamano
Quoted 5 lines
Eric DeCosta (4):
fsmonitor: refactor filesystem checks to common interface
fsmonitor: relocate socket file if .git directory is remote
fsmonitor: avoid socket location check if using hook
fsmonitor: deal with synthetic firmlinks on macOSIt looks like this one is organized quite differently from the last iteration <[email protected]>
Quoted 6 lines
Given that we are very close to -rc period, I am very tempted to say that this follow-up effort can and should be separated out of the previous "windows only" one, and we should merge the single patch that has been in 'next' to 'master' without waiting for this "macOS, too" topic. What do people think?
Thanks.
I'm totally fine with that, turned out to be a bit more involved than I initially anticipated.
-Eric
Quoted 29 lines · Eric DeCosta via GitGitGadget
Eric DeCosta (4):
fsmonitor: refactor filesystem checks to common interface
fsmonitor: relocate socket file if .git directory is remote
fsmonitor: avoid socket location check if using hook
fsmonitor: deal with synthetic firmlinks on macOSMakefile | 2 + builtin/fsmonitor--daemon.c | 8 ++ compat/fsmonitor/fsm-ipc-darwin.c | 46 ++++++++ compat/fsmonitor/fsm-ipc-win32.c | 4 + compat/fsmonitor/fsm-listen-darwin.c | 6 +- compat/fsmonitor/fsm-path-utils-darwin.c | 131 +++++++++++++++++++++++ compat/fsmonitor/fsm-path-utils-win32.c | 106 ++++++++++++++++++ compat/fsmonitor/fsm-settings-darwin.c | 70 ++++-------- compat/fsmonitor/fsm-settings-win32.c | 106 +----------------- contrib/buildsystems/CMakeLists.txt | 4 + fsmonitor--daemon.h | 6 ++ fsmonitor-ipc.c | 2 - fsmonitor-path-utils.h | 49 +++++++++ fsmonitor-settings.c | 58 +++++++++- fsmonitor-settings.h | 2 +- 15 files changed, 441 insertions(+), 159 deletions(-) create mode 100644 compat/fsmonitor/fsm-ipc-darwin.c create mode 100644 compat/fsmonitor/fsm-ipc-win32.c create mode 100644 compat/fsmonitor/fsm-path-utils-darwin.c create mode 100644 compat/fsmonitor/fsm-path-utils-win32.c create mode 100644 fsmonitor-path-utils.h
Quoted 1 line
base-commit:
be1a02a17ede4082a86dfbfee0f54f345e8b43ac
Another curious thing I need your help on.
The cover letter says that this work is a follow-up to extend the previous work for Windows, and I would have expected these patches to build on 85dc0da6 (fsmonitor: option to allow fsmonitor to run against network-mounted repos, 2022-08-11), which is not yet in be1a02a1 (The seventeenth batch, 2022-09-01).
What is going on?
Thanks.
Quoted 56 lines
-----Original Message-----
From: Junio C Hamano <[email protected]> On Behalf Of Junio C Hamano
Sent: Monday, September 12, 2022 11:36 AM
To: Eric DeCosta via GitGitGadget <[email protected]>
Cc: [email protected]; Jeff Hostetler <[email protected]>; Eric Sunshine <[email protected]>; Torsten Bögershausen <[email protected]>; Ævar Arnfjörð Bjarmason <[email protected]>; Ramsay Jones <[email protected]>; Johannes Schindelin <[email protected]>; Eric DeCosta <[email protected]>
Subject: Re: [PATCH v5 0/4] fsmonitor: option to allow fsmonitor to run against network-mounted reposQuoted 6 lines · Eric DeCosta via GitGitGadget
Eric DeCosta (4):
fsmonitor: refactor filesystem checks to common interface
fsmonitor: relocate socket file if .git directory is remote
fsmonitor: avoid socket location check if using hook
fsmonitor: deal with synthetic firmlinks on macOSQuoted 7 lines
Makefile | 2 + builtin/fsmonitor--daemon.c | 8 ++ compat/fsmonitor/fsm-ipc-darwin.c | 46 ++++++++ compat/fsmonitor/fsm-ipc-win32.c | 4 + compat/fsmonitor/fsm-listen-darwin.c | 6 +- compat/fsmonitor/fsm-path-utils-darwin.c | 131+++++++++++++++++++++++
compat/fsmonitor/fsm-path-utils-win32.c|106 ++++++++++++++++++
Quoted 16 lines
compat/fsmonitor/fsm-settings-darwin.c | 70 ++++-------- compat/fsmonitor/fsm-settings-win32.c | 106 +----------------- contrib/buildsystems/CMakeLists.txt | 4 + fsmonitor--daemon.h | 6 ++ fsmonitor-ipc.c | 2 - fsmonitor-path-utils.h | 49 +++++++++ fsmonitor-settings.c | 58 +++++++++- fsmonitor-settings.h | 2 +- 15 files changed, 441 insertions(+), 159 deletions(-) create mode 100644 compat/fsmonitor/fsm-ipc-darwin.c create mode 100644 compat/fsmonitor/fsm-ipc-win32.c create mode 100644 compat/fsmonitor/fsm-path-utils-darwin.c create mode 100644 compat/fsmonitor/fsm-path-utils-win32.c create mode 100644 fsmonitor-path-utils.hQuoted 1 line
base-commit:
be1a02a17ede4082a86dfbfee0f54f345e8b43acAnother curious thing I need your help on.
The cover letter says that this work is a follow-up to extend the previous work for Windows, and I would have expected these patches to build on
85dc0da6(fsmonitor: option to allow fsmonitor to run against network- mounted repos, 2022-08-11), which is not yet inbe1a02a1(The seventeenth batch, 2022-09-01).
Quoted 3 lines
What is going on?
Thanks.
I've been rolling that around in the back of my head over the last couple of days actually. Initially I though the macOS changes were going to be independent of my earlier changes, but with all the refactoring that has occurred, that is no longer the case. I've rebased my current patch set on top of those earlier changes. v6 will incorporate that, the review feedback that I've received so far and add doc for the new config options. I'll be sure to note all of this in the cover letter for v6.
-Eric
v6 cover fsmonitor: option to allow fsmonitor to run against network-mounted repos
In reply to Eric DeCosta via GitGitGadgetFollow-on to the work done to allow Windows to work against network-mounted repos for macOS.
Have macOS take advantage of the same configuration option, 'fsmonitor.allowRemote' that was introduced for Windows. Setting this option to true will override the default behavior (erroring-out) when a network-mounted repo is detected by fsmonitor.
The added wrinkle being that the Unix domain socket (UDS) file used for IPC cannot be created in a network location; instead $HOME is used if the default location is on the network. The user may, optionally, set the 'fsmonitor.socketDir' configuration option to a valid, local directory if $HOME itself is on the network or is simply not the desired location for the UDS file.
An additional issue is that for mount points in the root directory, FSEvents does not report a path that matches the worktree directory due to the introduction of 'synthetic firmlinks'. fsmonitor must map the FSEvents paths to the worktree directory by interrogating the root filesystem for synthetic firmlinks and using that information to translate the path.
v6 differs from v5:
* incorporates earlier, Windows-specific changes that have not made it back yet to the master branch
* incorporates code review feedback
* adds documentation for 'fsmonitor.allowRemote' and 'fsmonitor.socketDir'
v5 differs significantly from earlier versions:
* redesign of handling 'fsmonitor.allowRemote' and 'fsmonitor.socketDir' such that these options are no longer added to the settings data structure but are rather read from config at point of use
* refactoring of code for handling platform-specific file system checks via a common interface to avoid platform #ifdef in IPC code and be in-model with other platform-specific fsmonitor code
* dealing with 'synthetic firmlinks' on macOS
Eric DeCosta (6):
fsmonitor: option to allow fsmonitor to run against network-mounted
repos
fsmonitor: refactor filesystem checks to common interface
fsmonitor: relocate socket file if .git directory is remote
fsmonitor: avoid socket location check if using hook
fsmonitor: deal with synthetic firmlinks on macOS
fsmonitor: add documentation for allowRemote and socketDir options
Documentation/git-fsmonitor--daemon.txt | 35 ++++++ Makefile | 2 + builtin/fsmonitor--daemon.c | 8 ++ compat/fsmonitor/fsm-ipc-darwin.c | 46 +++++++ compat/fsmonitor/fsm-ipc-win32.c | 4 + compat/fsmonitor/fsm-listen-darwin.c | 6 +- compat/fsmonitor/fsm-path-utils-darwin.c | 132 +++++++++++++++++++++ compat/fsmonitor/fsm-path-utils-win32.c | 145 +++++++++++++++++++++++ compat/fsmonitor/fsm-settings-darwin.c | 70 ++++------- compat/fsmonitor/fsm-settings-win32.c | 106 +---------------- contrib/buildsystems/CMakeLists.txt | 4 + fsmonitor--daemon.h | 6 + fsmonitor-ipc.c | 2 - fsmonitor-path-utils.h | 59 +++++++++ fsmonitor-settings.c | 58 ++++++++- fsmonitor-settings.h | 2 +- 16 files changed, 526 insertions(+), 159 deletions(-) create mode 100644 compat/fsmonitor/fsm-ipc-darwin.c create mode 100644 compat/fsmonitor/fsm-ipc-win32.c create mode 100644 compat/fsmonitor/fsm-path-utils-darwin.c create mode 100644 compat/fsmonitor/fsm-path-utils-win32.c create mode 100644 fsmonitor-path-utils.h
base-commit: dd3f6c4cae7e3b15ce984dce8593ff7569650e24
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1326%2Fedecosta-mw%2Ffsmonitor_macos-v6
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1326/edecosta-mw/fsmonitor_macos-v6
Pull-Request: https://github.com/gitgitgadget/git/pull/1326
Range-diff vs v5:
-: ----------- > 1: 3233c908c4a fsmonitor: option to allow fsmonitor to run against network-mounted repos
1: 2f647b53e4d ! 2: d2a8fc6b707 fsmonitor: refactor filesystem checks to common interface
@@ compat/fsmonitor/fsm-path-utils-win32.c (new)
+#include "fsmonitor-path-utils.h"
+
+/*
++ * Check remote working directory protocol.
++ *
++ * Return -1 if client machine cannot get remote protocol information.
++ */
++static int check_remote_protocol(wchar_t *wpath)
++{
++ HANDLE h;
++ FILE_REMOTE_PROTOCOL_INFO proto_info;
++
++ h = CreateFileW(wpath, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING,
++ FILE_FLAG_BACKUP_SEMANTICS, NULL);
++
++ if (h == INVALID_HANDLE_VALUE) {
++ error(_("[GLE %ld] unable to open for read '%ls'"),
++ GetLastError(), wpath);
++ return -1;
++ }
++
++ if (!GetFileInformationByHandleEx(h, FileRemoteProtocolInfo,
++ &proto_info, sizeof(proto_info))) {
++ error(_("[GLE %ld] unable to get protocol information for '%ls'"),
++ GetLastError(), wpath);
++ CloseHandle(h);
++ return -1;
++ }
++
++ CloseHandle(h);
++
++ trace_printf_key(&trace_fsmonitor,
++ "check_remote_protocol('%ls') remote protocol %#8.8lx",
++ wpath, proto_info.Protocol);
++
++ return 0;
++}
++
++/*
+ * Notes for testing:
+ *
+ * (a) Windows allows a network share to be mapped to a drive letter.
@@ compat/fsmonitor/fsm-path-utils-win32.c (new)
+ "DriveType '%s' L'%ls' (%u)",
+ path, wfullpath, driveType);
+
-+ if (driveType == DRIVE_REMOTE)
++ if (driveType == DRIVE_REMOTE) {
+ fs_info->is_remote = 1;
-+ else
++ if (check_remote_protocol(wfullpath) < 0)
++ return -1;
++ } else {
+ fs_info->is_remote = 0;
++ }
+
+ trace_printf_key(&trace_fsmonitor,
+ "'%s' is_remote: %d",
@@ compat/fsmonitor/fsm-settings-win32.c: static enum fsmonitor_reason check_vfs4gi
return FSMONITOR_REASON_OK;
}
+-/*
+- * Check if monitoring remote working directories is allowed.
+- *
+- * By default, monitoring remote working directories is
+- * disabled. Users may override this behavior in enviroments where
+- * they have proper support.
+- */
+-static int check_config_allowremote(struct repository *r)
+-{
+- int allow;
+-
+- if (!repo_config_get_bool(r, "fsmonitor.allowremote", &allow))
+- return allow;
+-
+- return -1; /* fsmonitor.allowremote not set */
+-}
+-
+-/*
+- * Check remote working directory protocol.
+- *
+- * Error if client machine cannot get remote protocol information.
+- */
+-static int check_remote_protocol(wchar_t *wpath)
+-{
+- HANDLE h;
+- FILE_REMOTE_PROTOCOL_INFO proto_info;
+-
+- h = CreateFileW(wpath, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING,
+- FILE_FLAG_BACKUP_SEMANTICS, NULL);
+-
+- if (h == INVALID_HANDLE_VALUE) {
+- error(_("[GLE %ld] unable to open for read '%ls'"),
+- GetLastError(), wpath);
+- return -1;
+- }
+-
+- if (!GetFileInformationByHandleEx(h, FileRemoteProtocolInfo,
+- &proto_info, sizeof(proto_info))) {
+- error(_("[GLE %ld] unable to get protocol information for '%ls'"),
+- GetLastError(), wpath);
+- CloseHandle(h);
+- return -1;
+- }
+-
+- CloseHandle(h);
+-
+- trace_printf_key(&trace_fsmonitor,
+- "check_remote_protocol('%ls') remote protocol %#8.8lx",
+- wpath, proto_info.Protocol);
+-
+- return 0;
+-}
+-
-/*
- * Remote working directories are problematic for FSMonitor.
- *
@@ compat/fsmonitor/fsm-settings-win32.c: static enum fsmonitor_reason check_vfs4gi
- */
-static enum fsmonitor_reason check_remote(struct repository *r)
-{
+- int ret;
- wchar_t wpath[MAX_PATH];
- wchar_t wfullpath[MAX_PATH];
- size_t wlen;
@@ compat/fsmonitor/fsm-settings-win32.c: static enum fsmonitor_reason check_vfs4gi
- trace_printf_key(&trace_fsmonitor,
- "check_remote('%s') true",
- r->worktree);
+-
+- ret = check_remote_protocol(wfullpath);
+- if (ret < 0)
+- return FSMONITOR_REASON_ERROR;
+-
+- switch (check_config_allowremote(r)) {
+- case 0: /* config overrides and disables */
+- return FSMONITOR_REASON_REMOTE;
+- case 1: /* config overrides and enables */
+- return FSMONITOR_REASON_OK;
+- default:
+- break; /* config has no opinion */
+- }
+-
- return FSMONITOR_REASON_REMOTE;
- }
-
2: 28d08bcf808 = 3: edef029a298 fsmonitor: relocate socket file if .git directory is remote
3: ff54b6e0bb5 = 4: 3428bcf8763 fsmonitor: avoid socket location check if using hook
4: b7d6cf44695 ! 5: 9c1f408ae6d fsmonitor: deal with synthetic firmlinks on macOS
@@ Commit message
ability to create symlinks in the root directory and replaced them with
'synthetic firmlinks'. See 'man synthetic.conf'
- When FSEevents reports the path of changed files, if the path invloves
+ When FSEevents reports the path of changed files, if the path involves
a synthetic firmlink, the path is reported from the point of the
synthetic firmlink and not the real path. For example:
@@ builtin/fsmonitor--daemon.c: static int fsmonitor_run_daemon(void)
strbuf_addstr(&state.path_worktree_watch, absolute_path(get_git_work_tree()));
state.nr_paths_watching = 1;
-+ strbuf_init(&state.alias.alias, 0);
-+ strbuf_init(&state.alias.points_to, 0);
++ state.alias.alias = NULL;
++ state.alias.points_to = NULL;
+ if (fsmonitor__get_alias(state.path_worktree_watch.buf, &state.alias)) {
+ err = error(_("could not get worktree alias"));
+ goto done;
@@ compat/fsmonitor/fsm-path-utils-darwin.c: int fsmonitor__is_fs_remote(const char
+ * information, but for now there is only one caller of this function.
+ *
+ * If there is more than one alias for the path, that is another
-+ * matter altogteher.
++ * matter altogether.
+ */
+int fsmonitor__get_alias(const char *path, struct alias_info *info)
+{
@@ compat/fsmonitor/fsm-path-utils-darwin.c: int fsmonitor__is_fs_remote(const char
+ strbuf_setlen(&points_to, read);
+ if ((strncmp(points_to.buf, path, points_to.len) == 0)
+ && path[points_to.len] == '/') {
-+ strbuf_addbuf(&info->alias, &alias);
-+ strbuf_addbuf(&info->points_to, &points_to);
++ info->alias = strbuf_detach(&alias, NULL);
++ info->points_to = strbuf_detach(&points_to, NULL);
+ trace_printf_key(&trace_fsmonitor,
+ "Found alias for '%s' : '%s' -> '%s'",
-+ path, info->alias.buf, info->points_to.buf);
++ path, info->alias, info->points_to);
+ retval = 0;
+ goto done;
+ }
@@ compat/fsmonitor/fsm-path-utils-darwin.c: int fsmonitor__is_fs_remote(const char
+char *fsmonitor__resolve_alias(const char *path,
+ const struct alias_info *info)
+{
-+ int len = info->alias.len;
++ int len = info->alias ? strlen(info->alias) : 0;
+
+ if (!len)
+ return NULL;
+
-+ if ((strncmp(info->alias.buf, path, len) == 0)
++ if ((strncmp(info->alias, path, len) == 0)
+ && path[len] == '/') {
+ struct strbuf tmp;
+ const char *remainder = path + len;
++ int ptr_len = strlen(info->points_to);
+ int rem_len = strlen(remainder);
+
-+ strbuf_init(&tmp, info->points_to.len + rem_len);
-+ strbuf_addbuf(&tmp, &info->points_to);
++ strbuf_init(&tmp, ptr_len + rem_len);
++ strbuf_add(&tmp, info->points_to, ptr_len);
+ strbuf_add(&tmp, remainder, rem_len);
+ return strbuf_detach(&tmp, NULL);
+ }
@@ fsmonitor-path-utils.h
+
+struct alias_info
+{
-+ struct strbuf alias;
-+ struct strbuf points_to;
++ char *alias;
++ char *points_to;
+};
+
struct fs_info {
@@ fsmonitor-path-utils.h: int fsmonitor__get_fs_info(const char *path, struct fs_i
+ *
+ * Sets alias to the first alias that matches any part of the path.
+ *
++ * If an alias is found, info.alias and info.points_to are set to the
++ * found mapping.
++ *
+ * Returns -1 on error, 0 otherwise.
++ *
++ * The caller owns the storage that is occupied by set info.alias and
++ * info.points_to and is responsible for releasing it with `free(3)`
++ * when done.
+ */
+int fsmonitor__get_alias(const char *path, struct alias_info *info);
+
@@ fsmonitor-path-utils.h: int fsmonitor__get_fs_info(const char *path, struct fs_i
+ * Resolve the path against the given alias.
+ *
+ * Returns the resolved path if there is one, NULL otherwise.
++ *
++ * The caller owns the storage that the returned string occupies and
++ * is responsible for releasing it with `free(3)` when done.
+ */
+char *fsmonitor__resolve_alias(const char *path,
+ const struct alias_info *info); -: ----------- > 6: d2c95e34d3a fsmonitor: add documentation for allowRemote and socketDir options
Signature · 1 line
gitgitgadget
PATCH v6 1/6 fsmonitor: option to allow fsmonitor to run against network-mounted repos
From: Eric DeCosta <[email protected]>
Though perhaps not common, there are use cases where users have large, network-mounted repos. Having the ability to run fsmonitor against network paths would benefit those users.
Most modern Samba-based filers have the necessary support to enable fsmonitor on network-mounted repos. As a first step towards enabling fsmonitor to work against network-mounted repos, introduce a configuration option, 'fsmonitor.allowRemote'. Setting this option to true will override the default behavior (erroring-out) when a network-mounted repo is detected by fsmonitor.
--- compat/fsmonitor/fsm-settings-win32.c | 68 +++++++++++++++++++++++++++ 1 file changed, 68 insertions(+)
compat/fsmonitor/fsm-settings-win32.c +68 -0
@@ -24,6 +24,59 @@ static enum fsmonitor_reason check_vfs4git(struct repository *r)return FSMONITOR_REASON_OK ; } +/* +* Check if monitoring remote working directories is allowed. +* +* By default, monitoring remote working directories is +* disabled. Users may override this behavior in enviroments where +* they have proper support. +*/ +static int check_config_allowremote (struct repository *r) +{ +int allow; + +if (!repo_config_get_bool (r,"fsmonitor.allowremote" , &allow)) +return allow; + +return -1 ;/* fsmonitor.allowremote not set */ +} + +/* +* Check remote working directory protocol. +* +* Error if client machine cannot get remote protocol information. +*/ +static int check_remote_protocol (wchar_t *wpath) +{ +HANDLE h; +FILE_REMOTE_PROTOCOL_INFO proto_info; + + h =CreateFileW (wpath,GENERIC_READ ,FILE_SHARE_READ ,NULL ,OPEN_EXISTING , +FILE_FLAG_BACKUP_SEMANTICS ,NULL ); + +if (h ==INVALID_HANDLE_VALUE ) { +error (_ ("[GLE %ld] unable to open for read '%ls'" ), +GetLastError (), wpath); +return -1 ; + } + +if (!GetFileInformationByHandleEx (h, FileRemoteProtocolInfo, + &proto_info,sizeof (proto_info))) { +error (_ ("[GLE %ld] unable to get protocol information for '%ls'" ), +GetLastError (), wpath); +CloseHandle (h); +return -1 ; + } + +CloseHandle (h); + +trace_printf_key (&trace_fsmonitor, +"check_remote_protocol('%ls') remote protocol %#8.8lx" , + wpath, proto_info.Protocol); + +return 0 ; +} +/* * Remote working directories are problematic for FSMonitor. *
@@ -76,6 +129,7 @@ static enum fsmonitor_reason check_vfs4git(struct repository *r)*/ static enum fsmonitor_reason check_remote (struct repository *r) { +int ret;wchar_t wpath[MAX_PATH ];wchar_t wfullpath[MAX_PATH ];size_t wlen;
@@ -115,6 +169,20 @@ static enum fsmonitor_reason check_remote(struct repository *r)trace_printf_key (&trace_fsmonitor,"check_remote('%s') true" , r->worktree); + + ret =check_remote_protocol (wfullpath); +if (ret <0 ) +return FSMONITOR_REASON_ERROR ; + +switch (check_config_allowremote (r)) { +case 0 :/* config overrides and disables */ +return FSMONITOR_REASON_REMOTE ; +case 1 :/* config overrides and enables */ +return FSMONITOR_REASON_OK ; +default : +break ;/* config has no opinion */ + } +return FSMONITOR_REASON_REMOTE ; }
Signature · 1 line
gitgitgadget
PATCH v6 2/6 fsmonitor: refactor filesystem checks to common interface
In reply to Eric DeCosta via GitGitGadgetFrom: Eric DeCosta <[email protected]>
Provide a common interface for getting basic filesystem information including filesystem type and whether the filesystem is remote.
Refactor existing code for getting basic filesystem info and detecting remote file systems to the new interface.
Refactor filesystem checks to leverage new interface. For macOS, error-out if the Unix Domain socket (UDS) file is on a remote filesystem.
--- Makefile | 1 + compat/fsmonitor/fsm-path-utils-darwin.c | 40 ++++++ compat/fsmonitor/fsm-path-utils-win32.c | 128 +++++++++++++++++ compat/fsmonitor/fsm-settings-darwin.c | 62 +++----- compat/fsmonitor/fsm-settings-win32.c | 172 +---------------------- contrib/buildsystems/CMakeLists.txt | 2 + fsmonitor-path-utils.h | 23 +++ fsmonitor-settings.c | 50 +++++++ 8 files changed, 263 insertions(+), 215 deletions(-) create mode 100644 compat/fsmonitor/fsm-path-utils-darwin.c create mode 100644 compat/fsmonitor/fsm-path-utils-win32.c create mode 100644 fsmonitor-path-utils.h
Makefile +1 -0
@@ -2038,6 +2038,7 @@ endif ifdef FSMONITOR_OS_SETTINGS COMPAT_CFLAGS += -DHAVE_FSMONITOR_OS_SETTINGS COMPAT_OBJS += compat/fsmonitor/fsm-settings-$(FSMONITOR_OS_SETTINGS).o + COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_OS_SETTINGS).o endif ifeq ($(TCLTK_PATH),)
compat/fsmonitor/fsm-path-utils-darwin.c new file +40 -0
@@ -0,0 +1,40 @@ +#include "fsmonitor.h" +#include "fsmonitor-path-utils.h" +#include <sys/param.h> +#include <sys/mount.h> + +int fsmonitor__get_fs_info (const char *path,struct fs_info *fs_info) +{ +struct statfs fs; +if (statfs (path, &fs) ==-1 ) { +int saved_errno = errno; +trace_printf_key (&trace_fsmonitor,"statfs('%s') failed: %s" , + path,strerror (saved_errno)); + errno = saved_errno; +return -1 ; + } + +trace_printf_key (&trace_fsmonitor, +"statfs('%s') [type 0x%08x][flags 0x%08x] '%s'" , + path, fs.f_type, fs.f_flags, fs.f_fstypename); + +if (!(fs.f_flags &MNT_LOCAL )) + fs_info->is_remote =1 ; +else + fs_info->is_remote =0 ; + + fs_info->typename = fs.f_fstypename; + +trace_printf_key (&trace_fsmonitor, +"'%s' is_remote: %d" , + path, fs_info->is_remote); +return 0 ; +} + +int fsmonitor__is_fs_remote (const char *path) +{ +struct fs_info fs; +if (fsmonitor__get_fs_info (path, &fs)) +return -1 ; +return fs.is_remote; +}
compat/fsmonitor/fsm-path-utils-win32.c new file +128 -0
128 lines not on this page · raw message
compat/fsmonitor/fsm-settings-darwin.c +17 -45
96 lines not on this page · raw message
compat/fsmonitor/fsm-settings-win32.c +2 -170
190 lines not on this page · raw message
contrib/buildsystems/CMakeLists.txt +2 -0
@@ -308,6 +308,7 @@ if(SUPPORTS_SIMPLE_IPC) add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-win32.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-win32.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-win32.c)
@@ -315,6 +316,7 @@ if(SUPPORTS_SIMPLE_IPC) add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-darwin.c)
fsmonitor-path-utils.h new file +23 -0
@@ -0,0 +1,23 @@ +#ifndef FSM_PATH_UTILS_H +#define FSM_PATH_UTILS_H + +struct fs_info { +int is_remote; +char *typename; +}; + +/* +* Get some basic filesystem informtion for the given path +* +* Returns -1 on error, zero otherwise. +*/ +int fsmonitor__get_fs_info (const char *path,struct fs_info *fs_info); + +/* +* Determines if the filesystem that path resides on is remote. +* +* Returns -1 on error, 0 if not remote, 1 if remote. +*/ +int fsmonitor__is_fs_remote (const char *path); + +#endif
fsmonitor-settings.c +50 -0
68 lines not on this page · raw message
Signature · 1 line
gitgitgadget
PATCH v6 3/6 fsmonitor: relocate socket file if .git directory is remote
In reply to Eric DeCosta via GitGitGadgetFrom: Eric DeCosta <[email protected]>
If the .git directory is on a remote file system, create the socket file in 'fsmonitor.socketDir' if it is defined, else create it in $HOME.
--- Makefile | 1 + compat/fsmonitor/fsm-ipc-darwin.c | 46 +++++++++++++++++++++++++++++ compat/fsmonitor/fsm-ipc-win32.c | 4 +++ contrib/buildsystems/CMakeLists.txt | 2 ++ fsmonitor-ipc.c | 2 -- 5 files changed, 53 insertions(+), 2 deletions(-) create mode 100644 compat/fsmonitor/fsm-ipc-darwin.c create mode 100644 compat/fsmonitor/fsm-ipc-win32.c
Makefile +1 -0
@@ -2033,6 +2033,7 @@ ifdef FSMONITOR_DAEMON_BACKEND COMPAT_CFLAGS += -DHAVE_FSMONITOR_DAEMON_BACKEND COMPAT_OBJS += compat/fsmonitor/fsm-listen-$(FSMONITOR_DAEMON_BACKEND).o COMPAT_OBJS += compat/fsmonitor/fsm-health-$(FSMONITOR_DAEMON_BACKEND).o + COMPAT_OBJS += compat/fsmonitor/fsm-ipc-$(FSMONITOR_DAEMON_BACKEND).o endif ifdef FSMONITOR_OS_SETTINGS
compat/fsmonitor/fsm-ipc-darwin.c new file +46 -0
46 lines not on this page · raw message
compat/fsmonitor/fsm-ipc-win32.c new file +4 -0
@@ -0,0 +1,4 @@ +#include "cache.h" +#include "fsmonitor-ipc.h" + +GIT_PATH_FUNC (fsmonitor_ipc__get_path,"fsmonitor--daemon.ipc" )
contrib/buildsystems/CMakeLists.txt +2 -0
@@ -308,6 +308,7 @@ if(SUPPORTS_SIMPLE_IPC) add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-win32.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-win32.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-win32.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
@@ -316,6 +317,7 @@ if(SUPPORTS_SIMPLE_IPC) add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-darwin.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
fsmonitor-ipc.c +0 -2
@@ -47,8 +47,6 @@ int fsmonitor_ipc__is_supported(void)return 1 ; } -GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc") -enum ipc_active_state fsmonitor_ipc__get_state (void ) {return ipc_get_active_state (fsmonitor_ipc__get_path ());
Signature · 1 line
gitgitgadget
Re: [PATCH v6 3/6] fsmonitor: relocate socket file if .git directory is remote
compat/fsmonitor/fsm-ipc-darwin.c line 11
Quoted 2 lines · Eric DeCosta via GitGitGadget
compat/fsmonitor/fsm-ipc-darwin.c new file +2
-0+const char *fsmonitor_ipc__get_path (void ) +{
Looks like a bit klunky API. I would have expected it to take at least the path to the worktree or a pointer to struct repository.
Quoted 8 lines
compat/fsmonitor/fsm-ipc-darwin.c new file +8
-0+static const char *ipc_path; +SHA_CTX sha1ctx; +char *sock_dir; +struct strbuf ipc_file =STRBUF_INIT ; +unsigned char hash[SHA_DIGEST_LENGTH ]; + +if (ipc_path) +return ipc_path;
compat/fsmonitor/fsm-ipc-darwin.c line 29
Quoted 10 lines
compat/fsmonitor/fsm-ipc-darwin.c new file +10
-0+ + ipc_path =fsmonitor_ipc__get_default_path (); + +/* By default the socket file is created in the .git directory */ +if (fsmonitor__is_fs_remote (ipc_path) <1 ) +return ipc_path; + +SHA1_Init (&sha1ctx); +SHA1_Update (&sha1ctx, the_repository->worktree,strlen (the_repository->worktree)); +SHA1_Final (hash, &sha1ctx);
I would not worry about SHA-1 hash collision for this use case, but would worry more about .worktree possible being unique.
Can the .worktree string be aliased for the same directory in some way (e.g. depending on the initialization sequence, can it be a full pathname, a relative pathname, or can a pathname that looks like a full-pathname have symbolic link component in it?) that ends up creating two or more socket for the same worktree?
Quoted 16 lines
compat/fsmonitor/fsm-ipc-darwin.c new file +16
-0+repo_config_get_string (the_repository,"fsmonitor.socketdir" , &sock_dir); + +/* Create the socket file in either socketDir or $HOME */ +if (sock_dir && *sock_dir) +strbuf_addf (&ipc_file,"%s/.git-fsmonitor-%s" , + sock_dir,hash_to_hex (hash)); +else +strbuf_addf (&ipc_file,"~/.git-fsmonitor-%s" ,hash_to_hex (hash)); + + ipc_path =interpolate_path (ipc_file.buf,1 ); +if (!ipc_path) +die (_ ("Invalid path: %s" ), ipc_file.buf); + +strbuf_release (&ipc_file); +return ipc_path; +}
RE: [PATCH v6 3/6] fsmonitor: relocate socket file if .git directory is remote
Quoted 14 lines
-----Original Message-----
From: Junio C Hamano <[email protected]>
Sent: Tuesday, September 13, 2022 8:48 PM
To: Eric DeCosta via GitGitGadget <[email protected]>
Cc: [email protected]; Jeff Hostetler <[email protected]>; Eric Sunshine <[email protected]>; Torsten Bögershausen <[email protected]>; Ævar Arnfjörð Bjarmason <[email protected]>; Ramsay Jones <[email protected]>; Johannes Schindelin <[email protected]>; Eric DeCosta <[email protected]>
Subject: Re: [PATCH v6 3/6] fsmonitor: relocate socket file if .git directory is remote
Quoted 7 lines
Quoted 1 line · Eric DeCosta via GitGitGadget
+const char
*fsmonitor_ipc__get_path(void){Looks like a bit klunky API. I would have expected it to take at least the path to the worktree or a pointer to struct repository.
OK, I'll change it to take a pointer to struct repository.
Quoted 23 lines
Quoted 7 lines
compat/fsmonitor/fsm-ipc-darwin.c new file +7
-0+static const char *ipc_path; +SHA_CTX sha1ctx; +char *sock_dir; +struct strbuf ipc_file =STRBUF_INIT ; +unsigned char hash[SHA_DIGEST_LENGTH ]; + +if (ipc_path)compat/fsmonitor/fsm-ipc-darwin.c
Quoted 10 lines
compat/fsmonitor/fsm-ipc-darwin.c new file +10
-0+return ipc_path; + + ipc_path =fsmonitor_ipc__get_default_path (); + +/* By default the socket file is created in the .git directory */ +if (fsmonitor__is_fs_remote (ipc_path) <1 ) +return ipc_path; + +SHA1_Init (&sha1ctx); + SHA1_Update(&sha1ctx, the_repository->worktree,
strlen(the_repository->worktree));compat/fsmonitor/fsm-ipc-darwin.c line 29
Quoted 1 line
compat/fsmonitor/fsm-ipc-darwin.c new file +1
-0+SHA1_Final (hash, &sha1ctx);I would not worry about SHA-1 hash collision for this use case, but would worry more about .worktree possible being unique.
Quoted 6 lines
Can the .worktree string be aliased for the same directory in some way (e.g. depending on the initialization sequence, can it be a full pathname, a relative pathname, or can a pathname that looks like a full-pathname have symbolic link component in it?) that ends up creating two or more socket for the same worktree?
.worktree is set to the real path and since hardlinks are not allowed across file systems, I think we are OK.
Quoted 18 lines
compat/fsmonitor/fsm-ipc-darwin.c
Quoted 9 lines
compat/fsmonitor/fsm-ipc-darwin.c new file +9
-0+repo_config_get_string (the_repository,"fsmonitor.socketdir" , +&sock_dir); + +/* Create the socket file in either socketDir or $HOME */ +if (sock_dir && *sock_dir) +strbuf_addf (&ipc_file,"%s/.git-fsmonitor-%s" , + sock_dir,hash_to_hex (hash)); +else + strbuf_addf(&ipc_file, "~/.git-fsmonitor-%s",
hash_to_hex(hash));Quoted 8 lines
compat/fsmonitor/fsm-ipc-darwin.c new file +8
-0+ + ipc_path =interpolate_path (ipc_file.buf,1 ); +if (!ipc_path) +die (_ ("Invalid path: %s" ), ipc_file.buf); + +strbuf_release (&ipc_file); +return ipc_path; +}
PATCH v6 4/6 fsmonitor: avoid socket location check if using hook
In reply to Eric DeCosta via GitGitGadgetFrom: Eric DeCosta <[email protected]>
If monitoring is done via fsmonitor hook rather than IPC there is no need to check if the location of the Unix Domain socket (UDS) file is on a remote filesystem.
--- compat/fsmonitor/fsm-settings-darwin.c | 10 ++++++---- compat/fsmonitor/fsm-settings-win32.c | 2 +- fsmonitor-settings.c | 8 ++++---- fsmonitor-settings.h | 2 +- 4 files changed, 12 insertions(+), 10 deletions(-)
compat/fsmonitor/fsm-settings-darwin.c +6 -4
@@ -49,13 +49,15 @@ static enum fsmonitor_reason check_uds_volume(struct repository *r)return FSMONITOR_REASON_OK ; } -enum fsmonitor_reason fsm_os__incompatible (struct repository *r) +enum fsmonitor_reason fsm_os__incompatible (struct repository *r,int ipc) {enum fsmonitor_reason reason; - reason =check_uds_volume (r); -if (reason !=FSMONITOR_REASON_OK ) -return reason; +if (ipc) { + reason =check_uds_volume (r); +if (reason !=FSMONITOR_REASON_OK ) +return reason; + }return FSMONITOR_REASON_OK ; }
compat/fsmonitor/fsm-settings-win32.c +1 -1
@@ -25,7 +25,7 @@ static enum fsmonitor_reason check_vfs4git(struct repository *r)return FSMONITOR_REASON_OK ; } -enum fsmonitor_reason fsm_os__incompatible (struct repository *r) +enum fsmonitor_reason fsm_os__incompatible (struct repository *r,int ipc) {enum fsmonitor_reason reason;
fsmonitor-settings.c +4 -4
@@ -60,7 +60,7 @@ static enum fsmonitor_reason check_remote(struct repository *r) }#endif -static enum fsmonitor_reason check_for_incompatible (struct repository *r) +static enum fsmonitor_reason check_for_incompatible (struct repository *r,int ipc) {if (!r->worktree) {/*
@@ -77,7 +77,7 @@ static enum fsmonitor_reason check_for_incompatible(struct repository *r) reason =check_remote (r);if (reason !=FSMONITOR_REASON_OK )return reason; - reason =fsm_os__incompatible (r); + reason =fsm_os__incompatible (r, ipc);if (reason !=FSMONITOR_REASON_OK )return reason; }
@@ -162,7 +162,7 @@ const char *fsm_settings__get_hook_path(struct repository *r)void fsm_settings__set_ipc (struct repository *r) { -enum fsmonitor_reason reason =check_for_incompatible (r); +enum fsmonitor_reason reason =check_for_incompatible (r,1 );if (reason !=FSMONITOR_REASON_OK ) {fsm_settings__set_incompatible (r, reason);
@@ -185,7 +185,7 @@ void fsm_settings__set_ipc(struct repository *r)void fsm_settings__set_hook (struct repository *r,const char *path) { -enum fsmonitor_reason reason =check_for_incompatible (r); +enum fsmonitor_reason reason =check_for_incompatible (r,0 );if (reason !=FSMONITOR_REASON_OK ) {fsm_settings__set_incompatible (r, reason);
fsmonitor-settings.h +1 -1
@@ -48,7 +48,7 @@ struct fsmonitor_settings;* fsm_os__* routines should considered private to fsm_settings__ * routines. */ -enum fsmonitor_reason fsm_os__incompatible (struct repository *r); +enum fsmonitor_reason fsm_os__incompatible (struct repository *r,int ipc);#endif /* HAVE_FSMONITOR_OS_SETTINGS */ #endif /* FSMONITOR_SETTINGS_H */
Signature · 1 line
gitgitgadget
Re: [PATCH v6 4/6] fsmonitor: avoid socket location check if using hook
Quoted 5 lines · Eric DeCosta via GitGitGadget
From: Eric DeCosta <[email protected]>
If monitoring is done via fsmonitor hook rather than IPC there is no need to check if the location of the Unix Domain socket (UDS) file is on a remote filesystem.
Nicely done.
Quoted 99 lines
Signed-off-by Eric DeCosta <[email protected]>--- compat/fsmonitor/fsm-settings-darwin.c | 10 ++++++---- compat/fsmonitor/fsm-settings-win32.c | 2 +- fsmonitor-settings.c | 8 ++++---- fsmonitor-settings.h | 2 +- 4 files changed, 12 insertions(+), 10 deletions(-)compat/fsmonitor/fsm-settings-darwin.c +6
-4@@ -49,13 +49,15 @@ static enum fsmonitor_reason check_uds_volume(struct repository *r)return FSMONITOR_REASON_OK ; } -enum fsmonitor_reason fsm_os__incompatible (struct repository *r) +enum fsmonitor_reason fsm_os__incompatible (struct repository *r,int ipc) {enum fsmonitor_reason reason; - reason =check_uds_volume (r); -if (reason !=FSMONITOR_REASON_OK ) -return reason; +if (ipc) { + reason =check_uds_volume (r); +if (reason !=FSMONITOR_REASON_OK ) +return reason; + }return FSMONITOR_REASON_OK ; }compat/fsmonitor/fsm-settings-win32.c +1
-1@@ -25,7 +25,7 @@ static enum fsmonitor_reason check_vfs4git(struct repository *r)return FSMONITOR_REASON_OK ; } -enum fsmonitor_reason fsm_os__incompatible (struct repository *r) +enum fsmonitor_reason fsm_os__incompatible (struct repository *r,int ipc) {enum fsmonitor_reason reason;fsmonitor-settings.c +4
-4@@ -60,7 +60,7 @@ static enum fsmonitor_reason check_remote(struct repository *r) }#endif -static enum fsmonitor_reason check_for_incompatible (struct repository *r) +static enum fsmonitor_reason check_for_incompatible (struct repository *r,int ipc) {if (!r->worktree) {/* @@ -77,7 +77,7 @@ static enum fsmonitor_reason check_for_incompatible(struct repository *r) reason =check_remote (r);if (reason !=FSMONITOR_REASON_OK )return reason; - reason =fsm_os__incompatible (r); + reason =fsm_os__incompatible (r, ipc);if (reason !=FSMONITOR_REASON_OK )return reason; }@@ -162,7 +162,7 @@ const char *fsm_settings__get_hook_path(struct repository *r)void fsm_settings__set_ipc (struct repository *r) { -enum fsmonitor_reason reason =check_for_incompatible (r); +enum fsmonitor_reason reason =check_for_incompatible (r,1 );if (reason !=FSMONITOR_REASON_OK ) {fsm_settings__set_incompatible (r, reason);@@ -185,7 +185,7 @@ void fsm_settings__set_ipc(struct repository *r)void fsm_settings__set_hook (struct repository *r,const char *path) { -enum fsmonitor_reason reason =check_for_incompatible (r); +enum fsmonitor_reason reason =check_for_incompatible (r,0 );if (reason !=FSMONITOR_REASON_OK ) {fsm_settings__set_incompatible (r, reason);fsmonitor-settings.h +1
-1@@ -48,7 +48,7 @@ struct fsmonitor_settings;* fsm_os__* routines should considered private to fsm_settings__ * routines. */ -enum fsmonitor_reason fsm_os__incompatible (struct repository *r); +enum fsmonitor_reason fsm_os__incompatible (struct repository *r,int ipc);#endif /* HAVE_FSMONITOR_OS_SETTINGS */ #endif /* FSMONITOR_SETTINGS_H */
PATCH v6 5/6 fsmonitor: deal with synthetic firmlinks on macOS
In reply to Eric DeCosta via GitGitGadgetFrom: Eric DeCosta <[email protected]>
Starting with macOS 10.15 (Catalina), Apple introduced a new feature called 'firmlinks' in order to separate the boot volume into two volumes, one read-only and one writable but still present them to the user as a single volume. Along with this change, Apple removed the ability to create symlinks in the root directory and replaced them with 'synthetic firmlinks'. See 'man synthetic.conf'
When FSEevents reports the path of changed files, if the path involves a synthetic firmlink, the path is reported from the point of the synthetic firmlink and not the real path. For example:
Real path:
/System/Volumes/Data/network/working/directory/foo.txt
Synthetic firmlink:
/network -> /System/Volumes/Data/network
FSEvents path:
/network/working/directory/foo.txt
This causes the FSEvents path to not match against the worktree directory.
There are several ways in which synthetic firmlinks can be created:
they can be defined in /etc/synthetic.conf, the automounter can create them, and there may be other means. Simply reading /etc/synthetic.conf is insufficient. No matter what process creates synthetic firmlinks, they all get created in the root directory.
Therefore, in order to deal with synthetic firmlinks, the root directory is scanned and the first possible synthetic firmink that, when resolved, is a prefix of the worktree is used to map FSEvents paths to worktree paths.
--- builtin/fsmonitor--daemon.c | 8 +++ compat/fsmonitor/fsm-listen-darwin.c | 6 +- compat/fsmonitor/fsm-path-utils-darwin.c | 92 ++++++++++++++++++++++++ compat/fsmonitor/fsm-path-utils-win32.c | 17 +++++ fsmonitor--daemon.h | 6 ++ fsmonitor-path-utils.h | 36 ++++++++++ 6 files changed, 164 insertions(+), 1 deletion(-)
builtin/fsmonitor--daemon.c +8 -0
@@ -3,6 +3,7 @@#include "parse-options.h" #include "fsmonitor.h" #include "fsmonitor-ipc.h" +#include "fsmonitor-path-utils.h" #include "compat/fsmonitor/fsm-health.h" #include "compat/fsmonitor/fsm-listen.h" #include "fsmonitor--daemon.h"
@@ -1282,6 +1283,13 @@ static int fsmonitor_run_daemon(void)strbuf_addstr (&state.path_worktree_watch,absolute_path (get_git_work_tree ())); state.nr_paths_watching =1 ; + state.alias.alias =NULL ; + state.alias.points_to =NULL ; +if (fsmonitor__get_alias (state.path_worktree_watch.buf, &state.alias)) { + err =error (_ ("could not get worktree alias" )); + goto done; + } +/* * We create and delete cookie files somewhere inside the .git * directory to help us keep sync with the file system. If
compat/fsmonitor/fsm-listen-darwin.c +5 -1
@@ -26,6 +26,7 @@#include "fsmonitor.h" #include "fsm-listen.h" #include "fsmonitor--daemon.h" +#include "fsmonitor-path-utils.h" struct fsm_listen_data {
@@ -209,7 +210,9 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,/* * On Mac, we receive an array of absolute paths. */ - path_k = paths[k]; + path_k =fsmonitor__resolve_alias (paths[k], &state->alias); +if (!path_k) + path_k = paths[k];/* * If you want to debug FSEvents, log them to GIT_TRACE_FSMONITOR.
@@ -238,6 +241,7 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,fsmonitor_force_resync (state);fsmonitor_batch__free_list (batch);string_list_clear (&cookie_list,0 ); + batch =NULL ;/* * We assume that any events that we received
compat/fsmonitor/fsm-path-utils-darwin.c +92 -0
100 lines not on this page · raw message
compat/fsmonitor/fsm-path-utils-win32.c +17 -0
@@ -126,3 +126,20 @@ int fsmonitor__is_fs_remote(const char *path)return -1 ;return fs.is_remote; } + +/* +* No-op for now. +*/ +int fsmonitor__get_alias (const char *path,struct alias_info *info) +{ +return 0 ; +} + +/* +* No-op for now. +*/ +char *fsmonitor__resolve_alias (const char *path, +const struct alias_info *info) +{ +return NULL ; +}
fsmonitor--daemon.h +6 -0
@@ -8,6 +8,7 @@#include "run-command.h" #include "simple-ipc.h" #include "thread-utils.h" +#include "fsmonitor-path-utils.h" struct fsmonitor_batch ;struct fsmonitor_token_data ;
@@ -43,6 +44,7 @@ struct fsmonitor_daemon_state {struct strbuf path_worktree_watch;struct strbuf path_gitdir_watch; +struct alias_info alias;int nr_paths_watching;struct fsmonitor_token_data *current_token_data;
@@ -59,6 +61,7 @@ struct fsmonitor_daemon_state {struct ipc_server_data *ipc_server_data;struct strbuf path_ipc; + };/*
@@ -167,5 +170,8 @@ void fsmonitor_publish(struct fsmonitor_daemon_state *state,*/ void fsmonitor_force_resync (struct fsmonitor_daemon_state *state); +char *fsmonitor_resolve_alias (const char *path, +struct alias_info *alias); +#endif /* HAVE_FSMONITOR_DAEMON_BACKEND */ #endif /* FSMONITOR_DAEMON_H */
fsmonitor-path-utils.h +36 -0
@@ -1,6 +1,14 @@ #ifndef FSM_PATH_UTILS_H #define FSM_PATH_UTILS_H +#include "strbuf.h" + +struct alias_info +{ + char *alias; + char *points_to; +}; + struct fs_info { int is_remote; char *typename;
@@ -20,4 +28,32 @@ int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info);*/ int fsmonitor__is_fs_remote (const char *path); +/* +* Get the alias in given path, if any. +* +* Sets alias to the first alias that matches any part of the path. +* +* If an alias is found, info.alias and info.points_to are set to the +* found mapping. +* +* Returns -1 on error, 0 otherwise. +* +* The caller owns the storage that is occupied by set info.alias and +* info.points_to and is responsible for releasing it with `free(3)` +* when done. +*/ +int fsmonitor__get_alias (const char *path,struct alias_info *info); + +/* +* Resolve the path against the given alias. +* +* Returns the resolved path if there is one, NULL otherwise. +* +* The caller owns the storage that the returned string occupies and +* is responsible for releasing it with `free(3)` when done. +*/ +char *fsmonitor__resolve_alias (const char *path, +const struct alias_info *info); + +#endif
Signature · 1 line
gitgitgadget
PATCH v6 6/6 fsmonitor: add documentation for allowRemote and socketDir options
In reply to Eric DeCosta via GitGitGadgetFrom: Eric DeCosta <[email protected]>
Add documentation for 'fsmonitor.allowRemote' and 'fsmonitor.socketDir'. Call-out experimental nature of 'fsmonitor.allowRemote' and limited file system support for 'fsmonitor.socketDir'.
--- Documentation/git-fsmonitor--daemon.txt | 35 +++++++++++++++++++++++++ 1 file changed, 35 insertions(+)
Documentation/git-fsmonitor--daemon.txt +35 -0
@@ -70,6 +70,41 @@ the change (as happening against the super repo). However, the client will properly ignore these extra events, so performance may be affected but it will not cause an incorrect result. +By default, the fsmonitor daemon refuses to work against network-mounted +repositories; this my be overridden by setting `fsmonitor.allowRemote` to +`true`. Note, however, that the fsmonitor daemon is not guaranteed to work +correctly with all network-mounted repositores and such use is considered +experimental. + +On Mac OS, the inter-process communication (IPC) between various Git +commands and the fsmonitor daemon is done via a Unix domain socket (UDS). +Usage of UDS requires the creation of a file which, by default, is created +in the .git directory. If the fsmonitor daemon detects that the .git directory +is on a network-mounted file system, it will create the UDS file in $HOME. If +$HOME itself is on a network-mounted file system or if $HOME is not the desired +location for the UDS file, 'fsmonitor.socketDir' may be set to any valid, local +directory on a file system with proper support. Mac OS native file systems have +the required support. File systems known to lack support include FAT32 and +NTFS. Other file systems may or many not have the needed support; the fsmonitor +daemon is not guaranteed to work with these file systems and such use is +considered experimental. + +CONFIGURATION +------------- +When `core.fsmonitor` is set to `true` (see linkgit:git-config[1]) +the fsmonitor daemon will pay attention to the following configuration +variables: + +fsmonitor.allowRemote:: + By default, the daemon refuses to work against network-mounted + repositories. Setting `fsmonitor.allowRemote` to `true` overrides + this behavior. + +fsmonitor.socketDir:: + This option is only used by the Mac OS implementation of the fsmonitor + daemon. If set, 'fsmonitor.socketDir' must be set to a valid, local + directory on a file system that can support Unix domain sockets (UDS). + GIT --- Part of the linkgit:git[1] suite
Signature · 1 line
gitgitgadget
Quoted 30 lines · Eric DeCosta via GitGitGadget
Follow-on to the work done to allow Windows to work against network-mounted repos for macOS.
Have macOS take advantage of the same configuration option, 'fsmonitor.allowRemote' that was introduced for Windows. Setting this option to true will override the default behavior (erroring-out) when a network-mounted repo is detected by fsmonitor.
The added wrinkle being that the Unix domain socket (UDS) file used for IPC cannot be created in a network location; instead $HOME is used if the default location is on the network. The user may, optionally, set the 'fsmonitor.socketDir' configuration option to a valid, local directory if $HOME itself is on the network or is simply not the desired location for the UDS file.
An additional issue is that for mount points in the root directory, FSEvents does not report a path that matches the worktree directory due to the introduction of 'synthetic firmlinks'. fsmonitor must map the FSEvents paths to the worktree directory by interrogating the root filesystem for synthetic firmlinks and using that information to translate the path.
v6 differs from v5:
* incorporates earlier, Windows-specific changes that have not made it back yet to the master branch
* incorporates code review feedback
* adds documentation for 'fsmonitor.allowRemote' and 'fsmonitor.socketDir'v5 differs significantly from earlier versions:
Quoted 8 lines
* redesign of handling 'fsmonitor.allowRemote' and 'fsmonitor.socketDir' such that these options are no longer added to the settings data structure but are rather read from config at point of use
* refactoring of code for handling platform-specific file system checks via a common interface to avoid platform #ifdef in IPC code and be in-model with other platform-specific fsmonitor code
* dealing with 'synthetic firmlinks' on macOS
I've looked at v5 and v6 and I like the direction this is heading, so I'll mark this LGTM. (I'm still traveling back from Git Merge, so I haven't had time to test it out, but I think we should proceed with it.)
Thanks for you patience and attention to detail on this! Jeff
Quoted 59 lines
-----Original Message-----
From: Jeff Hostetler <[email protected]>
Sent: Friday, September 16, 2022 1:59 PM
To: Eric DeCosta via GitGitGadget <[email protected]>; [email protected]
Cc: Eric Sunshine <[email protected]>; Torsten Bögershausen <[email protected]>; Ævar Arnfjörð Bjarmason <[email protected]>; Ramsay Jones <[email protected]>; Johannes Schindelin <[email protected]>; Eric DeCosta <[email protected]>
Subject: Re: [PATCH v6 0/6] fsmonitor: option to allow fsmonitor to run against network-mounted reposQuoted 15 lines · Eric DeCosta via GitGitGadget
Follow-on to the work done to allow Windows to work against network-mounted repos for macOS.
Have macOS take advantage of the same configuration option, 'fsmonitor.allowRemote' that was introduced for Windows. Setting this option to true will override the default behavior (erroring-out) when a network-mounted repo is detected by fsmonitor.
The added wrinkle being that the Unix domain socket (UDS) file used for IPC cannot be created in a network location; instead $HOME is used if the default location is on the network. The user may, optionally, set the 'fsmonitor.socketDir' configuration option to a valid, local directory if $HOME itself is on the network or is simply not the desired location for the UDS file.
Quoted 5 lines
An additional issue is that for mount points in the root directory, FSEvents does not report a path that matches the worktree directory due to the introduction of 'synthetic firmlinks'. fsmonitor must map the FSEvents paths to the worktree directory by interrogating the root filesystem for synthetic firmlinks and using that information to translate the
path.
Quoted 4 lines
v6 differs from v5:
* incorporates earlier, Windows-specific changes that have not made it
back
Quoted 3 lines
yet to the master branch
* incorporates code review feedback
* adds documentation for 'fsmonitor.allowRemote' and'fsmonitor.socketDir'
Quoted 11 lines
v5 differs significantly from earlier versions:
* redesign of handling 'fsmonitor.allowRemote' and 'fsmonitor.socketDir' such that these options are no longer added to the settings data structure but are rather read from config at point of use
* refactoring of code for handling platform-specific file system checks via a common interface to avoid platform #ifdef in IPC code and be in-model with other platform-specific fsmonitor code
* dealing with 'synthetic firmlinks' on macOS
Quoted 6 lines
I've looked at v5 and v6 and I like the direction this is heading, so I'll mark this LGTM. (I'm still traveling back from Git Merge, so I haven't had time to test it out, but I think we should proceed with it.)
Thanks for you patience and attention to detail on this! Jeff
Great! I just created v7 with some small updates based on Junio's feedback.
I think this landed in a pretty good place. I don't know that the plans are for fsmonitor on Linux, but I've already started tinkering around with an inotify-based implementation for that platform. The devil is in the details, of course, but it looks fairly straight-forward.
-Eric
v7 cover fsmonitor: option to allow fsmonitor to run against network-mounted repos
In reply to Eric DeCosta via GitGitGadgetFollow-on to the work done to allow Windows to work against network-mounted repos for macOS.
Have macOS take advantage of the same configuration option, 'fsmonitor.allowRemote' that was introduced for Windows. Setting this option to true will override the default behavior (erroring-out) when a network-mounted repo is detected by fsmonitor.
The added wrinkle being that the Unix domain socket (UDS) file used for IPC cannot be created in a network location; instead $HOME is used if the default location is on the network. The user may, optionally, set the 'fsmonitor.socketDir' configuration option to a valid, local directory if $HOME itself is on the network or is simply not the desired location for the UDS file.
An additional issue is that for mount points in the root directory, FSEvents does not report a path that matches the worktree directory due to the introduction of 'synthetic firmlinks'. fsmonitor must map the FSEvents paths to the worktree directory by interrogating the root filesystem for synthetic firmlinks and using that information to translate the path.
v7 differs from v6:
* incorporates code review feedback
v6 differs from v5:
* incorporates earlier, Windows-specific changes that have not made it back yet to the master branch
* incorporates code review feedback
* adds documentation for 'fsmonitor.allowRemote' and 'fsmonitor.socketDir'
v5 differs significantly from earlier versions:
* redesign of handling 'fsmonitor.allowRemote' and 'fsmonitor.socketDir' such that these options are no longer added to the settings data structure but are rather read from config at point of use
* refactoring of code for handling platform-specific file system checks via a common interface to avoid platform #ifdef in IPC code and be in-model with other platform-specific fsmonitor code
* dealing with 'synthetic firmlinks' on macOS
Eric DeCosta (6):
fsmonitor: refactor filesystem checks to common interface
fsmonitor: relocate socket file if .git directory is remote
fsmonitor: avoid socket location check if using hook
fsmonitor: deal with synthetic firmlinks on macOS
fsmonitor: deal with synthetic firmlinks on macOS
fsmonitor: add documentation for allowRemote and socketDir options
Documentation/git-fsmonitor--daemon.txt | 35 +++++ Makefile | 2 + builtin/fsmonitor--daemon.c | 11 +- compat/fsmonitor/fsm-ipc-darwin.c | 49 +++++++ compat/fsmonitor/fsm-ipc-win32.c | 9 ++ compat/fsmonitor/fsm-listen-darwin.c | 6 +- compat/fsmonitor/fsm-path-utils-darwin.c | 132 +++++++++++++++++ compat/fsmonitor/fsm-path-utils-win32.c | 145 +++++++++++++++++++ compat/fsmonitor/fsm-settings-darwin.c | 70 +++------ compat/fsmonitor/fsm-settings-win32.c | 174 +---------------------- contrib/buildsystems/CMakeLists.txt | 4 + fsmonitor--daemon.h | 6 + fsmonitor-ipc.c | 18 ++- fsmonitor-ipc.h | 4 +- fsmonitor-path-utils.h | 59 ++++++++ fsmonitor-settings.c | 58 +++++++- fsmonitor-settings.h | 2 +- 17 files changed, 547 insertions(+), 237 deletions(-) create mode 100644 compat/fsmonitor/fsm-ipc-darwin.c create mode 100644 compat/fsmonitor/fsm-ipc-win32.c create mode 100644 compat/fsmonitor/fsm-path-utils-darwin.c create mode 100644 compat/fsmonitor/fsm-path-utils-win32.c create mode 100644 fsmonitor-path-utils.h
base-commit: d3fa443f97e3a8d75b51341e2d5bac380b7422df
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1326%2Fedecosta-mw%2Ffsmonitor_macos-v7
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1326/edecosta-mw/fsmonitor_macos-v7
Pull-Request: https://github.com/gitgitgadget/git/pull/1326
Range-diff vs v6:
1: 3233c908c4a < -: ----------- fsmonitor: option to allow fsmonitor to run against network-mounted repos
2: d2a8fc6b707 = 1: 155a6890806 fsmonitor: refactor filesystem checks to common interface
3: edef029a298 ! 2: 075340bd2a7 fsmonitor: relocate socket file if .git directory is remote
@@ Makefile: ifdef FSMONITOR_DAEMON_BACKEND
ifdef FSMONITOR_OS_SETTINGS
+ ## builtin/fsmonitor--daemon.c ##
+@@ builtin/fsmonitor--daemon.c: static int fsmonitor_run_daemon(void)
+ * directory.)
+ */
+ strbuf_init(&state.path_ipc, 0);
+- strbuf_addstr(&state.path_ipc, absolute_path(fsmonitor_ipc__get_path()));
++ strbuf_addstr(&state.path_ipc,
++ absolute_path(fsmonitor_ipc__get_path(the_repository)));
+
+ /*
+ * Confirm that we can create platform-specific resources for the
+
## compat/fsmonitor/fsm-ipc-darwin.c (new) ##
@@
+#include "cache.h"
@@ compat/fsmonitor/fsm-ipc-darwin.c (new)
+
+static GIT_PATH_FUNC(fsmonitor_ipc__get_default_path, "fsmonitor--daemon.ipc")
+
-+const char *fsmonitor_ipc__get_path(void)
++const char *fsmonitor_ipc__get_path(struct repository *r)
+{
+ static const char *ipc_path;
+ SHA_CTX sha1ctx;
@@ compat/fsmonitor/fsm-ipc-darwin.c (new)
+ if (ipc_path)
+ return ipc_path;
+
++ if (!r)
++ r = the_repository;
++
+ ipc_path = fsmonitor_ipc__get_default_path();
+
+ /* By default the socket file is created in the .git directory */
@@ compat/fsmonitor/fsm-ipc-darwin.c (new)
+ return ipc_path;
+
+ SHA1_Init(&sha1ctx);
-+ SHA1_Update(&sha1ctx, the_repository->worktree, strlen(the_repository->worktree));
++ SHA1_Update(&sha1ctx, r->worktree, strlen(r->worktree));
+ SHA1_Final(hash, &sha1ctx);
+
-+ repo_config_get_string(the_repository, "fsmonitor.socketdir", &sock_dir);
++ repo_config_get_string(r, "fsmonitor.socketdir", &sock_dir);
+
+ /* Create the socket file in either socketDir or $HOME */
+ if (sock_dir && *sock_dir)
@@ compat/fsmonitor/fsm-ipc-darwin.c (new)
## compat/fsmonitor/fsm-ipc-win32.c (new) ##
@@
-+#include "cache.h"
++#include "config.h"
+#include "fsmonitor-ipc.h"
+
-+GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc")
++const char *fsmonitor_ipc__get_path(struct repository *r) {
++ static char *ret;
++ if (!ret)
++ ret = git_pathdup("fsmonitor--daemon.ipc");
++ return ret;
++}
+ \ No newline at end of file
+
+ ## compat/fsmonitor/fsm-settings-darwin.c ##
+@@
+ static enum fsmonitor_reason check_uds_volume(struct repository *r)
+ {
+ struct fs_info fs;
+- const char *ipc_path = fsmonitor_ipc__get_path();
++ const char *ipc_path = fsmonitor_ipc__get_path(r);
+ struct strbuf path = STRBUF_INIT;
+ strbuf_add(&path, ipc_path, strlen(ipc_path));
+
## contrib/buildsystems/CMakeLists.txt ##
@@ contrib/buildsystems/CMakeLists.txt: if(SUPPORTS_SIMPLE_IPC)
@@ contrib/buildsystems/CMakeLists.txt: if(SUPPORTS_SIMPLE_IPC)
add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
## fsmonitor-ipc.c ##
+@@ fsmonitor-ipc.c: int fsmonitor_ipc__is_supported(void)
+ return 0;
+ }
+
+-const char *fsmonitor_ipc__get_path(void)
++const char *fsmonitor_ipc__get_path(struct repository *r)
+ {
+ return NULL;
+ }
@@ fsmonitor-ipc.c: int fsmonitor_ipc__is_supported(void)
return 1;
}
@@ fsmonitor-ipc.c: int fsmonitor_ipc__is_supported(void)
-
enum ipc_active_state fsmonitor_ipc__get_state(void)
{
- return ipc_get_active_state(fsmonitor_ipc__get_path());
+- return ipc_get_active_state(fsmonitor_ipc__get_path());
++ return ipc_get_active_state(fsmonitor_ipc__get_path(the_repository));
+ }
+
+ static int spawn_daemon(void)
+@@ fsmonitor-ipc.c: int fsmonitor_ipc__send_query(const char *since_token,
+ trace2_data_string("fsm_client", NULL, "query/command", tok);
+
+ try_again:
+- state = ipc_client_try_connect(fsmonitor_ipc__get_path(), &options,
+- &connection);
++ state = ipc_client_try_connect(fsmonitor_ipc__get_path(the_repository),
++ &options, &connection);
+
+ switch (state) {
+ case IPC_STATE__LISTENING:
+@@ fsmonitor-ipc.c: try_again:
+
+ case IPC_STATE__INVALID_PATH:
+ ret = error(_("fsmonitor_ipc__send_query: invalid path '%s'"),
+- fsmonitor_ipc__get_path());
++ fsmonitor_ipc__get_path(the_repository));
+ goto done;
+
+ case IPC_STATE__OTHER_ERROR:
+ default:
+ ret = error(_("fsmonitor_ipc__send_query: unspecified error on '%s'"),
+- fsmonitor_ipc__get_path());
++ fsmonitor_ipc__get_path(the_repository));
+ goto done;
+ }
+
+@@ fsmonitor-ipc.c: int fsmonitor_ipc__send_command(const char *command,
+ options.wait_if_busy = 1;
+ options.wait_if_not_found = 0;
+
+- state = ipc_client_try_connect(fsmonitor_ipc__get_path(), &options,
+- &connection);
++ state = ipc_client_try_connect(fsmonitor_ipc__get_path(the_repository),
++ &options, &connection);
+ if (state != IPC_STATE__LISTENING) {
+ die(_("fsmonitor--daemon is not running"));
+ return -1;
+
+ ## fsmonitor-ipc.h ##
+@@
+
+ #include "simple-ipc.h"
+
++struct repository;
++
+ /*
+ * Returns true if built-in file system monitor daemon is defined
+ * for this platform.
+@@ fsmonitor-ipc.h: int fsmonitor_ipc__is_supported(void);
+ *
+ * Returns NULL if the daemon is not supported on this platform.
+ */
+-const char *fsmonitor_ipc__get_path(void);
++const char *fsmonitor_ipc__get_path(struct repository *r);
+
+ /*
+ * Try to determine whether there is a `git-fsmonitor--daemon` process
4: 3428bcf8763 = 3: 5518d2f3e03 fsmonitor: avoid socket location check if using hook
5: 9c1f408ae6d ! 4: 3a9fe473cf4 fsmonitor: deal with synthetic firmlinks on macOS @@ fsmonitor--daemon.h: struct fsmonitor_daemon_state {
};
/*
-@@ fsmonitor--daemon.h: void fsmonitor_publish(struct fsmonitor_daemon_state *state,
- */
- void fsmonitor_force_resync(struct fsmonitor_daemon_state *state);
-
-+char *fsmonitor_resolve_alias(const char *path,
-+ struct alias_info *alias);
-+
- #endif /* HAVE_FSMONITOR_DAEMON_BACKEND */
- #endif /* FSMONITOR_DAEMON_H */
## fsmonitor-path-utils.h ##
@@
-: ----------- > 5: 4d00adb1deb fsmonitor: deal with synthetic firmlinks on macOS 6: d2c95e34d3a = 6: 260591f5820 fsmonitor: add documentation for allowRemote and socketDir options
Signature · 1 line
gitgitgadget
PATCH v7 1/6 fsmonitor: refactor filesystem checks to common interface
From: Eric DeCosta <[email protected]>
Provide a common interface for getting basic filesystem information including filesystem type and whether the filesystem is remote.
Refactor existing code for getting basic filesystem info and detecting remote file systems to the new interface.
Refactor filesystem checks to leverage new interface. For macOS, error-out if the Unix Domain socket (UDS) file is on a remote filesystem.
--- Makefile | 1 + compat/fsmonitor/fsm-path-utils-darwin.c | 40 ++++++ compat/fsmonitor/fsm-path-utils-win32.c | 128 +++++++++++++++++ compat/fsmonitor/fsm-settings-darwin.c | 62 +++----- compat/fsmonitor/fsm-settings-win32.c | 172 +---------------------- contrib/buildsystems/CMakeLists.txt | 2 + fsmonitor-path-utils.h | 23 +++ fsmonitor-settings.c | 50 +++++++ 8 files changed, 263 insertions(+), 215 deletions(-) create mode 100644 compat/fsmonitor/fsm-path-utils-darwin.c create mode 100644 compat/fsmonitor/fsm-path-utils-win32.c create mode 100644 fsmonitor-path-utils.h
Makefile +1 -0
@@ -2039,6 +2039,7 @@ endif ifdef FSMONITOR_OS_SETTINGS COMPAT_CFLAGS += -DHAVE_FSMONITOR_OS_SETTINGS COMPAT_OBJS += compat/fsmonitor/fsm-settings-$(FSMONITOR_OS_SETTINGS).o + COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_OS_SETTINGS).o endif ifeq ($(TCLTK_PATH),)
compat/fsmonitor/fsm-path-utils-darwin.c new file +40 -0
@@ -0,0 +1,40 @@ +#include "fsmonitor.h" +#include "fsmonitor-path-utils.h" +#include <sys/param.h> +#include <sys/mount.h> + +int fsmonitor__get_fs_info (const char *path,struct fs_info *fs_info) +{ +struct statfs fs; +if (statfs (path, &fs) ==-1 ) { +int saved_errno = errno; +trace_printf_key (&trace_fsmonitor,"statfs('%s') failed: %s" , + path,strerror (saved_errno)); + errno = saved_errno; +return -1 ; + } + +trace_printf_key (&trace_fsmonitor, +"statfs('%s') [type 0x%08x][flags 0x%08x] '%s'" , + path, fs.f_type, fs.f_flags, fs.f_fstypename); + +if (!(fs.f_flags &MNT_LOCAL )) + fs_info->is_remote =1 ; +else + fs_info->is_remote =0 ; + + fs_info->typename = fs.f_fstypename; + +trace_printf_key (&trace_fsmonitor, +"'%s' is_remote: %d" , + path, fs_info->is_remote); +return 0 ; +} + +int fsmonitor__is_fs_remote (const char *path) +{ +struct fs_info fs; +if (fsmonitor__get_fs_info (path, &fs)) +return -1 ; +return fs.is_remote; +}
compat/fsmonitor/fsm-path-utils-win32.c new file +128 -0
128 lines not on this page · raw message
compat/fsmonitor/fsm-settings-darwin.c +17 -45
96 lines not on this page · raw message
compat/fsmonitor/fsm-settings-win32.c +2 -170
190 lines not on this page · raw message
contrib/buildsystems/CMakeLists.txt +2 -0
@@ -308,6 +308,7 @@ if(SUPPORTS_SIMPLE_IPC) add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-win32.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-win32.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-win32.c)
@@ -315,6 +316,7 @@ if(SUPPORTS_SIMPLE_IPC) add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-darwin.c)
fsmonitor-path-utils.h new file +23 -0
@@ -0,0 +1,23 @@ +#ifndef FSM_PATH_UTILS_H +#define FSM_PATH_UTILS_H + +struct fs_info { +int is_remote; +char *typename; +}; + +/* +* Get some basic filesystem informtion for the given path +* +* Returns -1 on error, zero otherwise. +*/ +int fsmonitor__get_fs_info (const char *path,struct fs_info *fs_info); + +/* +* Determines if the filesystem that path resides on is remote. +* +* Returns -1 on error, 0 if not remote, 1 if remote. +*/ +int fsmonitor__is_fs_remote (const char *path); + +#endif
fsmonitor-settings.c +50 -0
68 lines not on this page · raw message
Signature · 1 line
gitgitgadget
PATCH v7 2/6 fsmonitor: relocate socket file if .git directory is remote
In reply to Eric DeCosta via GitGitGadgetFrom: Eric DeCosta <[email protected]>
If the .git directory is on a remote file system, create the socket file in 'fsmonitor.socketDir' if it is defined, else create it in $HOME.
--- Makefile | 1 + builtin/fsmonitor--daemon.c | 3 +- compat/fsmonitor/fsm-ipc-darwin.c | 49 ++++++++++++++++++++++++++ compat/fsmonitor/fsm-ipc-win32.c | 9 +++++ compat/fsmonitor/fsm-settings-darwin.c | 2 +- contrib/buildsystems/CMakeLists.txt | 2 ++ fsmonitor-ipc.c | 18 +++++----- fsmonitor-ipc.h | 4 ++- 8 files changed, 75 insertions(+), 13 deletions(-) create mode 100644 compat/fsmonitor/fsm-ipc-darwin.c create mode 100644 compat/fsmonitor/fsm-ipc-win32.c
Makefile +1 -0
@@ -2034,6 +2034,7 @@ ifdef FSMONITOR_DAEMON_BACKEND COMPAT_CFLAGS += -DHAVE_FSMONITOR_DAEMON_BACKEND COMPAT_OBJS += compat/fsmonitor/fsm-listen-$(FSMONITOR_DAEMON_BACKEND).o COMPAT_OBJS += compat/fsmonitor/fsm-health-$(FSMONITOR_DAEMON_BACKEND).o + COMPAT_OBJS += compat/fsmonitor/fsm-ipc-$(FSMONITOR_DAEMON_BACKEND).o endif ifdef FSMONITOR_OS_SETTINGS
builtin/fsmonitor--daemon.c +2 -1
@@ -1343,7 +1343,8 @@ static int fsmonitor_run_daemon(void)* directory.) */ strbuf_init (&state.path_ipc,0 ); -strbuf_addstr (&state.path_ipc,absolute_path (fsmonitor_ipc__get_path ())); +strbuf_addstr (&state.path_ipc, +absolute_path (fsmonitor_ipc__get_path (the_repository)));/* * Confirm that we can create platform-specific resources for the
compat/fsmonitor/fsm-ipc-darwin.c new file +49 -0
49 lines not on this page · raw message
compat/fsmonitor/fsm-ipc-win32.c new file +9 -0
@@ -0,0 +1,9 @@ +#include "config.h" +#include "fsmonitor-ipc.h" + +const char *fsmonitor_ipc__get_path (struct repository *r) { +static char *ret; +if (!ret) + ret =git_pathdup ("fsmonitor--daemon.ipc" ); +return ret; +}
compat/fsmonitor/fsm-settings-darwin.c +1 -1
@@ -26,7 +26,7 @@static enum fsmonitor_reason check_uds_volume (struct repository *r) {struct fs_info fs; -const char *ipc_path =fsmonitor_ipc__get_path (); +const char *ipc_path =fsmonitor_ipc__get_path (r);struct strbuf path =STRBUF_INIT ;strbuf_add (&path, ipc_path,strlen (ipc_path));
contrib/buildsystems/CMakeLists.txt +2 -0
@@ -308,6 +308,7 @@ if(SUPPORTS_SIMPLE_IPC) add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-win32.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-win32.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-win32.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
@@ -316,6 +317,7 @@ if(SUPPORTS_SIMPLE_IPC) add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-darwin.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
fsmonitor-ipc.c +8 -10
@@ -18,7 +18,7 @@ int fsmonitor_ipc__is_supported(void)return 0 ; } -const char *fsmonitor_ipc__get_path (void ) +const char *fsmonitor_ipc__get_path (struct repository *r) {return NULL ; }
@@ -47,11 +47,9 @@ int fsmonitor_ipc__is_supported(void)return 1 ; } -GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc") -enum ipc_active_state fsmonitor_ipc__get_state (void ) { -return ipc_get_active_state (fsmonitor_ipc__get_path ()); +return ipc_get_active_state (fsmonitor_ipc__get_path (the_repository)); }static int spawn_daemon (void )
@@ -81,8 +79,8 @@ int fsmonitor_ipc__send_query(const char *since_token,trace2_data_string ("fsm_client" ,NULL ,"query/command" , tok); try_again: - state =ipc_client_try_connect (fsmonitor_ipc__get_path (), &options, - &connection); + state =ipc_client_try_connect (fsmonitor_ipc__get_path (the_repository), + &options, &connection);switch (state) {case IPC_STATE__LISTENING :
@@ -117,13 +115,13 @@ try_again:case IPC_STATE__INVALID_PATH : ret =error (_ ("fsmonitor_ipc__send_query: invalid path '%s'" ), -fsmonitor_ipc__get_path ()); +fsmonitor_ipc__get_path (the_repository)); goto done;case IPC_STATE__OTHER_ERROR :default : ret =error (_ ("fsmonitor_ipc__send_query: unspecified error on '%s'" ), -fsmonitor_ipc__get_path ()); +fsmonitor_ipc__get_path (the_repository)); goto done; }
@@ -149,8 +147,8 @@ int fsmonitor_ipc__send_command(const char *command, options.wait_if_busy =1 ; options.wait_if_not_found =0 ; - state =ipc_client_try_connect (fsmonitor_ipc__get_path (), &options, - &connection); + state =ipc_client_try_connect (fsmonitor_ipc__get_path (the_repository), + &options, &connection);if (state !=IPC_STATE__LISTENING ) {die (_ ("fsmonitor--daemon is not running" ));return -1 ;
fsmonitor-ipc.h +3 -1
@@ -3,6 +3,8 @@#include "simple-ipc.h" +struct repository ; +/* * Returns true if built-in file system monitor daemon is defined * for this platform.
@@ -16,7 +18,7 @@ int fsmonitor_ipc__is_supported(void);* * Returns NULL if the daemon is not supported on this platform. */ -const char *fsmonitor_ipc__get_path (void ); +const char *fsmonitor_ipc__get_path (struct repository *r);/* * Try to determine whether there is a `git-fsmonitor--daemon` process
Signature · 1 line
gitgitgadget
Re: [PATCH v7 2/6] fsmonitor: relocate socket file if .git directory is remote
compat/fsmonitor/fsm-ipc-darwin.c line 22
Quoted 13 lines · Eric DeCosta via GitGitGadget
compat/fsmonitor/fsm-ipc-darwin.c new file +13
-0+const char *fsmonitor_ipc__get_path (struct repository *r) +{ +static const char *ipc_path; +SHA_CTX sha1ctx; +char *sock_dir; +struct strbuf ipc_file =STRBUF_INIT ; +unsigned char hash[SHA_DIGEST_LENGTH ]; + +if (ipc_path) +return ipc_path; + +if (!r) + r = the_repository;
I'd prefer not to see this "NULL means the_repository". It would be a different story if the caller does not necessarily have a ready access to the_repository, but it is a global, so the caller can pass the_repository and be more explicit. Giving two ways to the caller to express same thing is not a good idea.
Thanks.
Re: [PATCH v7 2/6] fsmonitor: relocate socket file if .git directory is remote
Quoted 22 lines · Junio C Hamano
compat/fsmonitor/fsm-ipc-darwin.c line 22
Quoted 13 lines · Eric DeCosta via GitGitGadget
compat/fsmonitor/fsm-ipc-darwin.c new file +13
-0+const char *fsmonitor_ipc__get_path (struct repository *r) +{ +static const char *ipc_path; +SHA_CTX sha1ctx; +char *sock_dir; +struct strbuf ipc_file =STRBUF_INIT ; +unsigned char hash[SHA_DIGEST_LENGTH ]; + +if (ipc_path) +return ipc_path; + +if (!r) + r = the_repository;I'd prefer not to see this "NULL means
the_repository". It would be a different story if the caller does not necessarily have a ready access tothe_repository, but it is a global, so the caller can passthe_repositoryand be more explicit. Giving two ways to the caller to express same thing is not a good idea.
Quoted 2 lines
Thanks.
To be fair, I added several "if (!r) r = the_repository;" statements to the original public FSMonitor routines. There were obscure cases where tests would sometimes randomly fail because "r" wasn't completely passed down via some hard to isolate call stack. Offlist, AEvar told me that he managed to isolate it and has a fix.
So eventually, we'll be able to get rid of all of these direct references to "the_repository" and properly assume that "r" is always passed down.
But for now, I think we should let this stay for safety.
Jeff
Re: [PATCH v7 2/6] fsmonitor: relocate socket file if .git directory is remote
Quoted 30 lines · Jeff Hostetler
Quoted 13 lines · Junio C Hamano
Quoted 11 lines · Eric DeCosta via GitGitGadget
compat/fsmonitor/fsm-ipc-darwin.c new file +11
-0+const char *fsmonitor_ipc__get_path (struct repository *r) +{ +static const char *ipc_path; +SHA_CTX sha1ctx; +char *sock_dir; +struct strbuf ipc_file =STRBUF_INIT ; +unsigned char hash[SHA_DIGEST_LENGTH ]; + +if (ipc_path) +return ipc_path; +Quoted 9 lines
compat/fsmonitor/fsm-ipc-darwin.c line 22
Quoted 2 lines
compat/fsmonitor/fsm-ipc-darwin.c new file +2
-0+if (!r) + r = the_repository;I'd prefer not to see this "NULL means
the_repository". It would be a different story if the caller does not necessarily have a ready access tothe_repository, but it is a global, so the caller can passthe_repositoryand be more explicit. Giving two ways to the caller to express same thing is not a good idea.
Thanks.To be fair, I added several "if (!r) r =
the_repository;" statements to the original public FSMonitor routines. There were obscure cases where tests would sometimes randomly fail because "r" wasn't completely passed down via some hard to isolate call stack. Offlist, AEvar told me that he managed to isolate it and has a fix.
Quoted 5 lines
So eventually, we'll be able to get rid of all of these direct references to "
the_repository" and properly assume that "r" is always passed down.But for now, I think we should let this stay for safety.
I wouldn't call "sweeping a breakage under the rug" a "safety", though. If the caller cannot decide which repository instance is the right thing to pass, or the caller does not yet have a good one to pass when making a call down the codepath, how can it be safer to use the_repository that may or may not be the appropriate one than noticing the problem by dying and stopping the spread of damage?
Re: [PATCH v7 2/6] fsmonitor: relocate socket file if .git directory is remote
Quoted 36 lines · Junio C Hamano
Quoted 34 lines · Jeff Hostetler
Quoted 13 lines · Junio C Hamano
Quoted 11 lines · Eric DeCosta via GitGitGadget
compat/fsmonitor/fsm-ipc-darwin.c new file +11
-0+const char *fsmonitor_ipc__get_path (struct repository *r) +{ +static const char *ipc_path; +SHA_CTX sha1ctx; +char *sock_dir; +struct strbuf ipc_file =STRBUF_INIT ; +unsigned char hash[SHA_DIGEST_LENGTH ]; + +if (ipc_path) +return ipc_path; +Quoted 9 lines
compat/fsmonitor/fsm-ipc-darwin.c line 22
Quoted 2 lines
compat/fsmonitor/fsm-ipc-darwin.c new file +2
-0+if (!r) + r = the_repository;I'd prefer not to see this "NULL means
the_repository". It would be a different story if the caller does not necessarily have a ready access tothe_repository, but it is a global, so the caller can passthe_repositoryand be more explicit. Giving two ways to the caller to express same thing is not a good idea.
Thanks.To be fair, I added several "if (!r) r =
the_repository;" statements to the original public FSMonitor routines. There were obscure cases where tests would sometimes randomly fail because "r" wasn't completely passed down via some hard to isolate call stack. Offlist, AEvar told me that he managed to isolate it and has a fix.So eventually, we'll be able to get rid of all of these direct references to "
the_repository" and properly assume that "r" is always passed down.
Quoted 9 lines
Quoted 1 line
But for now, I think we should let this stay for safety.
I wouldn't call "sweeping a breakage under the rug" a "safety", though. If the caller cannot decide which repository instance is the right thing to pass, or the caller does not yet have a good one to pass when making a call down the codepath, how can it be safer to use
the_repositorythat may or may not be the appropriate one than noticing the problem by dying and stopping the spread of damage?
Aren't we in the middle of a transition from always using the global "the_repository" to a passed "r" variable? We're getting closer to being able to hide the the global symbol, but we're not there yet, right?
I'm thinking that at as long as the global exists, we are not safe to have multiple "struct repository" instances, right?
All I'm saying is that there are obscure/edge code paths where a valid "r" is not being passed down. Or, more likely, someone has an "istate" that doesn't yet have "istate->repo" set at the point of the call, so they might be passing "istate->repo", but it is null.
Tracking down these nulls is important, but shouldn't it be independent of this one?
Jeff
Re: [PATCH v7 2/6] fsmonitor: relocate socket file if .git directory is remote
Quoted 4 lines · Jeff Hostetler
Aren't we in the middle of a transition from always using the global "
the_repository" to a passed "r" variable? We're getting closer to being able to hide the the global symbol, but we're not there yet, right?
We may still have code that works ONLY on the_repository, but letting a function take "r" and lettin it ignore is worse than leaving it explicitly limited to the_repository only, no?
Quoted 2 lines
I'm thinking that at as long as the global exists, we are not safe to have multiple "struct repository" instances, right?
By itself, Not at all. It is the code like I am criticizing that makes it unsafe.
I do not mind adding
if (!r) BUG(...);
at the place you have the "sweep it under the rug" band-aid, though.
RE: [PATCH v7 2/6] fsmonitor: relocate socket file if .git directory is remote
Quoted 29 lines
-----Original Message-----
From: Junio C Hamano <[email protected]>
Sent: Monday, September 19, 2022 1:49 PM
To: Jeff Hostetler <[email protected]>
Cc: Eric DeCosta via GitGitGadget <[email protected]>; [email protected]; Eric Sunshine <[email protected]>; Torsten Bögershausen <[email protected]>; Ævar Arnfjörð Bjarmason <[email protected]>; Ramsay Jones <[email protected]>; Johannes Schindelin <[email protected]>; Eric DeCosta <[email protected]>
Subject: Re: [PATCH v7 2/6] fsmonitor: relocate socket file if .git directory is remoteQuoted 4 lines · Jeff Hostetler
Aren't we in the middle of a transition from always using the global "
the_repository" to a passed "r" variable?
We're getting closer to being able to hide the the global symbol, but we're not there yet, right?We may still have code that works ONLY on
the_repository, but letting a function take "r" and lettin it ignore is worse than leaving it explicitly limited tothe_repositoryonly, no?Quoted 2 lines
I'm thinking that at as long as the global exists, we are not safe to have multiple "struct repository" instances, right?
Quoted 8 lines
By itself, Not at all. It is the code like I am criticizing that makes it unsafe.
I do not mind adding
if (!r) BUG(...);at the place you have the "sweep it under the rug" band-aid, though.
Appreciate all the insights and comments. Where are we landing with this? Very close to the finish line and I'd like to be able to push these changes over that line.
-Eric
Re: [PATCH v7 2/6] fsmonitor: relocate socket file if .git directory is remote
Quoted 33 lines · Eric DeCosta
Quoted 33 lines
-----Original Message-----
From: Junio C Hamano <[email protected]>
Sent: Monday, September 19, 2022 1:49 PM
To: Jeff Hostetler <[email protected]>
Cc: Eric DeCosta via GitGitGadget <[email protected]>; [email protected]; Eric Sunshine <[email protected]>; Torsten Bögershausen <[email protected]>; Ævar Arnfjörð Bjarmason <[email protected]>; Ramsay Jones <[email protected]>; Johannes Schindelin <[email protected]>; Eric DeCosta <[email protected]>
Subject: Re: [PATCH v7 2/6] fsmonitor: relocate socket file if .git directory is remoteQuoted 4 lines · Jeff Hostetler
Aren't we in the middle of a transition from always using the global "
the_repository" to a passed "r" variable?
We're getting closer to being able to hide the the global symbol, but we're not there yet, right?We may still have code that works ONLY on
the_repository, but letting a function take "r" and lettin it ignore is worse than leaving it explicitly limited tothe_repositoryonly, no?Quoted 2 lines
I'm thinking that at as long as the global exists, we are not safe to have multiple "struct repository" instances, right?
By itself, Not at all. It is the code like I am criticizing that makes it unsafe.
I do not mind adding
Quoted 9 lines
Quoted 4 lines
if (!r) BUG(...);at the place you have the "sweep it under the rug" band-aid, though.
Appreciate all the insights and comments. Where are we landing with this? Very close to the finish line and I'd like to be able to push these changes over that line.
-Eric
I'm OK doing it either way. Junio seems to prefer the BUG() version, so let's go with that. That lets us make progress on getting rid of direct references to "the_repository".
Jeff
RE: [PATCH v7 2/6] fsmonitor: relocate socket file if .git directory is remote
Quoted 60 lines
-----Original Message-----
From: Jeff Hostetler <[email protected]>
Sent: Tuesday, September 20, 2022 10:36 AM
To: Eric DeCosta <[email protected]>; Junio C Hamano <[email protected]>
Cc: Eric DeCosta via GitGitGadget <[email protected]>; [email protected]; Eric Sunshine <[email protected]>; Torsten Bögershausen <[email protected]>; Ævar Arnfjörð Bjarmason <[email protected]>; Ramsay Jones <[email protected]>; Johannes Schindelin <[email protected]>
Subject: Re: [PATCH v7 2/6] fsmonitor: relocate socket file if .git directory is remoteQuoted 31 lines · Eric DeCosta
Quoted 31 lines
-----Original Message-----
From: Junio C Hamano <[email protected]>
Sent: Monday, September 19, 2022 1:49 PM
To: Jeff Hostetler <[email protected]>
Cc: Eric DeCosta via GitGitGadget <[email protected]>; [email protected]; Eric Sunshine <[email protected]>; Torsten Bögershausen <[email protected]>; Ævar Arnfjörð Bjarmason <[email protected]>; Ramsay Jones <[email protected]>; Johannes Schindelin <[email protected]>; Eric DeCosta <[email protected]>
Subject: Re: [PATCH v7 2/6] fsmonitor: relocate socket file if .git directory is remoteQuoted 4 lines · Jeff Hostetler
Aren't we in the middle of a transition from always using the global "
the_repository" to a passed "r" variable?
We're getting closer to being able to hide the the global symbol, but we're not there yet, right?We may still have code that works ONLY on
the_repository, but letting a function take "r" and lettin it ignore is worse than leaving it explicitly limited tothe_repositoryonly, no?Quoted 2 lines
I'm thinking that at as long as the global exists, we are not safe to have multiple "struct repository" instances, right?
By itself, Not at all. It is the code like I am criticizing that makes it unsafe.
Quoted 8 lines
Quoted 6 lines
I do not mind adding
if (!r) BUG(...);at the place you have the "sweep it under the rug" band-aid, though.
Appreciate all the insights and comments. Where are we landing with this?
Very close to the finish line and I'd like to be able to push these changes over that line.
Quoted 1 line
Quoted 8 lines
Quoted 2 lines
-Eric
I'm OK doing it either way. Junio seems to prefer the
BUG()version, so let's go with that. That lets us make progress on getting rid of direct references to "the_repository".Jeff
Sounds like a plan!
-Eric
PATCH v7 3/6 fsmonitor: avoid socket location check if using hook
In reply to Eric DeCosta via GitGitGadgetFrom: Eric DeCosta <[email protected]>
If monitoring is done via fsmonitor hook rather than IPC there is no need to check if the location of the Unix Domain socket (UDS) file is on a remote filesystem.
--- compat/fsmonitor/fsm-settings-darwin.c | 10 ++++++---- compat/fsmonitor/fsm-settings-win32.c | 2 +- fsmonitor-settings.c | 8 ++++---- fsmonitor-settings.h | 2 +- 4 files changed, 12 insertions(+), 10 deletions(-)
compat/fsmonitor/fsm-settings-darwin.c +6 -4
@@ -49,13 +49,15 @@ static enum fsmonitor_reason check_uds_volume(struct repository *r)return FSMONITOR_REASON_OK ; } -enum fsmonitor_reason fsm_os__incompatible (struct repository *r) +enum fsmonitor_reason fsm_os__incompatible (struct repository *r,int ipc) {enum fsmonitor_reason reason; - reason =check_uds_volume (r); -if (reason !=FSMONITOR_REASON_OK ) -return reason; +if (ipc) { + reason =check_uds_volume (r); +if (reason !=FSMONITOR_REASON_OK ) +return reason; + }return FSMONITOR_REASON_OK ; }
compat/fsmonitor/fsm-settings-win32.c +1 -1
@@ -25,7 +25,7 @@ static enum fsmonitor_reason check_vfs4git(struct repository *r)return FSMONITOR_REASON_OK ; } -enum fsmonitor_reason fsm_os__incompatible (struct repository *r) +enum fsmonitor_reason fsm_os__incompatible (struct repository *r,int ipc) {enum fsmonitor_reason reason;
fsmonitor-settings.c +4 -4
@@ -60,7 +60,7 @@ static enum fsmonitor_reason check_remote(struct repository *r) }#endif -static enum fsmonitor_reason check_for_incompatible (struct repository *r) +static enum fsmonitor_reason check_for_incompatible (struct repository *r,int ipc) {if (!r->worktree) {/*
@@ -77,7 +77,7 @@ static enum fsmonitor_reason check_for_incompatible(struct repository *r) reason =check_remote (r);if (reason !=FSMONITOR_REASON_OK )return reason; - reason =fsm_os__incompatible (r); + reason =fsm_os__incompatible (r, ipc);if (reason !=FSMONITOR_REASON_OK )return reason; }
@@ -162,7 +162,7 @@ const char *fsm_settings__get_hook_path(struct repository *r)void fsm_settings__set_ipc (struct repository *r) { -enum fsmonitor_reason reason =check_for_incompatible (r); +enum fsmonitor_reason reason =check_for_incompatible (r,1 );if (reason !=FSMONITOR_REASON_OK ) {fsm_settings__set_incompatible (r, reason);
@@ -185,7 +185,7 @@ void fsm_settings__set_ipc(struct repository *r)void fsm_settings__set_hook (struct repository *r,const char *path) { -enum fsmonitor_reason reason =check_for_incompatible (r); +enum fsmonitor_reason reason =check_for_incompatible (r,0 );if (reason !=FSMONITOR_REASON_OK ) {fsm_settings__set_incompatible (r, reason);
fsmonitor-settings.h +1 -1
@@ -48,7 +48,7 @@ struct fsmonitor_settings;* fsm_os__* routines should considered private to fsm_settings__ * routines. */ -enum fsmonitor_reason fsm_os__incompatible (struct repository *r); +enum fsmonitor_reason fsm_os__incompatible (struct repository *r,int ipc);#endif /* HAVE_FSMONITOR_OS_SETTINGS */ #endif /* FSMONITOR_SETTINGS_H */
Signature · 1 line
gitgitgadget
PATCH v7 4/6 fsmonitor: deal with synthetic firmlinks on macOS
In reply to Eric DeCosta via GitGitGadgetFrom: Eric DeCosta <[email protected]>
Starting with macOS 10.15 (Catalina), Apple introduced a new feature called 'firmlinks' in order to separate the boot volume into two volumes, one read-only and one writable but still present them to the user as a single volume. Along with this change, Apple removed the ability to create symlinks in the root directory and replaced them with 'synthetic firmlinks'. See 'man synthetic.conf'
When FSEevents reports the path of changed files, if the path involves a synthetic firmlink, the path is reported from the point of the synthetic firmlink and not the real path. For example:
Real path:
/System/Volumes/Data/network/working/directory/foo.txt
Synthetic firmlink:
/network -> /System/Volumes/Data/network
FSEvents path:
/network/working/directory/foo.txt
This causes the FSEvents path to not match against the worktree directory.
There are several ways in which synthetic firmlinks can be created:
they can be defined in /etc/synthetic.conf, the automounter can create them, and there may be other means. Simply reading /etc/synthetic.conf is insufficient. No matter what process creates synthetic firmlinks, they all get created in the root directory.
Therefore, in order to deal with synthetic firmlinks, the root directory is scanned and the first possible synthetic firmink that, when resolved, is a prefix of the worktree is used to map FSEvents paths to worktree paths.
--- builtin/fsmonitor--daemon.c | 8 +++ compat/fsmonitor/fsm-listen-darwin.c | 6 +- compat/fsmonitor/fsm-path-utils-darwin.c | 92 ++++++++++++++++++++++++ compat/fsmonitor/fsm-path-utils-win32.c | 17 +++++ fsmonitor--daemon.h | 3 + fsmonitor-path-utils.h | 36 ++++++++++ 6 files changed, 161 insertions(+), 1 deletion(-)
builtin/fsmonitor--daemon.c +8 -0
@@ -3,6 +3,7 @@#include "parse-options.h" #include "fsmonitor.h" #include "fsmonitor-ipc.h" +#include "fsmonitor-path-utils.h" #include "compat/fsmonitor/fsm-health.h" #include "compat/fsmonitor/fsm-listen.h" #include "fsmonitor--daemon.h"
@@ -1282,6 +1283,13 @@ static int fsmonitor_run_daemon(void)strbuf_addstr (&state.path_worktree_watch,absolute_path (get_git_work_tree ())); state.nr_paths_watching =1 ; + state.alias.alias =NULL ; + state.alias.points_to =NULL ; +if (fsmonitor__get_alias (state.path_worktree_watch.buf, &state.alias)) { + err =error (_ ("could not get worktree alias" )); + goto done; + } +/* * We create and delete cookie files somewhere inside the .git * directory to help us keep sync with the file system. If
compat/fsmonitor/fsm-listen-darwin.c +5 -1
@@ -26,6 +26,7 @@#include "fsmonitor.h" #include "fsm-listen.h" #include "fsmonitor--daemon.h" +#include "fsmonitor-path-utils.h" struct fsm_listen_data {
@@ -209,7 +210,9 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,/* * On Mac, we receive an array of absolute paths. */ - path_k = paths[k]; + path_k =fsmonitor__resolve_alias (paths[k], &state->alias); +if (!path_k) + path_k = paths[k];/* * If you want to debug FSEvents, log them to GIT_TRACE_FSMONITOR.
@@ -238,6 +241,7 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,fsmonitor_force_resync (state);fsmonitor_batch__free_list (batch);string_list_clear (&cookie_list,0 ); + batch =NULL ;/* * We assume that any events that we received
compat/fsmonitor/fsm-path-utils-darwin.c +92 -0
100 lines not on this page · raw message
compat/fsmonitor/fsm-path-utils-win32.c +17 -0
@@ -126,3 +126,20 @@ int fsmonitor__is_fs_remote(const char *path)return -1 ;return fs.is_remote; } + +/* +* No-op for now. +*/ +int fsmonitor__get_alias (const char *path,struct alias_info *info) +{ +return 0 ; +} + +/* +* No-op for now. +*/ +char *fsmonitor__resolve_alias (const char *path, +const struct alias_info *info) +{ +return NULL ; +}
fsmonitor--daemon.h +3 -0
@@ -8,6 +8,7 @@#include "run-command.h" #include "simple-ipc.h" #include "thread-utils.h" +#include "fsmonitor-path-utils.h" struct fsmonitor_batch ;struct fsmonitor_token_data ;
@@ -43,6 +44,7 @@ struct fsmonitor_daemon_state {struct strbuf path_worktree_watch;struct strbuf path_gitdir_watch; +struct alias_info alias;int nr_paths_watching;struct fsmonitor_token_data *current_token_data;
@@ -59,6 +61,7 @@ struct fsmonitor_daemon_state {struct ipc_server_data *ipc_server_data;struct strbuf path_ipc; + };/*
fsmonitor-path-utils.h +36 -0
@@ -1,6 +1,14 @@ #ifndef FSM_PATH_UTILS_H #define FSM_PATH_UTILS_H +#include "strbuf.h" + +struct alias_info +{ + char *alias; + char *points_to; +}; + struct fs_info { int is_remote; char *typename;
@@ -20,4 +28,32 @@ int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info);*/ int fsmonitor__is_fs_remote (const char *path); +/* +* Get the alias in given path, if any. +* +* Sets alias to the first alias that matches any part of the path. +* +* If an alias is found, info.alias and info.points_to are set to the +* found mapping. +* +* Returns -1 on error, 0 otherwise. +* +* The caller owns the storage that is occupied by set info.alias and +* info.points_to and is responsible for releasing it with `free(3)` +* when done. +*/ +int fsmonitor__get_alias (const char *path,struct alias_info *info); + +/* +* Resolve the path against the given alias. +* +* Returns the resolved path if there is one, NULL otherwise. +* +* The caller owns the storage that the returned string occupies and +* is responsible for releasing it with `free(3)` when done. +*/ +char *fsmonitor__resolve_alias (const char *path, +const struct alias_info *info); + +#endif
Signature · 1 line
gitgitgadget
PATCH v7 5/6 fsmonitor: deal with synthetic firmlinks on macOS
In reply to Eric DeCosta via GitGitGadgetFrom: Eric DeCosta <[email protected]>
Starting with macOS 10.15 (Catalina), Apple introduced a new feature called 'firmlinks' in order to separate the boot volume into two volumes, one read-only and one writable but still present them to the user as a single volume. Along with this change, Apple removed the ability to create symlinks in the root directory and replaced them with 'synthetic firmlinks'. See 'man synthetic.conf'
When FSEevents reports the path of changed files, if the path involves a synthetic firmlink, the path is reported from the point of the synthetic firmlink and not the real path. For example:
Real path:
/System/Volumes/Data/network/working/directory/foo.txt
Synthetic firmlink:
/network -> /System/Volumes/Data/network
FSEvents path:
/network/working/directory/foo.txt
This causes the FSEvents path to not match against the worktree directory.
There are several ways in which synthetic firmlinks can be created:
they can be defined in /etc/synthetic.conf, the automounter can create them, and there may be other means. Simply reading /etc/synthetic.conf is insufficient. No matter what process creates synthetic firmlinks, they all get created in the root directory.
Therefore, in order to deal with synthetic firmlinks, the root directory is scanned and the first possible synthetic firmink that, when resolved, is a prefix of the worktree is used to map FSEvents paths to worktree paths.
--- fsmonitor--daemon.h | 3 +++ 1 file changed, 3 insertions(+)
fsmonitor--daemon.h +3 -0
@@ -170,5 +170,8 @@ void fsmonitor_publish(struct fsmonitor_daemon_state *state,*/ void fsmonitor_force_resync (struct fsmonitor_daemon_state *state); +char *fsmonitor_resolve_alias (const char *path, +struct alias_info *alias); +#endif /* HAVE_FSMONITOR_DAEMON_BACKEND */ #endif /* FSMONITOR_DAEMON_H */
Signature · 1 line
gitgitgadget
Re: [PATCH v7 5/6] fsmonitor: deal with synthetic firmlinks on macOS
Quoted 9 lines · Eric DeCosta via GitGitGadget
Starting with macOS 10.15 (Catalina), Apple introduced a new feature called 'firmlinks' in order to separate the boot volume into two volumes, one read-only and one writable but still present them to the user as a single volume. Along with this change, Apple removed the
...
Therefore, in order to deal with synthetic firmlinks, the root directory is scanned and the first possible synthetic firmink that, when resolved, is a prefix of the worktree is used to map FSEvents paths to worktree paths.
The proposed log message for this step seems fairly similar to the previous one.
fsmonitor--daemon.h void fsmonitor_publish(struct fsmonitor_daemon_state *state, · line 175
Quoted 11 lines
fsmonitor--daemon.h +3
-0@@ -170,5 +170,8 @@ void fsmonitor_publish(struct fsmonitor_daemon_state *state,*/ void fsmonitor_force_resync (struct fsmonitor_daemon_state *state); +char *fsmonitor_resolve_alias (const char *path, +struct alias_info *alias); +
After applying all the 6 steps, this function is declared here without any implementation. Should we drop this step, I have to wonder.
Thanks.
PATCH v7 6/6 fsmonitor: add documentation for allowRemote and socketDir options
In reply to Eric DeCosta via GitGitGadgetFrom: Eric DeCosta <[email protected]>
Add documentation for 'fsmonitor.allowRemote' and 'fsmonitor.socketDir'. Call-out experimental nature of 'fsmonitor.allowRemote' and limited file system support for 'fsmonitor.socketDir'.
--- Documentation/git-fsmonitor--daemon.txt | 35 +++++++++++++++++++++++++ 1 file changed, 35 insertions(+)
Documentation/git-fsmonitor--daemon.txt +35 -0
@@ -70,6 +70,41 @@ the change (as happening against the super repo). However, the client will properly ignore these extra events, so performance may be affected but it will not cause an incorrect result. +By default, the fsmonitor daemon refuses to work against network-mounted +repositories; this my be overridden by setting `fsmonitor.allowRemote` to +`true`. Note, however, that the fsmonitor daemon is not guaranteed to work +correctly with all network-mounted repositores and such use is considered +experimental. + +On Mac OS, the inter-process communication (IPC) between various Git +commands and the fsmonitor daemon is done via a Unix domain socket (UDS). +Usage of UDS requires the creation of a file which, by default, is created +in the .git directory. If the fsmonitor daemon detects that the .git directory +is on a network-mounted file system, it will create the UDS file in $HOME. If +$HOME itself is on a network-mounted file system or if $HOME is not the desired +location for the UDS file, 'fsmonitor.socketDir' may be set to any valid, local +directory on a file system with proper support. Mac OS native file systems have +the required support. File systems known to lack support include FAT32 and +NTFS. Other file systems may or many not have the needed support; the fsmonitor +daemon is not guaranteed to work with these file systems and such use is +considered experimental. + +CONFIGURATION +------------- +When `core.fsmonitor` is set to `true` (see linkgit:git-config[1]) +the fsmonitor daemon will pay attention to the following configuration +variables: + +fsmonitor.allowRemote:: + By default, the daemon refuses to work against network-mounted + repositories. Setting `fsmonitor.allowRemote` to `true` overrides + this behavior. + +fsmonitor.socketDir:: + This option is only used by the Mac OS implementation of the fsmonitor + daemon. If set, 'fsmonitor.socketDir' must be set to a valid, local + directory on a file system that can support Unix domain sockets (UDS). + GIT --- Part of the linkgit:git[1] suite
Signature · 1 line
gitgitgadget
Quoted 15 lines · Eric DeCosta via GitGitGadget
5:
9c1f408ae6d! 4:3a9fe473cf4fsmonitor: deal with synthetic firmlinks on macOS
@@fsmonitor--daemon.h: structfsmonitor_daemon_state{
};/*
-@@fsmonitor--daemon.h: void fsmonitor_publish(structfsmonitor_daemon_state*state,
- */
- voidfsmonitor_force_resync(struct fsmonitor_daemon_state *state);
-
-+char *fsmonitor_resolve_alias(const char *path,
-+ structalias_info*alias);
-+
- #endif /*HAVE_FSMONITOR_DAEMON_BACKEND*/
- #endif /*FSMONITOR_DAEMON_H*/
Quoted 3 lines
##
fsmonitor-path-utils.h##
@@
-: ----------- > 5:4d00adb1debfsmonitor: deal with synthetic firmlinks on macOS
"rebase -i" mistake or something?
v8 cover fsmonitor: option to allow fsmonitor to run against network-mounted repos
In reply to Eric DeCosta via GitGitGadgetFollow-on to the work done to allow Windows to work against network-mounted repos for macOS.
Have macOS take advantage of the same configuration option, 'fsmonitor.allowRemote' that was introduced for Windows. Setting this option to true will override the default behavior (erroring-out) when a network-mounted repo is detected by fsmonitor.
The added wrinkle being that the Unix domain socket (UDS) file used for IPC cannot be created in a network location; instead $HOME is used if the default location is on the network. The user may, optionally, set the 'fsmonitor.socketDir' configuration option to a valid, local directory if $HOME itself is on the network or is simply not the desired location for the UDS file.
An additional issue is that for mount points in the root directory, FSEvents does not report a path that matches the worktree directory due to the introduction of 'synthetic firmlinks'. fsmonitor must map the FSEvents paths to the worktree directory by interrogating the root filesystem for synthetic firmlinks and using that information to translate the path.
v8 differs from v7:
* incorporates code review feedback
* gets the rebase right
v7 differs from v6:
* incorporates code review feedback
v6 differs from v5:
* incorporates earlier, Windows-specific changes that have not made it back yet to the master branch
* incorporates code review feedback
* adds documentation for 'fsmonitor.allowRemote' and 'fsmonitor.socketDir'
v5 differs significantly from earlier versions:
* redesign of handling 'fsmonitor.allowRemote' and 'fsmonitor.socketDir' such that these options are no longer added to the settings data structure but are rather read from config at point of use
* refactoring of code for handling platform-specific file system checks via a common interface to avoid platform #ifdef in IPC code and be in-model with other platform-specific fsmonitor code
* dealing with 'synthetic firmlinks' on macOS
Eric DeCosta (5):
fsmonitor: refactor filesystem checks to common interface
fsmonitor: relocate socket file if .git directory is remote
fsmonitor: avoid socket location check if using hook
fsmonitor: deal with synthetic firmlinks on macOS
fsmonitor: add documentation for allowRemote and socketDir options
Documentation/git-fsmonitor--daemon.txt | 35 +++++ Makefile | 2 + builtin/fsmonitor--daemon.c | 11 +- compat/fsmonitor/fsm-ipc-darwin.c | 46 ++++++ compat/fsmonitor/fsm-ipc-win32.c | 9 ++ compat/fsmonitor/fsm-listen-darwin.c | 6 +- compat/fsmonitor/fsm-path-utils-darwin.c | 132 +++++++++++++++++ compat/fsmonitor/fsm-path-utils-win32.c | 145 +++++++++++++++++++ compat/fsmonitor/fsm-settings-darwin.c | 70 +++------ compat/fsmonitor/fsm-settings-win32.c | 174 +---------------------- contrib/buildsystems/CMakeLists.txt | 4 + fsmonitor--daemon.h | 3 + fsmonitor-ipc.c | 18 ++- fsmonitor-ipc.h | 4 +- fsmonitor-path-utils.h | 59 ++++++++ fsmonitor-settings.c | 58 +++++++- fsmonitor-settings.h | 2 +- 17 files changed, 541 insertions(+), 237 deletions(-) create mode 100644 compat/fsmonitor/fsm-ipc-darwin.c create mode 100644 compat/fsmonitor/fsm-ipc-win32.c create mode 100644 compat/fsmonitor/fsm-path-utils-darwin.c create mode 100644 compat/fsmonitor/fsm-path-utils-win32.c create mode 100644 fsmonitor-path-utils.h
base-commit: d3fa443f97e3a8d75b51341e2d5bac380b7422df
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1326%2Fedecosta-mw%2Ffsmonitor_macos-v8
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1326/edecosta-mw/fsmonitor_macos-v8
Pull-Request: https://github.com/gitgitgadget/git/pull/1326
Range-diff vs v7:
1: 155a6890806 = 1: 155a6890806 fsmonitor: refactor filesystem checks to common interface
2: 075340bd2a7 ! 2: b5356497228 fsmonitor: relocate socket file if .git directory is remote
@@ compat/fsmonitor/fsm-ipc-darwin.c (new)
+ if (ipc_path)
+ return ipc_path;
+
-+ if (!r)
-+ r = the_repository;
-+
+ ipc_path = fsmonitor_ipc__get_default_path();
+
+ /* By default the socket file is created in the .git directory */
3: 5518d2f3e03 = 3: 6719ca2b24d fsmonitor: avoid socket location check if using hook
4: 3a9fe473cf4 ! 4: d736cb8fa90 fsmonitor: deal with synthetic firmlinks on macOS@@ Commit message
+ fsmonitor: deal with synthetic firmlinks on macOS
+
+ Starting with macOS 10.15 (Catalina), Apple introduced a new feature
+ called 'firmlinks' in order to separate the boot volume into two
+ volumes, one read-only and one writable but still present them to the
+ user as a single volume. Along with this change, Apple removed the
+ ability to create symlinks in the root directory and replaced them with
+ 'synthetic firmlinks'. See 'man synthetic.conf'
+
+ When FSEevents reports the path of changed files, if the path involves
+ a synthetic firmlink, the path is reported from the point of the
+ synthetic firmlink and not the real path. For example:
+
+ Real path:
+ /System/Volumes/Data/network/working/directory/foo.txt
+
+ Synthetic firmlink:
+ /network -> /System/Volumes/Data/network
+
+ FSEvents path:
+ /network/working/directory/foo.txt
+
+ This causes the FSEvents path to not match against the worktree
+ directory.
+
+ There are several ways in which synthetic firmlinks can be created:
+ they can be defined in /etc/synthetic.conf, the automounter can create
+ them, and there may be other means. Simply reading /etc/synthetic.conf
+ is insufficient. No matter what process creates synthetic firmlinks,
+ they all get created in the root directory.
+
+ Therefore, in order to deal with synthetic firmlinks, the root directory
+ is scanned and the first possible synthetic firmink that, when resolved,
+ is a prefix of the worktree is used to map FSEvents paths to worktree
+ paths.
+
+ Signed-off-by: Eric DeCosta <[email protected]>
+
+ fsmonitor: deal with synthetic firmlinks on macOS
+
+ Starting with macOS 10.15 (Catalina), Apple introduced a new feature
+ called 'firmlinks' in order to separate the boot volume into two
+ volumes, one read-only and one writable but still present them to the
+ user as a single volume. Along with this change, Apple removed the
+ ability to create symlinks in the root directory and replaced them with
+ 'synthetic firmlinks'. See 'man synthetic.conf'
+
+ When FSEevents reports the path of changed files, if the path involves
+ a synthetic firmlink, the path is reported from the point of the
+ synthetic firmlink and not the real path. For example:
+
+ Real path:
+ /System/Volumes/Data/network/working/directory/foo.txt
+
+ Synthetic firmlink:
+ /network -> /System/Volumes/Data/network
+
+ FSEvents path:
+ /network/working/directory/foo.txt
+
+ This causes the FSEvents path to not match against the worktree
+ directory.
+
+ There are several ways in which synthetic firmlinks can be created:
+ they can be defined in /etc/synthetic.conf, the automounter can create
+ them, and there may be other means. Simply reading /etc/synthetic.conf
+ is insufficient. No matter what process creates synthetic firmlinks,
+ they all get created in the root directory.
+
+ Therefore, in order to deal with synthetic firmlinks, the root directory
+ is scanned and the first possible synthetic firmink that, when resolved,
+ is a prefix of the worktree is used to map FSEvents paths to worktree
+ paths.
+
+ Signed-off-by: Eric DeCosta <[email protected]>
+
## builtin/fsmonitor--daemon.c ##
@@
#include "parse-options.h"
5: 4d00adb1deb < -: ----------- fsmonitor: deal with synthetic firmlinks on macOS 6: 260591f5820 = 5: ddf4e3e6442 fsmonitor: add documentation for allowRemote and socketDir options
Signature · 1 line
gitgitgadget
PATCH v8 1/5 fsmonitor: refactor filesystem checks to common interface
From: Eric DeCosta <[email protected]>
Provide a common interface for getting basic filesystem information including filesystem type and whether the filesystem is remote.
Refactor existing code for getting basic filesystem info and detecting remote file systems to the new interface.
Refactor filesystem checks to leverage new interface. For macOS, error-out if the Unix Domain socket (UDS) file is on a remote filesystem.
--- Makefile | 1 + compat/fsmonitor/fsm-path-utils-darwin.c | 40 ++++++ compat/fsmonitor/fsm-path-utils-win32.c | 128 +++++++++++++++++ compat/fsmonitor/fsm-settings-darwin.c | 62 +++----- compat/fsmonitor/fsm-settings-win32.c | 172 +---------------------- contrib/buildsystems/CMakeLists.txt | 2 + fsmonitor-path-utils.h | 23 +++ fsmonitor-settings.c | 50 +++++++ 8 files changed, 263 insertions(+), 215 deletions(-) create mode 100644 compat/fsmonitor/fsm-path-utils-darwin.c create mode 100644 compat/fsmonitor/fsm-path-utils-win32.c create mode 100644 fsmonitor-path-utils.h
Makefile +1 -0
@@ -2039,6 +2039,7 @@ endif ifdef FSMONITOR_OS_SETTINGS COMPAT_CFLAGS += -DHAVE_FSMONITOR_OS_SETTINGS COMPAT_OBJS += compat/fsmonitor/fsm-settings-$(FSMONITOR_OS_SETTINGS).o + COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_OS_SETTINGS).o endif ifeq ($(TCLTK_PATH),)
compat/fsmonitor/fsm-path-utils-darwin.c new file +40 -0
@@ -0,0 +1,40 @@ +#include "fsmonitor.h" +#include "fsmonitor-path-utils.h" +#include <sys/param.h> +#include <sys/mount.h> + +int fsmonitor__get_fs_info (const char *path,struct fs_info *fs_info) +{ +struct statfs fs; +if (statfs (path, &fs) ==-1 ) { +int saved_errno = errno; +trace_printf_key (&trace_fsmonitor,"statfs('%s') failed: %s" , + path,strerror (saved_errno)); + errno = saved_errno; +return -1 ; + } + +trace_printf_key (&trace_fsmonitor, +"statfs('%s') [type 0x%08x][flags 0x%08x] '%s'" , + path, fs.f_type, fs.f_flags, fs.f_fstypename); + +if (!(fs.f_flags &MNT_LOCAL )) + fs_info->is_remote =1 ; +else + fs_info->is_remote =0 ; + + fs_info->typename = fs.f_fstypename; + +trace_printf_key (&trace_fsmonitor, +"'%s' is_remote: %d" , + path, fs_info->is_remote); +return 0 ; +} + +int fsmonitor__is_fs_remote (const char *path) +{ +struct fs_info fs; +if (fsmonitor__get_fs_info (path, &fs)) +return -1 ; +return fs.is_remote; +}
compat/fsmonitor/fsm-path-utils-win32.c new file +128 -0
128 lines not on this page · raw message
compat/fsmonitor/fsm-settings-darwin.c +17 -45
96 lines not on this page · raw message
compat/fsmonitor/fsm-settings-win32.c +2 -170
190 lines not on this page · raw message
contrib/buildsystems/CMakeLists.txt +2 -0
@@ -308,6 +308,7 @@ if(SUPPORTS_SIMPLE_IPC) add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-win32.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-win32.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-win32.c)
@@ -315,6 +316,7 @@ if(SUPPORTS_SIMPLE_IPC) add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-darwin.c)
fsmonitor-path-utils.h new file +23 -0
@@ -0,0 +1,23 @@ +#ifndef FSM_PATH_UTILS_H +#define FSM_PATH_UTILS_H + +struct fs_info { +int is_remote; +char *typename; +}; + +/* +* Get some basic filesystem informtion for the given path +* +* Returns -1 on error, zero otherwise. +*/ +int fsmonitor__get_fs_info (const char *path,struct fs_info *fs_info); + +/* +* Determines if the filesystem that path resides on is remote. +* +* Returns -1 on error, 0 if not remote, 1 if remote. +*/ +int fsmonitor__is_fs_remote (const char *path); + +#endif
fsmonitor-settings.c +50 -0
68 lines not on this page · raw message
Signature · 1 line
gitgitgadget
PATCH v8 2/5 fsmonitor: relocate socket file if .git directory is remote
In reply to Eric DeCosta via GitGitGadgetFrom: Eric DeCosta <[email protected]>
If the .git directory is on a remote file system, create the socket file in 'fsmonitor.socketDir' if it is defined, else create it in $HOME.
--- Makefile | 1 + builtin/fsmonitor--daemon.c | 3 +- compat/fsmonitor/fsm-ipc-darwin.c | 46 ++++++++++++++++++++++++++ compat/fsmonitor/fsm-ipc-win32.c | 9 +++++ compat/fsmonitor/fsm-settings-darwin.c | 2 +- contrib/buildsystems/CMakeLists.txt | 2 ++ fsmonitor-ipc.c | 18 +++++----- fsmonitor-ipc.h | 4 ++- 8 files changed, 72 insertions(+), 13 deletions(-) create mode 100644 compat/fsmonitor/fsm-ipc-darwin.c create mode 100644 compat/fsmonitor/fsm-ipc-win32.c
Makefile +1 -0
@@ -2034,6 +2034,7 @@ ifdef FSMONITOR_DAEMON_BACKEND COMPAT_CFLAGS += -DHAVE_FSMONITOR_DAEMON_BACKEND COMPAT_OBJS += compat/fsmonitor/fsm-listen-$(FSMONITOR_DAEMON_BACKEND).o COMPAT_OBJS += compat/fsmonitor/fsm-health-$(FSMONITOR_DAEMON_BACKEND).o + COMPAT_OBJS += compat/fsmonitor/fsm-ipc-$(FSMONITOR_DAEMON_BACKEND).o endif ifdef FSMONITOR_OS_SETTINGS
builtin/fsmonitor--daemon.c +2 -1
@@ -1343,7 +1343,8 @@ static int fsmonitor_run_daemon(void)* directory.) */ strbuf_init (&state.path_ipc,0 ); -strbuf_addstr (&state.path_ipc,absolute_path (fsmonitor_ipc__get_path ())); +strbuf_addstr (&state.path_ipc, +absolute_path (fsmonitor_ipc__get_path (the_repository)));/* * Confirm that we can create platform-specific resources for the
compat/fsmonitor/fsm-ipc-darwin.c new file +46 -0
46 lines not on this page · raw message
compat/fsmonitor/fsm-ipc-win32.c new file +9 -0
@@ -0,0 +1,9 @@ +#include "config.h" +#include "fsmonitor-ipc.h" + +const char *fsmonitor_ipc__get_path (struct repository *r) { +static char *ret; +if (!ret) + ret =git_pathdup ("fsmonitor--daemon.ipc" ); +return ret; +}
compat/fsmonitor/fsm-settings-darwin.c +1 -1
@@ -26,7 +26,7 @@static enum fsmonitor_reason check_uds_volume (struct repository *r) {struct fs_info fs; -const char *ipc_path =fsmonitor_ipc__get_path (); +const char *ipc_path =fsmonitor_ipc__get_path (r);struct strbuf path =STRBUF_INIT ;strbuf_add (&path, ipc_path,strlen (ipc_path));
contrib/buildsystems/CMakeLists.txt +2 -0
@@ -308,6 +308,7 @@ if(SUPPORTS_SIMPLE_IPC) add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-win32.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-win32.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-win32.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
@@ -316,6 +317,7 @@ if(SUPPORTS_SIMPLE_IPC) add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-darwin.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
fsmonitor-ipc.c +8 -10
@@ -18,7 +18,7 @@ int fsmonitor_ipc__is_supported(void)return 0 ; } -const char *fsmonitor_ipc__get_path (void ) +const char *fsmonitor_ipc__get_path (struct repository *r) {return NULL ; }
@@ -47,11 +47,9 @@ int fsmonitor_ipc__is_supported(void)return 1 ; } -GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc") -enum ipc_active_state fsmonitor_ipc__get_state (void ) { -return ipc_get_active_state (fsmonitor_ipc__get_path ()); +return ipc_get_active_state (fsmonitor_ipc__get_path (the_repository)); }static int spawn_daemon (void )
@@ -81,8 +79,8 @@ int fsmonitor_ipc__send_query(const char *since_token,trace2_data_string ("fsm_client" ,NULL ,"query/command" , tok); try_again: - state =ipc_client_try_connect (fsmonitor_ipc__get_path (), &options, - &connection); + state =ipc_client_try_connect (fsmonitor_ipc__get_path (the_repository), + &options, &connection);switch (state) {case IPC_STATE__LISTENING :
@@ -117,13 +115,13 @@ try_again:case IPC_STATE__INVALID_PATH : ret =error (_ ("fsmonitor_ipc__send_query: invalid path '%s'" ), -fsmonitor_ipc__get_path ()); +fsmonitor_ipc__get_path (the_repository)); goto done;case IPC_STATE__OTHER_ERROR :default : ret =error (_ ("fsmonitor_ipc__send_query: unspecified error on '%s'" ), -fsmonitor_ipc__get_path ()); +fsmonitor_ipc__get_path (the_repository)); goto done; }
@@ -149,8 +147,8 @@ int fsmonitor_ipc__send_command(const char *command, options.wait_if_busy =1 ; options.wait_if_not_found =0 ; - state =ipc_client_try_connect (fsmonitor_ipc__get_path (), &options, - &connection); + state =ipc_client_try_connect (fsmonitor_ipc__get_path (the_repository), + &options, &connection);if (state !=IPC_STATE__LISTENING ) {die (_ ("fsmonitor--daemon is not running" ));return -1 ;
fsmonitor-ipc.h +3 -1
@@ -3,6 +3,8 @@#include "simple-ipc.h" +struct repository ; +/* * Returns true if built-in file system monitor daemon is defined * for this platform.
@@ -16,7 +18,7 @@ int fsmonitor_ipc__is_supported(void);* * Returns NULL if the daemon is not supported on this platform. */ -const char *fsmonitor_ipc__get_path (void ); +const char *fsmonitor_ipc__get_path (struct repository *r);/* * Try to determine whether there is a `git-fsmonitor--daemon` process
Signature · 1 line
gitgitgadget
Re: [PATCH v8 2/5] fsmonitor: relocate socket file if .git directory is remote
Quoted 7 lines · Eric DeCosta via GitGitGadget
If the .git directory is on a remote file system, create the socket file in 'fsmonitor.socketDir' if it is defined, else create it in $HOME.
Signed-off-by Eric DeCosta <[email protected]>---
compat/fsmonitor/fsm-ipc-win32.c line 9
Quoted 10 lines
compat/fsmonitor/fsm-ipc-win32.c +9
-0@@ -0,0 +1,9 @@ +#include "config.h" +#include "fsmonitor-ipc.h" + +const char *fsmonitor_ipc__get_path (struct repository *r) { +static char *ret; +if (!ret) + ret =git_pathdup ("fsmonitor--daemon.ipc" ); +return ret; +}
This file probably wants a final line terminator.
Re: [PATCH v8 2/5] fsmonitor: relocate socket file if .git directory is remote
Quoted 6 lines · Eric Sunshine
Quoted 2 lines · Eric DeCosta via GitGitGadget
+}
\ No newline at end of fileThis file probably wants a final line terminator.
definitely ;-)
Re: [PATCH v8 2/5] fsmonitor: relocate socket file if .git directory is remote
In reply to Eric DeCosta via GitGitGadgetQuoted 28 lines · Eric DeCosta via GitGitGadget
If the .git directory is on a remote file system, create the socket file in 'fsmonitor.socketDir' if it is defined, else create it in $HOME.
Signed-off-by Eric DeCosta <[email protected]>---
compat/fsmonitor/fsm-ipc-darwin.c +21
-0@@ -0,0 +1,46 @@ +static GIT_PATH_FUNC(fsmonitor_ipc__get_default_path, "fsmonitor--daemon.ipc") + +const char *fsmonitor_ipc__get_path (struct repository *r) +{ +static const char *ipc_path; +SHA_CTX sha1ctx; +char *sock_dir; +struct strbuf ipc_file =STRBUF_INIT ; +unsigned char hash[SHA_DIGEST_LENGTH ]; + +if (ipc_path) +return ipc_path; + + ipc_path =fsmonitor_ipc__get_default_path (); + +/* By default the socket file is created in the .git directory */ +if (fsmonitor__is_fs_remote (ipc_path) <1 ) +return ipc_path; + +SHA1_Init (&sha1ctx); +SHA1_Update (&sha1ctx, r->worktree,strlen (r->worktree));
compat/fsmonitor/fsm-ipc-darwin.c line 31
Quoted 10 lines
compat/fsmonitor/fsm-ipc-darwin.c +10
-0+SHA1_Final (hash, &sha1ctx); + +repo_config_get_string (r,"fsmonitor.socketdir" , &sock_dir); + +/* Create the socket file in either socketDir or $HOME */ +if (sock_dir && *sock_dir) +strbuf_addf (&ipc_file,"%s/.git-fsmonitor-%s" , + sock_dir,hash_to_hex (hash)); +else +strbuf_addf (&ipc_file,"~/.git-fsmonitor-%s" ,hash_to_hex (hash));
A couple comments...
In my mind, the directory specified by fsmonitor.socketdir is likely to be dedicated to this purpose (i.e. housing Git administrative junk). As such, it feels somewhat odd for the socket file to be hidden; I would instead expect the socket name to be non-hidden (say, "git-fsmonitor-daemon-{hash}.ipc") rather than hidden (".git-fsmonitor-*"). The directory specified by fsmonitor.socketdir may or may not be hidden (i.e. start with a dot), but that's the user's decision. For the $HOME case, it almost feels cleaner to create a hidden directory (say, "$HOME/.git-fsmonitor") in which to house the socket files ("git-fsmonitor-daemon-{hash}.ipc"). Anyhow, this comment is quite subjective; perhaps not actionable.
What happens if either $HOME or fsmonitor.socketdir are network-mounted? Should this code be checking for that case? If they are network-mounted, should it error out? At minimum, I would think a warning is warranted in order to save users the headache of wondering why fsmonitor isn't working correctly.
compat/fsmonitor/fsm-ipc-darwin.c line 44
Quoted 5 lines
compat/fsmonitor/fsm-ipc-darwin.c +5
-0+ ipc_path =interpolate_path (ipc_file.buf,1 ); +if (!ipc_path) +die (_ ("Invalid path: %s" ), ipc_file.buf); + +strbuf_release (&ipc_file);
sock_dir is being leaked, isn't it?
compat/fsmonitor/fsm-ipc-win32.c line 6
Quoted 11 lines
compat/fsmonitor/fsm-ipc-darwin.c +2
-0+return ipc_path; +}compat/fsmonitor/fsm-ipc-win32.c +6
-0@@ -0,0 +1,9 @@ +const char *fsmonitor_ipc__get_path (struct repository *r) { +static char *ret; +if (!ret) + ret =git_pathdup ("fsmonitor--daemon.ipc" ); +return ret; +}
Mentioned already.
RE: [PATCH v8 2/5] fsmonitor: relocate socket file if .git directory is remote
Quoted 71 lines
-----Original Message-----
From: Eric Sunshine <[email protected]>
Sent: Saturday, September 17, 2022 2:30 AM
To: Eric DeCosta via GitGitGadget <[email protected]>
Cc: Git List <[email protected]>; Jeff Hostetler <[email protected]>; Torsten Bögershausen <[email protected]>; Ævar Arnfjörð Bjarmason <[email protected]>; Ramsay Jones <[email protected]>; Johannes Schindelin <[email protected]>; Eric DeCosta <[email protected]>
Subject: Re: [PATCH v8 2/5] fsmonitor: relocate socket file if .git directory is remoteQuoted 30 lines · Eric DeCosta via GitGitGadget
If the .git directory is on a remote file system, create the socket file in 'fsmonitor.socketDir' if it is defined, else create it in $HOME.
Signed-off-by Eric DeCosta <[email protected]>---
compat/fsmonitor/fsm-ipc-darwin.c +22
-0@@ -0,0 +1,46 @@ +static GIT_PATH_FUNC(fsmonitor_ipc__get_default_path, +"fsmonitor--daemon.ipc") + +const char *fsmonitor_ipc__get_path (struct repository *r) { +static const char *ipc_path; +SHA_CTX sha1ctx; +char *sock_dir; +struct strbuf ipc_file =STRBUF_INIT ; +unsigned char hash[SHA_DIGEST_LENGTH ]; + +if (ipc_path) +return ipc_path; + + ipc_path =fsmonitor_ipc__get_default_path (); + +/* By default the socket file is created in the .git directory */ +if (fsmonitor__is_fs_remote (ipc_path) <1 ) +return ipc_path; + +SHA1_Init (&sha1ctx); +SHA1_Update (&sha1ctx, r->worktree,strlen (r->worktree)); +SHA1_Final (hash, &sha1ctx);compat/fsmonitor/fsm-ipc-darwin.c line 32
Quoted 10 lines
compat/fsmonitor/fsm-ipc-darwin.c +10
-0+ +repo_config_get_string (r,"fsmonitor.socketdir" , &sock_dir); + +/* Create the socket file in either socketDir or $HOME */ +if (sock_dir && *sock_dir) +strbuf_addf (&ipc_file,"%s/.git-fsmonitor-%s" , + sock_dir,hash_to_hex (hash)); +else +strbuf_addf (&ipc_file,"~/.git-fsmonitor-%s" , +hash_to_hex (hash));A couple comments...
In my mind, the directory specified by
fsmonitor.socketdiris likely to be dedicated to this purpose (i.e. housing Git administrative junk). As such, it feels somewhat odd for the socket file to be hidden; I would instead expect the socket name to be non-hidden (say, "git-fsmonitor-daemon-{hash}.ipc") rather than hidden (".git-fsmonitor-*"). The directory specified byfsmonitor.socketdirmay or may not be hidden (i.e. start with a dot), but that's the user's decision. For the $HOME case, it almost feels cleaner to create a hidden directory (say, "$HOME/.git- fsmonitor") in which to house the socket files ("git-fsmonitor-daemon- {hash}.ipc"). Anyhow, this comment is quite subjective; perhaps not actionable.
Quoted 6 lines
What happens if either $HOME or
fsmonitor.socketdirare network- mounted? Should this code be checking for that case? If they are network- mounted, should it error out? At minimum, I would think a warning is warranted in order to save users the headache of wondering why fsmonitor isn't working correctly.
Ultimately, the UDS file location is checked by check_uds_volume in fsm-settings-darwin as part of the overall settings checks; it will error-out there if the path is on the network.
Quoted 8 lines
compat/fsmonitor/fsm-ipc-darwin.c line 44
Quoted 5 lines
compat/fsmonitor/fsm-ipc-darwin.c +5
-0+ ipc_path =interpolate_path (ipc_file.buf,1 ); +if (!ipc_path) +die (_ ("Invalid path: %s" ), ipc_file.buf); + +strbuf_release (&ipc_file);
sock_diris being leaked, isn't it?
Sure is, thanks.
Quoted 14 lines
compat/fsmonitor/fsm-ipc-win32.c line 6
Quoted 12 lines
compat/fsmonitor/fsm-ipc-darwin.c +2
-0+return ipc_path; +}compat/fsmonitor/fsm-ipc-win32.c +6
-0@@ -0,0 +1,9 @@ +const char *fsmonitor_ipc__get_path (struct repository *r) { +static char *ret; +if (!ret) + ret =git_pathdup ("fsmonitor--daemon.ipc" ); +return ret; +}Mentioned already.
Re: [PATCH v8 2/5] fsmonitor: relocate socket file if .git directory is remote
In reply to Eric SunshineQuoted 13 lines · Eric Sunshine
A couple comments...
In my mind, the directory specified by
fsmonitor.socketdiris likely to be dedicated to this purpose (i.e. housing Git administrative junk). As such, it feels somewhat odd for the socket file to be hidden; I would instead expect the socket name to be non-hidden (say, "git-fsmonitor-daemon-{hash}.ipc") rather than hidden (".git-fsmonitor-*"). The directory specified byfsmonitor.socketdirmay or may not be hidden (i.e. start with a dot), but that's the user's decision. For the $HOME case, it almost feels cleaner to create a hidden directory (say, "$HOME/.git-fsmonitor") in which to house the socket files ("git-fsmonitor-daemon-{hash}.ipc"). Anyhow, this comment is quite subjective; perhaps not actionable.
Yeah, dot-prefixed files are appropriate if they are to be placed at the top of some tree without the user having any say in how that tree is chosen (e.g. the working tree or $HOME). If the user has the power to specify the location, the equation changes.
Quoted 5 lines
What happens if either $HOME or
fsmonitor.socketdirare network-mounted? Should this code be checking for that case? If they are network-mounted, should it error out? At minimum, I would think a warning is warranted in order to save users the headache of wondering why fsmonitor isn't working correctly.
That's a good point. If one default position (e.g. repository) is checked if it is usable and can be rejected if it isn't, the fallback position should at least satisfy the same "is it usable?" criteria.
Thanks.
PATCH v8 3/5 fsmonitor: avoid socket location check if using hook
In reply to Eric DeCosta via GitGitGadgetFrom: Eric DeCosta <[email protected]>
If monitoring is done via fsmonitor hook rather than IPC there is no need to check if the location of the Unix Domain socket (UDS) file is on a remote filesystem.
--- compat/fsmonitor/fsm-settings-darwin.c | 10 ++++++---- compat/fsmonitor/fsm-settings-win32.c | 2 +- fsmonitor-settings.c | 8 ++++---- fsmonitor-settings.h | 2 +- 4 files changed, 12 insertions(+), 10 deletions(-)
compat/fsmonitor/fsm-settings-darwin.c +6 -4
@@ -49,13 +49,15 @@ static enum fsmonitor_reason check_uds_volume(struct repository *r)return FSMONITOR_REASON_OK ; } -enum fsmonitor_reason fsm_os__incompatible (struct repository *r) +enum fsmonitor_reason fsm_os__incompatible (struct repository *r,int ipc) {enum fsmonitor_reason reason; - reason =check_uds_volume (r); -if (reason !=FSMONITOR_REASON_OK ) -return reason; +if (ipc) { + reason =check_uds_volume (r); +if (reason !=FSMONITOR_REASON_OK ) +return reason; + }return FSMONITOR_REASON_OK ; }
compat/fsmonitor/fsm-settings-win32.c +1 -1
@@ -25,7 +25,7 @@ static enum fsmonitor_reason check_vfs4git(struct repository *r)return FSMONITOR_REASON_OK ; } -enum fsmonitor_reason fsm_os__incompatible (struct repository *r) +enum fsmonitor_reason fsm_os__incompatible (struct repository *r,int ipc) {enum fsmonitor_reason reason;
fsmonitor-settings.c +4 -4
@@ -60,7 +60,7 @@ static enum fsmonitor_reason check_remote(struct repository *r) }#endif -static enum fsmonitor_reason check_for_incompatible (struct repository *r) +static enum fsmonitor_reason check_for_incompatible (struct repository *r,int ipc) {if (!r->worktree) {/*
@@ -77,7 +77,7 @@ static enum fsmonitor_reason check_for_incompatible(struct repository *r) reason =check_remote (r);if (reason !=FSMONITOR_REASON_OK )return reason; - reason =fsm_os__incompatible (r); + reason =fsm_os__incompatible (r, ipc);if (reason !=FSMONITOR_REASON_OK )return reason; }
@@ -162,7 +162,7 @@ const char *fsm_settings__get_hook_path(struct repository *r)void fsm_settings__set_ipc (struct repository *r) { -enum fsmonitor_reason reason =check_for_incompatible (r); +enum fsmonitor_reason reason =check_for_incompatible (r,1 );if (reason !=FSMONITOR_REASON_OK ) {fsm_settings__set_incompatible (r, reason);
@@ -185,7 +185,7 @@ void fsm_settings__set_ipc(struct repository *r)void fsm_settings__set_hook (struct repository *r,const char *path) { -enum fsmonitor_reason reason =check_for_incompatible (r); +enum fsmonitor_reason reason =check_for_incompatible (r,0 );if (reason !=FSMONITOR_REASON_OK ) {fsm_settings__set_incompatible (r, reason);
fsmonitor-settings.h +1 -1
@@ -48,7 +48,7 @@ struct fsmonitor_settings;* fsm_os__* routines should considered private to fsm_settings__ * routines. */ -enum fsmonitor_reason fsm_os__incompatible (struct repository *r); +enum fsmonitor_reason fsm_os__incompatible (struct repository *r,int ipc);#endif /* HAVE_FSMONITOR_OS_SETTINGS */ #endif /* FSMONITOR_SETTINGS_H */
Signature · 1 line
gitgitgadget
PATCH v8 4/5 fsmonitor: deal with synthetic firmlinks on macOS
In reply to Eric DeCosta via GitGitGadgetFrom: Eric DeCosta <[email protected]>
Starting with macOS 10.15 (Catalina), Apple introduced a new feature called 'firmlinks' in order to separate the boot volume into two volumes, one read-only and one writable but still present them to the user as a single volume. Along with this change, Apple removed the ability to create symlinks in the root directory and replaced them with 'synthetic firmlinks'. See 'man synthetic.conf'
When FSEevents reports the path of changed files, if the path involves a synthetic firmlink, the path is reported from the point of the synthetic firmlink and not the real path. For example:
Real path:
/System/Volumes/Data/network/working/directory/foo.txt
Synthetic firmlink:
/network -> /System/Volumes/Data/network
FSEvents path:
/network/working/directory/foo.txt
This causes the FSEvents path to not match against the worktree directory.
There are several ways in which synthetic firmlinks can be created:
they can be defined in /etc/synthetic.conf, the automounter can create them, and there may be other means. Simply reading /etc/synthetic.conf is insufficient. No matter what process creates synthetic firmlinks, they all get created in the root directory.
Therefore, in order to deal with synthetic firmlinks, the root directory is scanned and the first possible synthetic firmink that, when resolved, is a prefix of the worktree is used to map FSEvents paths to worktree paths.
fsmonitor: deal with synthetic firmlinks on macOS
Starting with macOS 10.15 (Catalina), Apple introduced a new feature called 'firmlinks' in order to separate the boot volume into two volumes, one read-only and one writable but still present them to the user as a single volume. Along with this change, Apple removed the ability to create symlinks in the root directory and replaced them with 'synthetic firmlinks'. See 'man synthetic.conf'
When FSEevents reports the path of changed files, if the path involves a synthetic firmlink, the path is reported from the point of the synthetic firmlink and not the real path. For example:
Real path:
/System/Volumes/Data/network/working/directory/foo.txt
Synthetic firmlink:
/network -> /System/Volumes/Data/network
FSEvents path:
/network/working/directory/foo.txt
This causes the FSEvents path to not match against the worktree directory.
There are several ways in which synthetic firmlinks can be created:
they can be defined in /etc/synthetic.conf, the automounter can create them, and there may be other means. Simply reading /etc/synthetic.conf is insufficient. No matter what process creates synthetic firmlinks, they all get created in the root directory.
Therefore, in order to deal with synthetic firmlinks, the root directory is scanned and the first possible synthetic firmink that, when resolved, is a prefix of the worktree is used to map FSEvents paths to worktree paths.
fsmonitor: deal with synthetic firmlinks on macOS
Starting with macOS 10.15 (Catalina), Apple introduced a new feature called 'firmlinks' in order to separate the boot volume into two volumes, one read-only and one writable but still present them to the user as a single volume. Along with this change, Apple removed the ability to create symlinks in the root directory and replaced them with 'synthetic firmlinks'. See 'man synthetic.conf'
When FSEevents reports the path of changed files, if the path involves a synthetic firmlink, the path is reported from the point of the synthetic firmlink and not the real path. For example:
Real path:
/System/Volumes/Data/network/working/directory/foo.txt
Synthetic firmlink:
/network -> /System/Volumes/Data/network
FSEvents path:
/network/working/directory/foo.txt
This causes the FSEvents path to not match against the worktree directory.
There are several ways in which synthetic firmlinks can be created:
they can be defined in /etc/synthetic.conf, the automounter can create them, and there may be other means. Simply reading /etc/synthetic.conf is insufficient. No matter what process creates synthetic firmlinks, they all get created in the root directory.
Therefore, in order to deal with synthetic firmlinks, the root directory is scanned and the first possible synthetic firmink that, when resolved, is a prefix of the worktree is used to map FSEvents paths to worktree paths.
--- builtin/fsmonitor--daemon.c | 8 +++ compat/fsmonitor/fsm-listen-darwin.c | 6 +- compat/fsmonitor/fsm-path-utils-darwin.c | 92 ++++++++++++++++++++++++ compat/fsmonitor/fsm-path-utils-win32.c | 17 +++++ fsmonitor--daemon.h | 3 + fsmonitor-path-utils.h | 36 ++++++++++ 6 files changed, 161 insertions(+), 1 deletion(-)
builtin/fsmonitor--daemon.c +8 -0
@@ -3,6 +3,7 @@#include "parse-options.h" #include "fsmonitor.h" #include "fsmonitor-ipc.h" +#include "fsmonitor-path-utils.h" #include "compat/fsmonitor/fsm-health.h" #include "compat/fsmonitor/fsm-listen.h" #include "fsmonitor--daemon.h"
@@ -1282,6 +1283,13 @@ static int fsmonitor_run_daemon(void)strbuf_addstr (&state.path_worktree_watch,absolute_path (get_git_work_tree ())); state.nr_paths_watching =1 ; + state.alias.alias =NULL ; + state.alias.points_to =NULL ; +if (fsmonitor__get_alias (state.path_worktree_watch.buf, &state.alias)) { + err =error (_ ("could not get worktree alias" )); + goto done; + } +/* * We create and delete cookie files somewhere inside the .git * directory to help us keep sync with the file system. If
compat/fsmonitor/fsm-listen-darwin.c +5 -1
@@ -26,6 +26,7 @@#include "fsmonitor.h" #include "fsm-listen.h" #include "fsmonitor--daemon.h" +#include "fsmonitor-path-utils.h" struct fsm_listen_data {
@@ -209,7 +210,9 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,/* * On Mac, we receive an array of absolute paths. */ - path_k = paths[k]; + path_k =fsmonitor__resolve_alias (paths[k], &state->alias); +if (!path_k) + path_k = paths[k];/* * If you want to debug FSEvents, log them to GIT_TRACE_FSMONITOR.
@@ -238,6 +241,7 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,fsmonitor_force_resync (state);fsmonitor_batch__free_list (batch);string_list_clear (&cookie_list,0 ); + batch =NULL ;/* * We assume that any events that we received
compat/fsmonitor/fsm-path-utils-darwin.c +92 -0
100 lines not on this page · raw message
compat/fsmonitor/fsm-path-utils-win32.c +17 -0
@@ -126,3 +126,20 @@ int fsmonitor__is_fs_remote(const char *path)return -1 ;return fs.is_remote; } + +/* +* No-op for now. +*/ +int fsmonitor__get_alias (const char *path,struct alias_info *info) +{ +return 0 ; +} + +/* +* No-op for now. +*/ +char *fsmonitor__resolve_alias (const char *path, +const struct alias_info *info) +{ +return NULL ; +}
fsmonitor--daemon.h +3 -0
@@ -8,6 +8,7 @@#include "run-command.h" #include "simple-ipc.h" #include "thread-utils.h" +#include "fsmonitor-path-utils.h" struct fsmonitor_batch ;struct fsmonitor_token_data ;
@@ -43,6 +44,7 @@ struct fsmonitor_daemon_state {struct strbuf path_worktree_watch;struct strbuf path_gitdir_watch; +struct alias_info alias;int nr_paths_watching;struct fsmonitor_token_data *current_token_data;
@@ -59,6 +61,7 @@ struct fsmonitor_daemon_state {struct ipc_server_data *ipc_server_data;struct strbuf path_ipc; + };/*
fsmonitor-path-utils.h +36 -0
@@ -1,6 +1,14 @@ #ifndef FSM_PATH_UTILS_H #define FSM_PATH_UTILS_H +#include "strbuf.h" + +struct alias_info +{ + char *alias; + char *points_to; +}; + struct fs_info { int is_remote; char *typename;
@@ -20,4 +28,32 @@ int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info);*/ int fsmonitor__is_fs_remote (const char *path); +/* +* Get the alias in given path, if any. +* +* Sets alias to the first alias that matches any part of the path. +* +* If an alias is found, info.alias and info.points_to are set to the +* found mapping. +* +* Returns -1 on error, 0 otherwise. +* +* The caller owns the storage that is occupied by set info.alias and +* info.points_to and is responsible for releasing it with `free(3)` +* when done. +*/ +int fsmonitor__get_alias (const char *path,struct alias_info *info); + +/* +* Resolve the path against the given alias. +* +* Returns the resolved path if there is one, NULL otherwise. +* +* The caller owns the storage that the returned string occupies and +* is responsible for releasing it with `free(3)` when done. +*/ +char *fsmonitor__resolve_alias (const char *path, +const struct alias_info *info); + +#endif
Signature · 1 line
gitgitgadget
PATCH v8 5/5 fsmonitor: add documentation for allowRemote and socketDir options
In reply to Eric DeCosta via GitGitGadgetFrom: Eric DeCosta <[email protected]>
Add documentation for 'fsmonitor.allowRemote' and 'fsmonitor.socketDir'. Call-out experimental nature of 'fsmonitor.allowRemote' and limited file system support for 'fsmonitor.socketDir'.
--- Documentation/git-fsmonitor--daemon.txt | 35 +++++++++++++++++++++++++ 1 file changed, 35 insertions(+)
Documentation/git-fsmonitor--daemon.txt +35 -0
@@ -70,6 +70,41 @@ the change (as happening against the super repo). However, the client will properly ignore these extra events, so performance may be affected but it will not cause an incorrect result. +By default, the fsmonitor daemon refuses to work against network-mounted +repositories; this my be overridden by setting `fsmonitor.allowRemote` to +`true`. Note, however, that the fsmonitor daemon is not guaranteed to work +correctly with all network-mounted repositores and such use is considered +experimental. + +On Mac OS, the inter-process communication (IPC) between various Git +commands and the fsmonitor daemon is done via a Unix domain socket (UDS). +Usage of UDS requires the creation of a file which, by default, is created +in the .git directory. If the fsmonitor daemon detects that the .git directory +is on a network-mounted file system, it will create the UDS file in $HOME. If +$HOME itself is on a network-mounted file system or if $HOME is not the desired +location for the UDS file, 'fsmonitor.socketDir' may be set to any valid, local +directory on a file system with proper support. Mac OS native file systems have +the required support. File systems known to lack support include FAT32 and +NTFS. Other file systems may or many not have the needed support; the fsmonitor +daemon is not guaranteed to work with these file systems and such use is +considered experimental. + +CONFIGURATION +------------- +When `core.fsmonitor` is set to `true` (see linkgit:git-config[1]) +the fsmonitor daemon will pay attention to the following configuration +variables: + +fsmonitor.allowRemote:: + By default, the daemon refuses to work against network-mounted + repositories. Setting `fsmonitor.allowRemote` to `true` overrides + this behavior. + +fsmonitor.socketDir:: + This option is only used by the Mac OS implementation of the fsmonitor + daemon. If set, 'fsmonitor.socketDir' must be set to a valid, local + directory on a file system that can support Unix domain sockets (UDS). + GIT --- Part of the linkgit:git[1] suite
Signature · 1 line
gitgitgadget
Re: [PATCH v8 5/5] fsmonitor: add documentation for allowRemote and socketDir options
Documentation/git-fsmonitor--daemon.txt the change (as happening against the super repo). However, the client · line 74
Quoted 13 lines · Eric DeCosta via GitGitGadget
Add documentation for 'fsmonitor.allowRemote' and 'fsmonitor.socketDir'. Call-out experimental nature of 'fsmonitor.allowRemote' and limited file system support for 'fsmonitor.socketDir'.
Signed-off-by Eric DeCosta <[email protected]>---
Documentation/git-fsmonitor--daemon.txt +5
-0@@ -70,6 +70,41 @@ the change (as happening against the super repo). However, the client +By default, the fsmonitor daemon refuses to work against network-mounted +repositories; this my be overridden by setting `fsmonitor.allowRemote` to +`true`. Note, however, that the fsmonitor daemon is not guaranteed to work +correctly with all network-mounted repositores and such use is considered +experimental.
s/repositores/repositories/
Documentation/git-fsmonitor--daemon.txt the change (as happening against the super repo). However, the client · line 82
Quoted 4 lines
Documentation/git-fsmonitor--daemon.txt +4
-0+On Mac OS, the inter-process communication (IPC) between various Git +commands and the fsmonitor daemon is done via a Unix domain socket (UDS). +Usage of UDS requires the creation of a file which, by default, is created +in the .git directory. If the fsmonitor daemon detects that the .git directory
Typesetting: s/.git/.git/g
Documentation/git-fsmonitor--daemon.txt the change (as happening against the super repo). However, the client · line 83
Quoted 1 line
Documentation/git-fsmonitor--daemon.txt +1
-0+is on a network-mounted file system, it will create the UDS file in $HOME. If
There's a gap in the explanation as to _why_ fsmonitor won't use the .git directory for this file when on a network-mounted filesystem and instead chooses $HOME. For the reader who is not well-versed in Unix sockets/filesystems, it may be difficult to understand the logic behind this. This gap is somewhat filled in a few sentences later, but it makes for potentially confusing reading until then.
Should the reader know the name of the socket file or at least the templated form of the name? The first question which popped into my head upon reading this was whether it was going to pollute my home directory with non-hidden files. If this had mentioned something along the lines of "creation of a file named .git-fsmonitor-*" or "creation of a hidden file" then I would have understood immediately that the file would have been hidden.
Documentation/git-fsmonitor--daemon.txt the change (as happening against the super repo). However, the client · line 84
Quoted 1 line
Documentation/git-fsmonitor--daemon.txt +1
-0+$HOME itself is on a network-mounted file system or if $HOME is not the desired
To be consistent with formatting elsewhere in the Git documentation, let's typeset this as $HOME (with backticks).
Aside: The spelling "filesystem" appears almost five times as often as "file system" in Git documentation, however, this particular file already uses "file system" and does so consistently, so it makes sense to follow suit as you do here. Changing to use "filesystem" instead, if such a task is desirable, is outside the scope of this patch series.
Documentation/git-fsmonitor--daemon.txt the change (as happening against the super repo). However, the client · line 85
Quoted 1 line
Documentation/git-fsmonitor--daemon.txt +1
-0+location for the UDS file, 'fsmonitor.socketDir' may be set to any valid, local
For consistency, let's use backticks here, as well: fsmonitor.socketDir
Documentation/git-fsmonitor--daemon.txt the change (as happening against the super repo). However, the client · line 86
Quoted 1 line
Documentation/git-fsmonitor--daemon.txt +1
-0+directory on a file system with proper support. Mac OS native file systems have
Together with the above comment about a gap in the explanation, I found myself scratching my head about what "proper support" meant (when pretending to read this as a person not particularly familiar with Unix sockets or filesystems).
Also, although this explains how to work around the case when $HOME is itself network-mounted, what happens if $HOME is network-mounted and the user does not set fsmonitor.socketDir? Does it error out? Does it simply misbehave in some way? Should it error out? (I would think "yes".)
Documentation/git-fsmonitor--daemon.txt the change (as happening against the super repo). However, the client · line 88
Quoted 2 lines
Documentation/git-fsmonitor--daemon.txt +2
-0+the required support. File systems known to lack support include FAT32 and +NTFS. Other file systems may or many not have the needed support; the fsmonitor
s/many/may/
Documentation/git-fsmonitor--daemon.txt the change (as happening against the super repo). However, the client · line 90
Quoted 2 lines
Documentation/git-fsmonitor--daemon.txt +2
-0+daemon is not guaranteed to work with these file systems and such use is +considered experimental.
Taking the above comments into account, here's my attempt at a rewrite:
On Mac OS, the inter-process communication (IPC) between various Git commands and the fsmonitor daemon occurs via a Unix domain socket (UDS) -- a special type of file -- which is supported by the native Mac OS filesystems but not by network-mounted filesystems, NTFS or FAT32. Other file systems may or many not have the needed support; the fsmonitor daemon is not guaranteed to work with these file systems and such use is considered experimental.
By default, the socket is created in the .git directory, however, if the .git directory is on a network-mounted file system, it will instead be created at $HOME/.git-fsmonitor-* unless $HOME itself is on a network-mounted file system, in which case you must set the configuration variable fsmonitor.socketDir to the path of a directory on a Mac OS native filesystem in which to create the socket file.
Documentation/git-fsmonitor--daemon.txt the change (as happening against the super repo). However, the client · line 96
Quoted 5 lines
Documentation/git-fsmonitor--daemon.txt +5
-0+CONFIGURATION +------------- +When `core.fsmonitor` is set to `true` (see linkgit:git-config[1]) +the fsmonitor daemon will pay attention to the following configuration +variables:
We probably want a blank line after the header underline and before this paragraph.
Documentation/git-fsmonitor--daemon.txt the change (as happening against the super repo). However, the client · line 106
Quoted 9 lines
Documentation/git-fsmonitor--daemon.txt +9
-0+fsmonitor.allowRemote:: + By default, the daemon refuses to work against network-mounted + repositories. Setting `fsmonitor.allowRemote` to `true` overrides + this behavior. + +fsmonitor.socketDir:: + This option is only used by the Mac OS implementation of the fsmonitor + daemon. If set, 'fsmonitor.socketDir' must be set to a valid, local + directory on a file system that can support Unix domain sockets (UDS).
Typeset with backticks: fsmonitor.socketDir
The word "valid" seems unnecessary. A possible rewrite:
This Mac OS-specific option, if set, specifies the directory in which to create the Unix domain socket used for communication between fsmonitor and various Git commands. The directory must reside on a native Mac OS filesystem as discussed above.
RE: [PATCH v8 5/5] fsmonitor: add documentation for allowRemote and socketDir options
Quoted 141 lines
-----Original Message-----
From: Eric Sunshine <[email protected]>
Sent: Saturday, September 17, 2022 2:09 AM
To: Eric DeCosta via GitGitGadget <[email protected]>
Cc: Git List <[email protected]>; Jeff Hostetler <[email protected]>; Torsten Bögershausen <[email protected]>; Ævar Arnfjörð Bjarmason <[email protected]>; Ramsay Jones <[email protected]>; Johannes Schindelin <[email protected]>; Eric DeCosta <[email protected]>
Subject: Re: [PATCH v8 5/5] fsmonitor: add documentation for allowRemote and socketDir optionsQuoted 10 lines · Eric DeCosta via GitGitGadget
Add documentation for 'fsmonitor.allowRemote' and 'fsmonitor.socketDir'. Call-out experimental nature of 'fsmonitor.allowRemote' and limited file system support for 'fsmonitor.socketDir'.
Signed-off-by Eric DeCosta <[email protected]>---
Documentation/git-fsmonitor--daemon.txt +0
-0@@ -70,6 +70,41 @@ the change (as happening against the super repo).However, the client
Documentation/git-fsmonitor--daemon.txt
Quoted 5 lines
Documentation/git-fsmonitor--daemon.txt +5
-0+By default, the fsmonitor daemon refuses to work against +network-mounted repositories; this my be overridden by setting +`fsmonitor.allowRemote` to `true`. Note, however, that the fsmonitor +daemon is not guaranteed to work correctly with all network-mounted +repositores and such use is considered experimental.s/repositores/repositories/
Documentation/git-fsmonitor--daemon.txt
Quoted 2 lines
Documentation/git-fsmonitor--daemon.txt +2
-0+On Mac OS, the inter-process communication (IPC) between various Git +commands and the fsmonitor daemon is done via a Unix domain socket(UDS).
Documentation/git-fsmonitor--daemon.txt
Quoted 3 lines
Documentation/git-fsmonitor--daemon.txt +3
-0+Usage of UDS requires the creation of a file which, by default, is +created in the .git directory. If the fsmonitor daemon detects that +the .git directoryTypesetting: s/.git/
.git/gQuoted 2 lines
+is on a network-mounted file system, it will create the UDS file in +$HOME. If
There's a gap in the explanation as to _why_ fsmonitor won't use the .git directory for this file when on a network-mounted filesystem and instead chooses $HOME. For the reader who is not well-versed in Unix sockets/filesystems, it may be difficult to understand the logic behind this. This gap is somewhat filled in a few sentences later, but it makes for potentially confusing reading until then.
Should the reader know the name of the socket file or at least the templated form of the name? The first question which popped into my head upon reading this was whether it was going to pollute my home directory with non-hidden files. If this had mentioned something along the lines of "creation of a file named
.git-fsmonitor-*" or "creation of a hidden file" then I would have understood immediately that the file would have been hidden.Quoted 2 lines
+$HOME itself is on a network-mounted file system or if $HOME is not +the desired
To be consistent with formatting elsewhere in the Git documentation, let's typeset this as
$HOME(with backticks).Aside: The spelling "filesystem" appears almost five times as often as "file system" in Git documentation, however, this particular file already uses "file system" and does so consistently, so it makes sense to follow suit as you do here. Changing to use "filesystem" instead, if such a task is desirable, is outside the scope of this patch series.
Documentation/git-fsmonitor--daemon.txt
Quoted 2 lines
Documentation/git-fsmonitor--daemon.txt +2
-0+location for the UDS file, 'fsmonitor.socketDir' may be set to any +valid, localFor consistency, let's use backticks here, as well:
fsmonitor.socketDirDocumentation/git-fsmonitor--daemon.txt
Quoted 2 lines
Documentation/git-fsmonitor--daemon.txt +2
-0+directory on a file system with proper support. Mac OS native file +systems haveTogether with the above comment about a gap in the explanation, I found myself scratching my head about what "proper support" meant (when pretending to read this as a person not particularly familiar with Unix sockets or filesystems).
Also, although this explains how to work around the case when $HOME is itself network-mounted, what happens if $HOME is network-mounted and the user does not set
fsmonitor.socketDir? Does it error out? Does it simply misbehave in some way? Should it error out? (I would think "yes".)Documentation/git-fsmonitor--daemon.txt
Quoted 3 lines
Documentation/git-fsmonitor--daemon.txt +3
-0+the required support. File systems known to lack support include +FAT32 and NTFS. Other file systems may or many not have the needed +support; the fsmonitors/many/may/
Documentation/git-fsmonitor--daemon.txt
Quoted 2 lines
Documentation/git-fsmonitor--daemon.txt +2
-0+daemon is not guaranteed to work with these file systems and such use +is considered experimental.Taking the above comments into account, here's my attempt at a rewrite:
On Mac OS, the inter-process communication (IPC) between various Git commands and the fsmonitor daemon occurs via a Unix domain socket (UDS) -- a special type of file -- which is supported by the native Mac OS filesystems but not by network-mounted filesystems, NTFS or FAT32. Other file systems may or many not have the needed support; the fsmonitor daemon is not guaranteed to work with these file systems and such use is considered experimental.
By default, the socket is created in the
.gitdirectory, however, if the.gitdirectory is on a network-mounted file system, it will instead be created at$HOME/.git-fsmonitor-*unless$HOMEitself is on a network-mounted file system, in which case you must set the configuration variablefsmonitor.socketDirto the path of a directory on a Mac OS native filesystem in which to create the socket file.Documentation/git-fsmonitor--daemon.txt the change (as happening against the super repo). However, the client · line 96
Quoted 6 lines
Documentation/git-fsmonitor--daemon.txt +6
-0+CONFIGURATION +------------- +When `core.fsmonitor` is set to `true` (see linkgit:git-config[1]) +the fsmonitor daemon will pay attention to the following +configuration +variables:We probably want a blank line after the header underline and before this paragraph.
Documentation/git-fsmonitor--daemon.txt the change (as happening against the super repo). However, the client
Quoted 7 lines
Documentation/git-fsmonitor--daemon.txt +7
-0+fsmonitor.allowRemote:: + By default, the daemon refuses to work against network-mounted + repositories. Setting `fsmonitor.allowRemote` to `true` overrides + this behavior. + +fsmonitor.socketDir:: + This option is only used by the Mac OS implementation of thefsmonitor
Documentation/git-fsmonitor--daemon.txt the change (as happening against the super repo). However, the client · line 106
Quoted 2 lines
Documentation/git-fsmonitor--daemon.txt +2
-0+ daemon. If set, 'fsmonitor.socketDir' must be set to a valid, local + directory on a file system that can support Unix domain sockets (UDS).
Quoted 8 lines
Typeset with backticks:
fsmonitor.socketDirThe word "valid" seems unnecessary. A possible rewrite:
This Mac OS-specific option, if set, specifies the directory in which to create the Unix domain socket used for communication between fsmonitor and various Git commands. The directory must reside on a native Mac OS filesystem as discussed above.
The way you've re-written things is so much clearer! Much appreciated.
-Eric
v9 cover fsmonitor: option to allow fsmonitor to run against network-mounted repos
In reply to Eric DeCosta via GitGitGadgetFollow-on to the work done to allow Windows to work against network-mounted repos for macOS.
Have macOS take advantage of the same configuration option, 'fsmonitor.allowRemote' that was introduced for Windows. Setting this option to true will override the default behavior (erroring-out) when a network-mounted repo is detected by fsmonitor.
The added wrinkle being that the Unix domain socket (UDS) file used for IPC cannot be created in a network location; instead $HOME is used if the default location is on the network. The user may, optionally, set the 'fsmonitor.socketDir' configuration option to a valid, local directory if $HOME itself is on the network or is simply not the desired location for the UDS file.
An additional issue is that for mount points in the root directory, FSEvents does not report a path that matches the worktree directory due to the introduction of 'synthetic firmlinks'. fsmonitor must map the FSEvents paths to the worktree directory by interrogating the root filesystem for synthetic firmlinks and using that information to translate the path.
v9 differs from v8:
* incorporates code review feedback
* check for incompatibility before communicating with fsmonitor
v8 differs from v7:
* incorporates code review feedback
* gets the rebase right
v7 differs from v6:
* incorporates code review feedback
v6 differs from v5:
* incorporates earlier, Windows-specific changes that have not made it back yet to the master branch
* incorporates code review feedback
* adds documentation for 'fsmonitor.allowRemote' and 'fsmonitor.socketDir'
v5 differs significantly from earlier versions:
* redesign of handling 'fsmonitor.allowRemote' and 'fsmonitor.socketDir' such that these options are no longer added to the settings data structure but are rather read from config at point of use
* refactoring of code for handling platform-specific file system checks via a common interface to avoid platform #ifdef in IPC code and be in-model with other platform-specific fsmonitor code
* dealing with 'synthetic firmlinks' on macOS
Eric DeCosta (6):
fsmonitor: refactor filesystem checks to common interface
fsmonitor: relocate socket file if .git directory is remote
fsmonitor: avoid socket location check if using hook
fsmonitor: deal with synthetic firmlinks on macOS
fsmonitor: check for compatability before communicating with fsmonitor
fsmonitor: add documentation for allowRemote and socketDir options
Documentation/git-fsmonitor--daemon.txt | 48 ++++++- Makefile | 2 + builtin/fsmonitor--daemon.c | 11 +- compat/fsmonitor/fsm-ipc-darwin.c | 48 +++++++ compat/fsmonitor/fsm-ipc-win32.c | 9 ++ compat/fsmonitor/fsm-listen-darwin.c | 6 +- compat/fsmonitor/fsm-path-utils-darwin.c | 132 +++++++++++++++++ compat/fsmonitor/fsm-path-utils-win32.c | 145 +++++++++++++++++++ compat/fsmonitor/fsm-settings-darwin.c | 70 +++------ compat/fsmonitor/fsm-settings-win32.c | 174 +---------------------- contrib/buildsystems/CMakeLists.txt | 4 + fsmonitor--daemon.h | 3 + fsmonitor-ipc.c | 18 ++- fsmonitor-ipc.h | 4 +- fsmonitor-path-utils.h | 59 ++++++++ fsmonitor-settings.c | 60 +++++++- fsmonitor-settings.h | 2 +- fsmonitor.c | 4 + 18 files changed, 559 insertions(+), 240 deletions(-) create mode 100644 compat/fsmonitor/fsm-ipc-darwin.c create mode 100644 compat/fsmonitor/fsm-ipc-win32.c create mode 100644 compat/fsmonitor/fsm-path-utils-darwin.c create mode 100644 compat/fsmonitor/fsm-path-utils-win32.c create mode 100644 fsmonitor-path-utils.h
base-commit: d3fa443f97e3a8d75b51341e2d5bac380b7422df
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1326%2Fedecosta-mw%2Ffsmonitor_macos-v9
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1326/edecosta-mw/fsmonitor_macos-v9
Pull-Request: https://github.com/gitgitgadget/git/pull/1326
Range-diff vs v8:
1: 155a6890806 = 1: 155a6890806 fsmonitor: refactor filesystem checks to common interface
2: b5356497228 ! 2: 68709d788d5 fsmonitor: relocate socket file if .git directory is remote
@@ compat/fsmonitor/fsm-ipc-darwin.c (new)
+ repo_config_get_string(r, "fsmonitor.socketdir", &sock_dir);
+
+ /* Create the socket file in either socketDir or $HOME */
-+ if (sock_dir && *sock_dir)
++ if (sock_dir && *sock_dir) {
+ strbuf_addf(&ipc_file, "%s/.git-fsmonitor-%s",
+ sock_dir, hash_to_hex(hash));
-+ else
++ free(sock_dir);
++ } else {
+ strbuf_addf(&ipc_file, "~/.git-fsmonitor-%s", hash_to_hex(hash));
++ }
+
+ ipc_path = interpolate_path(ipc_file.buf, 1);
+ if (!ipc_path)
3: 6719ca2b24d = 3: 6ddd922917a fsmonitor: avoid socket location check if using hook
4: d736cb8fa90 ! 4: 73afd9f3122 fsmonitor: deal with synthetic firmlinks on macOS@@ Commit message
- fsmonitor: deal with synthetic firmlinks on macOS
-
- Starting with macOS 10.15 (Catalina), Apple introduced a new feature
- called 'firmlinks' in order to separate the boot volume into two
- volumes, one read-only and one writable but still present them to the
- user as a single volume. Along with this change, Apple removed the
- ability to create symlinks in the root directory and replaced them with
- 'synthetic firmlinks'. See 'man synthetic.conf'
-
- When FSEevents reports the path of changed files, if the path involves
- a synthetic firmlink, the path is reported from the point of the
- synthetic firmlink and not the real path. For example:
-
- Real path:
- /System/Volumes/Data/network/working/directory/foo.txt
-
- Synthetic firmlink:
- /network -> /System/Volumes/Data/network
-
- FSEvents path:
- /network/working/directory/foo.txt
-
- This causes the FSEvents path to not match against the worktree
- directory.
-
- There are several ways in which synthetic firmlinks can be created:
- they can be defined in /etc/synthetic.conf, the automounter can create
- them, and there may be other means. Simply reading /etc/synthetic.conf
- is insufficient. No matter what process creates synthetic firmlinks,
- they all get created in the root directory.
-
- Therefore, in order to deal with synthetic firmlinks, the root directory
- is scanned and the first possible synthetic firmink that, when resolved,
- is a prefix of the worktree is used to map FSEvents paths to worktree
- paths.
-
- Signed-off-by: Eric DeCosta <[email protected]>
-
- fsmonitor: deal with synthetic firmlinks on macOS
-
- Starting with macOS 10.15 (Catalina), Apple introduced a new feature
- called 'firmlinks' in order to separate the boot volume into two
- volumes, one read-only and one writable but still present them to the
- user as a single volume. Along with this change, Apple removed the
- ability to create symlinks in the root directory and replaced them with
- 'synthetic firmlinks'. See 'man synthetic.conf'
-
- When FSEevents reports the path of changed files, if the path involves
- a synthetic firmlink, the path is reported from the point of the
- synthetic firmlink and not the real path. For example:
-
- Real path:
- /System/Volumes/Data/network/working/directory/foo.txt
-
- Synthetic firmlink:
- /network -> /System/Volumes/Data/network
-
- FSEvents path:
- /network/working/directory/foo.txt
-
- This causes the FSEvents path to not match against the worktree
- directory.
-
- There are several ways in which synthetic firmlinks can be created:
- they can be defined in /etc/synthetic.conf, the automounter can create
- them, and there may be other means. Simply reading /etc/synthetic.conf
- is insufficient. No matter what process creates synthetic firmlinks,
- they all get created in the root directory.
-
- Therefore, in order to deal with synthetic firmlinks, the root directory
- is scanned and the first possible synthetic firmink that, when resolved,
- is a prefix of the worktree is used to map FSEvents paths to worktree
- paths.
-
- Signed-off-by: Eric DeCosta <[email protected]>
-
## builtin/fsmonitor--daemon.c ##
@@
#include "parse-options.h"
-: ----------- > 5: 02afeaa01be fsmonitor: check for compatability before communicating with fsmonitor 5: ddf4e3e6442 ! 6: 0e8ea28acc1 fsmonitor: add documentation for allowRemote and socketDir options
@@ Commit message
## Documentation/git-fsmonitor--daemon.txt ##
-@@ Documentation/git-fsmonitor--daemon.txt: the change (as happening against the super repo). However, the client +@@ Documentation/git-fsmonitor--daemon.txt: git-fsmonitor{litdd}daemon(1)
+
+ NAME
+ ----
+-git-fsmonitor--daemon - A Built-in File System Monitor
++git-fsmonitor--daemon - A Built-in Filesystem Monitor
+
+ SYNOPSIS
+ --------
+@@ Documentation/git-fsmonitor--daemon.txt: DESCRIPTION
+ -----------
+
+ A daemon to watch the working directory for file and directory
+-changes using platform-specific file system notification facilities.
++changes using platform-specific filesystem notification facilities.
+
+ This daemon communicates directly with commands like git status
+ using the link:technical/api-simple-ipc.html[simple IPC] interface
+@@ Documentation/git-fsmonitor--daemon.txt: CAVEATS
+ -------
+
+ The fsmonitor daemon does not currently know about submodules and does
+-not know to filter out file system events that happen within a
++not know to filter out filesystem events that happen within a
+ submodule. If fsmonitor daemon is watching a super repo and a file is
+ modified within the working directory of a submodule, it will report
+ the change (as happening against the super repo). However, the client
will properly ignore these extra events, so performance may be affected
but it will not cause an incorrect result.
+By default, the fsmonitor daemon refuses to work against network-mounted
-+repositories; this my be overridden by setting fsmonitor.allowRemote to ++repositories; this may be overridden by setting fsmonitor.allowRemote to
+true. Note, however, that the fsmonitor daemon is not guaranteed to work
-+correctly with all network-mounted repositores and such use is considered ++correctly with all network-mounted repositories and such use is considered
+experimental.
+
+On Mac OS, the inter-process communication (IPC) between various Git
-+commands and the fsmonitor daemon is done via a Unix domain socket (UDS). -+Usage of UDS requires the creation of a file which, by default, is created -+in the .git directory. If the fsmonitor daemon detects that the .git directory -+is on a network-mounted file system, it will create the UDS file in $HOME. If -+$HOME itself is on a network-mounted file system or if $HOME is not the desired -+location for the UDS file, 'fsmonitor.socketDir' may be set to any valid, local -+directory on a file system with proper support. Mac OS native file systems have -+the required support. File systems known to lack support include FAT32 and -+NTFS. Other file systems may or many not have the needed support; the fsmonitor -+daemon is not guaranteed to work with these file systems and such use is -+considered experimental.
++commands and the fsmonitor daemon is done via a Unix domain socket (UDS) -- a ++special type of file -- which is supported by native Mac OS filesystems, ++but not on network-mounted filesystems, NTFS, or FAT32. Other filesystems ++may or may not have the needed support; the fsmonitor daemon is not guaranteed ++to work with these filesystems and such use is considered experimental. ++
++By default, the socket is created in the .git directory, however, if the ++.git directory is on a network-mounted filesystem, it will be instead be ++created at $HOME/.git-fsmonitor-* unless $HOME itself is on a ++network-mounted filesystem in which case you must set the configuration ++variable fsmonitor.socketDir to the path of a directory on a Mac OS native ++filesystem in which to create the socket file.
++
++If none of the above directories (.git, $HOME, or fsmonitor.socketDir) ++is on a native Mac OS file filesystem the fsmonitor daemon will report an ++error that will cause the daemon and the currently running command to exit.
+
+CONFIGURATION
+-------------
++
+When core.fsmonitor is set to true (see linkgit:git-config[1]) +the fsmonitor daemon will pay attention to the following configuration +variables:
+
-+fsmonitor.allowRemote::
++fsmonitor.allowRemote::
+ By default, the daemon refuses to work against network-mounted
+ repositories. Setting fsmonitor.allowRemote to true overrides
+ this behavior.
+
-+fsmonitor.socketDir::
-+ This option is only used by the Mac OS implementation of the fsmonitor
-+ daemon. If set, 'fsmonitor.socketDir' must be set to a valid, local
-+ directory on a file system that can support Unix domain sockets (UDS).
++fsmonitor.socketDir::
++ This Mac OS-specific option, if set, specifies the directory in
++ which to create the Unix domain socket used for communication
++ between fsmonitor and various Git commands. The directory must
++ reside on a native Mac OS filesystem as discussed above.
+
GIT
---
Signature · 1 line
gitgitgadget
PATCH v9 1/6 fsmonitor: refactor filesystem checks to common interface
From: Eric DeCosta <[email protected]>
Provide a common interface for getting basic filesystem information including filesystem type and whether the filesystem is remote.
Refactor existing code for getting basic filesystem info and detecting remote file systems to the new interface.
Refactor filesystem checks to leverage new interface. For macOS, error-out if the Unix Domain socket (UDS) file is on a remote filesystem.
--- Makefile | 1 + compat/fsmonitor/fsm-path-utils-darwin.c | 40 ++++++ compat/fsmonitor/fsm-path-utils-win32.c | 128 +++++++++++++++++ compat/fsmonitor/fsm-settings-darwin.c | 62 +++----- compat/fsmonitor/fsm-settings-win32.c | 172 +---------------------- contrib/buildsystems/CMakeLists.txt | 2 + fsmonitor-path-utils.h | 23 +++ fsmonitor-settings.c | 50 +++++++ 8 files changed, 263 insertions(+), 215 deletions(-) create mode 100644 compat/fsmonitor/fsm-path-utils-darwin.c create mode 100644 compat/fsmonitor/fsm-path-utils-win32.c create mode 100644 fsmonitor-path-utils.h
Makefile +1 -0
@@ -2039,6 +2039,7 @@ endif ifdef FSMONITOR_OS_SETTINGS COMPAT_CFLAGS += -DHAVE_FSMONITOR_OS_SETTINGS COMPAT_OBJS += compat/fsmonitor/fsm-settings-$(FSMONITOR_OS_SETTINGS).o + COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_OS_SETTINGS).o endif ifeq ($(TCLTK_PATH),)
compat/fsmonitor/fsm-path-utils-darwin.c new file +40 -0
@@ -0,0 +1,40 @@ +#include "fsmonitor.h" +#include "fsmonitor-path-utils.h" +#include <sys/param.h> +#include <sys/mount.h> + +int fsmonitor__get_fs_info (const char *path,struct fs_info *fs_info) +{ +struct statfs fs; +if (statfs (path, &fs) ==-1 ) { +int saved_errno = errno; +trace_printf_key (&trace_fsmonitor,"statfs('%s') failed: %s" , + path,strerror (saved_errno)); + errno = saved_errno; +return -1 ; + } + +trace_printf_key (&trace_fsmonitor, +"statfs('%s') [type 0x%08x][flags 0x%08x] '%s'" , + path, fs.f_type, fs.f_flags, fs.f_fstypename); + +if (!(fs.f_flags &MNT_LOCAL )) + fs_info->is_remote =1 ; +else + fs_info->is_remote =0 ; + + fs_info->typename = fs.f_fstypename; + +trace_printf_key (&trace_fsmonitor, +"'%s' is_remote: %d" , + path, fs_info->is_remote); +return 0 ; +} + +int fsmonitor__is_fs_remote (const char *path) +{ +struct fs_info fs; +if (fsmonitor__get_fs_info (path, &fs)) +return -1 ; +return fs.is_remote; +}
compat/fsmonitor/fsm-path-utils-win32.c new file +128 -0
128 lines not on this page · raw message
compat/fsmonitor/fsm-settings-darwin.c +17 -45
96 lines not on this page · raw message
compat/fsmonitor/fsm-settings-win32.c +2 -170
190 lines not on this page · raw message
contrib/buildsystems/CMakeLists.txt +2 -0
@@ -308,6 +308,7 @@ if(SUPPORTS_SIMPLE_IPC) add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-win32.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-win32.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-win32.c)
@@ -315,6 +316,7 @@ if(SUPPORTS_SIMPLE_IPC) add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-darwin.c)
fsmonitor-path-utils.h new file +23 -0
@@ -0,0 +1,23 @@ +#ifndef FSM_PATH_UTILS_H +#define FSM_PATH_UTILS_H + +struct fs_info { +int is_remote; +char *typename; +}; + +/* +* Get some basic filesystem informtion for the given path +* +* Returns -1 on error, zero otherwise. +*/ +int fsmonitor__get_fs_info (const char *path,struct fs_info *fs_info); + +/* +* Determines if the filesystem that path resides on is remote. +* +* Returns -1 on error, 0 if not remote, 1 if remote. +*/ +int fsmonitor__is_fs_remote (const char *path); + +#endif
fsmonitor-settings.c +50 -0
68 lines not on this page · raw message
Signature · 1 line
gitgitgadget
PATCH v9 2/6 fsmonitor: relocate socket file if .git directory is remote
In reply to Eric DeCosta via GitGitGadgetFrom: Eric DeCosta <[email protected]>
If the .git directory is on a remote file system, create the socket file in 'fsmonitor.socketDir' if it is defined, else create it in $HOME.
--- Makefile | 1 + builtin/fsmonitor--daemon.c | 3 +- compat/fsmonitor/fsm-ipc-darwin.c | 48 ++++++++++++++++++++++++++ compat/fsmonitor/fsm-ipc-win32.c | 9 +++++ compat/fsmonitor/fsm-settings-darwin.c | 2 +- contrib/buildsystems/CMakeLists.txt | 2 ++ fsmonitor-ipc.c | 18 +++++----- fsmonitor-ipc.h | 4 ++- 8 files changed, 74 insertions(+), 13 deletions(-) create mode 100644 compat/fsmonitor/fsm-ipc-darwin.c create mode 100644 compat/fsmonitor/fsm-ipc-win32.c
Makefile +1 -0
@@ -2034,6 +2034,7 @@ ifdef FSMONITOR_DAEMON_BACKEND COMPAT_CFLAGS += -DHAVE_FSMONITOR_DAEMON_BACKEND COMPAT_OBJS += compat/fsmonitor/fsm-listen-$(FSMONITOR_DAEMON_BACKEND).o COMPAT_OBJS += compat/fsmonitor/fsm-health-$(FSMONITOR_DAEMON_BACKEND).o + COMPAT_OBJS += compat/fsmonitor/fsm-ipc-$(FSMONITOR_DAEMON_BACKEND).o endif ifdef FSMONITOR_OS_SETTINGS
builtin/fsmonitor--daemon.c +2 -1
@@ -1343,7 +1343,8 @@ static int fsmonitor_run_daemon(void)* directory.) */ strbuf_init (&state.path_ipc,0 ); -strbuf_addstr (&state.path_ipc,absolute_path (fsmonitor_ipc__get_path ())); +strbuf_addstr (&state.path_ipc, +absolute_path (fsmonitor_ipc__get_path (the_repository)));/* * Confirm that we can create platform-specific resources for the
compat/fsmonitor/fsm-ipc-darwin.c new file +48 -0
48 lines not on this page · raw message
compat/fsmonitor/fsm-ipc-win32.c new file +9 -0
@@ -0,0 +1,9 @@ +#include "config.h" +#include "fsmonitor-ipc.h" + +const char *fsmonitor_ipc__get_path (struct repository *r) { +static char *ret; +if (!ret) + ret =git_pathdup ("fsmonitor--daemon.ipc" ); +return ret; +}
compat/fsmonitor/fsm-settings-darwin.c +1 -1
@@ -26,7 +26,7 @@static enum fsmonitor_reason check_uds_volume (struct repository *r) {struct fs_info fs; -const char *ipc_path =fsmonitor_ipc__get_path (); +const char *ipc_path =fsmonitor_ipc__get_path (r);struct strbuf path =STRBUF_INIT ;strbuf_add (&path, ipc_path,strlen (ipc_path));
contrib/buildsystems/CMakeLists.txt +2 -0
@@ -308,6 +308,7 @@ if(SUPPORTS_SIMPLE_IPC) add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-win32.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-win32.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-win32.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
@@ -316,6 +317,7 @@ if(SUPPORTS_SIMPLE_IPC) add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-darwin.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
fsmonitor-ipc.c +8 -10
@@ -18,7 +18,7 @@ int fsmonitor_ipc__is_supported(void)return 0 ; } -const char *fsmonitor_ipc__get_path (void ) +const char *fsmonitor_ipc__get_path (struct repository *r) {return NULL ; }
@@ -47,11 +47,9 @@ int fsmonitor_ipc__is_supported(void)return 1 ; } -GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc") -enum ipc_active_state fsmonitor_ipc__get_state (void ) { -return ipc_get_active_state (fsmonitor_ipc__get_path ()); +return ipc_get_active_state (fsmonitor_ipc__get_path (the_repository)); }static int spawn_daemon (void )
@@ -81,8 +79,8 @@ int fsmonitor_ipc__send_query(const char *since_token,trace2_data_string ("fsm_client" ,NULL ,"query/command" , tok); try_again: - state =ipc_client_try_connect (fsmonitor_ipc__get_path (), &options, - &connection); + state =ipc_client_try_connect (fsmonitor_ipc__get_path (the_repository), + &options, &connection);switch (state) {case IPC_STATE__LISTENING :
@@ -117,13 +115,13 @@ try_again:case IPC_STATE__INVALID_PATH : ret =error (_ ("fsmonitor_ipc__send_query: invalid path '%s'" ), -fsmonitor_ipc__get_path ()); +fsmonitor_ipc__get_path (the_repository)); goto done;case IPC_STATE__OTHER_ERROR :default : ret =error (_ ("fsmonitor_ipc__send_query: unspecified error on '%s'" ), -fsmonitor_ipc__get_path ()); +fsmonitor_ipc__get_path (the_repository)); goto done; }
@@ -149,8 +147,8 @@ int fsmonitor_ipc__send_command(const char *command, options.wait_if_busy =1 ; options.wait_if_not_found =0 ; - state =ipc_client_try_connect (fsmonitor_ipc__get_path (), &options, - &connection); + state =ipc_client_try_connect (fsmonitor_ipc__get_path (the_repository), + &options, &connection);if (state !=IPC_STATE__LISTENING ) {die (_ ("fsmonitor--daemon is not running" ));return -1 ;
fsmonitor-ipc.h +3 -1
@@ -3,6 +3,8 @@#include "simple-ipc.h" +struct repository ; +/* * Returns true if built-in file system monitor daemon is defined * for this platform.
@@ -16,7 +18,7 @@ int fsmonitor_ipc__is_supported(void);* * Returns NULL if the daemon is not supported on this platform. */ -const char *fsmonitor_ipc__get_path (void ); +const char *fsmonitor_ipc__get_path (struct repository *r);/* * Try to determine whether there is a `git-fsmonitor--daemon` process
Signature · 1 line
gitgitgadget
Re: [PATCH v9 2/6] fsmonitor: relocate socket file if .git directory is remote
Quoted 11 lines · Eric DeCosta via GitGitGadget
If the .git directory is on a remote file system, create the socket file in 'fsmonitor.socketDir' if it is defined, else create it in $HOME.
Signed-off-by Eric DeCosta <[email protected]>---
compat/fsmonitor/fsm-ipc-darwin.c +4
-0@@ -0,0 +1,48 @@ +const char *fsmonitor_ipc__get_path (struct repository *r) +{ +char *sock_dir; +
compat/fsmonitor/fsm-ipc-darwin.c line 14
Quoted 10 lines
compat/fsmonitor/fsm-ipc-darwin.c +10
-0+repo_config_get_string (r,"fsmonitor.socketdir" , &sock_dir); + +/* Create the socket file in either socketDir or $HOME */ +if (sock_dir && *sock_dir) { +strbuf_addf (&ipc_file,"%s/.git-fsmonitor-%s" , + sock_dir,hash_to_hex (hash)); +free (sock_dir); + }else { +strbuf_addf (&ipc_file,"~/.git-fsmonitor-%s" ,hash_to_hex (hash)); + }
This is still leaking sock_dir, isn't it, if repo_config_get_string() returns it as a zero-length string?
Probably want to move free() below the entire conditional.
compat/fsmonitor/fsm-ipc-win32.c line 9
Quoted 12 lines
compat/fsmonitor/fsm-ipc-win32.c +9
-0@@ -0,0 +1,9 @@ +#include "config.h" +#include "fsmonitor-ipc.h" + +const char *fsmonitor_ipc__get_path (struct repository *r) { +static char *ret; +if (!ret) + ret =git_pathdup ("fsmonitor--daemon.ipc" ); +return ret; +}
Still an incomplete line.
PATCH v9 3/6 fsmonitor: avoid socket location check if using hook
In reply to Eric DeCosta via GitGitGadgetFrom: Eric DeCosta <[email protected]>
If monitoring is done via fsmonitor hook rather than IPC there is no need to check if the location of the Unix Domain socket (UDS) file is on a remote filesystem.
--- compat/fsmonitor/fsm-settings-darwin.c | 10 ++++++---- compat/fsmonitor/fsm-settings-win32.c | 2 +- fsmonitor-settings.c | 8 ++++---- fsmonitor-settings.h | 2 +- 4 files changed, 12 insertions(+), 10 deletions(-)
compat/fsmonitor/fsm-settings-darwin.c +6 -4
@@ -49,13 +49,15 @@ static enum fsmonitor_reason check_uds_volume(struct repository *r)return FSMONITOR_REASON_OK ; } -enum fsmonitor_reason fsm_os__incompatible (struct repository *r) +enum fsmonitor_reason fsm_os__incompatible (struct repository *r,int ipc) {enum fsmonitor_reason reason; - reason =check_uds_volume (r); -if (reason !=FSMONITOR_REASON_OK ) -return reason; +if (ipc) { + reason =check_uds_volume (r); +if (reason !=FSMONITOR_REASON_OK ) +return reason; + }return FSMONITOR_REASON_OK ; }
compat/fsmonitor/fsm-settings-win32.c +1 -1
@@ -25,7 +25,7 @@ static enum fsmonitor_reason check_vfs4git(struct repository *r)return FSMONITOR_REASON_OK ; } -enum fsmonitor_reason fsm_os__incompatible (struct repository *r) +enum fsmonitor_reason fsm_os__incompatible (struct repository *r,int ipc) {enum fsmonitor_reason reason;
fsmonitor-settings.c +4 -4
@@ -60,7 +60,7 @@ static enum fsmonitor_reason check_remote(struct repository *r) }#endif -static enum fsmonitor_reason check_for_incompatible (struct repository *r) +static enum fsmonitor_reason check_for_incompatible (struct repository *r,int ipc) {if (!r->worktree) {/*
@@ -77,7 +77,7 @@ static enum fsmonitor_reason check_for_incompatible(struct repository *r) reason =check_remote (r);if (reason !=FSMONITOR_REASON_OK )return reason; - reason =fsm_os__incompatible (r); + reason =fsm_os__incompatible (r, ipc);if (reason !=FSMONITOR_REASON_OK )return reason; }
@@ -162,7 +162,7 @@ const char *fsm_settings__get_hook_path(struct repository *r)void fsm_settings__set_ipc (struct repository *r) { -enum fsmonitor_reason reason =check_for_incompatible (r); +enum fsmonitor_reason reason =check_for_incompatible (r,1 );if (reason !=FSMONITOR_REASON_OK ) {fsm_settings__set_incompatible (r, reason);
@@ -185,7 +185,7 @@ void fsm_settings__set_ipc(struct repository *r)void fsm_settings__set_hook (struct repository *r,const char *path) { -enum fsmonitor_reason reason =check_for_incompatible (r); +enum fsmonitor_reason reason =check_for_incompatible (r,0 );if (reason !=FSMONITOR_REASON_OK ) {fsm_settings__set_incompatible (r, reason);
fsmonitor-settings.h +1 -1
@@ -48,7 +48,7 @@ struct fsmonitor_settings;* fsm_os__* routines should considered private to fsm_settings__ * routines. */ -enum fsmonitor_reason fsm_os__incompatible (struct repository *r); +enum fsmonitor_reason fsm_os__incompatible (struct repository *r,int ipc);#endif /* HAVE_FSMONITOR_OS_SETTINGS */ #endif /* FSMONITOR_SETTINGS_H */
Signature · 1 line
gitgitgadget
PATCH v9 4/6 fsmonitor: deal with synthetic firmlinks on macOS
In reply to Eric DeCosta via GitGitGadgetFrom: Eric DeCosta <[email protected]>
Starting with macOS 10.15 (Catalina), Apple introduced a new feature called 'firmlinks' in order to separate the boot volume into two volumes, one read-only and one writable but still present them to the user as a single volume. Along with this change, Apple removed the ability to create symlinks in the root directory and replaced them with 'synthetic firmlinks'. See 'man synthetic.conf'
When FSEevents reports the path of changed files, if the path involves a synthetic firmlink, the path is reported from the point of the synthetic firmlink and not the real path. For example:
Real path:
/System/Volumes/Data/network/working/directory/foo.txt
Synthetic firmlink:
/network -> /System/Volumes/Data/network
FSEvents path:
/network/working/directory/foo.txt
This causes the FSEvents path to not match against the worktree directory.
There are several ways in which synthetic firmlinks can be created:
they can be defined in /etc/synthetic.conf, the automounter can create them, and there may be other means. Simply reading /etc/synthetic.conf is insufficient. No matter what process creates synthetic firmlinks, they all get created in the root directory.
Therefore, in order to deal with synthetic firmlinks, the root directory is scanned and the first possible synthetic firmink that, when resolved, is a prefix of the worktree is used to map FSEvents paths to worktree paths.
--- builtin/fsmonitor--daemon.c | 8 +++ compat/fsmonitor/fsm-listen-darwin.c | 6 +- compat/fsmonitor/fsm-path-utils-darwin.c | 92 ++++++++++++++++++++++++ compat/fsmonitor/fsm-path-utils-win32.c | 17 +++++ fsmonitor--daemon.h | 3 + fsmonitor-path-utils.h | 36 ++++++++++ 6 files changed, 161 insertions(+), 1 deletion(-)
builtin/fsmonitor--daemon.c +8 -0
@@ -3,6 +3,7 @@#include "parse-options.h" #include "fsmonitor.h" #include "fsmonitor-ipc.h" +#include "fsmonitor-path-utils.h" #include "compat/fsmonitor/fsm-health.h" #include "compat/fsmonitor/fsm-listen.h" #include "fsmonitor--daemon.h"
@@ -1282,6 +1283,13 @@ static int fsmonitor_run_daemon(void)strbuf_addstr (&state.path_worktree_watch,absolute_path (get_git_work_tree ())); state.nr_paths_watching =1 ; + state.alias.alias =NULL ; + state.alias.points_to =NULL ; +if (fsmonitor__get_alias (state.path_worktree_watch.buf, &state.alias)) { + err =error (_ ("could not get worktree alias" )); + goto done; + } +/* * We create and delete cookie files somewhere inside the .git * directory to help us keep sync with the file system. If
compat/fsmonitor/fsm-listen-darwin.c +5 -1
@@ -26,6 +26,7 @@#include "fsmonitor.h" #include "fsm-listen.h" #include "fsmonitor--daemon.h" +#include "fsmonitor-path-utils.h" struct fsm_listen_data {
@@ -209,7 +210,9 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,/* * On Mac, we receive an array of absolute paths. */ - path_k = paths[k]; + path_k =fsmonitor__resolve_alias (paths[k], &state->alias); +if (!path_k) + path_k = paths[k];/* * If you want to debug FSEvents, log them to GIT_TRACE_FSMONITOR.
@@ -238,6 +241,7 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,fsmonitor_force_resync (state);fsmonitor_batch__free_list (batch);string_list_clear (&cookie_list,0 ); + batch =NULL ;/* * We assume that any events that we received
compat/fsmonitor/fsm-path-utils-darwin.c +92 -0
100 lines not on this page · raw message
compat/fsmonitor/fsm-path-utils-win32.c +17 -0
@@ -126,3 +126,20 @@ int fsmonitor__is_fs_remote(const char *path)return -1 ;return fs.is_remote; } + +/* +* No-op for now. +*/ +int fsmonitor__get_alias (const char *path,struct alias_info *info) +{ +return 0 ; +} + +/* +* No-op for now. +*/ +char *fsmonitor__resolve_alias (const char *path, +const struct alias_info *info) +{ +return NULL ; +}
fsmonitor--daemon.h +3 -0
@@ -8,6 +8,7 @@#include "run-command.h" #include "simple-ipc.h" #include "thread-utils.h" +#include "fsmonitor-path-utils.h" struct fsmonitor_batch ;struct fsmonitor_token_data ;
@@ -43,6 +44,7 @@ struct fsmonitor_daemon_state {struct strbuf path_worktree_watch;struct strbuf path_gitdir_watch; +struct alias_info alias;int nr_paths_watching;struct fsmonitor_token_data *current_token_data;
@@ -59,6 +61,7 @@ struct fsmonitor_daemon_state {struct ipc_server_data *ipc_server_data;struct strbuf path_ipc; + };/*
fsmonitor-path-utils.h +36 -0
@@ -1,6 +1,14 @@ #ifndef FSM_PATH_UTILS_H #define FSM_PATH_UTILS_H +#include "strbuf.h" + +struct alias_info +{ + char *alias; + char *points_to; +}; + struct fs_info { int is_remote; char *typename;
@@ -20,4 +28,32 @@ int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info);*/ int fsmonitor__is_fs_remote (const char *path); +/* +* Get the alias in given path, if any. +* +* Sets alias to the first alias that matches any part of the path. +* +* If an alias is found, info.alias and info.points_to are set to the +* found mapping. +* +* Returns -1 on error, 0 otherwise. +* +* The caller owns the storage that is occupied by set info.alias and +* info.points_to and is responsible for releasing it with `free(3)` +* when done. +*/ +int fsmonitor__get_alias (const char *path,struct alias_info *info); + +/* +* Resolve the path against the given alias. +* +* Returns the resolved path if there is one, NULL otherwise. +* +* The caller owns the storage that the returned string occupies and +* is responsible for releasing it with `free(3)` when done. +*/ +char *fsmonitor__resolve_alias (const char *path, +const struct alias_info *info); + +#endif
Signature · 1 line
gitgitgadget
PATCH v9 5/6 fsmonitor: check for compatability before communicating with fsmonitor
In reply to Eric DeCosta via GitGitGadgetFrom: Eric DeCosta <[email protected]>
If fsmonitor is not in a compatible state, die with an appropriate error messge.
--- fsmonitor-settings.c | 2 ++ fsmonitor.c | 4 ++++ 2 files changed, 6 insertions(+)
fsmonitor-settings.c +2 -0
@@ -247,6 +247,8 @@ char *fsm_settings__get_incompatible_msg(const struct repository *r, {struct strbuf msg =STRBUF_INIT ; +strbuf_add (&msg,"fsmonitor: " ,strlen ("fsmonitor: " )); +switch (reason) {case FSMONITOR_REASON_UNTESTED :case FSMONITOR_REASON_OK :
fsmonitor.c +4 -0
@@ -305,6 +305,10 @@ void refresh_fsmonitor(struct index_state *istate)int is_trivial =0 ;struct repository *r = istate->repo ? istate->repo : the_repository;enum fsmonitor_mode fsm_mode =fsm_settings__get_mode (r); +enum fsmonitor_reason reason =fsm_settings__get_reason (r); + +if (reason >FSMONITOR_REASON_OK ) +die ("%s" ,fsm_settings__get_incompatible_msg (r, reason));if (fsm_mode <=FSMONITOR_MODE_DISABLED || istate->fsmonitor_has_run_once)
Signature · 1 line
gitgitgadget
PATCH v9 6/6 fsmonitor: add documentation for allowRemote and socketDir options
In reply to Eric DeCosta via GitGitGadgetFrom: Eric DeCosta <[email protected]>
Add documentation for 'fsmonitor.allowRemote' and 'fsmonitor.socketDir'. Call-out experimental nature of 'fsmonitor.allowRemote' and limited file system support for 'fsmonitor.socketDir'.
--- Documentation/git-fsmonitor--daemon.txt | 48 +++++++++++++++++++++++-- 1 file changed, 45 insertions(+), 3 deletions(-)
Documentation/git-fsmonitor--daemon.txt +45 -3
@@ -3,7 +3,7 @@ git-fsmonitor{litdd}daemon(1) NAME ---- -git-fsmonitor--daemon - A Built-in File System Monitor +git-fsmonitor--daemon - A Built-in Filesystem Monitor SYNOPSIS --------
@@ -17,7 +17,7 @@ DESCRIPTION ----------- A daemon to watch the working directory for file and directory -changes using platform-specific file system notification facilities. +changes using platform-specific filesystem notification facilities. This daemon communicates directly with commands like `git status` using the link:technical/api-simple-ipc.html[simple IPC] interface
@@ -63,13 +63,55 @@ CAVEATS ------- The fsmonitor daemon does not currently know about submodules and does -not know to filter out file system events that happen within a +not know to filter out filesystem events that happen within a submodule. If fsmonitor daemon is watching a super repo and a file is modified within the working directory of a submodule, it will report the change (as happening against the super repo). However, the client will properly ignore these extra events, so performance may be affected but it will not cause an incorrect result. +By default, the fsmonitor daemon refuses to work against network-mounted +repositories; this may be overridden by setting `fsmonitor.allowRemote` to +`true`. Note, however, that the fsmonitor daemon is not guaranteed to work +correctly with all network-mounted repositories and such use is considered +experimental. + +On Mac OS, the inter-process communication (IPC) between various Git +commands and the fsmonitor daemon is done via a Unix domain socket (UDS) -- a +special type of file -- which is supported by native Mac OS filesystems, +but not on network-mounted filesystems, NTFS, or FAT32. Other filesystems +may or may not have the needed support; the fsmonitor daemon is not guaranteed +to work with these filesystems and such use is considered experimental. + +By default, the socket is created in the `.git` directory, however, if the +`.git` directory is on a network-mounted filesystem, it will be instead be +created at `$HOME/.git-fsmonitor-*` unless `$HOME` itself is on a +network-mounted filesystem in which case you must set the configuration +variable `fsmonitor.socketDir` to the path of a directory on a Mac OS native +filesystem in which to create the socket file. + +If none of the above directories (`.git`, `$HOME`, or `fsmonitor.socketDir`) +is on a native Mac OS file filesystem the fsmonitor daemon will report an +error that will cause the daemon and the currently running command to exit. + +CONFIGURATION +------------- + +When `core.fsmonitor` is set to `true` (see linkgit:git-config[1]) +the fsmonitor daemon will pay attention to the following configuration +variables: + +`fsmonitor.allowRemote`:: + By default, the daemon refuses to work against network-mounted + repositories. Setting `fsmonitor.allowRemote` to `true` overrides + this behavior. + +`fsmonitor.socketDir`:: + This Mac OS-specific option, if set, specifies the directory in + which to create the Unix domain socket used for communication + between fsmonitor and various Git commands. The directory must + reside on a native Mac OS filesystem as discussed above. + GIT --- Part of the linkgit:git[1] suite
Signature · 1 line
gitgitgadget
v10 cover fsmonitor: option to allow fsmonitor to run against network-mounted repos
In reply to Eric DeCosta via GitGitGadgetFollow-on to the work done to allow Windows to work against network-mounted repos for macOS.
Have macOS take advantage of the same configuration option, 'fsmonitor.allowRemote' that was introduced for Windows. Setting this option to true will override the default behavior (erroring-out) when a network-mounted repo is detected by fsmonitor.
The added wrinkle being that the Unix domain socket (UDS) file used for IPC cannot be created in a network location; instead $HOME is used if the default location is on the network. The user may, optionally, set the 'fsmonitor.socketDir' configuration option to a valid, local directory if $HOME itself is on the network or is simply not the desired location for the UDS file.
An additional issue is that for mount points in the root directory, FSEvents does not report a path that matches the worktree directory due to the introduction of 'synthetic firmlinks'. fsmonitor must map the FSEvents paths to the worktree directory by interrogating the root filesystem for synthetic firmlinks and using that information to translate the path.
v10 differs from v9:
* incorporates code review feedback
* improves error messaging for incompatible socket directory
v9 differs from v8:
* incorporates code review feedback
* check for incompatibility before communicating with fsmonitor
v8 differs from v7:
* incorporates code review feedback
* gets the rebase right
v7 differs from v6:
* incorporates code review feedback
v6 differs from v5:
* incorporates earlier, Windows-specific changes that have not made it back yet to the master branch
* incorporates code review feedback
* adds documentation for 'fsmonitor.allowRemote' and 'fsmonitor.socketDir'
v5 differs significantly from earlier versions:
* redesign of handling 'fsmonitor.allowRemote' and 'fsmonitor.socketDir' such that these options are no longer added to the settings data structure but are rather read from config at point of use
* refactoring of code for handling platform-specific file system checks via a common interface to avoid platform #ifdef in IPC code and be in-model with other platform-specific fsmonitor code
* dealing with 'synthetic firmlinks' on macOS
Eric DeCosta (6):
fsmonitor: refactor filesystem checks to common interface
fsmonitor: relocate socket file if .git directory is remote
fsmonitor: avoid socket location check if using hook
fsmonitor: deal with synthetic firmlinks on macOS
fsmonitor: check for compatability before communicating with fsmonitor
fsmonitor: add documentation for allowRemote and socketDir options
Documentation/git-fsmonitor--daemon.txt | 48 ++++++- Makefile | 2 + builtin/fsmonitor--daemon.c | 11 +- compat/fsmonitor/fsm-ipc-darwin.c | 51 +++++++ compat/fsmonitor/fsm-ipc-win32.c | 9 ++ compat/fsmonitor/fsm-listen-darwin.c | 6 +- compat/fsmonitor/fsm-path-utils-darwin.c | 132 +++++++++++++++++ compat/fsmonitor/fsm-path-utils-win32.c | 145 +++++++++++++++++++ compat/fsmonitor/fsm-settings-darwin.c | 72 +++------- compat/fsmonitor/fsm-settings-win32.c | 174 +---------------------- contrib/buildsystems/CMakeLists.txt | 4 + fsmonitor--daemon.h | 3 + fsmonitor-ipc.c | 18 ++- fsmonitor-ipc.h | 4 +- fsmonitor-path-utils.h | 59 ++++++++ fsmonitor-settings.c | 68 ++++++++- fsmonitor-settings.h | 4 +- fsmonitor.c | 4 + 18 files changed, 569 insertions(+), 245 deletions(-) create mode 100644 compat/fsmonitor/fsm-ipc-darwin.c create mode 100644 compat/fsmonitor/fsm-ipc-win32.c create mode 100644 compat/fsmonitor/fsm-path-utils-darwin.c create mode 100644 compat/fsmonitor/fsm-path-utils-win32.c create mode 100644 fsmonitor-path-utils.h
base-commit: d3fa443f97e3a8d75b51341e2d5bac380b7422df
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1326%2Fedecosta-mw%2Ffsmonitor_macos-v10
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1326/edecosta-mw/fsmonitor_macos-v10
Pull-Request: https://github.com/gitgitgadget/git/pull/1326
Range-diff vs v9:
1: 155a6890806 = 1: 155a6890806 fsmonitor: refactor filesystem checks to common interface
2: 68709d788d5 ! 2: dbe113abb87 fsmonitor: relocate socket file if .git directory is remote
@@ compat/fsmonitor/fsm-ipc-darwin.c (new)
+{
+ static const char *ipc_path;
+ SHA_CTX sha1ctx;
-+ char *sock_dir;
++ char *sock_dir = NULL;
+ struct strbuf ipc_file = STRBUF_INIT;
+ unsigned char hash[SHA_DIGEST_LENGTH];
+
++ if (!r)
++ BUG("No repository passed into fsmonitor_ipc__get_path");
++
+ if (ipc_path)
+ return ipc_path;
+
@@ compat/fsmonitor/fsm-ipc-darwin.c (new)
+ if (sock_dir && *sock_dir) {
+ strbuf_addf(&ipc_file, "%s/.git-fsmonitor-%s",
+ sock_dir, hash_to_hex(hash));
-+ free(sock_dir);
+ } else {
+ strbuf_addf(&ipc_file, "~/.git-fsmonitor-%s", hash_to_hex(hash));
+ }
++ free(sock_dir);
+
+ ipc_path = interpolate_path(ipc_file.buf, 1);
+ if (!ipc_path)
@@ compat/fsmonitor/fsm-ipc-win32.c (new)
+ ret = git_pathdup("fsmonitor--daemon.ipc");
+ return ret;
+}
- \ No newline at end of file
## compat/fsmonitor/fsm-settings-darwin.c ##
@@
3: 6ddd922917a = 3: 86c15299ae8 fsmonitor: avoid socket location check if using hook
4: 73afd9f3122 = 4: 4f6c98cf834 fsmonitor: deal with synthetic firmlinks on macOS
5: 02afeaa01be ! 5: 8f6c1fbacbf fsmonitor: check for compatability before communicating with fsmonitor@@ Commit message
+ ## compat/fsmonitor/fsm-settings-darwin.c ##
+@@ compat/fsmonitor/fsm-settings-darwin.c: static enum fsmonitor_reason check_uds_volume(struct repository *r)
+ strbuf_release(&path);
+
+ if (fs.is_remote)
+- return FSMONITOR_REASON_REMOTE;
++ return FSMONITOR_REASON_NOSOCKETS;
+
+ if (!strcmp(fs.typename, "msdos")) /* aka FAT32 */
+ return FSMONITOR_REASON_NOSOCKETS;
+
## fsmonitor-settings.c ##
-@@ fsmonitor-settings.c: char *fsm_settings__get_incompatible_msg(const struct repository *r, +@@
+ #include "cache.h"
+ #include "config.h"
+ #include "repository.h"
++#include "fsmonitor-ipc.h"
+ #include "fsmonitor-settings.h"
+ #include "fsmonitor-path-utils.h"
+
+@@ fsmonitor-settings.c: enum fsmonitor_reason fsm_settings__get_reason(struct repository *r)
+ return r->settings.fsmonitor->reason;
+ }
+
+-char *fsm_settings__get_incompatible_msg(const struct repository *r,
++char *fsm_settings__get_incompatible_msg(struct repository *r,
+ enum fsmonitor_reason reason)
{
struct strbuf msg = STRBUF_INIT;
++ const char *socket_dir;
-+ strbuf_add(&msg, "fsmonitor: ", strlen("fsmonitor: "));
-+
switch (reason) {
case FSMONITOR_REASON_UNTESTED:
- case FSMONITOR_REASON_OK:
+@@ fsmonitor-settings.c: char *fsm_settings__get_incompatible_msg(const struct repository *r,
+ goto done;
+
+ case FSMONITOR_REASON_NOSOCKETS:
++ socket_dir = dirname((char *)fsmonitor_ipc__get_path(r));
+ strbuf_addf(&msg,
+- _("repository '%s' is incompatible with fsmonitor due to lack of Unix sockets"),
+- r->worktree);
++ _("socket directory '%s' is incompatible with fsmonitor due"),
++ socket_dir);
++ strbuf_add(&msg, _(" to lack of Unix sockets support"), 32);
+ goto done;
+ }
+
+
+ ## fsmonitor-settings.h ##
+@@ fsmonitor-settings.h: enum fsmonitor_mode fsm_settings__get_mode(struct repository *r);
+ const char *fsm_settings__get_hook_path(struct repository *r);
+
+ enum fsmonitor_reason fsm_settings__get_reason(struct repository *r);
+-char *fsm_settings__get_incompatible_msg(const struct repository *r,
++char *fsm_settings__get_incompatible_msg(struct repository *r,
+ enum fsmonitor_reason reason);
+
+ struct fsmonitor_settings;
## fsmonitor.c ##
@@ fsmonitor.c: void refresh_fsmonitor(struct index_state *istate)
6: 0e8ea28acc1 = 6: d7c25bf96c6 fsmonitor: add documentation for allowRemote and socketDir options
Signature · 1 line
gitgitgadget
PATCH v10 1/6 fsmonitor: refactor filesystem checks to common interface
From: Eric DeCosta <[email protected]>
Provide a common interface for getting basic filesystem information including filesystem type and whether the filesystem is remote.
Refactor existing code for getting basic filesystem info and detecting remote file systems to the new interface.
Refactor filesystem checks to leverage new interface. For macOS, error-out if the Unix Domain socket (UDS) file is on a remote filesystem.
--- Makefile | 1 + compat/fsmonitor/fsm-path-utils-darwin.c | 40 ++++++ compat/fsmonitor/fsm-path-utils-win32.c | 128 +++++++++++++++++ compat/fsmonitor/fsm-settings-darwin.c | 62 +++----- compat/fsmonitor/fsm-settings-win32.c | 172 +---------------------- contrib/buildsystems/CMakeLists.txt | 2 + fsmonitor-path-utils.h | 23 +++ fsmonitor-settings.c | 50 +++++++ 8 files changed, 263 insertions(+), 215 deletions(-) create mode 100644 compat/fsmonitor/fsm-path-utils-darwin.c create mode 100644 compat/fsmonitor/fsm-path-utils-win32.c create mode 100644 fsmonitor-path-utils.h
Makefile +1 -0
@@ -2039,6 +2039,7 @@ endif ifdef FSMONITOR_OS_SETTINGS COMPAT_CFLAGS += -DHAVE_FSMONITOR_OS_SETTINGS COMPAT_OBJS += compat/fsmonitor/fsm-settings-$(FSMONITOR_OS_SETTINGS).o + COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_OS_SETTINGS).o endif ifeq ($(TCLTK_PATH),)
compat/fsmonitor/fsm-path-utils-darwin.c new file +40 -0
@@ -0,0 +1,40 @@ +#include "fsmonitor.h" +#include "fsmonitor-path-utils.h" +#include <sys/param.h> +#include <sys/mount.h> + +int fsmonitor__get_fs_info (const char *path,struct fs_info *fs_info) +{ +struct statfs fs; +if (statfs (path, &fs) ==-1 ) { +int saved_errno = errno; +trace_printf_key (&trace_fsmonitor,"statfs('%s') failed: %s" , + path,strerror (saved_errno)); + errno = saved_errno; +return -1 ; + } + +trace_printf_key (&trace_fsmonitor, +"statfs('%s') [type 0x%08x][flags 0x%08x] '%s'" , + path, fs.f_type, fs.f_flags, fs.f_fstypename); + +if (!(fs.f_flags &MNT_LOCAL )) + fs_info->is_remote =1 ; +else + fs_info->is_remote =0 ; + + fs_info->typename = fs.f_fstypename; + +trace_printf_key (&trace_fsmonitor, +"'%s' is_remote: %d" , + path, fs_info->is_remote); +return 0 ; +} + +int fsmonitor__is_fs_remote (const char *path) +{ +struct fs_info fs; +if (fsmonitor__get_fs_info (path, &fs)) +return -1 ; +return fs.is_remote; +}
compat/fsmonitor/fsm-path-utils-win32.c new file +128 -0
128 lines not on this page · raw message
compat/fsmonitor/fsm-settings-darwin.c +17 -45
96 lines not on this page · raw message
compat/fsmonitor/fsm-settings-win32.c +2 -170
190 lines not on this page · raw message
contrib/buildsystems/CMakeLists.txt +2 -0
@@ -308,6 +308,7 @@ if(SUPPORTS_SIMPLE_IPC) add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-win32.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-win32.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-win32.c)
@@ -315,6 +316,7 @@ if(SUPPORTS_SIMPLE_IPC) add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-darwin.c)
fsmonitor-path-utils.h new file +23 -0
@@ -0,0 +1,23 @@ +#ifndef FSM_PATH_UTILS_H +#define FSM_PATH_UTILS_H + +struct fs_info { +int is_remote; +char *typename; +}; + +/* +* Get some basic filesystem informtion for the given path +* +* Returns -1 on error, zero otherwise. +*/ +int fsmonitor__get_fs_info (const char *path,struct fs_info *fs_info); + +/* +* Determines if the filesystem that path resides on is remote. +* +* Returns -1 on error, 0 if not remote, 1 if remote. +*/ +int fsmonitor__is_fs_remote (const char *path); + +#endif
fsmonitor-settings.c +50 -0
68 lines not on this page · raw message
Signature · 1 line
gitgitgadget
PATCH v10 2/6 fsmonitor: relocate socket file if .git directory is remote
In reply to Eric DeCosta via GitGitGadgetFrom: Eric DeCosta <[email protected]>
If the .git directory is on a remote file system, create the socket file in 'fsmonitor.socketDir' if it is defined, else create it in $HOME.
--- Makefile | 1 + builtin/fsmonitor--daemon.c | 3 +- compat/fsmonitor/fsm-ipc-darwin.c | 51 ++++++++++++++++++++++++++ compat/fsmonitor/fsm-ipc-win32.c | 9 +++++ compat/fsmonitor/fsm-settings-darwin.c | 2 +- contrib/buildsystems/CMakeLists.txt | 2 + fsmonitor-ipc.c | 18 ++++----- fsmonitor-ipc.h | 4 +- 8 files changed, 77 insertions(+), 13 deletions(-) create mode 100644 compat/fsmonitor/fsm-ipc-darwin.c create mode 100644 compat/fsmonitor/fsm-ipc-win32.c
Makefile +1 -0
@@ -2034,6 +2034,7 @@ ifdef FSMONITOR_DAEMON_BACKEND COMPAT_CFLAGS += -DHAVE_FSMONITOR_DAEMON_BACKEND COMPAT_OBJS += compat/fsmonitor/fsm-listen-$(FSMONITOR_DAEMON_BACKEND).o COMPAT_OBJS += compat/fsmonitor/fsm-health-$(FSMONITOR_DAEMON_BACKEND).o + COMPAT_OBJS += compat/fsmonitor/fsm-ipc-$(FSMONITOR_DAEMON_BACKEND).o endif ifdef FSMONITOR_OS_SETTINGS
builtin/fsmonitor--daemon.c +2 -1
@@ -1343,7 +1343,8 @@ static int fsmonitor_run_daemon(void)* directory.) */ strbuf_init (&state.path_ipc,0 ); -strbuf_addstr (&state.path_ipc,absolute_path (fsmonitor_ipc__get_path ())); +strbuf_addstr (&state.path_ipc, +absolute_path (fsmonitor_ipc__get_path (the_repository)));/* * Confirm that we can create platform-specific resources for the
compat/fsmonitor/fsm-ipc-darwin.c new file +51 -0
51 lines not on this page · raw message
compat/fsmonitor/fsm-ipc-win32.c new file +9 -0
@@ -0,0 +1,9 @@ +#include "config.h" +#include "fsmonitor-ipc.h" + +const char *fsmonitor_ipc__get_path (struct repository *r) { +static char *ret; +if (!ret) + ret =git_pathdup ("fsmonitor--daemon.ipc" ); +return ret; +}
compat/fsmonitor/fsm-settings-darwin.c +1 -1
@@ -26,7 +26,7 @@static enum fsmonitor_reason check_uds_volume (struct repository *r) {struct fs_info fs; -const char *ipc_path =fsmonitor_ipc__get_path (); +const char *ipc_path =fsmonitor_ipc__get_path (r);struct strbuf path =STRBUF_INIT ;strbuf_add (&path, ipc_path,strlen (ipc_path));
contrib/buildsystems/CMakeLists.txt +2 -0
@@ -308,6 +308,7 @@ if(SUPPORTS_SIMPLE_IPC) add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-win32.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-win32.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-win32.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
@@ -316,6 +317,7 @@ if(SUPPORTS_SIMPLE_IPC) add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-darwin.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
fsmonitor-ipc.c +8 -10
@@ -18,7 +18,7 @@ int fsmonitor_ipc__is_supported(void)return 0 ; } -const char *fsmonitor_ipc__get_path (void ) +const char *fsmonitor_ipc__get_path (struct repository *r) {return NULL ; }
@@ -47,11 +47,9 @@ int fsmonitor_ipc__is_supported(void)return 1 ; } -GIT_PATH_FUNC(fsmonitor_ipc__get_path, "fsmonitor--daemon.ipc") -enum ipc_active_state fsmonitor_ipc__get_state (void ) { -return ipc_get_active_state (fsmonitor_ipc__get_path ()); +return ipc_get_active_state (fsmonitor_ipc__get_path (the_repository)); }static int spawn_daemon (void )
@@ -81,8 +79,8 @@ int fsmonitor_ipc__send_query(const char *since_token,trace2_data_string ("fsm_client" ,NULL ,"query/command" , tok); try_again: - state =ipc_client_try_connect (fsmonitor_ipc__get_path (), &options, - &connection); + state =ipc_client_try_connect (fsmonitor_ipc__get_path (the_repository), + &options, &connection);switch (state) {case IPC_STATE__LISTENING :
@@ -117,13 +115,13 @@ try_again:case IPC_STATE__INVALID_PATH : ret =error (_ ("fsmonitor_ipc__send_query: invalid path '%s'" ), -fsmonitor_ipc__get_path ()); +fsmonitor_ipc__get_path (the_repository)); goto done;case IPC_STATE__OTHER_ERROR :default : ret =error (_ ("fsmonitor_ipc__send_query: unspecified error on '%s'" ), -fsmonitor_ipc__get_path ()); +fsmonitor_ipc__get_path (the_repository)); goto done; }
@@ -149,8 +147,8 @@ int fsmonitor_ipc__send_command(const char *command, options.wait_if_busy =1 ; options.wait_if_not_found =0 ; - state =ipc_client_try_connect (fsmonitor_ipc__get_path (), &options, - &connection); + state =ipc_client_try_connect (fsmonitor_ipc__get_path (the_repository), + &options, &connection);if (state !=IPC_STATE__LISTENING ) {die (_ ("fsmonitor--daemon is not running" ));return -1 ;
fsmonitor-ipc.h +3 -1
@@ -3,6 +3,8 @@#include "simple-ipc.h" +struct repository ; +/* * Returns true if built-in file system monitor daemon is defined * for this platform.
@@ -16,7 +18,7 @@ int fsmonitor_ipc__is_supported(void);* * Returns NULL if the daemon is not supported on this platform. */ -const char *fsmonitor_ipc__get_path (void ); +const char *fsmonitor_ipc__get_path (struct repository *r);/* * Try to determine whether there is a `git-fsmonitor--daemon` process
Signature · 1 line
gitgitgadget
PATCH v10 3/6 fsmonitor: avoid socket location check if using hook
In reply to Eric DeCosta via GitGitGadgetFrom: Eric DeCosta <[email protected]>
If monitoring is done via fsmonitor hook rather than IPC there is no need to check if the location of the Unix Domain socket (UDS) file is on a remote filesystem.
--- compat/fsmonitor/fsm-settings-darwin.c | 10 ++++++---- compat/fsmonitor/fsm-settings-win32.c | 2 +- fsmonitor-settings.c | 8 ++++---- fsmonitor-settings.h | 2 +- 4 files changed, 12 insertions(+), 10 deletions(-)
compat/fsmonitor/fsm-settings-darwin.c +6 -4
@@ -49,13 +49,15 @@ static enum fsmonitor_reason check_uds_volume(struct repository *r)return FSMONITOR_REASON_OK ; } -enum fsmonitor_reason fsm_os__incompatible (struct repository *r) +enum fsmonitor_reason fsm_os__incompatible (struct repository *r,int ipc) {enum fsmonitor_reason reason; - reason =check_uds_volume (r); -if (reason !=FSMONITOR_REASON_OK ) -return reason; +if (ipc) { + reason =check_uds_volume (r); +if (reason !=FSMONITOR_REASON_OK ) +return reason; + }return FSMONITOR_REASON_OK ; }
compat/fsmonitor/fsm-settings-win32.c +1 -1
@@ -25,7 +25,7 @@ static enum fsmonitor_reason check_vfs4git(struct repository *r)return FSMONITOR_REASON_OK ; } -enum fsmonitor_reason fsm_os__incompatible (struct repository *r) +enum fsmonitor_reason fsm_os__incompatible (struct repository *r,int ipc) {enum fsmonitor_reason reason;
fsmonitor-settings.c +4 -4
@@ -60,7 +60,7 @@ static enum fsmonitor_reason check_remote(struct repository *r) }#endif -static enum fsmonitor_reason check_for_incompatible (struct repository *r) +static enum fsmonitor_reason check_for_incompatible (struct repository *r,int ipc) {if (!r->worktree) {/*
@@ -77,7 +77,7 @@ static enum fsmonitor_reason check_for_incompatible(struct repository *r) reason =check_remote (r);if (reason !=FSMONITOR_REASON_OK )return reason; - reason =fsm_os__incompatible (r); + reason =fsm_os__incompatible (r, ipc);if (reason !=FSMONITOR_REASON_OK )return reason; }
@@ -162,7 +162,7 @@ const char *fsm_settings__get_hook_path(struct repository *r)void fsm_settings__set_ipc (struct repository *r) { -enum fsmonitor_reason reason =check_for_incompatible (r); +enum fsmonitor_reason reason =check_for_incompatible (r,1 );if (reason !=FSMONITOR_REASON_OK ) {fsm_settings__set_incompatible (r, reason);
@@ -185,7 +185,7 @@ void fsm_settings__set_ipc(struct repository *r)void fsm_settings__set_hook (struct repository *r,const char *path) { -enum fsmonitor_reason reason =check_for_incompatible (r); +enum fsmonitor_reason reason =check_for_incompatible (r,0 );if (reason !=FSMONITOR_REASON_OK ) {fsm_settings__set_incompatible (r, reason);
fsmonitor-settings.h +1 -1
@@ -48,7 +48,7 @@ struct fsmonitor_settings;* fsm_os__* routines should considered private to fsm_settings__ * routines. */ -enum fsmonitor_reason fsm_os__incompatible (struct repository *r); +enum fsmonitor_reason fsm_os__incompatible (struct repository *r,int ipc);#endif /* HAVE_FSMONITOR_OS_SETTINGS */ #endif /* FSMONITOR_SETTINGS_H */
Signature · 1 line
gitgitgadget
PATCH v10 4/6 fsmonitor: deal with synthetic firmlinks on macOS
In reply to Eric DeCosta via GitGitGadgetFrom: Eric DeCosta <[email protected]>
Starting with macOS 10.15 (Catalina), Apple introduced a new feature called 'firmlinks' in order to separate the boot volume into two volumes, one read-only and one writable but still present them to the user as a single volume. Along with this change, Apple removed the ability to create symlinks in the root directory and replaced them with 'synthetic firmlinks'. See 'man synthetic.conf'
When FSEevents reports the path of changed files, if the path involves a synthetic firmlink, the path is reported from the point of the synthetic firmlink and not the real path. For example:
Real path:
/System/Volumes/Data/network/working/directory/foo.txt
Synthetic firmlink:
/network -> /System/Volumes/Data/network
FSEvents path:
/network/working/directory/foo.txt
This causes the FSEvents path to not match against the worktree directory.
There are several ways in which synthetic firmlinks can be created:
they can be defined in /etc/synthetic.conf, the automounter can create them, and there may be other means. Simply reading /etc/synthetic.conf is insufficient. No matter what process creates synthetic firmlinks, they all get created in the root directory.
Therefore, in order to deal with synthetic firmlinks, the root directory is scanned and the first possible synthetic firmink that, when resolved, is a prefix of the worktree is used to map FSEvents paths to worktree paths.
--- builtin/fsmonitor--daemon.c | 8 +++ compat/fsmonitor/fsm-listen-darwin.c | 6 +- compat/fsmonitor/fsm-path-utils-darwin.c | 92 ++++++++++++++++++++++++ compat/fsmonitor/fsm-path-utils-win32.c | 17 +++++ fsmonitor--daemon.h | 3 + fsmonitor-path-utils.h | 36 ++++++++++ 6 files changed, 161 insertions(+), 1 deletion(-)
builtin/fsmonitor--daemon.c +8 -0
@@ -3,6 +3,7 @@#include "parse-options.h" #include "fsmonitor.h" #include "fsmonitor-ipc.h" +#include "fsmonitor-path-utils.h" #include "compat/fsmonitor/fsm-health.h" #include "compat/fsmonitor/fsm-listen.h" #include "fsmonitor--daemon.h"
@@ -1282,6 +1283,13 @@ static int fsmonitor_run_daemon(void)strbuf_addstr (&state.path_worktree_watch,absolute_path (get_git_work_tree ())); state.nr_paths_watching =1 ; + state.alias.alias =NULL ; + state.alias.points_to =NULL ; +if (fsmonitor__get_alias (state.path_worktree_watch.buf, &state.alias)) { + err =error (_ ("could not get worktree alias" )); + goto done; + } +/* * We create and delete cookie files somewhere inside the .git * directory to help us keep sync with the file system. If
compat/fsmonitor/fsm-listen-darwin.c +5 -1
@@ -26,6 +26,7 @@#include "fsmonitor.h" #include "fsm-listen.h" #include "fsmonitor--daemon.h" +#include "fsmonitor-path-utils.h" struct fsm_listen_data {
@@ -209,7 +210,9 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,/* * On Mac, we receive an array of absolute paths. */ - path_k = paths[k]; + path_k =fsmonitor__resolve_alias (paths[k], &state->alias); +if (!path_k) + path_k = paths[k];/* * If you want to debug FSEvents, log them to GIT_TRACE_FSMONITOR.
@@ -238,6 +241,7 @@ static void fsevent_callback(ConstFSEventStreamRef streamRef,fsmonitor_force_resync (state);fsmonitor_batch__free_list (batch);string_list_clear (&cookie_list,0 ); + batch =NULL ;/* * We assume that any events that we received
compat/fsmonitor/fsm-path-utils-darwin.c +92 -0
100 lines not on this page · raw message
compat/fsmonitor/fsm-path-utils-win32.c +17 -0
@@ -126,3 +126,20 @@ int fsmonitor__is_fs_remote(const char *path)return -1 ;return fs.is_remote; } + +/* +* No-op for now. +*/ +int fsmonitor__get_alias (const char *path,struct alias_info *info) +{ +return 0 ; +} + +/* +* No-op for now. +*/ +char *fsmonitor__resolve_alias (const char *path, +const struct alias_info *info) +{ +return NULL ; +}
fsmonitor--daemon.h +3 -0
@@ -8,6 +8,7 @@#include "run-command.h" #include "simple-ipc.h" #include "thread-utils.h" +#include "fsmonitor-path-utils.h" struct fsmonitor_batch ;struct fsmonitor_token_data ;
@@ -43,6 +44,7 @@ struct fsmonitor_daemon_state {struct strbuf path_worktree_watch;struct strbuf path_gitdir_watch; +struct alias_info alias;int nr_paths_watching;struct fsmonitor_token_data *current_token_data;
@@ -59,6 +61,7 @@ struct fsmonitor_daemon_state {struct ipc_server_data *ipc_server_data;struct strbuf path_ipc; + };/*
fsmonitor-path-utils.h +36 -0
@@ -1,6 +1,14 @@ #ifndef FSM_PATH_UTILS_H #define FSM_PATH_UTILS_H +#include "strbuf.h" + +struct alias_info +{ + char *alias; + char *points_to; +}; + struct fs_info { int is_remote; char *typename;
@@ -20,4 +28,32 @@ int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info);*/ int fsmonitor__is_fs_remote (const char *path); +/* +* Get the alias in given path, if any. +* +* Sets alias to the first alias that matches any part of the path. +* +* If an alias is found, info.alias and info.points_to are set to the +* found mapping. +* +* Returns -1 on error, 0 otherwise. +* +* The caller owns the storage that is occupied by set info.alias and +* info.points_to and is responsible for releasing it with `free(3)` +* when done. +*/ +int fsmonitor__get_alias (const char *path,struct alias_info *info); + +/* +* Resolve the path against the given alias. +* +* Returns the resolved path if there is one, NULL otherwise. +* +* The caller owns the storage that the returned string occupies and +* is responsible for releasing it with `free(3)` when done. +*/ +char *fsmonitor__resolve_alias (const char *path, +const struct alias_info *info); + +#endif
Signature · 1 line
gitgitgadget
PATCH v10 5/6 fsmonitor: check for compatability before communicating with fsmonitor
In reply to Eric DeCosta via GitGitGadgetFrom: Eric DeCosta <[email protected]>
If fsmonitor is not in a compatible state, die with an appropriate error messge.
--- compat/fsmonitor/fsm-settings-darwin.c | 2 +- fsmonitor-settings.c | 10 +++++++--- fsmonitor-settings.h | 2 +- fsmonitor.c | 4 ++++ 4 files changed, 13 insertions(+), 5 deletions(-)
compat/fsmonitor/fsm-settings-darwin.c +1 -1
@@ -38,7 +38,7 @@ static enum fsmonitor_reason check_uds_volume(struct repository *r)strbuf_release (&path);if (fs.is_remote) -return FSMONITOR_REASON_REMOTE ; +return FSMONITOR_REASON_NOSOCKETS ;if (!strcmp (fs.typename,"msdos" ))/* aka FAT32 */ return FSMONITOR_REASON_NOSOCKETS ;
fsmonitor-settings.c +7 -3
@@ -1,6 +1,7 @@#include "cache.h" #include "config.h" #include "repository.h" +#include "fsmonitor-ipc.h" #include "fsmonitor-settings.h" #include "fsmonitor-path-utils.h"
@@ -242,10 +243,11 @@ enum fsmonitor_reason fsm_settings__get_reason(struct repository *r)return r->settings.fsmonitor->reason; } -char *fsm_settings__get_incompatible_msg (const struct repository *r, +char *fsm_settings__get_incompatible_msg (struct repository *r,enum fsmonitor_reason reason) {struct strbuf msg =STRBUF_INIT ; +const char *socket_dir;switch (reason) {case FSMONITOR_REASON_UNTESTED :
@@ -281,9 +283,11 @@ char *fsm_settings__get_incompatible_msg(const struct repository *r, goto done;case FSMONITOR_REASON_NOSOCKETS : + socket_dir =dirname ((char *)fsmonitor_ipc__get_path (r));strbuf_addf (&msg, -_ ("repository '%s' is incompatible with fsmonitor due to lack of Unix sockets" ), - r->worktree); +_ ("socket directory '%s' is incompatible with fsmonitor due" ), + socket_dir); +strbuf_add (&msg,_ (" to lack of Unix sockets support" ),32 ); goto done; }
fsmonitor-settings.h +1 -1
@@ -33,7 +33,7 @@ enum fsmonitor_mode fsm_settings__get_mode(struct repository *r);const char *fsm_settings__get_hook_path (struct repository *r);enum fsmonitor_reason fsm_settings__get_reason (struct repository *r); -char *fsm_settings__get_incompatible_msg (const struct repository *r, +char *fsm_settings__get_incompatible_msg (struct repository *r,enum fsmonitor_reason reason);struct fsmonitor_settings ;
fsmonitor.c +4 -0
@@ -305,6 +305,10 @@ void refresh_fsmonitor(struct index_state *istate)int is_trivial =0 ;struct repository *r = istate->repo ? istate->repo : the_repository;enum fsmonitor_mode fsm_mode =fsm_settings__get_mode (r); +enum fsmonitor_reason reason =fsm_settings__get_reason (r); + +if (reason >FSMONITOR_REASON_OK ) +die ("%s" ,fsm_settings__get_incompatible_msg (r, reason));if (fsm_mode <=FSMONITOR_MODE_DISABLED || istate->fsmonitor_has_run_once)
Signature · 1 line
gitgitgadget
Re: [PATCH v10 5/6] fsmonitor: check for compatability before communicating with fsmonitor
Quoted 4 lines · Eric DeCosta via GitGitGadget
From: Eric DeCosta <[email protected]>
If fsmonitor is not in a compatible state, die with an appropriate error messge.
[...]
Quoted 4 lines
fsmonitor-settings.c +0
-0
[...]
Quoted 10 lines
fsmonitor-settings.c +2
-0+char *fsm_settings__get_incompatible_msg (struct repository *r,enum fsmonitor_reason reason) {struct strbuf msg =STRBUF_INIT ; +const char *socket_dir;switch (reason) {case FSMONITOR_REASON_UNTESTED :@@ -281,9 +283,11 @@ char *fsm_settings__get_incompatible_msg(const struct repository *r, goto done;
fsmonitor-settings.c char *fsm_settings__get_incompatible_msg(const struct repository *r, · line 291
Quoted 10 lines
fsmonitor-settings.c +4
-2case FSMONITOR_REASON_NOSOCKETS : + socket_dir =dirname ((char *)fsmonitor_ipc__get_path (r));strbuf_addf (&msg, -_ ("repository '%s' is incompatible with fsmonitor due to lack of Unix sockets" ), - r->worktree); +_ ("socket directory '%s' is incompatible with fsmonitor due" ), + socket_dir); +strbuf_add (&msg,_ (" to lack of Unix sockets support" ),32 ); goto done;
I don't think we should split the error message between two calls to strbuf_add(). I realize that this was probably done because of line length concerns. But this makes assumptions on language word order during translations.
Instead, we can use C string literal joining before passing it to the translation macro. Something like:
strbuf_addf(&msg,
_("socket directory '%s' is incompatible with "
"fsmonitor due to lack of Unix sockets support"),
socket_dir);
[...]
fsmonitor.c void refresh_fsmonitor(struct index_state *istate) · line 311
Quoted 12 lines
fsmonitor.c +4
-0@@ -305,6 +305,10 @@ void refresh_fsmonitor(struct index_state *istate)int is_trivial =0 ;struct repository *r = istate->repo ? istate->repo : the_repository;enum fsmonitor_mode fsm_mode =fsm_settings__get_mode (r); +enum fsmonitor_reason reason =fsm_settings__get_reason (r); + +if (reason >FSMONITOR_REASON_OK ) +die ("%s" ,fsm_settings__get_incompatible_msg (r, reason));
We don't want to call die() here. Maybe just silently return without doing anything or issue a warning() and return. (But I'm favoring a silent return here.)
From the clients' (git status, git diff, etc.) point of view, they just want a speed-up, if possible, but we shouldn't kill them; we should just let them do the normal scan that would have done if the feature were turned off.
Jeff
RE: [PATCH v10 5/6] fsmonitor: check for compatability before communicating with fsmonitor
Quoted 82 lines
-----Original Message-----
From: Jeff Hostetler <[email protected]>
Sent: Wednesday, September 21, 2022 7:22 AM
To: Eric DeCosta via GitGitGadget <[email protected]>; [email protected]
Cc: Eric Sunshine <[email protected]>; Torsten Bögershausen <[email protected]>; Ævar Arnfjörð Bjarmason <[email protected]>; Ramsay Jones <[email protected]>; Johannes Schindelin <[email protected]>; Eric DeCosta <[email protected]>
Subject: Re: [PATCH v10 5/6] fsmonitor: check for compatability before communicating with fsmonitorQuoted 4 lines · Eric DeCosta via GitGitGadget
From: Eric DeCosta <[email protected]>
If fsmonitor is not in a compatible state, die with an appropriate error messge.
[...]
Quoted 4 lines
fsmonitor-settings.c index renamed from fsmonitor-settings.c +0
-0
531a1b6f956..24480b9806d100644fsmonitor-settings.c +0
-0[...]
fsmonitor-settings.c char *fsm_settings__get_incompatible_msg(const
Quoted 9 lines
fsmonitor-settings.c +2
-0+char *fsm_settings__get_incompatible_msg (struct repository *r,enum fsmonitor_reason reason) {struct strbuf msg =STRBUF_INIT ; +const char *socket_dir;switch (reason) {case FSMONITOR_REASON_UNTESTED :@@ -281,9 +283,11 @@ char *fsm_settings__get_incompatible_msg(conststruct repository *r,
fsmonitor-settings.c char *fsm_settings__get_incompatible_msg(const struct repository *r,
Quoted 6 lines
fsmonitor-settings.c +1
-1goto done;case FSMONITOR_REASON_NOSOCKETS : + socket_dir =dirname ((char *)fsmonitor_ipc__get_path (r)); strbuf_addf(&msg, - _("repository '%s' is incompatible with fsmonitordue to lack of Unix sockets"),
fsmonitor-settings.c char *fsm_settings__get_incompatible_msg(const · line 288
Quoted 2 lines
fsmonitor-settings.c +1
-1- r->worktree); + _("socket directory '%s' is incompatible withfsmonitor due"),
fsmonitor-settings.c char *fsm_settings__get_incompatible_msg(const struct repository *r, · line 291
Quoted 3 lines
fsmonitor-settings.c +2
-0+ socket_dir); +strbuf_add (&msg,_ (" to lack of Unix sockets support" ),32 ); goto done;I don't think we should split the error message between two calls to
strbuf_add(). I realize that this was probably done because of line length concerns. But this makes assumptions on language word order during translations.Instead, we can use C string literal joining before passing it to the translation macro. Something like:
strbuf_addf(&msg,
_("socket directory '%s' is incompatible with "
"fsmonitor due to lack of Unix sockets support"),
socket_dir);[...]
fsmonitor.c void refresh_fsmonitor(struct index_state *istate) · line 311
Quoted 12 lines
fsmonitor.c index 57d6a483bee..43d580132fb renamed from fsmonitor.c +0
-0100644f
fsmonitor.c +4
-0@@ -305,6 +305,10 @@ void refresh_fsmonitor(struct index_state *istate)int is_trivial =0 ;struct repository *r = istate->repo ? istate->repo : the_repository;enum fsmonitor_mode fsm_mode =fsm_settings__get_mode (r); +enum fsmonitor_reason reason =fsm_settings__get_reason (r); + +if (reason >FSMONITOR_REASON_OK ) +die ("%s" ,fsm_settings__get_incompatible_msg (r, reason));We don't want to call
die()here. Maybe just silently return without doing anything or issue awarning()and return. (But I'm favoring a silent return here.)
Quoted 5 lines
From the clients' (
git status,git diff, etc.) point of view, they just want a
speed-up, if possible, but we shouldn't kill them; we should just let them do the normal scan that would have done if the feature were turned off.Jeff
If we just silently return then fsmonitor is in a perpetual incompatible state and the user gets no benefit from fsmonitor (in fact it could be worse as fsmonitor will attempt to spawn over and over again). I would think that it would be better to at least inform the user so that they can update fsmonitor's settings and have a more pleasant experience going forward.
-Eric
PATCH v10 6/6 fsmonitor: add documentation for allowRemote and socketDir options
In reply to Eric DeCosta via GitGitGadgetFrom: Eric DeCosta <[email protected]>
Add documentation for 'fsmonitor.allowRemote' and 'fsmonitor.socketDir'. Call-out experimental nature of 'fsmonitor.allowRemote' and limited file system support for 'fsmonitor.socketDir'.
--- Documentation/git-fsmonitor--daemon.txt | 48 +++++++++++++++++++++++-- 1 file changed, 45 insertions(+), 3 deletions(-)
Documentation/git-fsmonitor--daemon.txt +45 -3
@@ -3,7 +3,7 @@ git-fsmonitor{litdd}daemon(1) NAME ---- -git-fsmonitor--daemon - A Built-in File System Monitor +git-fsmonitor--daemon - A Built-in Filesystem Monitor SYNOPSIS --------
@@ -17,7 +17,7 @@ DESCRIPTION ----------- A daemon to watch the working directory for file and directory -changes using platform-specific file system notification facilities. +changes using platform-specific filesystem notification facilities. This daemon communicates directly with commands like `git status` using the link:technical/api-simple-ipc.html[simple IPC] interface
@@ -63,13 +63,55 @@ CAVEATS ------- The fsmonitor daemon does not currently know about submodules and does -not know to filter out file system events that happen within a +not know to filter out filesystem events that happen within a submodule. If fsmonitor daemon is watching a super repo and a file is modified within the working directory of a submodule, it will report the change (as happening against the super repo). However, the client will properly ignore these extra events, so performance may be affected but it will not cause an incorrect result. +By default, the fsmonitor daemon refuses to work against network-mounted +repositories; this may be overridden by setting `fsmonitor.allowRemote` to +`true`. Note, however, that the fsmonitor daemon is not guaranteed to work +correctly with all network-mounted repositories and such use is considered +experimental. + +On Mac OS, the inter-process communication (IPC) between various Git +commands and the fsmonitor daemon is done via a Unix domain socket (UDS) -- a +special type of file -- which is supported by native Mac OS filesystems, +but not on network-mounted filesystems, NTFS, or FAT32. Other filesystems +may or may not have the needed support; the fsmonitor daemon is not guaranteed +to work with these filesystems and such use is considered experimental. + +By default, the socket is created in the `.git` directory, however, if the +`.git` directory is on a network-mounted filesystem, it will be instead be +created at `$HOME/.git-fsmonitor-*` unless `$HOME` itself is on a +network-mounted filesystem in which case you must set the configuration +variable `fsmonitor.socketDir` to the path of a directory on a Mac OS native +filesystem in which to create the socket file. + +If none of the above directories (`.git`, `$HOME`, or `fsmonitor.socketDir`) +is on a native Mac OS file filesystem the fsmonitor daemon will report an +error that will cause the daemon and the currently running command to exit. + +CONFIGURATION +------------- + +When `core.fsmonitor` is set to `true` (see linkgit:git-config[1]) +the fsmonitor daemon will pay attention to the following configuration +variables: + +`fsmonitor.allowRemote`:: + By default, the daemon refuses to work against network-mounted + repositories. Setting `fsmonitor.allowRemote` to `true` overrides + this behavior. + +`fsmonitor.socketDir`:: + This Mac OS-specific option, if set, specifies the directory in + which to create the Unix domain socket used for communication + between fsmonitor and various Git commands. The directory must + reside on a native Mac OS filesystem as discussed above. + GIT --- Part of the linkgit:git[1] suite
Signature · 1 line
gitgitgadget
v11 cover fsmonitor: option to allow fsmonitor to run against network-mounted repos
In reply to Eric DeCosta via GitGitGadgetFollow-on to the work done to allow Windows to work against network-mounted repos for macOS.
Have macOS take advantage of the same configuration option, 'fsmonitor.allowRemote' that was introduced for Windows. Setting this option to true will override the default behavior (erroring-out) when a network-mounted repo is detected by fsmonitor.
The added wrinkle being that the Unix domain socket (UDS) file used for IPC cannot be created in a network location; instead $HOME is used if the default location is on the network. The user may, optionally, set the 'fsmonitor.socketDir' configuration option to a valid, local directory if $HOME itself is on the network or is simply not the desired location for the UDS file.
An additional issue is that for mount points in the root directory, FSEvents does not report a path that matches the worktree directory due to the introduction of 'synthetic firmlinks'. fsmonitor must map the FSEvents paths to the worktree directory by interrogating the root filesystem for synthetic firmlinks and using that information to translate the path.
v11 differs from v10:
* incorporates code review feedback
* fix memory leak in fsm-listen-darwin.c
v10 differs from v9:
* incorporates code review feedback
* improves error messaging for incompatible socket directory
v9 differs from v8:
* incorporates code review feedback
* check for incompatibility before communicating with fsmonitor
v8 differs from v7:
* incorporates code review feedback
* gets the rebase right
v7 differs from v6:
* incorporates code review feedback
v6 differs from v5:
* incorporates earlier, Windows-specific changes that have not made it back yet to the master branch
* incorporates code review feedback
* adds documentation for 'fsmonitor.allowRemote' and 'fsmonitor.socketDir'
v5 differs significantly from earlier versions:
* redesign of handling 'fsmonitor.allowRemote' and 'fsmonitor.socketDir' such that these options are no longer added to the settings data structure but are rather read from config at point of use
* refactoring of code for handling platform-specific file system checks via a common interface to avoid platform #ifdef in IPC code and be in-model with other platform-specific fsmonitor code
* dealing with 'synthetic firmlinks' on macOS
Eric DeCosta (6):
fsmonitor: refactor filesystem checks to common interface
fsmonitor: relocate socket file if .git directory is remote
fsmonitor: avoid socket location check if using hook
fsmonitor: deal with synthetic firmlinks on macOS
fsmonitor: check for compatability before communicating with fsmonitor
fsmonitor: add documentation for allowRemote and socketDir options
Documentation/git-fsmonitor--daemon.txt | 48 ++++++- Makefile | 2 + builtin/fsmonitor--daemon.c | 11 +- compat/fsmonitor/fsm-ipc-darwin.c | 51 +++++++ compat/fsmonitor/fsm-ipc-win32.c | 9 ++ compat/fsmonitor/fsm-listen-darwin.c | 14 +- compat/fsmonitor/fsm-path-utils-darwin.c | 132 +++++++++++++++++ compat/fsmonitor/fsm-path-utils-win32.c | 145 +++++++++++++++++++ compat/fsmonitor/fsm-settings-darwin.c | 72 +++------- compat/fsmonitor/fsm-settings-win32.c | 174 +---------------------- contrib/buildsystems/CMakeLists.txt | 4 + fsmonitor--daemon.h | 3 + fsmonitor-ipc.c | 18 ++- fsmonitor-ipc.h | 4 +- fsmonitor-path-utils.h | 59 ++++++++ fsmonitor-settings.c | 68 ++++++++- fsmonitor-settings.h | 4 +- fsmonitor.c | 4 + 18 files changed, 576 insertions(+), 246 deletions(-) create mode 100644 compat/fsmonitor/fsm-ipc-darwin.c create mode 100644 compat/fsmonitor/fsm-ipc-win32.c create mode 100644 compat/fsmonitor/fsm-path-utils-darwin.c create mode 100644 compat/fsmonitor/fsm-path-utils-win32.c create mode 100644 fsmonitor-path-utils.h
base-commit: d3fa443f97e3a8d75b51341e2d5bac380b7422df
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1326%2Fedecosta-mw%2Ffsmonitor_macos-v11
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1326/edecosta-mw/fsmonitor_macos-v11
Pull-Request: https://github.com/gitgitgadget/git/pull/1326
Range-diff vs v10:
1: 155a6890806 = 1: 155a6890806 fsmonitor: refactor filesystem checks to common interface
2: dbe113abb87 = 2: dbe113abb87 fsmonitor: relocate socket file if .git directory is remote
3: 86c15299ae8 = 3: 86c15299ae8 fsmonitor: avoid socket location check if using hook
4: 4f6c98cf834 ! 4: 1a516fd9214 fsmonitor: deal with synthetic firmlinks on macOS
@@ compat/fsmonitor/fsm-listen-darwin.c
struct fsm_listen_data
{
+@@ compat/fsmonitor/fsm-listen-darwin.c: static void fsevent_callback(ConstFSEventStreamRef streamRef,
+ struct string_list cookie_list = STRING_LIST_INIT_DUP;
+ const char *path_k;
+ const char *slash;
+- int k;
++ char *resolved = NULL;
+ struct strbuf tmp = STRBUF_INIT;
++ int k;
+
+ /*
+ * Build a list of all filesystem changes into a private/local
@@ compat/fsmonitor/fsm-listen-darwin.c: static void fsevent_callback(ConstFSEventStreamRef streamRef,
/*
* On Mac, we receive an array of absolute paths.
*/
- path_k = paths[k];
-+ path_k = fsmonitor__resolve_alias(paths[k], &state->alias);
-+ if (!path_k)
++ free(resolved);
++ resolved = fsmonitor__resolve_alias(paths[k], &state->alias);
++ if (resolved)
++ path_k = resolved;
++ else
+ path_k = paths[k];
/*
@@ compat/fsmonitor/fsm-listen-darwin.c: static void fsevent_callback(ConstFSEventS
/*
* We assume that any events that we received
+@@ compat/fsmonitor/fsm-listen-darwin.c: static void fsevent_callback(ConstFSEventStreamRef streamRef,
+ }
+ }
+
++ free(resolved);
+ fsmonitor_publish(state, batch, &cookie_list);
+ string_list_clear(&cookie_list, 0);
+ strbuf_release(&tmp);
+ return;
+
+ force_shutdown:
++ free(resolved);
+ fsmonitor_batch__free_list(batch);
+ string_list_clear(&cookie_list, 0);
+
## compat/fsmonitor/fsm-path-utils-darwin.c ##
@@
5: 8f6c1fbacbf ! 5: e0fe05dabef fsmonitor: check for compatability before communicating with fsmonitor
@@ fsmonitor-settings.c: char *fsm_settings__get_incompatible_msg(const struct repo
strbuf_addf(&msg,
- _("repository '%s' is incompatible with fsmonitor due to lack of Unix sockets"),
- r->worktree);
-+ _("socket directory '%s' is incompatible with fsmonitor due"),
++ _("socket directory '%s' is incompatible with fsmonitor due"
++ " to lack of Unix sockets support"),
+ socket_dir);
-+ strbuf_add(&msg, _(" to lack of Unix sockets support"), 32);
goto done;
}
@@ fsmonitor.c: void refresh_fsmonitor(struct index_state *istate)
+ enum fsmonitor_reason reason = fsm_settings__get_reason(r);
+
+ if (reason > FSMONITOR_REASON_OK)
-+ die("%s", fsm_settings__get_incompatible_msg(r, reason));
++ warning("%s", fsm_settings__get_incompatible_msg(r, reason));
if (fsm_mode <= FSMONITOR_MODE_DISABLED ||
istate->fsmonitor_has_run_once) 6: d7c25bf96c6 = 6: 3200b505988 fsmonitor: add documentation for allowRemote and socketDir options
Signature · 1 line
gitgitgadget
PATCH v11 1/6 fsmonitor: refactor filesystem checks to common interface
From: Eric DeCosta <[email protected]>
Provide a common interface for getting basic filesystem information including filesystem type and whether the filesystem is remote.
Refactor existing code for getting basic filesystem info and detecting remote file systems to the new interface.
Refactor filesystem checks to leverage new interface. For macOS, error-out if the Unix Domain socket (UDS) file is on a remote filesystem.
--- Makefile | 1 + compat/fsmonitor/fsm-path-utils-darwin.c | 40 ++++++ compat/fsmonitor/fsm-path-utils-win32.c | 128 +++++++++++++++++ compat/fsmonitor/fsm-settings-darwin.c | 62 +++----- compat/fsmonitor/fsm-settings-win32.c | 172 +---------------------- contrib/buildsystems/CMakeLists.txt | 2 + fsmonitor-path-utils.h | 23 +++ fsmonitor-settings.c | 50 +++++++ 8 files changed, 263 insertions(+), 215 deletions(-) create mode 100644 compat/fsmonitor/fsm-path-utils-darwin.c create mode 100644 compat/fsmonitor/fsm-path-utils-win32.c create mode 100644 fsmonitor-path-utils.h
Makefile +1 -0
@@ -2039,6 +2039,7 @@ endif ifdef FSMONITOR_OS_SETTINGS COMPAT_CFLAGS += -DHAVE_FSMONITOR_OS_SETTINGS COMPAT_OBJS += compat/fsmonitor/fsm-settings-$(FSMONITOR_OS_SETTINGS).o + COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_OS_SETTINGS).o endif ifeq ($(TCLTK_PATH),)
compat/fsmonitor/fsm-path-utils-darwin.c new file +40 -0
@@ -0,0 +1,40 @@ +#include "fsmonitor.h" +#include "fsmonitor-path-utils.h" +#include <sys/param.h> +#include <sys/mount.h> + +int fsmonitor__get_fs_info (const char *path,struct fs_info *fs_info) +{ +struct statfs fs; +if (statfs (path, &fs) ==-1 ) { +int saved_errno = errno; +trace_printf_key (&trace_fsmonitor,"statfs('%s') failed: %s" , + path,strerror (saved_errno)); + errno = saved_errno; +return -1 ; + } + +trace_printf_key (&trace_fsmonitor, +"statfs('%s') [type 0x%08x][flags 0x%08x] '%s'" , + path, fs.f_type, fs.f_flags, fs.f_fstypename); + +if (!(fs.f_flags &MNT_LOCAL )) + fs_info->is_remote =1 ; +else + fs_info->is_remote =0 ; + + fs_info->typename = fs.f_fstypename; + +trace_printf_key (&trace_fsmonitor, +"'%s' is_remote: %d" , + path, fs_info->is_remote); +return 0 ; +} + +int fsmonitor__is_fs_remote (const char *path) +{ +struct fs_info fs; +if (fsmonitor__get_fs_info (path, &fs)) +return -1 ; +return fs.is_remote; +}
compat/fsmonitor/fsm-path-utils-win32.c new file +128 -0
128 lines not on this page · raw message
compat/fsmonitor/fsm-settings-darwin.c +17 -45
96 lines not on this page · raw message
compat/fsmonitor/fsm-settings-win32.c +2 -170
190 lines not on this page · raw message
contrib/buildsystems/CMakeLists.txt +2 -0
@@ -308,6 +308,7 @@ if(SUPPORTS_SIMPLE_IPC) add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-win32.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-win32.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-win32.c)
@@ -315,6 +316,7 @@ if(SUPPORTS_SIMPLE_IPC) add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-darwin.c)
fsmonitor-path-utils.h new file +23 -0
@@ -0,0 +1,23 @@ +#ifndef FSM_PATH_UTILS_H +#define FSM_PATH_UTILS_H + +struct fs_info { +int is_remote; +char *typename; +}; + +/* +* Get some basic filesystem informtion for the given path +* +* Returns -1 on error, zero otherwise. +*/ +int fsmonitor__get_fs_info (const char *path,struct fs_info *fs_info); + +/* +* Determines if the filesystem that path resides on is remote. +* +* Returns -1 on error, 0 if not remote, 1 if remote. +*/ +int fsmonitor__is_fs_remote (const char *path); + +#endif
fsmonitor-settings.c +50 -0
68 lines not on this page · raw message
Signature · 1 line
gitgitgadget