Convert USE_NSEC to runtime config
Subsystems (3)
cover Convert USE_NSEC to runtime config
Topic name: dk/use-nsec-runtime
Topic summary: Expose USE_NSEC as a runtime configuration, since build-time is too early for distributing Git [1]. As a result, common index-related options, like git-diff, are less likely to hit "racy git" problems on supported filesystems.
[1]: https://git.github.io/rev_news/2026/07/31/edition-137/
Built on master (2c78326f81 (The 11th batch, 2026-08-05)).
Hi all, this series follows up on the previous racy Git/USE_NSEC conversations.
- The first patch is a mostly-unrelated documentation fix for Meson, but it came out of something I spotted while reviewing the outputs of the final (main) patch.
- The second patch is a preliminary no-op reorganization of repo_config_values_init.
- The third patch is the meat, converting USE_NSEC into core.useNanosec.
There is a small textual and semantic conflict with 'ty/repo-config-cleanups' in 'seen', since that branch removes the comments in 'struct repo_config_values' which this series adds to. (The semantic conflict is that, if we drop those comments, we should probably not add them to repo_config_values_init like I do in patch 2.)
Todo: I haven't touched any tests; I saw a bunch of hits for "git grep racy t" but wasn't sure how to fit this particular change in, especially since it won't be equally valid on all systems? Advice welcome.
Todo: I wonder if "useNanosec" paints us into too much of a corner; that is (slightly more abstractly), we are using *extended precision* in the index. Maybe the name and documentation should reflect that, so we aren't too committed to "nanoseconds"?
- Some platforms could offer extended precision that is not as precise as nanoseconds
- Some could offer precision _beyond_ nanoseconds
idk.
[1/3] meson: expose knob for xmlto relative links in manuals [2/3] environment: align repo_config_values_init with struct declaration [3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
Documentation/config/core.adoc | 6 ++++++ Documentation/meson.build | 7 ++++++- Documentation/technical/racy-git.adoc | 11 ++++++----- Makefile | 12 +----------- builtin/update-index.c | 2 +- compat/posix.h | 1 - configure.ac | 6 ------ environment.c | 25 ++++++++++++++++++------- environment.h | 1 + meson_options.txt | 2 ++ read-cache.c | 17 +++++++++-------- statinfo.c | 14 +++++++------- 12 files changed, 57 insertions(+), 47 deletions(-)
base-commit: 2c78326f810173a4f3aefd8021f1e07575412481
Signature · 1 line
2.55.0.340.g8e2bf96aa5.dirty
PATCH 1/3 meson: expose knob for xmlto relative links in manuals
Makefile-based builds have had this knob for most of the project's life, since a479a564dc (Documentation/Makefile: allow man.base.url.for.relative.link to be set from Make, 2009-12-03).
Meson, however, hard-codes the equivalent of $prefix/$mandir, which is not really where all the HTML docs are stored in most distro builds. Plus, this value is missing a trailing slash, so links come out broken, like this in git.1:
1. Git User’s Manual
/usr/share/manuser-manual.html
Of course we can do better:
1. Change the default to match Make: use file://$(htmldir)/ (with trailing slash!) to form a local URL pointing at the HTML docs. This is safe because all current uses of link:<relative> point at HTML docs:
git grep 'link:[[:alnum:]]' Documentation | grep -ve html -e http
produces only a single result (Documentation/howto/howto-index.sh) which can be ignored. Since nothing else [*] in the normal build sets MAN_BASE_URL, this seems like the right default.
2. Provide a configurable knob, just like the Makefile, so distributions that build with Meson (like Gentoo) can decide where to make the links if they need to. Those that set htmldir probably won't need to tweak this any further, though.
[*]: Well, Git's todo branch has a script dodoc.sh to build and archive
docs for kernel.org; these docs are pulled by Homebrew installations, for example. It sets MAN_BASE_URL to "git_htmldocs", so the equivalent note on macOS + Homebrew is
1. Git User’s Manual
git-htmldocs/user-manual.html
which is not functional either, but that's a problem for downstream. In any case, users can recover the right path with "git --html-path".
---
Notes (benknoble/commits):
This patch is mostly because I noticed the link I added in a later patch didn't come out right.
I did an internet search for "MAN_BASE_URL" and got no real hits, so I'm not sure if any distros today actually use it, but that's not a proper audit in that I didn't look at any distro _code_ besides Gentoo (which, as noted, uses Meson).
Documentation/meson.build | 7 ++++++- meson_options.txt | 2 ++ 2 files changed, 8 insertions(+), 1 deletion(-)
Documentation/meson.build +6 -1
@@ -379,13 +379,18 @@ foreach manpage, category : manpages output: fs.stem(manpage) + '.xml', ) + man_base_url = 'file://' + htmldir + '/' + if get_option('man_base_url') != '' + man_base_url = get_option('man_base_url') + endif + doc_targets += custom_target( command: [ xmlto, '-m', '@INPUT0@', '-m', '@INPUT1@', '--stringparam', - 'man.base.url.for.relative.links=' + get_option('prefix') / get_option('mandir'), + 'man.base.url.for.relative.links=' + man_base_url, 'man', manpage_xml_target, '-o',
meson_options.txt +2 -0
@@ -111,6 +111,8 @@ option('default_help_format', type: 'combo', choices: ['man', 'html', 'platform' description: 'Default format used when executing git-help(1).') option('docs_backend', type: 'combo', choices: ['asciidoc', 'asciidoctor', 'auto'], value: 'auto', description: 'Which backend to use to generate documentation.') +option('man_base_url', type: 'string', value: '', + description: 'The base URL to use for relative links in manuals') # Testing. option('benchmarks', type: 'feature', value: 'auto',
Signature · 1 line
2.55.0.340.g8e2bf96aa5.dirty
PATCH 2/3 environment: align repo_config_values_init with struct declaration
In reply to D. Ben KnobleThe order of assignments in repo_config_values_init is chaotic and hard to follow, especially when comparing with the struct definition to ensure all members are initialized. As new members will be added in the future, make it easier to validate changes by aligning the two.
Refactor assignment order with no behavioral changes.
--- environment.c | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-)
environment.c +12 -7
@@ -745,6 +745,7 @@ int git_default_config(const char *var, const char *value,void repo_config_values_init (struct repo_config_values *cfg) { +/* section "core" config values */ cfg->attributes_file =NULL ; cfg->excludes_file =NULL ; cfg->editor_program =NULL ;
@@ -756,20 +757,24 @@ void repo_config_values_init(struct repo_config_values *cfg) cfg->autorebase =AUTOREBASE_NEVER ; cfg->object_creation_mode =OBJECT_CREATION_MODE ; cfg->apply_sparse_checkout =0 ; - cfg->protect_hfs =PROTECT_HFS_DEFAULT ; - cfg->protect_ntfs =PROTECT_NTFS_DEFAULT ; - cfg->ignore_case =0 ; - cfg->trust_executable_bit =1 ; - cfg->has_symlinks =platform_has_symlinks (); - cfg->branch_track =BRANCH_TRACK_REMOTE ; cfg->trust_ctime =1 ; cfg->check_stat =1 ; cfg->zlib_compression_level =Z_BEST_SPEED ; cfg->pack_compression_level =Z_DEFAULT_COMPRESSION ; cfg->precomposed_unicode =-1 ;/* see probe_utf8_pathname_composition() */ cfg->core_sparse_checkout_cone =0 ; - cfg->sparse_expect_files_outside_of_patterns =0 ; cfg->warn_on_object_refname_ambiguity =1 ; + cfg->protect_hfs =PROTECT_HFS_DEFAULT ; + cfg->protect_ntfs =PROTECT_NTFS_DEFAULT ; + cfg->ignore_case =0 ; + cfg->trust_executable_bit =1 ; + cfg->has_symlinks =platform_has_symlinks (); + +/* section "sparse" config values */ + cfg->sparse_expect_files_outside_of_patterns =0 ; + +/* section "branch" config values */ + cfg->branch_track =BRANCH_TRACK_REMOTE ; }void repo_config_values_clear (struct repo_config_values *cfg)
Signature · 1 line
2.55.0.340.g8e2bf96aa5.dirty
PATCH 3/3 core: convert build-time USE_NSEC into runtime core.useNanosec
In reply to D. Ben KnobleRacy Git problems persist today, manifesting themselves in the performance of commands like "git diff" in new worktrees [1]. We have long had a build knob "USE_NSEC" to tell Git to use in-core nanosecond precision when available, which mitigates most if not all racy issues, but most builds we know about it don't use it. In part, that's because someone distributing Git can't safely enable it at compile-time if they don't know exactly what platforms their distribution will be used on.
[1]: https://lore.kernel.org/git/CALnO6CADMJSixqYvL1Yo8qKX5rWhKQ+2OoSEuPUh-yoeK9TseQ@mail.gmail.com
These days, most platforms are likely to be safe for the USE_NSEC code. Regardless, we want to give users the ability to benefit from it. This requires exposing the compile-time gated code as a runtime option.
In addition, update the Racy Git documentation and other mentions of USE_NSEC in the code.
Best-viewed-with: --ignore-space-change
---
Notes (benknoble/commits):
Repeating the benchmark from my original mail [1] shows an improvement
# git worktree add -d ../perf-test HEAD
# hyperfine -N --warmup=10 './build/bin-wrappers/git diff'
Benchmark 1: ./build/bin-wrappers/git diff
Time (mean ± σ): 3.8 ms ± 0.4 ms [User: 4.7 ms, System: 4.4 ms]
Range (min … max): 3.2 ms … 5.6 ms 780 runs
# (pushd ../perf-test && hyperfine -N --warmup=10 $OLDPWD/'./build/bin-wrappers/git diff')
Benchmark 1: /home/benknoble/code/git/./build/bin-wrappers/git diff
Time (mean ± σ): 217.5 ms ± 2.9 ms [User: 202.1 ms, System: 23.4 ms]
Range (min … max): 213.9 ms … 223.3 ms 13 runs
# (pushd ../perf-test && hyperfine -N --warmup=10 $OLDPWD/'./build/bin-wrappers/git -c core.useNanosec=true diff')
Benchmark 1: /home/benknoble/code/git/./build/bin-wrappers/git -c core.useNanosec=true diff
Time (mean ± σ): 3.8 ms ± 0.4 ms [User: 5.3 ms, System: 4.2 ms]
Range (min … max): 3.2 ms … 6.9 ms 541 runs
[1]: <CALnO6CADMJSixqYvL1Yo8qKX5rWhKQ+2OoSEuPUh-yoeK9TseQ@mail.gmail.com>
Passing CI: https://github.com/benknoble/git/actions/runs/31104581195Documentation/config/core.adoc | 6 ++++++ Documentation/technical/racy-git.adoc | 11 ++++++----- Makefile | 12 +----------- builtin/update-index.c | 2 +- compat/posix.h | 1 - configure.ac | 6 ------ environment.c | 6 ++++++ environment.h | 1 + read-cache.c | 17 +++++++++-------- statinfo.c | 14 +++++++------- 10 files changed, 37 insertions(+), 39 deletions(-)
Documentation/config/core.adoc +6 -0
@@ -118,6 +118,12 @@ core.trustctime:: crawlers and some backup systems). See linkgit:git-update-index[1]. True by default. +core.useNanosec:: + If true, use nanosecond precision for ctime and mtime + comparisions between the index and the working tree (if Git + was compiled to store it). + See link:technical/racy-git.html[Racy Git]. False by default. + core.splitIndex:: If true, the split-index feature of the index will be used. See linkgit:git-update-index[1]. False by default.
Documentation/technical/racy-git.adoc +6 -5
@@ -39,8 +39,8 @@ files) from `st_mode` member, `st_mtime` and `st_ctime` timestamps, `st_uid`, `st_gid`, `st_ino`, and `st_size` members. With a `USE_STDEV` compile-time option, `st_dev` is also compared, but this is not enabled by default because this member -is not stable on network filesystems. With `USE_NSEC` -compile-time option, `st_mtim.tv_nsec` and `st_ctim.tv_nsec` +is not stable on network filesystems. With 'core.useNanosec' +config setting, `st_mtim.tv_nsec` and `st_ctim.tv_nsec` members are also compared. On Linux, this is not enabled by default because in-core timestamps can have finer granularity than on-disk timestamps, resulting in meaningless changes when an
@@ -49,9 +49,10 @@ of git://git.kernel.org/pub/scm/linux/kernel/git/tglx/history.git ([PATCH] Sync in core time granularity with filesystems, 2005-01-04). This patch is included in kernel 2.6.11 and newer, but only fixes the issue for file systems with exactly 1 ns or 1 s -resolution. Other file systems are still broken in current Linux -kernels (e.g. CEPH, CIFS, NTFS, UDF), see -https://lore.kernel.org/lkml/[email protected]/ +resolution. As of kernel 4.3, other file systems (CEPH, CIFS, NTFS, UFS, FUSE) +were fixed; see https://public-inbox.org/git/5605D88A.20104%40gmail.com/. FAT +has been fixed since 2015. The usual suspects (ext2, ext4, XFS) are known to +work, too. Racy Git --------
Makefile +1 -11
@@ -197,18 +197,11 @@ include shared.mak # Define NO_NORETURN if using buggy versions of gcc 4.6+ and profile feedback, # as the compiler can crash (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=49299) # -# Define USE_NSEC below if you want git to care about sub-second file mtimes -# and ctimes. Note that you need recent glibc (at least 2.2.4) for this. On -# Linux, kernel 2.6.11 or newer is required for reliable sub-second file times -# on file systems with exactly 1 ns or 1 s resolution. If you intend to use Git -# on other file systems (e.g. CEPH, CIFS, NTFS, UDF), don't enable USE_NSEC. See -# Documentation/technical/racy-git.adoc for details. -# # Define USE_ST_TIMESPEC if your "struct stat" uses "st_ctimespec" instead of # "st_ctim" # # Define NO_NSEC if your "struct stat" does not have "st_ctim.tv_nsec" -# available. This automatically turns USE_NSEC off. +# available. # # Define USE_STDEV below if you want git to care about the underlying device # change being considered an inode change from the update-index perspective.
@@ -1935,9 +1928,6 @@ endif ifdef NO_ST_BLOCKS_IN_STRUCT_STAT BASIC_CFLAGS += -DNO_ST_BLOCKS_IN_STRUCT_STAT endif -ifdef USE_NSEC - BASIC_CFLAGS += -DUSE_NSEC -endif ifdef USE_ST_TIMESPEC BASIC_CFLAGS += -DUSE_ST_TIMESPEC endif
builtin/update-index.c +1 -1
@@ -130,7 +130,7 @@ static void xrmdir(const char *path)static void avoid_racy (void ) {/* -* not use if we could usleep(10) if USE_NSEC is defined. The +* not use if we could usleep(10) if core.useNanosec is defined. The * field nsec could be there, but the OS could choose to * ignore it? */
compat/posix.h +0 -1
@@ -487,7 +487,6 @@ int git_qsort_s(void *base, size_t nmemb, size_t size, } while (0)#ifdef NO_NSEC -#undef USE_NSEC#define ST_CTIME_NSEC (st) 0#define ST_MTIME_NSEC (st) 0#else
configure.ac +0 -6
@@ -351,12 +351,6 @@ GIT_PARSE_WITH(iconv)) ## --enable-FEATURE[=ARG] and --disable-FEATURE # -# Define USE_NSEC below if you want git to care about sub-second file mtimes -# and ctimes. Note that you need recent glibc (at least 2.2.4) for this, and -# it will BREAK YOUR LOCAL DIFFS! show-diff and anything using it will likely -# randomly break unless your underlying filesystem supports those sub-second -# times (my ext3 doesn't). -# # Define USE_STDEV below if you want git to care about the underlying device # change being considered an inode change from the update-index perspective.
environment.c +6 -0
@@ -571,6 +571,11 @@ int git_default_core_config(const char *var, const char *value,return 0 ; } +if (!strcmp (var,"core.usenanosec" )) { + cfg->use_nanosec =git_config_bool (var, value); +return 0 ; + } +/* Add other config variables here and to Documentation/config.adoc. */ return platform_core_config (var, value, ctx, cb); }
@@ -769,6 +774,7 @@ void repo_config_values_init(struct repo_config_values *cfg) cfg->ignore_case =0 ; cfg->trust_executable_bit =1 ; cfg->has_symlinks =platform_has_symlinks (); + cfg->use_nanosec =0 ;/* section "sparse" config values */ cfg->sparse_expect_files_outside_of_patterns =0 ;
environment.h +1 -0
@@ -139,6 +139,7 @@ struct repo_config_values {int ignore_case;int trust_executable_bit;int has_symlinks; +int use_nanosec;/* section "sparse" config values */ int sparse_expect_files_outside_of_patterns;
read-cache.c +9 -8
@@ -353,15 +353,16 @@ static int ce_match_stat_basic(const struct cache_entry *ce, struct stat *st)static int is_racy_stat (const struct index_state *istate,const struct stat_data *sd) { +int use_nsec =0 ; +repo_config_get_bool (the_repository,"core.useNanosec" , &use_nsec); +return (istate->timestamp.sec && -#ifdef USE_NSEC - /* nanosecond timestamped files can also be racy! */ - (istate->timestamp.sec < sd->sd_mtime.sec || - (istate->timestamp.sec == sd->sd_mtime.sec && - istate->timestamp.nsec <= sd->sd_mtime.nsec)) -#else - istate->timestamp.sec <= sd->sd_mtime.sec -#endif +/* nanosecond timestamped files can also be racy! */ + use_nsec + ? (istate->timestamp.sec < sd->sd_mtime.sec || + (istate->timestamp.sec == sd->sd_mtime.sec && + istate->timestamp.nsec <= sd->sd_mtime.nsec)) + : istate->timestamp.sec <= sd->sd_mtime.sec ); }
statinfo.c +7 -7
@@ -72,13 +72,13 @@ int match_stat_data(const struct stat_data *sd, struct stat *st) sd->sd_ctime.sec != (unsigned int)st->st_ctime) changed |=CTIME_CHANGED ; -#ifdef USE_NSEC -if (cfg->check_stat && sd->sd_mtime.nsec !=ST_MTIME_NSEC (*st)) - changed |=MTIME_CHANGED ; -if (cfg->trust_ctime && cfg->check_stat && - sd->sd_ctime.nsec !=ST_CTIME_NSEC (*st)) - changed |=CTIME_CHANGED ; -#endif +if (cfg->use_nanosec) { +if (cfg->check_stat && sd->sd_mtime.nsec !=ST_MTIME_NSEC (*st)) + changed |=MTIME_CHANGED ; +if (cfg->trust_ctime && cfg->check_stat && + sd->sd_ctime.nsec !=ST_CTIME_NSEC (*st)) + changed |=CTIME_CHANGED ; + }if (cfg->check_stat) { if (sd->sd_uid != (unsigned int) st->st_uid ||
Signature · 1 line
2.55.0.340.g8e2bf96aa5.dirty
Re: [PATCH 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
Quoted 10 lines · D. Ben Knoble
Racy Git problems persist today, manifesting themselves in the performance of commands like "git diff" in new worktrees [1]. We have long had a build knob "
USE_NSEC" to tell Git to use in-core nanosecond precision when available, which mitigates most if not all racy issues, but most builds we know about it don't use it. In part, that's because someone distributing Git can't safely enable it at compile-time if they don't know exactly what platforms their distribution will be used on.[1]: https://lore.kernel.org/git/CALnO6CADMJSixqYvL1Yo8qKX5rWhKQ+2OoSEuPUh-yoeK9TseQ@mail.gmail.com
Quoted 8 lines
These days, most platforms are likely to be safe for the
USE_NSECcode. Regardless, we want to give users the ability to benefit from it. This requires exposing the compile-time gated code as a runtime option.In addition, update the Racy Git documentation and other mentions of
USE_NSECin the code.Best-viewed-with: --ignore-space-change
Don't do this. It probably is helpful to have something like that below the three-dash lines, though.
Quoted 8 lines
Signed-off-by D. Ben Knoble <[email protected]>---
environment.c +0
-0
environment.c int git_default_core_config(const char *var, const char *value, · line 577
Quoted 7 lines
environment.c +4
-0@@ -571,6 +571,11 @@ int git_default_core_config(const char *var, const char *value,return 0 ; } +if (!strcmp (var,"core.usenanosec" )) { + cfg->use_nanosec =git_config_bool (var, value); +return 0 ; + }
OK.
read-cache.c static int ce_match_stat_basic(const struct cache_entry *ce, struct stat *st) · line 357
Quoted 10 lines
read-cache.c +2
-0@@ -353,15 +353,16 @@ static int ce_match_stat_basic(const struct cache_entry *ce, struct stat *st)static int is_racy_stat (const struct index_state *istate,const struct stat_data *sd) { +int use_nsec =0 ; +repo_config_get_bool (the_repository,"core.useNanosec" , &use_nsec);
Yeek. Isn't this a relatively hot code path? If it is, it is criminal to force string parsing and matching like this, every time somebody calls the function.
Doesn't istate know what repository it is working with and in there you should be able find its repo_settings struct cheaply, no?
Re: [PATCH 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
Quoted 16 lines · Junio C Hamano
Quoted 14 lines · D. Ben Knoble
Racy Git problems persist today, manifesting themselves in the performance of commands like "git diff" in new worktrees [1]. We have long had a build knob "
USE_NSEC" to tell Git to use in-core nanosecond precision when available, which mitigates most if not all racy issues, but most builds we know about it don't use it. In part, that's because someone distributing Git can't safely enable it at compile-time if they don't know exactly what platforms their distribution will be used on.[1]: https://lore.kernel.org/git/CALnO6CADMJSixqYvL1Yo8qKX5rWhKQ+2OoSEuPUh-yoeK9TseQ@mail.gmail.com
These days, most platforms are likely to be safe for the
USE_NSECcode. Regardless, we want to give users the ability to benefit from it. This requires exposing the compile-time gated code as a runtime option.
Quoted 7 lines
Quoted 4 lines
In addition, update the Racy Git documentation and other mentions of
USE_NSECin the code.Best-viewed-with: --ignore-space-change
Don't do this. It probably is helpful to have something like that below the three-dash lines, though.
Including this hint in the commit message could be useful for anyone who stumbles upon this commit in a couple of months or years time. Whether it should be a trailer or not is another question.
Re: [PATCH 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
Quoted 21 lines · SZEDER Gábor
Quoted 19 lines · Junio C Hamano
Quoted 17 lines · D. Ben Knoble
Racy Git problems persist today, manifesting themselves in the performance of commands like "git diff" in new worktrees [1]. We have long had a build knob "
USE_NSEC" to tell Git to use in-core nanosecond precision when available, which mitigates most if not all racy issues, but most builds we know about it don't use it. In part, that's because someone distributing Git can't safely enable it at compile-time if they don't know exactly what platforms their distribution will be used on.[1]: https://lore.kernel.org/git/CALnO6CADMJSixqYvL1Yo8qKX5rWhKQ+2OoSEuPUh-yoeK9TseQ@mail.gmail.com
These days, most platforms are likely to be safe for the
USE_NSECcode. Regardless, we want to give users the ability to benefit from it. This requires exposing the compile-time gated code as a runtime option.In addition, update the Racy Git documentation and other mentions of
USE_NSECin the code.
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
Best-viewed-with: --ignore-space-change
Don't do this. It probably is helpful to have something like that below the three-dash lines, though.
Including this hint in the commit message could be useful for anyone who stumbles upon this commit in a couple of months or years time. Whether it should be a trailer or not is another question.
Yep, the trailer is a force-of-habit for me. I'll move it into the commit message body in the next version.
I do find it helpful when the author of a patch---who presumably knows the changes best---provides some guidance on making sense of the diff. In this case, some code is re-indented as '#ifdef's change to runtime 'if's, so ignoring whitespace changes makes it easier to see there was no change there.
Signature · 1 line
D. Ben Knoble
Re: [PATCH 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
In reply to Junio C HamanoQuoted 17 lines · Junio C Hamano
Quoted 14 lines · D. Ben Knoble
Racy Git problems persist today, manifesting themselves in the performance of commands like "git diff" in new worktrees [1]. We have long had a build knob "
USE_NSEC" to tell Git to use in-core nanosecond precision when available, which mitigates most if not all racy issues, but most builds we know about it don't use it. In part, that's because someone distributing Git can't safely enable it at compile-time if they don't know exactly what platforms their distribution will be used on.[1]: https://lore.kernel.org/git/CALnO6CADMJSixqYvL1Yo8qKX5rWhKQ+2OoSEuPUh-yoeK9TseQ@mail.gmail.com
These days, most platforms are likely to be safe for the
USE_NSECcode. Regardless, we want to give users the ability to benefit from it. This requires exposing the compile-time gated code as a runtime option.
Quoted 7 lines
Quoted 4 lines
In addition, update the Racy Git documentation and other mentions of
USE_NSECin the code.Best-viewed-with: --ignore-space-change
Don't do this. It probably is helpful to have something like that below the three-dash lines, though.
[replied to SZEDER down-thread]
Quoted 34 lines
Quoted 9 lines
Signed-off-by D. Ben Knoble <[email protected]>---
environment.c +0
-0environment.c int git_default_core_config(const char *var, const char *value, · line 577
Quoted 7 lines
environment.c +4
-0@@ -571,6 +571,11 @@ int git_default_core_config(const char *var, const char *value,return 0 ; } +if (!strcmp (var,"core.usenanosec" )) { + cfg->use_nanosec =git_config_bool (var, value); +return 0 ; + }OK.
read-cache.c static int ce_match_stat_basic(const struct cache_entry *ce, struct stat *st) · line 357
Quoted 10 lines
read-cache.c +2
-0@@ -353,15 +353,16 @@ static int ce_match_stat_basic(const struct cache_entry *ce, struct stat *st)static int is_racy_stat (const struct index_state *istate,const struct stat_data *sd) { +int use_nsec =0 ; +repo_config_get_bool (the_repository,"core.useNanosec" , &use_nsec);Yeek. Isn't this a relatively hot code path? If it is, it is criminal to force string parsing and matching like this, every time somebody calls the function.
Quoted 2 lines
Doesn't istate know what repository it is working with and in there you should be able find its
repo_settingsstruct cheaply, no?
TL;DR yes, but the patch series doesn't currently put the member in repo_settings (repo_config_values). End of mail contains some commentary there; folks from <[email protected]> cc'd.
I did some benchmarking on linux.git @ 848acc8ffe1b7cd5f1bf427b93069becfebc2c9d.
Brand-new worktree, without refreshing the index:
hyperfine -N --warmup=10 \
-n core.useNanosec=false ~c/'git/build/bin-wrappers/git diff' \
-n core.useNanosec=true ~c/'git/build/bin-wrappers/git -c core.useNanosec=true diff' \
-n v2.55.0_USE_NSEC_disabled ~c/'perf-test/build/bin-wrappers/git diff' Benchmark 1: core.useNanosec=false
Time (mean ± σ): 853.7 ms ± 23.2 ms [User: 823.2 ms, System: 159.3 ms]
Range (min … max): 839.5 ms … 904.7 ms 10 runs
Warning: Statistical outliers were detected. Consider re-running
this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.
Benchmark 2: core.useNanosec=true
Time (mean ± σ): 20.4 ms ± 1.5 ms [User: 41.0 ms, System: 102.2 ms] Range (min … max): 17.6 ms … 24.8 ms 122 runs
Benchmark 3: v2.55.0_USE_NSEC_disabled
Time (mean ± σ): 839.2 ms ± 12.7 ms [User: 796.8 ms, System: 158.8 ms]
Range (min … max): 830.5 ms … 864.4 ms 10 runs
Warning: Statistical outliers were detected. Consider re-running
this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.
Summary
core.useNanosec=true ran
41.06 ± 3.03 times faster than v2.55.0_USE_NSEC_disabled 41.77 ± 3.22 times faster than core.useNanosec=false
Same worktree after "git update-index --refresh":
hyperfine -N --warmup=10 \
-n core.useNanosec=false ~c/'git/build/bin-wrappers/git diff' \
-n core.useNanosec=true ~c/'git/build/bin-wrappers/git -c core.useNanosec=true diff' \
-n v2.55.0_USE_NSEC_disabled ~c/'perf-test/build/bin-wrappers/git diff' Benchmark 1: core.useNanosec=false
Time (mean ± σ): 20.8 ms ± 2.1 ms [User: 41.8 ms, System: 102.9 ms] Range (min … max): 17.4 ms … 26.4 ms 126 runs
Benchmark 2: core.useNanosec=true
Time (mean ± σ): 20.0 ms ± 1.0 ms [User: 40.4 ms, System: 101.8 ms] Range (min … max): 18.0 ms … 23.6 ms 158 runs
Benchmark 3: v2.55.0_USE_NSEC_disabled
Time (mean ± σ): 19.2 ms ± 1.1 ms [User: 27.3 ms, System: 100.2 ms] Range (min … max): 16.9 ms … 23.4 ms 160 runs
Summary
v2.55.0_USE_NSEC_disabled ran
1.04 ± 0.08 times faster than core.useNanosec=true 1.08 ± 0.13 times faster than core.useNanosec=false
So yeah, when we don't need the nanosec timings, this ends up minutely slower than without it. When I apply the attached patch (sorry, GMail) on top to poke through istate->repo->config_values_private_.use_nanosec:
New worktree, no index refresh:
hyperfine -N --warmup=10 \
-n v2_core.useNanosec=false ~c/'git/build/bin-wrappers/git diff' \
-n v2_core.useNanosec=true ~c/'git/build/bin-wrappers/git -c core.useNanosec=true diff' \
-n v2.55.0_USE_NSEC_disabled ~c/'perf-test/build/bin-wrappers/git diff' Benchmark 1: v2_core.useNanosec=false
Time (mean ± σ): 148.0 ms ± 2.8 ms [User: 142.5 ms, System: 124.0 ms]
Range (min … max): 144.3 ms … 155.3 ms 20 runs
Benchmark 2: v2_core.useNanosec=true
Time (mean ± σ): 21.2 ms ± 2.0 ms [User: 27.6 ms, System: 101.4 ms] Range (min … max): 17.5 ms … 28.8 ms 123 runs
Benchmark 3: v2.55.0_USE_NSEC_disabled
Time (mean ± σ): 148.4 ms ± 8.6 ms [User: 141.0 ms, System: 122.8 ms]
Range (min … max): 140.9 ms … 179.6 ms 21 runs
Summary
v2_core.useNanosec=true ran
7.00 ± 0.67 times faster than v2_core.useNanosec=false 7.01 ± 0.77 times faster than v2.55.0_USE_NSEC_disabled
(We can see the raciness in the variability of the timings, neat)
After "git update-index --refresh":
hyperfine -N --warmup=10 \
-n v2_core.useNanosec=false ~c/'git/build/bin-wrappers/git diff' \
-n v2_core.useNanosec=true ~c/'git/build/bin-wrappers/git -c core.useNanosec=true diff' \
-n v2.55.0_USE_NSEC_disabled ~c/'perf-test/build/bin-wrappers/git diff' Benchmark 1: v2_core.useNanosec=false
Time (mean ± σ): 20.8 ms ± 2.6 ms [User: 27.9 ms, System: 103.8 ms] Range (min … max): 17.2 ms … 29.1 ms 132 runs
Benchmark 2: v2_core.useNanosec=true
Time (mean ± σ): 19.7 ms ± 1.5 ms [User: 29.2 ms, System: 100.0 ms] Range (min … max): 17.0 ms … 28.2 ms 170 runs
Benchmark 3: v2.55.0_USE_NSEC_disabled
Time (mean ± σ): 19.7 ms ± 1.6 ms [User: 27.8 ms, System: 99.1 ms] Range (min … max): 16.8 ms … 25.0 ms 154 runs
Summary
v2_core.useNanosec=true ran
1.00 ± 0.11 times faster than v2.55.0_USE_NSEC_disabled 1.05 ± 0.15 times faster than v2_core.useNanosec=false
Back down to being on-par with original code. So that's good. The next version will include some variant that reads a struct member instead of going through repo_config_get_bool().
But which? Reading the private_ member is obviously wrong; I suppose I'm supposed to use repo_config_values() there. Or, rework the series to put this member in repo_settings. I think I originally assumed that struct is for things that are settings that aren't configured by git-config, but… now I'm not sure. Looking at prepare_repo_settings() shows lots of repo_cfg_*() calls. So I think I see how to adapt to using repo_settings,
Patrick, Junio, and Tian had a brief discussion in <[email protected]> about the split creating confusion. I don't really want to wait for it to settle to land this change, but we might want to work together on identifying the best path forward for core.useNanosec :)
I don't suppose it really matters to me which struct I put the member in. As I said, v2 will definitely fix the hot path lookup here. Just a matter of input on which struct we want to use this time, I guess.
Signature · 1 line
D. Ben Knoble
read-cache.c +3 -2
@@ -353,8 +353,9 @@ static int ce_match_stat_basic(const struct cache_entry *ce, struct stat *st)static int is_racy_stat (const struct index_state *istate,const struct stat_data *sd) { -int use_nsec =0 ; -repo_config_get_bool (the_repository,"core.useNanosec" , &use_nsec); +/* supposed to use repo_config_values(), probably? +* or we should move this member to struct repo_settings */ +int use_nsec = istate->repo->config_values_private_.use_nanosec; return (istate->timestamp.sec &&/* nanosecond timestamped files can also be racy! */
Re: [PATCH 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
On Mon, Aug 10, 2026 at 08:27:51AM -0400, D. Ben Knoble wrote:
[snip]
Quoted 18 lines
Back down to being on-par with original code. So that's good. The next version will include some variant that reads a struct member instead of going through
repo_config_get_bool().But which? Reading the
private_member is obviously wrong; I suppose I'm supposed to userepo_config_values()there. Or, rework the series to put this member inrepo_settings. I think I originally assumed that struct is for things that are settings that aren't configured by git-config, but… now I'm not sure. Looking atprepare_repo_settings()shows lots of repo_cfg_*() calls. So I think I see how to adapt to usingrepo_settings,Patrick, Junio, and Tian had a brief discussion in <[email protected]> about the split creating confusion. I don't really want to wait for it to settle to land this change, but we might want to work together on identifying the best path forward for core.useNanosec :)
Quoted 3 lines
I don't suppose it really matters to me which struct I put the member in. As I said, v2 will definitely fix the hot path lookup here. Just a matter of input on which struct we want to use this time, I guess.
I think repo_config_values() is the modern variant that we're slowly migrating stuff into. But that struct only works with the_repository, so the question is whether we ever use "core.useNsec" for a different repository. My hunch would be yes, for example when recusing into submodules, but I'm not sure.
Patrick
Re: [PATCH 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
In reply to D. Ben Knoblecompat/posix.h int git_qsort_s(void *base, size_t nmemb, size_t size, · line 492
Quoted 12 lines · D. Ben Knoble
compat/posix.h +0
-1@@ -487,7 +487,6 @@ int git_qsort_s(void *base, size_t nmemb, size_t size, } while (0)#ifdef NO_NSEC -#undef USE_NSEC#define ST_CTIME_NSEC (st) 0#define ST_MTIME_NSEC (st) 0#else
Ah, I was about to ask whether all platforms even support nanoseconds. But I wasn't aware that we have both NO_NSEC and USE_NSEC, so we still know to not use nanoseconds if unsupported by the platform.
Patrick
Re: [PATCH 1/3] meson: expose knob for xmlto relative links in manuals
In reply to D. Ben KnobleQuoted 29 lines · D. Ben Knoble
Documentation/meson.build +6
-1@@ -379,13 +379,18 @@ foreach manpage, category : manpages output: fs.stem(manpage) + '.xml', ) + man_base_url = 'file://' + htmldir + '/' + if get_option('man_base_url') != '' + man_base_url = get_option('man_base_url') + endif + doc_targets += custom_target( command: [ xmlto, '-m', '@INPUT0@', '-m', '@INPUT1@', '--stringparam', - 'man.base.url.for.relative.links=' + get_option('prefix') / get_option('mandir'), + 'man.base.url.for.relative.links=' + man_base_url, 'man', manpage_xml_target, '-o',meson_options.txt +0
-0
meson_options.txt option('default_help_format', type: 'combo', choices: ['man', 'html', 'platform' · line 115
Quoted 5 lines
meson_options.txt +2
-0@@ -111,6 +111,8 @@ option('default_help_format', type: 'combo', choices: ['man', 'html', 'platform' description: 'Default format used when executing git-help(1).') option('docs_backend', type: 'combo', choices: ['asciidoc', 'asciidoctor', 'auto'], value: 'auto', description: 'Which backend to use to generate documentation.') +option('man_base_url', type: 'string', value: '', + description: 'The base URL to use for relative links in manuals')
Makes sense. I also verified that we indeed use the "file://" prefix by default in our Makefile.
Patrick
Re: [PATCH 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
In reply to Patrick SteinhardtQuoted 27 lines · Patrick Steinhardt
On Mon, Aug 10, 2026 at 08:27:51AM -0400, D. Ben Knoble wrote:
[snip]Quoted 18 lines
Back down to being on-par with original code. So that's good. The next version will include some variant that reads a struct member instead of going through
repo_config_get_bool().But which? Reading the
private_member is obviously wrong; I suppose I'm supposed to userepo_config_values()there. Or, rework the series to put this member inrepo_settings. I think I originally assumed that struct is for things that are settings that aren't configured by git-config, but… now I'm not sure. Looking atprepare_repo_settings()shows lots of repo_cfg_*() calls. So I think I see how to adapt to usingrepo_settings,Patrick, Junio, and Tian had a brief discussion in <[email protected]> about the split creating confusion. I don't really want to wait for it to settle to land this change, but we might want to work together on identifying the best path forward for core.useNanosec :)
Quoted 3 lines
I don't suppose it really matters to me which struct I put the member in. As I said, v2 will definitely fix the hot path lookup here. Just a matter of input on which struct we want to use this time, I guess.
Quoted 7 lines
I think
repo_config_values()is the modern variant that we're slowly migrating stuff into. But that struct only works withthe_repository, so the question is whether we ever use "core.useNsec" for a different repository. My hunch would be yes, for example when recusing into submodules, but I'm not sure.Patrick
Thanks. I’m working on control-flow analysis to see what kinds of repo values end up there. Of course I’ll also run the test suite and so on with the repo_config_values change. But the analysis will take some time.
Re: [PATCH 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
Quoted 34 lines · Ben Knoble
Quoted 34 lines · Patrick Steinhardt
On Mon, Aug 10, 2026 at 08:27:51AM -0400, D. Ben Knoble wrote:
[snip]Quoted 18 lines
Back down to being on-par with original code. So that's good. The next version will include some variant that reads a struct member instead of going through
repo_config_get_bool().But which? Reading the
private_member is obviously wrong; I suppose I'm supposed to userepo_config_values()there. Or, rework the series to put this member inrepo_settings. I think I originally assumed that struct is for things that are settings that aren't configured by git-config, but… now I'm not sure. Looking atprepare_repo_settings()shows lots of repo_cfg_*() calls. So I think I see how to adapt to usingrepo_settings,Patrick, Junio, and Tian had a brief discussion in <[email protected]> about the split creating confusion. I don't really want to wait for it to settle to land this change, but we might want to work together on identifying the best path forward for core.useNanosec :)
Quoted 3 lines
I don't suppose it really matters to me which struct I put the member in. As I said, v2 will definitely fix the hot path lookup here. Just a matter of input on which struct we want to use this time, I guess.
I think
repo_config_values()is the modern variant that we're slowly migrating stuff into. But that struct only works withthe_repository, so the question is whether we ever use "core.useNsec" for a different repository. My hunch would be yes, for example when recusing into submodules, but I'm not sure.
Quoted 3 lines
Quoted 1 line
Patrick
Thanks. I’m working on control-flow analysis to see what kinds of repo values end up there. Of course I’ll also run the test suite and so on with the
repo_config_valueschange. But the analysis will take some time.
Ok, CI run: https://github.com/benknoble/git/actions/runs/31701945211.
This demonstrates that nothing our test suite does across the many CI configurations ends up where with a non-the_repository-repository (ahem).
I have been working on control-flow analysis by hand in my Git time this week. It's of the form "Z calls Y calls X …" until we can see what the repository that's (eventually) fed to repo_config_values() here in is_racy_stat() is. My notes are one node per line, which indentation showing callee relationships. Some lines are pointers to other nodes to avoid duplicating work.
With that in mind, filtering out the pointer nodes, I've analyzed 214 nodes in the graph. If I'm lucky, I'm approaching the halfway mark, but I somewhat doubt it.
But since CI shows things work… I'd rather not continue the analysis if we're satisfied for now. (Esp. since that will give me more Git time back for reviewing ;) It being outside-of-work time, I only have so much of it.)
A few other related things:
- Some of the edges of the graph appear to be public libgit.a interfaces. That means we can't guarantee that only the_repository is used.
- On a related note, I don't know how large the current "must only use the_repository" (e.g., via repo_config_values()) surface area is right now. Based on the partial analysis I mentioned above, this feels like it's introducing (or at least contributing to) a rather large surface area. So, this change might make it more critical to resolve the limitation mentioned in the other thread. OTOH, I don't think this change is likely to represent the only pervasive the_repository-only limitation, and I'm afraid it will never land if it must be the_repository clean (unless repo_settings is the_repository clean and we decide that's an acceptable place for this member).
So, idk. If we're happy with the CI run + use of repo_config_values() overall, I can send a v2 shortly (in next 24h), I think.
Thoughts? Strong opinions?
Signature · 1 line
D. Ben Knoble
Re: [PATCH 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
Quoted 72 lines · D. Ben Knoble
Quoted 31 lines · Ben Knoble
Quoted 31 lines · Patrick Steinhardt
On Mon, Aug 10, 2026 at 08:27:51AM -0400, D. Ben Knoble wrote:
[snip]Quoted 18 lines
Back down to being on-par with original code. So that's good. The next version will include some variant that reads a struct member instead of going through
repo_config_get_bool().But which? Reading the
private_member is obviously wrong; I suppose I'm supposed to userepo_config_values()there. Or, rework the series to put this member inrepo_settings. I think I originally assumed that struct is for things that are settings that aren't configured by git-config, but… now I'm not sure. Looking atprepare_repo_settings()shows lots of repo_cfg_*() calls. So I think I see how to adapt to usingrepo_settings,Patrick, Junio, and Tian had a brief discussion in <[email protected]> about the split creating confusion. I don't really want to wait for it to settle to land this change, but we might want to work together on identifying the best path forward for core.useNanosec :)
Quoted 3 lines
I don't suppose it really matters to me which struct I put the member in. As I said, v2 will definitely fix the hot path lookup here. Just a matter of input on which struct we want to use this time, I guess.
I think
repo_config_values()is the modern variant that we're slowly migrating stuff into. But that struct only works withthe_repository, so the question is whether we ever use "core.useNsec" for a different repository. My hunch would be yes, for example when recusing into submodules, but I'm not sure.Quoted 3 lines
Quoted 1 line
Patrick
Thanks. I’m working on control-flow analysis to see what kinds of repo values end up there. Of course I’ll also run the test suite and so on with the
repo_config_valueschange. But the analysis will take some time.Ok, CI run: https://github.com/benknoble/git/actions/runs/31701945211.
This demonstrates that nothing our test suite does across the many CI configurations ends up where with a non-the_repository-repository (ahem).I have been working on control-flow analysis by hand in my Git time this week. It's of the form "Z calls Y calls X …" until we can see what the repository that's (eventually) fed to
repo_config_values()here inis_racy_stat()is. My notes are one node per line, which indentation showing callee relationships. Some lines are pointers to other nodes to avoid duplicating work.With that in mind, filtering out the pointer nodes, I've analyzed 214 nodes in the graph. If I'm lucky, I'm approaching the halfway mark, but I somewhat doubt it.
But since CI shows things work… I'd rather not continue the analysis if we're satisfied for now. (Esp. since that will give me more Git time back for reviewing ;) It being outside-of-work time, I only have so much of it.)
A few other related things:
- Some of the edges of the graph appear to be public libgit.a interfaces. That means we can't guarantee that onlythe_repositoryis used.
- On a related note, I don't know how large the current "must only usethe_repository" (e.g., viarepo_config_values()) surface area is right now. Based on the partial analysis I mentioned above, this feels like it's introducing (or at least contributing to) a rather large surface area. So, this change might make it more critical to resolve the limitation mentioned in the other thread. OTOH, I don't think this change is likely to represent the only pervasive the_repository-only limitation, and I'm afraid it will never land if it must bethe_repositoryclean (unlessrepo_settingsisthe_repositoryclean and we decide that's an acceptable place for this member).
Quoted 4 lines
So, idk. If we're happy with the CI run + use of
repo_config_values()overall, I can send a v2 shortly (in next 24h), I think.Thoughts? Strong opinions?
No strong opinions from my side, other than that we should stop converting everything to repo_config_values() until we have a plan for how to make it work with repositories other than the_repository.
I don't feel like holding this series in hostage though, so if your analysis and the test suite both say that this is probably fine then we may want to pursue it. Or we just use a global variable for it for the time being and then wait until the repo_config_values() dust has settled.
Patrick
Re: [PATCH 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
Quoted 51 lines · Patrick Steinhardt
Quoted 38 lines · D. Ben Knoble
Ok, CI run: https://github.com/benknoble/git/actions/runs/31701945211.
This demonstrates that nothing our test suite does across the many CI configurations ends up where with a non-the_repository-repository (ahem).I have been working on control-flow analysis by hand in my Git time this week. It's of the form "Z calls Y calls X …" until we can see what the repository that's (eventually) fed to
repo_config_values()here inis_racy_stat()is. My notes are one node per line, which indentation showing callee relationships. Some lines are pointers to other nodes to avoid duplicating work.With that in mind, filtering out the pointer nodes, I've analyzed 214 nodes in the graph. If I'm lucky, I'm approaching the halfway mark, but I somewhat doubt it.
But since CI shows things work… I'd rather not continue the analysis if we're satisfied for now. (Esp. since that will give me more Git time back for reviewing ;) It being outside-of-work time, I only have so much of it.)
A few other related things:
- Some of the edges of the graph appear to be public libgit.a interfaces. That means we can't guarantee that onlythe_repositoryis used.
- On a related note, I don't know how large the current "must only usethe_repository" (e.g., viarepo_config_values()) surface area is right now. Based on the partial analysis I mentioned above, this feels like it's introducing (or at least contributing to) a rather large surface area. So, this change might make it more critical to resolve the limitation mentioned in the other thread. OTOH, I don't think this change is likely to represent the only pervasive the_repository-only limitation, and I'm afraid it will never land if it must bethe_repositoryclean (unlessrepo_settingsisthe_repositoryclean and we decide that's an acceptable place for this member).Quoted 4 lines
So, idk. If we're happy with the CI run + use of
repo_config_values()overall, I can send a v2 shortly (in next 24h), I think.Thoughts? Strong opinions?
No strong opinions from my side, other than that we should stop converting everything to
repo_config_values()until we have a plan for how to make it work with repositories other thanthe_repository.
Quoted 7 lines
I don't feel like holding this series in hostage though, so if your analysis and the test suite both say that this is probably fine then we may want to pursue it. Or we just use a global variable for it for the time being and then wait until the
repo_config_values()dust has settled.Patrick
Makes sense. I should have also mentioned that, of the nodes I’ve analyzed so far, they all terminate in a path that uses the_repository (some intermediate nodes are also exposed, though, as written previously).
v2 cover Convert USE_NSEC to runtime config
In reply to D. Ben KnobleTopic name: dk/use-nsec-runtime (applied)
Topic summary: Expose USE_NSEC as a runtime configuration, since build-time is too early for distributing Git [1]. As a result, common index-related options, like git-diff, are less likely to hit "racy git" problems on supported filesystems.
[1]: https://git.github.io/rev_news/2026/07/31/edition-137/
Built on master (2c78326f81 (The 11th batch, 2026-08-05)).
Changes in v2:
- move Best-viewed-with trailer into message body as descriptive text.
- read core.useNanosec through struct repo instead of parsing config strings. The test suite passes locally this way, though that skipped 151 tests.
- CI run: https://github.com/benknoble/git/actions/runs/31701945211
Original cover letter:
Hi all, this series follows up on the previous racy Git/USE_NSEC conversations.
- The first patch is a mostly-unrelated documentation fix for Meson, but it came out of something I spotted while reviewing the outputs of the final (main) patch.
- The second patch is a preliminary no-op reorganization of repo_config_values_init.
- The third patch is the meat, converting USE_NSEC into core.useNanosec.
There is a small textual and semantic conflict with 'ty/repo-config-cleanups' in 'seen', since that branch removes the comments in 'struct repo_config_values' which this series adds to. (The semantic conflict is that, if we drop those comments, we should probably not add them to repo_config_values_init like I do in patch 2.)
Todo: I haven't touched any tests; I saw a bunch of hits for "git grep racy t" but wasn't sure how to fit this particular change in, especially since it won't be equally valid on all systems? Advice welcome.
Todo: I wonder if "useNanosec" paints us into too much of a corner; that is (slightly more abstractly), we are using *extended precision* in the index. Maybe the name and documentation should reflect that, so we aren't too committed to "nanoseconds"?
- Some platforms could offer extended precision that is not as precise as nanoseconds
- Some could offer precision _beyond_ nanoseconds
idk.
v1: <[email protected]>
[1/3] meson: expose knob for xmlto relative links in manuals [2/3] environment: align repo_config_values_init with struct declaration [3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
Documentation/config/core.adoc | 6 ++++++ Documentation/meson.build | 7 ++++++- Documentation/technical/racy-git.adoc | 11 ++++++----- Makefile | 12 +----------- builtin/update-index.c | 2 +- compat/posix.h | 1 - configure.ac | 6 ------ environment.c | 25 ++++++++++++++++++------- environment.h | 1 + meson_options.txt | 2 ++ read-cache.c | 16 ++++++++-------- statinfo.c | 14 +++++++------- 12 files changed, 56 insertions(+), 47 deletions(-)
Diff-intervalle contre v1 :
1: d612de6c2d = 1: d612de6c2d meson: expose knob for xmlto relative links in manuals 2: 5693baa992 = 2: 5693baa992 environment: align repo_config_values_init with struct declaration 3: dbbd96d508 ! 3: 2d1424732a core: convert build-time USE_NSEC into runtime core.useNanosec
@@ Commit message
In addition, update the Racy Git documentation and other mentions of USE_NSEC in the code.
- Best-viewed-with: --ignore-space-change
+ Due to the conversion from #ifdef to runtime check, using the flag
+ "--ignore-space-change" may be particularly helpful when viewing changes
+ from this patch.
## Notes (benknoble/commits) ##
- Repeating the benchmark from my original mail [1] shows an improvement
-
- # git worktree add -d ../perf-test HEAD
- # hyperfine -N --warmup=10 './build/bin-wrappers/git diff'
- Benchmark 1: ./build/bin-wrappers/git diff
- Time (mean ± σ): 3.8 ms ± 0.4 ms [User: 4.7 ms, System: 4.4 ms]
- Range (min … max): 3.2 ms … 5.6 ms 780 runs
- # (pushd ../perf-test && hyperfine -N --warmup=10 $OLDPWD/'./build/bin-wrappers/git diff')
- Benchmark 1: /home/benknoble/code/git/./build/bin-wrappers/git diff
- Time (mean ± σ): 217.5 ms ± 2.9 ms [User: 202.1 ms, System: 23.4 ms]
- Range (min … max): 213.9 ms … 223.3 ms 13 runs
- # (pushd ../perf-test && hyperfine -N --warmup=10 $OLDPWD/'./build/bin-wrappers/git -c core.useNanosec=true diff')
- Benchmark 1: /home/benknoble/code/git/./build/bin-wrappers/git -c core.useNanosec=true diff
- Time (mean ± σ): 3.8 ms ± 0.4 ms [User: 5.3 ms, System: 4.2 ms]
- Range (min … max): 3.2 ms … 6.9 ms 541 runs
-
- [1]: <CALnO6CADMJSixqYvL1Yo8qKX5rWhKQ+2OoSEuPUh-yoeK9TseQ@mail.gmail.com>
-
- Passing CI: https://github.com/benknoble/git/actions/runs/31104581195
+ Related benchmarks: <https://lore.kernel.org/git/CALnO6CBm4g27mWBvD9m6yL0e5YZu3M9_zcUeLZk7QwTgnxMLQA@mail.gmail.com/>
+ CI: <https://github.com/benknoble/git/actions/runs/31701945211>
## Documentation/config/core.adoc ##
@@ Documentation/config/core.adoc: core.trustctime::
@@ read-cache.c: static int ce_match_stat_basic(const struct cache_entry *ce, struc
static int is_racy_stat(const struct index_state *istate,
const struct stat_data *sd)
{
-+ int use_nsec = 0;
-+ repo_config_get_bool(the_repository, "core.useNanosec", &use_nsec);
++ int use_nsec = repo_config_values(istate->repo)->use_nanosec;
+
return (istate->timestamp.sec &&
-#ifdef USE_NSEC
base-commit: 2c78326f810173a4f3aefd8021f1e07575412481
Signature · 1 line
2.55.0.699.gb54405d56f.dirty
PATCH v2 1/3 meson: expose knob for xmlto relative links in manuals
Makefile-based builds have had this knob for most of the project's life, since a479a564dc (Documentation/Makefile: allow man.base.url.for.relative.link to be set from Make, 2009-12-03).
Meson, however, hard-codes the equivalent of $prefix/$mandir, which is not really where all the HTML docs are stored in most distro builds. Plus, this value is missing a trailing slash, so links come out broken, like this in git.1:
1. Git User’s Manual
/usr/share/manuser-manual.html
Of course we can do better:
1. Change the default to match Make: use file://$(htmldir)/ (with trailing slash!) to form a local URL pointing at the HTML docs. This is safe because all current uses of link:<relative> point at HTML docs:
git grep 'link:[[:alnum:]]' Documentation | grep -ve html -e http
produces only a single result (Documentation/howto/howto-index.sh) which can be ignored. Since nothing else [*] in the normal build sets MAN_BASE_URL, this seems like the right default.
2. Provide a configurable knob, just like the Makefile, so distributions that build with Meson (like Gentoo) can decide where to make the links if they need to. Those that set htmldir probably won't need to tweak this any further, though.
[*]: Well, Git's todo branch has a script dodoc.sh to build and archive
docs for kernel.org; these docs are pulled by Homebrew installations, for example. It sets MAN_BASE_URL to "git_htmldocs", so the equivalent note on macOS + Homebrew is
1. Git User’s Manual
git-htmldocs/user-manual.html
which is not functional either, but that's a problem for downstream. In any case, users can recover the right path with "git --html-path".
---
Notes (benknoble/commits):
This patch is mostly because I noticed the link I added in a later patch didn't come out right.
I did an internet search for "MAN_BASE_URL" and got no real hits, so I'm not sure if any distros today actually use it, but that's not a proper audit in that I didn't look at any distro _code_ besides Gentoo (which, as noted, uses Meson).
Documentation/meson.build | 7 ++++++- meson_options.txt | 2 ++ 2 files changed, 8 insertions(+), 1 deletion(-)
Documentation/meson.build +6 -1
@@ -379,13 +379,18 @@ foreach manpage, category : manpages output: fs.stem(manpage) + '.xml', ) + man_base_url = 'file://' + htmldir + '/' + if get_option('man_base_url') != '' + man_base_url = get_option('man_base_url') + endif + doc_targets += custom_target( command: [ xmlto, '-m', '@INPUT0@', '-m', '@INPUT1@', '--stringparam', - 'man.base.url.for.relative.links=' + get_option('prefix') / get_option('mandir'), + 'man.base.url.for.relative.links=' + man_base_url, 'man', manpage_xml_target, '-o',
meson_options.txt +2 -0
@@ -111,6 +111,8 @@ option('default_help_format', type: 'combo', choices: ['man', 'html', 'platform' description: 'Default format used when executing git-help(1).') option('docs_backend', type: 'combo', choices: ['asciidoc', 'asciidoctor', 'auto'], value: 'auto', description: 'Which backend to use to generate documentation.') +option('man_base_url', type: 'string', value: '', + description: 'The base URL to use for relative links in manuals') # Testing. option('benchmarks', type: 'feature', value: 'auto',
Signature · 1 line
2.55.0.699.gb54405d56f.dirty
PATCH v2 2/3 environment: align repo_config_values_init with struct declaration
In reply to D. Ben KnobleThe order of assignments in repo_config_values_init is chaotic and hard to follow, especially when comparing with the struct definition to ensure all members are initialized. As new members will be added in the future, make it easier to validate changes by aligning the two.
Refactor assignment order with no behavioral changes.
--- environment.c | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-)
environment.c +12 -7
@@ -745,6 +745,7 @@ int git_default_config(const char *var, const char *value,void repo_config_values_init (struct repo_config_values *cfg) { +/* section "core" config values */ cfg->attributes_file =NULL ; cfg->excludes_file =NULL ; cfg->editor_program =NULL ;
@@ -756,20 +757,24 @@ void repo_config_values_init(struct repo_config_values *cfg) cfg->autorebase =AUTOREBASE_NEVER ; cfg->object_creation_mode =OBJECT_CREATION_MODE ; cfg->apply_sparse_checkout =0 ; - cfg->protect_hfs =PROTECT_HFS_DEFAULT ; - cfg->protect_ntfs =PROTECT_NTFS_DEFAULT ; - cfg->ignore_case =0 ; - cfg->trust_executable_bit =1 ; - cfg->has_symlinks =platform_has_symlinks (); - cfg->branch_track =BRANCH_TRACK_REMOTE ; cfg->trust_ctime =1 ; cfg->check_stat =1 ; cfg->zlib_compression_level =Z_BEST_SPEED ; cfg->pack_compression_level =Z_DEFAULT_COMPRESSION ; cfg->precomposed_unicode =-1 ;/* see probe_utf8_pathname_composition() */ cfg->core_sparse_checkout_cone =0 ; - cfg->sparse_expect_files_outside_of_patterns =0 ; cfg->warn_on_object_refname_ambiguity =1 ; + cfg->protect_hfs =PROTECT_HFS_DEFAULT ; + cfg->protect_ntfs =PROTECT_NTFS_DEFAULT ; + cfg->ignore_case =0 ; + cfg->trust_executable_bit =1 ; + cfg->has_symlinks =platform_has_symlinks (); + +/* section "sparse" config values */ + cfg->sparse_expect_files_outside_of_patterns =0 ; + +/* section "branch" config values */ + cfg->branch_track =BRANCH_TRACK_REMOTE ; }void repo_config_values_clear (struct repo_config_values *cfg)
Signature · 1 line
2.55.0.699.gb54405d56f.dirty
PATCH v2 3/3 core: convert build-time USE_NSEC into runtime core.useNanosec
In reply to D. Ben KnobleRacy Git problems persist today, manifesting themselves in the performance of commands like "git diff" in new worktrees [1]. We have long had a build knob "USE_NSEC" to tell Git to use in-core nanosecond precision when available, which mitigates most if not all racy issues, but most builds we know about it don't use it. In part, that's because someone distributing Git can't safely enable it at compile-time if they don't know exactly what platforms their distribution will be used on.
[1]: https://lore.kernel.org/git/CALnO6CADMJSixqYvL1Yo8qKX5rWhKQ+2OoSEuPUh-yoeK9TseQ@mail.gmail.com
These days, most platforms are likely to be safe for the USE_NSEC code. Regardless, we want to give users the ability to benefit from it. This requires exposing the compile-time gated code as a runtime option.
In addition, update the Racy Git documentation and other mentions of USE_NSEC in the code.
Due to the conversion from #ifdef to runtime check, using the flag "--ignore-space-change" may be particularly helpful when viewing changes from this patch.
---
Notes (benknoble/commits):
Related benchmarks: <https://lore.kernel.org/git/CALnO6CBm4g27mWBvD9m6yL0e5YZu3M9_zcUeLZk7QwTgnxMLQA@mail.gmail.com/>
CI: <https://github.com/benknoble/git/actions/runs/31701945211>
Documentation/config/core.adoc | 6 ++++++ Documentation/technical/racy-git.adoc | 11 ++++++----- Makefile | 12 +----------- builtin/update-index.c | 2 +- compat/posix.h | 1 - configure.ac | 6 ------ environment.c | 6 ++++++ environment.h | 1 + read-cache.c | 16 ++++++++-------- statinfo.c | 14 +++++++------- 10 files changed, 36 insertions(+), 39 deletions(-)
Documentation/config/core.adoc +6 -0
@@ -118,6 +118,12 @@ core.trustctime:: crawlers and some backup systems). See linkgit:git-update-index[1]. True by default. +core.useNanosec:: + If true, use nanosecond precision for ctime and mtime + comparisions between the index and the working tree (if Git + was compiled to store it). + See link:technical/racy-git.html[Racy Git]. False by default. + core.splitIndex:: If true, the split-index feature of the index will be used. See linkgit:git-update-index[1]. False by default.
Documentation/technical/racy-git.adoc +6 -5
@@ -39,8 +39,8 @@ files) from `st_mode` member, `st_mtime` and `st_ctime` timestamps, `st_uid`, `st_gid`, `st_ino`, and `st_size` members. With a `USE_STDEV` compile-time option, `st_dev` is also compared, but this is not enabled by default because this member -is not stable on network filesystems. With `USE_NSEC` -compile-time option, `st_mtim.tv_nsec` and `st_ctim.tv_nsec` +is not stable on network filesystems. With 'core.useNanosec' +config setting, `st_mtim.tv_nsec` and `st_ctim.tv_nsec` members are also compared. On Linux, this is not enabled by default because in-core timestamps can have finer granularity than on-disk timestamps, resulting in meaningless changes when an
@@ -49,9 +49,10 @@ of git://git.kernel.org/pub/scm/linux/kernel/git/tglx/history.git ([PATCH] Sync in core time granularity with filesystems, 2005-01-04). This patch is included in kernel 2.6.11 and newer, but only fixes the issue for file systems with exactly 1 ns or 1 s -resolution. Other file systems are still broken in current Linux -kernels (e.g. CEPH, CIFS, NTFS, UDF), see -https://lore.kernel.org/lkml/[email protected]/ +resolution. As of kernel 4.3, other file systems (CEPH, CIFS, NTFS, UFS, FUSE) +were fixed; see https://public-inbox.org/git/5605D88A.20104%40gmail.com/. FAT +has been fixed since 2015. The usual suspects (ext2, ext4, XFS) are known to +work, too. Racy Git --------
Makefile +1 -11
@@ -197,18 +197,11 @@ include shared.mak # Define NO_NORETURN if using buggy versions of gcc 4.6+ and profile feedback, # as the compiler can crash (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=49299) # -# Define USE_NSEC below if you want git to care about sub-second file mtimes -# and ctimes. Note that you need recent glibc (at least 2.2.4) for this. On -# Linux, kernel 2.6.11 or newer is required for reliable sub-second file times -# on file systems with exactly 1 ns or 1 s resolution. If you intend to use Git -# on other file systems (e.g. CEPH, CIFS, NTFS, UDF), don't enable USE_NSEC. See -# Documentation/technical/racy-git.adoc for details. -# # Define USE_ST_TIMESPEC if your "struct stat" uses "st_ctimespec" instead of # "st_ctim" # # Define NO_NSEC if your "struct stat" does not have "st_ctim.tv_nsec" -# available. This automatically turns USE_NSEC off. +# available. # # Define USE_STDEV below if you want git to care about the underlying device # change being considered an inode change from the update-index perspective.
@@ -1935,9 +1928,6 @@ endif ifdef NO_ST_BLOCKS_IN_STRUCT_STAT BASIC_CFLAGS += -DNO_ST_BLOCKS_IN_STRUCT_STAT endif -ifdef USE_NSEC - BASIC_CFLAGS += -DUSE_NSEC -endif ifdef USE_ST_TIMESPEC BASIC_CFLAGS += -DUSE_ST_TIMESPEC endif
builtin/update-index.c +1 -1
@@ -130,7 +130,7 @@ static void xrmdir(const char *path)static void avoid_racy (void ) {/* -* not use if we could usleep(10) if USE_NSEC is defined. The +* not use if we could usleep(10) if core.useNanosec is defined. The * field nsec could be there, but the OS could choose to * ignore it? */
compat/posix.h +0 -1
@@ -487,7 +487,6 @@ int git_qsort_s(void *base, size_t nmemb, size_t size, } while (0)#ifdef NO_NSEC -#undef USE_NSEC#define ST_CTIME_NSEC (st) 0#define ST_MTIME_NSEC (st) 0#else
configure.ac +0 -6
@@ -351,12 +351,6 @@ GIT_PARSE_WITH(iconv)) ## --enable-FEATURE[=ARG] and --disable-FEATURE # -# Define USE_NSEC below if you want git to care about sub-second file mtimes -# and ctimes. Note that you need recent glibc (at least 2.2.4) for this, and -# it will BREAK YOUR LOCAL DIFFS! show-diff and anything using it will likely -# randomly break unless your underlying filesystem supports those sub-second -# times (my ext3 doesn't). -# # Define USE_STDEV below if you want git to care about the underlying device # change being considered an inode change from the update-index perspective.
environment.c +6 -0
@@ -571,6 +571,11 @@ int git_default_core_config(const char *var, const char *value,return 0 ; } +if (!strcmp (var,"core.usenanosec" )) { + cfg->use_nanosec =git_config_bool (var, value); +return 0 ; + } +/* Add other config variables here and to Documentation/config.adoc. */ return platform_core_config (var, value, ctx, cb); }
@@ -769,6 +774,7 @@ void repo_config_values_init(struct repo_config_values *cfg) cfg->ignore_case =0 ; cfg->trust_executable_bit =1 ; cfg->has_symlinks =platform_has_symlinks (); + cfg->use_nanosec =0 ;/* section "sparse" config values */ cfg->sparse_expect_files_outside_of_patterns =0 ;
environment.h +1 -0
@@ -139,6 +139,7 @@ struct repo_config_values {int ignore_case;int trust_executable_bit;int has_symlinks; +int use_nanosec;/* section "sparse" config values */ int sparse_expect_files_outside_of_patterns;
read-cache.c +8 -8
@@ -353,15 +353,15 @@ static int ce_match_stat_basic(const struct cache_entry *ce, struct stat *st)static int is_racy_stat (const struct index_state *istate,const struct stat_data *sd) { +int use_nsec =repo_config_values (istate->repo)->use_nanosec; +return (istate->timestamp.sec && -#ifdef USE_NSEC - /* nanosecond timestamped files can also be racy! */ - (istate->timestamp.sec < sd->sd_mtime.sec || - (istate->timestamp.sec == sd->sd_mtime.sec && - istate->timestamp.nsec <= sd->sd_mtime.nsec)) -#else - istate->timestamp.sec <= sd->sd_mtime.sec -#endif +/* nanosecond timestamped files can also be racy! */ + use_nsec + ? (istate->timestamp.sec < sd->sd_mtime.sec || + (istate->timestamp.sec == sd->sd_mtime.sec && + istate->timestamp.nsec <= sd->sd_mtime.nsec)) + : istate->timestamp.sec <= sd->sd_mtime.sec ); }
statinfo.c +7 -7
@@ -72,13 +72,13 @@ int match_stat_data(const struct stat_data *sd, struct stat *st) sd->sd_ctime.sec != (unsigned int)st->st_ctime) changed |=CTIME_CHANGED ; -#ifdef USE_NSEC -if (cfg->check_stat && sd->sd_mtime.nsec !=ST_MTIME_NSEC (*st)) - changed |=MTIME_CHANGED ; -if (cfg->trust_ctime && cfg->check_stat && - sd->sd_ctime.nsec !=ST_CTIME_NSEC (*st)) - changed |=CTIME_CHANGED ; -#endif +if (cfg->use_nanosec) { +if (cfg->check_stat && sd->sd_mtime.nsec !=ST_MTIME_NSEC (*st)) + changed |=MTIME_CHANGED ; +if (cfg->trust_ctime && cfg->check_stat && + sd->sd_ctime.nsec !=ST_CTIME_NSEC (*st)) + changed |=CTIME_CHANGED ; + }if (cfg->check_stat) { if (sd->sd_uid != (unsigned int) st->st_uid ||
Signature · 1 line
2.55.0.699.gb54405d56f.dirty
Re: [PATCH v2 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
Quoted 7 lines · D. Ben Knoble
statinfo.c +0
-7-#ifdef USE_NSEC -if (cfg->check_stat && sd->sd_mtime.nsec !=ST_MTIME_NSEC (*st)) - changed |=MTIME_CHANGED ; -if (cfg->trust_ctime && cfg->check_stat && - sd->sd_ctime.nsec !=ST_CTIME_NSEC (*st)) - changed |=CTIME_CHANGED ; -#endif
statinfo.c int match_stat_data(const struct stat_data *sd, struct stat *st) · line 84
Quoted 10 lines
statinfo.c +7
-0+if (cfg->use_nanosec) { +if (cfg->check_stat && sd->sd_mtime.nsec !=ST_MTIME_NSEC (*st)) + changed |=MTIME_CHANGED ; +if (cfg->trust_ctime && cfg->check_stat && + sd->sd_ctime.nsec !=ST_CTIME_NSEC (*st)) + changed |=CTIME_CHANGED ; + }if (cfg->check_stat) { if (sd->sd_uid != (unsigned int) st->st_uid ||
This is iffy.
If you have core.usenanosec=true in a networked $HOME/.gitconfig mounted on both USE_NSEC-capable and incapable platforms, what would ST_CTIME_NSEC() yield on the latter? I wonder if cfg's '.use_nanosec' should be force-disabled in NO_NSEC builds, or something similar?
Re: [PATCH v2 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
Quoted 18 lines · Junio C Hamano
Quoted 15 lines · D. Ben Knoble
statinfo.c +7
-7-#ifdef USE_NSEC -if (cfg->check_stat && sd->sd_mtime.nsec !=ST_MTIME_NSEC (*st)) - changed |=MTIME_CHANGED ; -if (cfg->trust_ctime && cfg->check_stat && - sd->sd_ctime.nsec !=ST_CTIME_NSEC (*st)) - changed |=CTIME_CHANGED ; -#endif +if (cfg->use_nanosec) { +if (cfg->check_stat && sd->sd_mtime.nsec !=ST_MTIME_NSEC (*st)) + changed |=MTIME_CHANGED ; +if (cfg->trust_ctime && cfg->check_stat && + sd->sd_ctime.nsec !=ST_CTIME_NSEC (*st)) + changed |=CTIME_CHANGED ; + }
Quoted 8 lines
Quoted 2 lines
if (cfg->check_stat) { if (sd->sd_uid != (unsigned int) st->st_uid ||This is iffy.
If you have core.usenanosec=true in a networked $HOME/.gitconfig mounted on both USE_NSEC-capable and incapable platforms, what would
ST_CTIME_NSEC()yield on the latter?
Perhaps "if it hurts, don't do that"? This config is definitely about exposing the underlying system's capabilities to Git, so if you cannot confidently do so globally, you probably shouldn't. That might limit the usefulness of the optimization for folks that share filesystems between multiple machines in this way, I suppose. Or maybe it will incentivize folks to be nsec-compatible in more places ;) Either way, users that can benefit from it will have the option.
Quoted 3 lines
I wonder if cfg's
'.use_nanosec' should be force-disabled inNO_NSECbuilds, or something similar?
This does, however, make some sense to me:
- Git today with NO_NSEC doesn't bother with the USE_NSEC paths, since we #undef USE_NSEC in that case.
- Git "tomorrow" should probably say "I was built with NO_NSEC, so I will (continue) ignoring platform-specific nanosecond optimizations."
I'll queue this change locally until I send out the next version, unless someone objects.
Signature · 1 line
D. Ben Knoble
v3 cover Convert USE_NSEC to runtime config
In reply to D. Ben KnobleTopic name: dk/use-nsec-runtime (applied)
Topic summary: Expose USE_NSEC as a runtime configuration, since build-time is too early for distributing Git [1]. As a result, common index-related options, like git-diff, are less likely to hit "racy git" problems on supported filesystems.
[1]: https://git.github.io/rev_news/2026/07/31/edition-137/
Built on master (2c78326f81 (The 11th batch, 2026-08-05)).
Changes in v3:
- #ifdef out use_nanosec when NO_NSEC is requested
As I have heard no comments about the "Todo" lines below, which perhaps could more clearly be marked "RFC"/"RFH", I've added this line to call them out ;) and renamed them "Comments welcome"
Changes in v2:
- move Best-viewed-with trailer into message body as descriptive text.
- read core.useNanosec through struct repo instead of parsing config strings. The test suite passes locally this way, though that skipped 151 tests.
- CI run: https://github.com/benknoble/git/actions/runs/31701945211
Original cover letter:
Hi all, this series follows up on the previous racy Git/USE_NSEC conversations.
- The first patch is a mostly-unrelated documentation fix for Meson, but it came out of something I spotted while reviewing the outputs of the final (main) patch.
- The second patch is a preliminary no-op reorganization of repo_config_values_init.
- The third patch is the meat, converting USE_NSEC into core.useNanosec.
There is a small textual and semantic conflict with 'ty/repo-config-cleanups' in 'seen', since that branch removes the comments in 'struct repo_config_values' which this series adds to. (The semantic conflict is that, if we drop those comments, we should probably not add them to repo_config_values_init like I do in patch 2.)
Comments welcome: I haven't touched any tests; I saw a bunch of hits for "git grep racy t" but wasn't sure how to fit this particular change in, especially since it won't be equally valid on all systems? Advice welcome.
Comments welcome: I wonder if "useNanosec" paints us into too much of a corner; that is (slightly more abstractly), we are using *extended precision* in the index. Maybe the name and documentation should reflect that, so we aren't too committed to "nanoseconds"?
- Some platforms could offer extended precision that is not as precise as nanoseconds
- Some could offer precision _beyond_ nanoseconds
idk.
v1: <[email protected]>
v2: <[email protected]>
[1/3] meson: expose knob for xmlto relative links in manuals [2/3] environment: align repo_config_values_init with struct declaration [3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
Documentation/config/core.adoc | 6 ++++++ Documentation/meson.build | 7 ++++++- Documentation/technical/racy-git.adoc | 11 +++++----- Makefile | 12 +---------- builtin/update-index.c | 2 +- compat/posix.h | 1 - configure.ac | 6 ------ environment.c | 29 ++++++++++++++++++++------- environment.h | 1 + meson_options.txt | 2 ++ read-cache.c | 16 ++++++++++----- statinfo.c | 14 +++++++------ 12 files changed, 64 insertions(+), 43 deletions(-)
Diff-intervalle contre v2 :
1: d612de6c2d = 1: d612de6c2d meson: expose knob for xmlto relative links in manuals
2: 5693baa992 = 2: 5693baa992 environment: align repo_config_values_init with struct declaration
3: 2d1424732a ! 3: 48fceb4b57 core: convert build-time USE_NSEC into runtime core.useNanosec
@@ Commit message
## Notes (benknoble/commits) ##
Related benchmarks: <https://lore.kernel.org/git/CALnO6CBm4g27mWBvD9m6yL0e5YZu3M9_zcUeLZk7QwTgnxMLQA@mail.gmail.com/>
- CI: <https://github.com/benknoble/git/actions/runs/31701945211>
+ CI: <https://github.com/benknoble/git/actions/runs/32137191115>
+
+ v3:
+ We could perhaps be cute in read-cache.c:is_racy_stat() by writing
+ the preprocessor directive like
+
+ return (istate->timestamp.sec &&
+ #ifndef NO_NSEC
+ /* nanosecond timestamped files can also be racy! */
+ use_nsec
+ ? (istate->timestamp.sec < sd->sd_mtime.sec ||
+ (istate->timestamp.sec == sd->sd_mtime.sec &&
+ istate->timestamp.nsec <= sd->sd_mtime.nsec))
+ :
+ #endif
+ istate->timestamp.sec <= sd->sd_mtime.sec
+
+ but that seemed maybe too clever?
## Documentation/config/core.adoc ##
@@ Documentation/config/core.adoc: core.trustctime::
@@ environment.c: int git_default_core_config(const char *var, const char *value,
return 0;
}
++#ifndef NO_NSEC
+ if (!strcmp(var, "core.usenanosec")) {
+ cfg->use_nanosec = git_config_bool(var, value);
+ return 0;
+ }
++#endif
+
/* Add other config variables here and to Documentation/config.adoc. */
return platform_core_config(var, value, ctx, cb);
@@ environment.c: void repo_config_values_init(struct repo_config_values *cfg)
cfg->ignore_case = 0;
cfg->trust_executable_bit = 1;
cfg->has_symlinks = platform_has_symlinks();
++#ifndef NO_NSEC
+ cfg->use_nanosec = 0;
++#endif
/* section "sparse" config values */
cfg->sparse_expect_files_outside_of_patterns = 0;
@@ read-cache.c: static int ce_match_stat_basic(const struct cache_entry *ce, struc
static int is_racy_stat(const struct index_state *istate,
const struct stat_data *sd)
{
++#ifndef NO_NSEC
+ int use_nsec = repo_config_values(istate->repo)->use_nanosec;
++#endif
+
return (istate->timestamp.sec &&
-#ifdef USE_NSEC
@@ read-cache.c: static int ce_match_stat_basic(const struct cache_entry *ce, struc
- (istate->timestamp.sec < sd->sd_mtime.sec ||
- (istate->timestamp.sec == sd->sd_mtime.sec &&
- istate->timestamp.nsec <= sd->sd_mtime.nsec))
--#else
-- istate->timestamp.sec <= sd->sd_mtime.sec
--#endif
++#ifndef NO_NSEC
+ /* nanosecond timestamped files can also be racy! */
+ use_nsec
+ ? (istate->timestamp.sec < sd->sd_mtime.sec ||
+ (istate->timestamp.sec == sd->sd_mtime.sec &&
+ istate->timestamp.nsec <= sd->sd_mtime.nsec))
+ : istate->timestamp.sec <= sd->sd_mtime.sec
- );
- }
-
+ #else
+ istate->timestamp.sec <= sd->sd_mtime.sec
+ #endif
## statinfo.c ##
@@ statinfo.c: int match_stat_data(const struct stat_data *sd, struct stat *st)
@@ statinfo.c: int match_stat_data(const struct stat_data *sd, struct stat *st)
- if (cfg->trust_ctime && cfg->check_stat &&
- sd->sd_ctime.nsec != ST_CTIME_NSEC(*st))
- changed |= CTIME_CHANGED;
--#endif
++#ifndef NO_NSEC
+ if (cfg->use_nanosec) {
+ if (cfg->check_stat && sd->sd_mtime.nsec != ST_MTIME_NSEC(*st))
+ changed |= MTIME_CHANGED;
@@ statinfo.c: int match_stat_data(const struct stat_data *sd, struct stat *st)
+ sd->sd_ctime.nsec != ST_CTIME_NSEC(*st))
+ changed |= CTIME_CHANGED;
+ }
+ #endif
if (cfg->check_stat) {
- if (sd->sd_uid != (unsigned int) st->st_uid ||base-commit: 2c78326f810173a4f3aefd8021f1e07575412481
Signature · 1 line
2.55.0.860.g4b6b3295ed.dirty
PATCH v3 1/3 meson: expose knob for xmlto relative links in manuals
Makefile-based builds have had this knob for most of the project's life, since a479a564dc (Documentation/Makefile: allow man.base.url.for.relative.link to be set from Make, 2009-12-03).
Meson, however, hard-codes the equivalent of $prefix/$mandir, which is not really where all the HTML docs are stored in most distro builds. Plus, this value is missing a trailing slash, so links come out broken, like this in git.1:
1. Git User’s Manual
/usr/share/manuser-manual.html
Of course we can do better:
1. Change the default to match Make: use file://$(htmldir)/ (with trailing slash!) to form a local URL pointing at the HTML docs. This is safe because all current uses of link:<relative> point at HTML docs:
git grep 'link:[[:alnum:]]' Documentation | grep -ve html -e http
produces only a single result (Documentation/howto/howto-index.sh) which can be ignored. Since nothing else [*] in the normal build sets MAN_BASE_URL, this seems like the right default.
2. Provide a configurable knob, just like the Makefile, so distributions that build with Meson (like Gentoo) can decide where to make the links if they need to. Those that set htmldir probably won't need to tweak this any further, though.
[*]: Well, Git's todo branch has a script dodoc.sh to build and archive
docs for kernel.org; these docs are pulled by Homebrew installations, for example. It sets MAN_BASE_URL to "git_htmldocs", so the equivalent note on macOS + Homebrew is
1. Git User’s Manual
git-htmldocs/user-manual.html
which is not functional either, but that's a problem for downstream. In any case, users can recover the right path with "git --html-path".
---
Notes (benknoble/commits):
This patch is mostly because I noticed the link I added in a later patch didn't come out right.
I did an internet search for "MAN_BASE_URL" and got no real hits, so I'm not sure if any distros today actually use it, but that's not a proper audit in that I didn't look at any distro _code_ besides Gentoo (which, as noted, uses Meson).
Documentation/meson.build | 7 ++++++- meson_options.txt | 2 ++ 2 files changed, 8 insertions(+), 1 deletion(-)
Documentation/meson.build +6 -1
@@ -379,13 +379,18 @@ foreach manpage, category : manpages output: fs.stem(manpage) + '.xml', ) + man_base_url = 'file://' + htmldir + '/' + if get_option('man_base_url') != '' + man_base_url = get_option('man_base_url') + endif + doc_targets += custom_target( command: [ xmlto, '-m', '@INPUT0@', '-m', '@INPUT1@', '--stringparam', - 'man.base.url.for.relative.links=' + get_option('prefix') / get_option('mandir'), + 'man.base.url.for.relative.links=' + man_base_url, 'man', manpage_xml_target, '-o',
meson_options.txt +2 -0
@@ -111,6 +111,8 @@ option('default_help_format', type: 'combo', choices: ['man', 'html', 'platform' description: 'Default format used when executing git-help(1).') option('docs_backend', type: 'combo', choices: ['asciidoc', 'asciidoctor', 'auto'], value: 'auto', description: 'Which backend to use to generate documentation.') +option('man_base_url', type: 'string', value: '', + description: 'The base URL to use for relative links in manuals') # Testing. option('benchmarks', type: 'feature', value: 'auto',
Signature · 1 line
2.55.0.860.g4b6b3295ed.dirty
PATCH v3 2/3 environment: align repo_config_values_init with struct declaration
In reply to D. Ben KnobleThe order of assignments in repo_config_values_init is chaotic and hard to follow, especially when comparing with the struct definition to ensure all members are initialized. As new members will be added in the future, make it easier to validate changes by aligning the two.
Refactor assignment order with no behavioral changes.
--- environment.c | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-)
environment.c +12 -7
@@ -745,6 +745,7 @@ int git_default_config(const char *var, const char *value,void repo_config_values_init (struct repo_config_values *cfg) { +/* section "core" config values */ cfg->attributes_file =NULL ; cfg->excludes_file =NULL ; cfg->editor_program =NULL ;
@@ -756,20 +757,24 @@ void repo_config_values_init(struct repo_config_values *cfg) cfg->autorebase =AUTOREBASE_NEVER ; cfg->object_creation_mode =OBJECT_CREATION_MODE ; cfg->apply_sparse_checkout =0 ; - cfg->protect_hfs =PROTECT_HFS_DEFAULT ; - cfg->protect_ntfs =PROTECT_NTFS_DEFAULT ; - cfg->ignore_case =0 ; - cfg->trust_executable_bit =1 ; - cfg->has_symlinks =platform_has_symlinks (); - cfg->branch_track =BRANCH_TRACK_REMOTE ; cfg->trust_ctime =1 ; cfg->check_stat =1 ; cfg->zlib_compression_level =Z_BEST_SPEED ; cfg->pack_compression_level =Z_DEFAULT_COMPRESSION ; cfg->precomposed_unicode =-1 ;/* see probe_utf8_pathname_composition() */ cfg->core_sparse_checkout_cone =0 ; - cfg->sparse_expect_files_outside_of_patterns =0 ; cfg->warn_on_object_refname_ambiguity =1 ; + cfg->protect_hfs =PROTECT_HFS_DEFAULT ; + cfg->protect_ntfs =PROTECT_NTFS_DEFAULT ; + cfg->ignore_case =0 ; + cfg->trust_executable_bit =1 ; + cfg->has_symlinks =platform_has_symlinks (); + +/* section "sparse" config values */ + cfg->sparse_expect_files_outside_of_patterns =0 ; + +/* section "branch" config values */ + cfg->branch_track =BRANCH_TRACK_REMOTE ; }void repo_config_values_clear (struct repo_config_values *cfg)
Signature · 1 line
2.55.0.860.g4b6b3295ed.dirty
PATCH v3 3/3 core: convert build-time USE_NSEC into runtime core.useNanosec
In reply to D. Ben KnobleRacy Git problems persist today, manifesting themselves in the performance of commands like "git diff" in new worktrees [1]. We have long had a build knob "USE_NSEC" to tell Git to use in-core nanosecond precision when available, which mitigates most if not all racy issues, but most builds we know about it don't use it. In part, that's because someone distributing Git can't safely enable it at compile-time if they don't know exactly what platforms their distribution will be used on.
[1]: https://lore.kernel.org/git/CALnO6CADMJSixqYvL1Yo8qKX5rWhKQ+2OoSEuPUh-yoeK9TseQ@mail.gmail.com
These days, most platforms are likely to be safe for the USE_NSEC code. Regardless, we want to give users the ability to benefit from it. This requires exposing the compile-time gated code as a runtime option.
In addition, update the Racy Git documentation and other mentions of USE_NSEC in the code.
Due to the conversion from #ifdef to runtime check, using the flag "--ignore-space-change" may be particularly helpful when viewing changes from this patch.
---
Notes (benknoble/commits):
Related benchmarks: <https://lore.kernel.org/git/CALnO6CBm4g27mWBvD9m6yL0e5YZu3M9_zcUeLZk7QwTgnxMLQA@mail.gmail.com/> CI: <https://github.com/benknoble/git/actions/runs/32137191115> v3: We could perhaps be cute in read-cache.c:is_racy_stat() by writing the preprocessor directive like return (istate->timestamp.sec && #ifndef NO_NSEC /* nanosecond timestamped files can also be racy! */ use_nsec ? (istate->timestamp.sec < sd->sd_mtime.sec || (istate->timestamp.sec == sd->sd_mtime.sec && istate->timestamp.nsec <= sd->sd_mtime.nsec)) : #endif istate->timestamp.sec <= sd->sd_mtime.sec but that seemed maybe too clever?
Documentation/config/core.adoc | 6 ++++++ Documentation/technical/racy-git.adoc | 11 ++++++----- Makefile | 12 +----------- builtin/update-index.c | 2 +- compat/posix.h | 1 - configure.ac | 6 ------ environment.c | 10 ++++++++++ environment.h | 1 + read-cache.c | 16 +++++++++++----- statinfo.c | 14 ++++++++------ 10 files changed, 44 insertions(+), 35 deletions(-)
Documentation/config/core.adoc +6 -0
@@ -118,6 +118,12 @@ core.trustctime:: crawlers and some backup systems). See linkgit:git-update-index[1]. True by default. +core.useNanosec:: + If true, use nanosecond precision for ctime and mtime + comparisions between the index and the working tree (if Git + was compiled to store it). + See link:technical/racy-git.html[Racy Git]. False by default. + core.splitIndex:: If true, the split-index feature of the index will be used. See linkgit:git-update-index[1]. False by default.
Documentation/technical/racy-git.adoc +6 -5
@@ -39,8 +39,8 @@ files) from `st_mode` member, `st_mtime` and `st_ctime` timestamps, `st_uid`, `st_gid`, `st_ino`, and `st_size` members. With a `USE_STDEV` compile-time option, `st_dev` is also compared, but this is not enabled by default because this member -is not stable on network filesystems. With `USE_NSEC` -compile-time option, `st_mtim.tv_nsec` and `st_ctim.tv_nsec` +is not stable on network filesystems. With 'core.useNanosec' +config setting, `st_mtim.tv_nsec` and `st_ctim.tv_nsec` members are also compared. On Linux, this is not enabled by default because in-core timestamps can have finer granularity than on-disk timestamps, resulting in meaningless changes when an
@@ -49,9 +49,10 @@ of git://git.kernel.org/pub/scm/linux/kernel/git/tglx/history.git ([PATCH] Sync in core time granularity with filesystems, 2005-01-04). This patch is included in kernel 2.6.11 and newer, but only fixes the issue for file systems with exactly 1 ns or 1 s -resolution. Other file systems are still broken in current Linux -kernels (e.g. CEPH, CIFS, NTFS, UDF), see -https://lore.kernel.org/lkml/[email protected]/ +resolution. As of kernel 4.3, other file systems (CEPH, CIFS, NTFS, UFS, FUSE) +were fixed; see https://public-inbox.org/git/5605D88A.20104%40gmail.com/. FAT +has been fixed since 2015. The usual suspects (ext2, ext4, XFS) are known to +work, too. Racy Git --------
Makefile +1 -11
@@ -197,18 +197,11 @@ include shared.mak # Define NO_NORETURN if using buggy versions of gcc 4.6+ and profile feedback, # as the compiler can crash (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=49299) # -# Define USE_NSEC below if you want git to care about sub-second file mtimes -# and ctimes. Note that you need recent glibc (at least 2.2.4) for this. On -# Linux, kernel 2.6.11 or newer is required for reliable sub-second file times -# on file systems with exactly 1 ns or 1 s resolution. If you intend to use Git -# on other file systems (e.g. CEPH, CIFS, NTFS, UDF), don't enable USE_NSEC. See -# Documentation/technical/racy-git.adoc for details. -# # Define USE_ST_TIMESPEC if your "struct stat" uses "st_ctimespec" instead of # "st_ctim" # # Define NO_NSEC if your "struct stat" does not have "st_ctim.tv_nsec" -# available. This automatically turns USE_NSEC off. +# available. # # Define USE_STDEV below if you want git to care about the underlying device # change being considered an inode change from the update-index perspective.
@@ -1935,9 +1928,6 @@ endif ifdef NO_ST_BLOCKS_IN_STRUCT_STAT BASIC_CFLAGS += -DNO_ST_BLOCKS_IN_STRUCT_STAT endif -ifdef USE_NSEC - BASIC_CFLAGS += -DUSE_NSEC -endif ifdef USE_ST_TIMESPEC BASIC_CFLAGS += -DUSE_ST_TIMESPEC endif
builtin/update-index.c +1 -1
@@ -130,7 +130,7 @@ static void xrmdir(const char *path)static void avoid_racy (void ) {/* -* not use if we could usleep(10) if USE_NSEC is defined. The +* not use if we could usleep(10) if core.useNanosec is defined. The * field nsec could be there, but the OS could choose to * ignore it? */
compat/posix.h +0 -1
@@ -487,7 +487,6 @@ int git_qsort_s(void *base, size_t nmemb, size_t size, } while (0)#ifdef NO_NSEC -#undef USE_NSEC#define ST_CTIME_NSEC (st) 0#define ST_MTIME_NSEC (st) 0#else
configure.ac +0 -6
@@ -351,12 +351,6 @@ GIT_PARSE_WITH(iconv)) ## --enable-FEATURE[=ARG] and --disable-FEATURE # -# Define USE_NSEC below if you want git to care about sub-second file mtimes -# and ctimes. Note that you need recent glibc (at least 2.2.4) for this, and -# it will BREAK YOUR LOCAL DIFFS! show-diff and anything using it will likely -# randomly break unless your underlying filesystem supports those sub-second -# times (my ext3 doesn't). -# # Define USE_STDEV below if you want git to care about the underlying device # change being considered an inode change from the update-index perspective.
environment.c +10 -0
@@ -571,6 +571,13 @@ int git_default_core_config(const char *var, const char *value,return 0 ; } +#ifndef NO_NSEC +if (!strcmp (var,"core.usenanosec" )) { + cfg->use_nanosec =git_config_bool (var, value); +return 0 ; + } +#endif +/* Add other config variables here and to Documentation/config.adoc. */ return platform_core_config (var, value, ctx, cb); }
@@ -769,6 +776,9 @@ void repo_config_values_init(struct repo_config_values *cfg) cfg->ignore_case =0 ; cfg->trust_executable_bit =1 ; cfg->has_symlinks =platform_has_symlinks (); +#ifndef NO_NSEC + cfg->use_nanosec =0 ; +#endif /* section "sparse" config values */ cfg->sparse_expect_files_outside_of_patterns =0 ;
environment.h +1 -0
@@ -139,6 +139,7 @@ struct repo_config_values {int ignore_case;int trust_executable_bit;int has_symlinks; +int use_nanosec;/* section "sparse" config values */ int sparse_expect_files_outside_of_patterns;
read-cache.c +11 -5
@@ -353,12 +353,18 @@ static int ce_match_stat_basic(const struct cache_entry *ce, struct stat *st)static int is_racy_stat (const struct index_state *istate,const struct stat_data *sd) { +#ifndef NO_NSEC +int use_nsec =repo_config_values (istate->repo)->use_nanosec; +#endif + return (istate->timestamp.sec && -#ifdef USE_NSEC - /* nanosecond timestamped files can also be racy! */ - (istate->timestamp.sec < sd->sd_mtime.sec || - (istate->timestamp.sec == sd->sd_mtime.sec && - istate->timestamp.nsec <= sd->sd_mtime.nsec)) +#ifndef NO_NSEC +/* nanosecond timestamped files can also be racy! */ + use_nsec + ? (istate->timestamp.sec < sd->sd_mtime.sec || + (istate->timestamp.sec == sd->sd_mtime.sec && + istate->timestamp.nsec <= sd->sd_mtime.nsec)) + : istate->timestamp.sec <= sd->sd_mtime.sec#else istate->timestamp.sec <= sd->sd_mtime.sec#endif
statinfo.c +8 -6
@@ -72,12 +72,14 @@ int match_stat_data(const struct stat_data *sd, struct stat *st) sd->sd_ctime.sec != (unsigned int)st->st_ctime) changed |=CTIME_CHANGED ; -#ifdef USE_NSEC -if (cfg->check_stat && sd->sd_mtime.nsec !=ST_MTIME_NSEC (*st)) - changed |=MTIME_CHANGED ; -if (cfg->trust_ctime && cfg->check_stat && - sd->sd_ctime.nsec !=ST_CTIME_NSEC (*st)) - changed |=CTIME_CHANGED ; +#ifndef NO_NSEC +if (cfg->use_nanosec) { +if (cfg->check_stat && sd->sd_mtime.nsec !=ST_MTIME_NSEC (*st)) + changed |=MTIME_CHANGED ; +if (cfg->trust_ctime && cfg->check_stat && + sd->sd_ctime.nsec !=ST_CTIME_NSEC (*st)) + changed |=CTIME_CHANGED ; + }#endif if (cfg->check_stat) {
Signature · 1 line
2.55.0.860.g4b6b3295ed.dirty
Re: [PATCH v3 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
Quoted 5 lines · D. Ben Knoble
environment.c +0
-0
environment.c int git_default_core_config(const char *var, const char *value, · line 579
Quoted 9 lines
environment.c +6
-0@@ -571,6 +571,13 @@ int git_default_core_config(const char *var, const char *value,return 0 ; } +#ifndef NO_NSEC +if (!strcmp (var,"core.usenanosec" )) { + cfg->use_nanosec =git_config_bool (var, value); +return 0 ; + } +#endif
What this hunk tells us: At build time, you could choose to ignore core.usenanosec configuration variable, preventing cfg->use_nanosec from getting flipped to true by the configured value.
environment.c void repo_config_values_init(struct repo_config_values *cfg) · line 781
Quoted 7 lines
environment.c +3
-0@@ -769,6 +776,9 @@ void repo_config_values_init(struct repo_config_values *cfg) cfg->ignore_case =0 ; cfg->trust_executable_bit =1 ; cfg->has_symlinks =platform_has_symlinks (); +#ifndef NO_NSEC + cfg->use_nanosec =0 ; +#endif
I think we want to unconditionally initialize it to 0, unless the definition of the .use_nanosec member itself in the structure is conditional on NO_NSEC. And ...
environment.h struct repo_config_values { · line 142
Quoted 12 lines
environment.c +0
-0/* section "sparse" config values */ cfg->sparse_expect_files_outside_of_patterns =0 ;environment.h +1
-0@@ -139,6 +139,7 @@ struct repo_config_values {int ignore_case;int trust_executable_bit;int has_symlinks; +int use_nanosec;
... that is not the case.
Which means that git_default_core_config() does keep the initial value of the member without getting affected by the configuration, but it does not necessarily be keeping "false". It may be keeping the uninitialized state instead ;-).
Quoted 18 lines
read-cache.c +4
-5@@ -353,12 +353,18 @@ static int ce_match_stat_basic(const struct cache_entry *ce, struct stat *st)static int is_racy_stat (const struct index_state *istate,const struct stat_data *sd) { +#ifndef NO_NSEC +int use_nsec =repo_config_values (istate->repo)->use_nanosec; +#endif + return (istate->timestamp.sec && -#ifdef USE_NSEC -/* nanosecond timestamped files can also be racy! */ - (istate->timestamp.sec < sd->sd_mtime.sec || - (istate->timestamp.sec == sd->sd_mtime.sec && - istate->timestamp.nsec <= sd->sd_mtime.nsec))
read-cache.c static int ce_match_stat_basic(const struct cache_entry *ce, struct stat *st) · line 370
Quoted 10 lines
read-cache.c +7
-0+#ifndef NO_NSEC +/* nanosecond timestamped files can also be racy! */ + use_nsec + ? (istate->timestamp.sec < sd->sd_mtime.sec || + (istate->timestamp.sec == sd->sd_mtime.sec && + istate->timestamp.nsec <= sd->sd_mtime.nsec)) + : istate->timestamp.sec <= sd->sd_mtime.sec#else istate->timestamp.sec <= sd->sd_mtime.sec#endif
Ugly. How about getting rid of the latter #ifndef/#else/#endif and instead keeping the "if use_nsec, pay attention to nsec, otherwise only the seconds part" ternary? As to the early part, as you can arrange cfg's '.use_nanosec' to always hold a sensible value, the function can become
return (istate->timestamp.sec &&
(repo_config_values(istate->repo)->use_nanosec
? (istate->timestamp.sec < sd->sd_mtime.sec ||
(istate->timestamp.sec == sd->sd_mtime.sec &&
istate->timestamp.nsec <= sd->sd_mtime.nsec))
: istate->timestamp.sec <= sd->sd_mtime.sec));I think.
The code you presented here for is_racy_stat() sprinkled with #ifndef/#else/#endif would be sensible if repo_config_values struct defined the '.use_nanosec' member conditionally. But that is not what is happening here.
Re: [PATCH v3 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
In reply to D. Ben KnobleQuoted 5 lines · D. Ben Knoble
Racy Git problems persist today, manifesting themselves in the performance of commands like "git diff" in new worktrees [1]. We have long had a build knob "
USE_NSEC" to tell Git to use in-core nanosecond precision when available, which mitigates most if not all racy issues, but most builds we know about it don't use it. In part, that's because
s/about it/about/
Documentation/config/core.adoc core.trustctime:: · line 125
Quoted 13 lines
Documentation/config/core.adoc +5
-0@@ -118,6 +118,12 @@ core.trustctime:: crawlers and some backup systems). See linkgit:git-update-index[1]. True by default. +core.useNanosec:: + If true, use nanosecond precision for ctime and mtime + comparisions between the index and the working tree (if Git + was compiled to store it). + See link:technical/racy-git.html[Racy Git]. False by default.
Should we mentino here that this may not be safe on all platforms and/or filesystems, in addition to linking to racy-hit?
And do we really want to link to the HTML page here? The user may be reading a manpage, so doing so feels a bit weird to me.
Quoted 5 lines
environment.c +0
-0
environment.c int git_default_core_config(const char *var, const char *value, · line 579
Quoted 9 lines
environment.c +6
-0@@ -571,6 +571,13 @@ int git_default_core_config(const char *var, const char *value,return 0 ; } +#ifndef NO_NSEC +if (!strcmp (var,"core.usenanosec" )) { + cfg->use_nanosec =git_config_bool (var, value); +return 0 ; + } +#endif
Do we want to omit a warning in case the config is enabled and we have NO_SEC set? Or would that be too obnoxious?
environment.c void repo_config_values_init(struct repo_config_values *cfg) · line 781
Quoted 7 lines
environment.c +3
-0@@ -769,6 +776,9 @@ void repo_config_values_init(struct repo_config_values *cfg) cfg->ignore_case =0 ; cfg->trust_executable_bit =1 ; cfg->has_symlinks =platform_has_symlinks (); +#ifndef NO_NSEC + cfg->use_nanosec =0 ; +#endif
Can't we set this unconditionally? The respective field exists unconditionally, too.
Quoted 18 lines
read-cache.c +4
-5@@ -353,12 +353,18 @@ static int ce_match_stat_basic(const struct cache_entry *ce, struct stat *st)static int is_racy_stat (const struct index_state *istate,const struct stat_data *sd) { +#ifndef NO_NSEC +int use_nsec =repo_config_values (istate->repo)->use_nanosec; +#endif + return (istate->timestamp.sec && -#ifdef USE_NSEC -/* nanosecond timestamped files can also be racy! */ - (istate->timestamp.sec < sd->sd_mtime.sec || - (istate->timestamp.sec == sd->sd_mtime.sec && - istate->timestamp.nsec <= sd->sd_mtime.nsec))
read-cache.c static int ce_match_stat_basic(const struct cache_entry *ce, struct stat *st) · line 370
Quoted 10 lines
read-cache.c +7
-0+#ifndef NO_NSEC +/* nanosecond timestamped files can also be racy! */ + use_nsec + ? (istate->timestamp.sec < sd->sd_mtime.sec || + (istate->timestamp.sec == sd->sd_mtime.sec && + istate->timestamp.nsec <= sd->sd_mtime.nsec)) + : istate->timestamp.sec <= sd->sd_mtime.sec#else istate->timestamp.sec <= sd->sd_mtime.sec#endif
I think this would be a bit more readable if we had a single NO_NSEC block.
Quoted 13 lines
statinfo.c +0
-5@@ -72,12 +72,14 @@ int match_stat_data(const struct stat_data *sd, struct stat *st) sd->sd_ctime.sec != (unsigned int)st->st_ctime) changed |=CTIME_CHANGED ; -#ifdef USE_NSEC -if (cfg->check_stat && sd->sd_mtime.nsec !=ST_MTIME_NSEC (*st)) - changed |=MTIME_CHANGED ; -if (cfg->trust_ctime && cfg->check_stat && - sd->sd_ctime.nsec !=ST_CTIME_NSEC (*st))
statinfo.c int match_stat_data(const struct stat_data *sd, struct stat *st) · line 83
Quoted 10 lines
statinfo.c +8
-1- changed |=CTIME_CHANGED ; +#ifndef NO_NSEC +if (cfg->use_nanosec) { +if (cfg->check_stat && sd->sd_mtime.nsec !=ST_MTIME_NSEC (*st)) + changed |=MTIME_CHANGED ; +if (cfg->trust_ctime && cfg->check_stat && + sd->sd_ctime.nsec !=ST_CTIME_NSEC (*st)) + changed |=CTIME_CHANGED ; + }#endif
There's one more site in "builtin/update-index.c" where we mention USE_NSEC that wasn't updated as part of this patch.
Thanks!
Patrick
Re: [PATCH v3 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
In reply to Junio C Hamano[Patrick, the below probably helps answer some of your questions as well.]
Quoted 35 lines · Junio C Hamano
Quoted 5 lines · D. Ben Knoble
environment.c +0
-0environment.c int git_default_core_config(const char *var, const char *value, · line 579
Quoted 9 lines
environment.c +6
-0@@ -571,6 +571,13 @@ int git_default_core_config(const char *var, const char *value,return 0 ; } +#ifndef NO_NSEC +if (!strcmp (var,"core.usenanosec" )) { + cfg->use_nanosec =git_config_bool (var, value); +return 0 ; + } +#endif What this hunk tells us: At build time, you could choose to ignore core.usenanosec configuration variable, preventing
cfg->use_nanosecfrom getting flipped to true by the configured value.environment.c void repo_config_values_init(struct repo_config_values *cfg) · line 781
Quoted 7 lines
environment.c +3
-0@@ -769,6 +776,9 @@ void repo_config_values_init(struct repo_config_values *cfg) cfg->ignore_case =0 ; cfg->trust_executable_bit =1 ; cfg->has_symlinks =platform_has_symlinks (); +#ifndef NO_NSEC + cfg->use_nanosec =0 ; +#endif I think we want to unconditionally initialize it to 0, unless the definition of the .use_nanosec member itself in the structure is conditional on
NO_NSEC. And ...Quoted 1 line
Quoted 13 lines
environment.h struct repo_config_values { · line 142
Quoted 11 lines
/* section "sparse" config values */ cfg->sparse_expect_files_outside_of_patterns = 0;environment.h +1
-0@@ -139,6 +139,7 @@ struct repo_config_values {int ignore_case;int trust_executable_bit;int has_symlinks; +int use_nanosec;... that is not the case.
Doh! I actually intended to send this version with a compiled-out member when NO_NSEC, since that was the only path I had come up with. No point in running around with code that's been asked to be ignored, eh? However…
Quoted 4 lines
Which means that
git_default_core_config()does keep the initial value of the member without getting affected by the configuration, but it does not necessarily be keeping "false". It may be keeping the uninitialized state instead ;-).
[ugly #ifdef trimmed]
Quoted 13 lines
Ugly. How about getting rid of the latter #ifndef/#else/#endif and instead keeping the "if
use_nsec, pay attention to nsec, otherwise only the seconds part" ternary? As to the early part, as you can arrange cfg's '.use_nanosec' to always hold a sensible value, the function can becomereturn (istate->timestamp.sec && (repo_config_values(istate->repo)->use_nanosec ? (istate->timestamp.sec < sd->sd_mtime.sec || (istate->timestamp.sec == sd->sd_mtime.sec && istate->timestamp.nsec <= sd->sd_mtime.nsec)) : istate->timestamp.sec <= sd->sd_mtime.sec));
Quoted 6 lines
I think.
The code you presented here for
is_racy_stat()sprinkled with #ifndef/#else/#endif would be sensible ifrepo_config_valuesstruct defined the '.use_nanosec' member conditionally. But that is not what is happening here.
…I now see a world where we could avoid quite a bit of headache:
- use #if[n]def NO_NSEC to ignore the config variable, but otherwise
- unconditionally compile the cfg->use_nanosec checks
That is, future readers/writers won't have to remember that they can only use the use_nanosec member under compiler conditionals; it will always be initialized to a safe value (either always false or from config). If we're lucky, the compiler will optimize the checks away in NO_NSEC builds ;)
I think this is what you are suggesting Junio, so let me see what I can come up with.
Signature · 1 line
D. Ben Knoble
Re: [PATCH v3 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
In reply to Patrick SteinhardtQuoted 9 lines · Patrick Steinhardt
Quoted 5 lines · D. Ben Knoble
Racy Git problems persist today, manifesting themselves in the performance of commands like "git diff" in new worktrees [1]. We have long had a build knob "
USE_NSEC" to tell Git to use in-core nanosecond precision when available, which mitigates most if not all racy issues, but most builds we know about it don't use it. In part, that's becauses/about it/about/
Thanks; fixed locally.
Quoted 8 lines
Quoted 8 lines
Documentation/config/core.adoc +0
-0@@ -118,6 +118,12 @@ core.trustctime:: crawlers and some backup systems). See linkgit:git-update-index[1]. True by default.
Quoted 8 lines
Documentation/config/core.adoc core.trustctime:: · line 125
Quoted 5 lines
Documentation/config/core.adoc +5
-0+core.useNanosec:: + If true, use nanosecond precision for ctime and mtime + comparisions between the index and the working tree (if Git + was compiled to store it). + See link:technical/racy-git.html[Racy Git]. False by default.Should we mentino here that this may not be safe on all platforms and/or filesystems, in addition to linking to racy-hit?
Yeah, a brief mention here is probably warranted.
Quoted 2 lines
And do we really want to link to the HTML page here? The user may be reading a manpage, so doing so feels a bit weird to me.
See a variation on the grep done in patch 1; we link lots of HTML documentation in our manuals (including when rendered to manpage format).
AFAICT, the idea is that we produce manual pages for commands and a few other "special" documents; we produce HTML of everything. So there isn't a good non-HTML link target for, e.g., the Racy Git document. In particular, even "git help" doesn't know about Racy Git. I have a script [1] that opens files out of "git --html-path", so that provides one way to access the Racy Git document (aside: neither of my systems---Homebrew macOS or Portage Gentoo---install anything into "git --info-path", so that would not make a good link target even if I knew how to write it). Patch 1/3 makes it easier to get the correct link in the manual for folks who can click links in their terminal emulators (or copy-paste).
[1]: https://github.com/benknoble/Dotfiles/tree/master/links/bin/git-doc
(with completion!
https://github.com/benknoble/Dotfiles/tree/master/links/zshfns/_git_doc)
TBH, I am not sure what other folks do for these HTML links in manuals. As I mention in patch 1, the Homebrew links are broken. If you know about "git --html-path" you can find the documents, or use the Git SCM website's rendered versions.
Anyway, this is the current "normal" style for linking, I think.
Quoted 8 lines
Quoted 8 lines
environment.c +0
-0@@ -571,6 +571,13 @@ int git_default_core_config(const char *var, const char *value,return 0 ; }
Quoted 9 lines
environment.c int git_default_core_config(const char *var, const char *value, · line 579
Quoted 6 lines
environment.c +6
-0+#ifndef NO_NSEC +if (!strcmp (var,"core.usenanosec" )) { + cfg->use_nanosec =git_config_bool (var, value); +return 0 ; + } +#endif Do we want to omit a warning in case the config is enabled and we have
NO_SECset? Or would that be too obnoxious?
I would say that can always be done later ;) Perhaps it should be better documented, though, so let me try that, too.
Quoted 11 lines
environment.c void repo_config_values_init(struct repo_config_values *cfg) · line 781
Quoted 8 lines
environment.c +3
-0@@ -769,6 +776,9 @@ void repo_config_values_init(struct repo_config_values *cfg) cfg->ignore_case =0 ; cfg->trust_executable_bit =1 ; cfg->has_symlinks =platform_has_symlinks (); +#ifndef NO_NSEC + cfg->use_nanosec =0 ; +#endif Can't we set this unconditionally? The respective field exists unconditionally, too.
Yep, see reply to Junio.
Quoted 23 lines
Quoted 23 lines
read-cache.c +9
-5@@ -353,12 +353,18 @@ static int ce_match_stat_basic(const struct cache_entry *ce, struct stat *st)static int is_racy_stat (const struct index_state *istate,const struct stat_data *sd) { +#ifndef NO_NSEC +int use_nsec =repo_config_values (istate->repo)->use_nanosec; +#endif + return (istate->timestamp.sec && -#ifdef USE_NSEC -/* nanosecond timestamped files can also be racy! */ - (istate->timestamp.sec < sd->sd_mtime.sec || - (istate->timestamp.sec == sd->sd_mtime.sec && - istate->timestamp.nsec <= sd->sd_mtime.nsec)) +#ifndef NO_NSEC +/* nanosecond timestamped files can also be racy! */ + use_nsec + ? (istate->timestamp.sec < sd->sd_mtime.sec || + (istate->timestamp.sec == sd->sd_mtime.sec &&
Quoted 8 lines
read-cache.c static int ce_match_stat_basic(const struct cache_entry *ce, struct stat *st) · line 370
Quoted 5 lines
read-cache.c +2
-0+ istate->timestamp.nsec <= sd->sd_mtime.nsec)) + : istate->timestamp.sec <= sd->sd_mtime.sec#else istate->timestamp.sec <= sd->sd_mtime.sec#endif I think this would be a bit more readable if we had a single
NO_NSECblock.
I'm not sure what "single block" means here, but I think the plan (see reply to Junio) is to make this more readable by not needing pre-processor directives at all.
[snip]
Quoted 2 lines
There's one more site in "
builtin/update-index.c" where we mentionUSE_NSECthat wasn't updated as part of this patch.
Oh, did I miss one? The only spot I saw in builtin/update-index.c that mentions USE_NSEC is a comment that I'm sure patch 3 updated. Maybe you were thinking of that, or maybe you know of something I left out? (That is, locally on this branch, "git grep USE_NSEC" returns one hit in Documentation/RelNotes/2.5.0.adoc.)
Thanks!
Signature · 1 line
D. Ben Knoble
Re: [PATCH v3 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
In reply to Patrick SteinhardtQuoted 8 lines · Patrick Steinhardt
Quoted 8 lines
environment.c +0
-0@@ -571,6 +571,13 @@ int git_default_core_config(const char *var, const char *value,return 0 ; }
Quoted 9 lines
environment.c int git_default_core_config(const char *var, const char *value, · line 579
Quoted 6 lines
environment.c +6
-0+#ifndef NO_NSEC +if (!strcmp (var,"core.usenanosec" )) { + cfg->use_nanosec =git_config_bool (var, value); +return 0 ; + } +#endif Do we want to omit a warning in case the config is enabled and we have
NO_SECset? Or would that be too obnoxious?
Those who use a $HOME/.gitconfig shared across two machines with different builds would be annoyed with one of them constantly complaining, I am afraid.
Re: [PATCH v3 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
Quoted 21 lines · Junio C Hamano
Quoted 8 lines · Patrick Steinhardt
Quoted 8 lines
environment.c +0
-0@@ -571,6 +571,13 @@ int git_default_core_config(const char *var, const char *value,return 0 ; }Quoted 9 lines
environment.c int git_default_core_config(const char *var, const char *value, · line 579
Quoted 6 lines
environment.c +6
-0+#ifndef NO_NSEC +if (!strcmp (var,"core.usenanosec" )) { + cfg->use_nanosec =git_config_bool (var, value); +return 0 ; + } +#endif Do we want to omit a warning in case the config is enabled and we have
NO_SECset? Or would that be too obnoxious?
Quoted 3 lines
Those who use a $HOME/.gitconfig shared across two machines with different builds would be annoyed with one of them constantly complaining, I am afraid.
Ah, that reminds me; my shared ~/.gitconfig includes a "site-local" config path, which could then be used to set this option (or not) only where supported.
Signature · 1 line
D. Ben Knoble
Re: [PATCH v3 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
In reply to D. Ben KnobleQuoted 34 lines · D. Ben Knoble
Quoted 24 lines · Patrick Steinhardt
Quoted 23 lines · D. Ben Knoble
read-cache.c +9
-5@@ -353,12 +353,18 @@ static int ce_match_stat_basic(const struct cache_entry *ce, struct stat *st)static int is_racy_stat (const struct index_state *istate,const struct stat_data *sd) { +#ifndef NO_NSEC +int use_nsec =repo_config_values (istate->repo)->use_nanosec; +#endif + return (istate->timestamp.sec && -#ifdef USE_NSEC -/* nanosecond timestamped files can also be racy! */ - (istate->timestamp.sec < sd->sd_mtime.sec || - (istate->timestamp.sec == sd->sd_mtime.sec && - istate->timestamp.nsec <= sd->sd_mtime.nsec)) +#ifndef NO_NSEC +/* nanosecond timestamped files can also be racy! */ + use_nsec + ? (istate->timestamp.sec < sd->sd_mtime.sec || + (istate->timestamp.sec == sd->sd_mtime.sec &&Quoted 8 lines
read-cache.c static int ce_match_stat_basic(const struct cache_entry *ce, struct stat *st) · line 370
Quoted 5 lines
read-cache.c +2
-0+ istate->timestamp.nsec <= sd->sd_mtime.nsec)) + : istate->timestamp.sec <= sd->sd_mtime.sec#else istate->timestamp.sec <= sd->sd_mtime.sec#endif I think this would be a bit more readable if we had a single
NO_NSECblock.
Quoted 3 lines
I'm not sure what "single block" means here, but I think the plan (see reply to Junio) is to make this more readable by not needing pre-processor directives at all.
That'd be quite welcome indeed. The less ifdeffery the bettery. :)
Quoted 8 lines
Quoted 2 lines
There's one more site in "
builtin/update-index.c" where we mentionUSE_NSECthat wasn't updated as part of this patch.Oh, did I miss one? The only spot I saw in
builtin/update-index.cthat mentionsUSE_NSECis a comment that I'm sure patch 3 updated. Maybe you were thinking of that, or maybe you know of something I left out? (That is, locally on this branch, "git grepUSE_NSEC" returns one hit inDocumentation/RelNotes/2.5.0.adoc.)
Oh, I guess I just missed it because I already trimmed context of this mail. Never mind then.
Patrick
Re: [PATCH v3 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
In reply to Junio C HamanoQuoted 20 lines · Junio C Hamano
Quoted 8 lines · Patrick Steinhardt
Quoted 8 lines
environment.c +0
-0@@ -571,6 +571,13 @@ int git_default_core_config(const char *var, const char *value,return 0 ; }Quoted 9 lines
environment.c int git_default_core_config(const char *var, const char *value, · line 579
Quoted 6 lines
environment.c +6
-0+#ifndef NO_NSEC +if (!strcmp (var,"core.usenanosec" )) { + cfg->use_nanosec =git_config_bool (var, value); +return 0 ; + } +#endif Do we want to omit a warning in case the config is enabled and we have
NO_SECset? Or would that be too obnoxious?
Quoted 3 lines
Those who use a $HOME/.gitconfig shared across two machines with different builds would be annoyed with one of them constantly complaining, I am afraid.
Yeah, that's what I was hinting at with "too obnovious". So I agree, let's not add one.
Patrick
Re: [PATCH v3 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
In reply to Patrick SteinhardtQuoted 2 lines · Patrick Steinhardt
That'd be quite welcome indeed. The less ifdeffery the bettery. :)
Agreed!
Quoted 11 lines · D. Ben Knoble
Quoted 9 lines · Patrick Steinhardt
Quoted 2 lines
There's one more site in "
builtin/update-index.c" where we mentionUSE_NSECthat wasn't updated as part of this patch.Oh, did I miss one? The only spot I saw in
builtin/update-index.cthat mentionsUSE_NSECis a comment that I'm sure patch 3 updated. Maybe you were thinking of that, or maybe you know of something I left out? (That is, locally on this branch, "git grepUSE_NSEC" returns one hit inDocumentation/RelNotes/2.5.0.adoc.)
Quoted 4 lines
Oh, I guess I just missed it because I already trimmed context of this mail. Never mind then.
Patrick
No worries, I appreciate the careful read!
Signature · 1 line
D. Ben Knoble
v4 cover Convert USE_NSEC to runtime config
In reply to D. Ben KnobleTopic name: dk/use-nsec-runtime (applied)
Topic summary: Expose USE_NSEC as a runtime configuration, since build-time is too early for distributing Git [1]. As a result, common index-related options, like git-diff, are less likely to hit "racy git" problems on supported filesystems.
[1]: https://git.github.io/rev_news/2026/07/31/edition-137/
Built on master (2c78326f81 (The 11th batch, 2026-08-05)).
Changes in v4:
- fix message typo
- change #ifdef strategy: only ignore the config variable. Otherwise, use the use_nanosec member unconditionally. Also clarify that config might be ignore depending on build options in the docs.
- mention potential platform unsafety directly in config doc in addition to the link to Racy Git
Changes in v3:
- #ifdef out use_nanosec when NO_NSEC is requested
As I have heard no comments about the "Todo" lines below, which perhaps could more clearly be marked "RFC"/"RFH", I've added this line to call them out ;) and renamed them "Comments welcome"
Changes in v2:
- move Best-viewed-with trailer into message body as descriptive text.
- read core.useNanosec through struct repo instead of parsing config strings. The test suite passes locally this way, though that skipped 151 tests.
- CI run: https://github.com/benknoble/git/actions/runs/31701945211
Original cover letter:
Hi all, this series follows up on the previous racy Git/USE_NSEC conversations.
- The first patch is a mostly-unrelated documentation fix for Meson, but it came out of something I spotted while reviewing the outputs of the final (main) patch.
- The second patch is a preliminary no-op reorganization of repo_config_values_init.
- The third patch is the meat, converting USE_NSEC into core.useNanosec.
There is a small textual and semantic conflict with 'ty/repo-config-cleanups' in 'seen', since that branch removes the comments in 'struct repo_config_values' which this series adds to. (The semantic conflict is that, if we drop those comments, we should probably not add them to repo_config_values_init like I do in patch 2.)
Comments welcome: I haven't touched any tests; I saw a bunch of hits for "git grep racy t" but wasn't sure how to fit this particular change in, especially since it won't be equally valid on all systems? Advice welcome.
Comments welcome: I wonder if "useNanosec" paints us into too much of a corner; that is (slightly more abstractly), we are using *extended precision* in the index. Maybe the name and documentation should reflect that, so we aren't too committed to "nanoseconds"?
- Some platforms could offer extended precision that is not as precise as nanoseconds
- Some could offer precision _beyond_ nanoseconds
idk.
v1: <[email protected]>
v2: <[email protected]>
v3: <[email protected]>
[1/3] meson: expose knob for xmlto relative links in manuals [2/3] environment: align repo_config_values_init with struct declaration [3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
Documentation/config/core.adoc | 7 +++++++ Documentation/meson.build | 7 ++++++- Documentation/technical/racy-git.adoc | 11 ++++++----- Makefile | 12 +----------- builtin/update-index.c | 2 +- compat/posix.h | 1 - configure.ac | 6 ------ environment.c | 27 ++++++++++++++++++++------- environment.h | 1 + meson_options.txt | 2 ++ read-cache.c | 15 ++++++--------- statinfo.c | 14 +++++++------- 12 files changed, 57 insertions(+), 48 deletions(-)
Diff-intervalle contre v3 :
1: d612de6c2d = 1: d612de6c2d meson: expose knob for xmlto relative links in manuals 2: 5693baa992 = 2: 5693baa992 environment: align repo_config_values_init with struct declaration 3: 48fceb4b57 ! 3: 0aa0e9fc17 core: convert build-time USE_NSEC into runtime core.useNanosec
@@ Commit message
performance of commands like "git diff" in new worktrees [1]. We have long had a build knob "USE_NSEC" to tell Git to use in-core nanosecond precision when available, which mitigates most if not all racy issues,
- but most builds we know about it don't use it. In part, that's because
+ but most builds we know about don't use it. In part, that's because
someone distributing Git can't safely enable it at compile-time if they don't know exactly what platforms their distribution will be used on.
@@ Commit message
## Notes (benknoble/commits) ##
Related benchmarks: <https://lore.kernel.org/git/CALnO6CBm4g27mWBvD9m6yL0e5YZu3M9_zcUeLZk7QwTgnxMLQA@mail.gmail.com/>
- CI: <https://github.com/benknoble/git/actions/runs/32137191115>
-
- v3:
- We could perhaps be cute in read-cache.c:is_racy_stat() by writing
- the preprocessor directive like
-
- return (istate->timestamp.sec &&
- #ifndef NO_NSEC
- /* nanosecond timestamped files can also be racy! */
- use_nsec
- ? (istate->timestamp.sec < sd->sd_mtime.sec ||
- (istate->timestamp.sec == sd->sd_mtime.sec &&
- istate->timestamp.nsec <= sd->sd_mtime.nsec))
- :
- #endif
- istate->timestamp.sec <= sd->sd_mtime.sec
-
- but that seemed maybe too clever?
+ CI: <https://github.com/benknoble/git/actions/runs/32365602564>
## Documentation/config/core.adoc ##
@@ Documentation/config/core.adoc: core.trustctime::
@@ Documentation/config/core.adoc: core.trustctime::
+core.useNanosec::
+ If true, use nanosecond precision for ctime and mtime
+ comparisions between the index and the working tree (if Git
-+ was compiled to store it).
-+ See link:technical/racy-git.html[Racy Git]. False by default.
++ was compiled to respect this option).
++ This is unsafe on some platforms;
++ see link:technical/racy-git.html[Racy Git]. False by default.
+
core.splitIndex::
If true, the split-index feature of the index will be used.
@@ environment.c: void repo_config_values_init(struct repo_config_values *cfg)
cfg->ignore_case = 0;
cfg->trust_executable_bit = 1;
cfg->has_symlinks = platform_has_symlinks();
-+#ifndef NO_NSEC
+ cfg->use_nanosec = 0;
-+#endif
/* section "sparse" config values */ cfg->sparse_expect_files_outside_of_patterns = 0;
@@ environment.h: struct repo_config_values {
int sparse_expect_files_outside_of_patterns;
## read-cache.c ##
-@@ read-cache.c: static int ce_match_stat_basic(const struct cache_entry *ce, struct stat *st)
- static int is_racy_stat(const struct index_state *istate,
+@@ read-cache.c: static int is_racy_stat(const struct index_state *istate,
const struct stat_data *sd)
{
-+#ifndef NO_NSEC
-+ int use_nsec = repo_config_values(istate->repo)->use_nanosec;
-+#endif
-+
return (istate->timestamp.sec &&
-#ifdef USE_NSEC
- /* nanosecond timestamped files can also be racy! */
- (istate->timestamp.sec < sd->sd_mtime.sec ||
- (istate->timestamp.sec == sd->sd_mtime.sec &&
- istate->timestamp.nsec <= sd->sd_mtime.nsec))
-+#ifndef NO_NSEC
+-#else
+- istate->timestamp.sec <= sd->sd_mtime.sec
+-#endif
+- );
+ /* nanosecond timestamped files can also be racy! */
-+ use_nsec
-+ ? (istate->timestamp.sec < sd->sd_mtime.sec ||
-+ (istate->timestamp.sec == sd->sd_mtime.sec &&
-+ istate->timestamp.nsec <= sd->sd_mtime.nsec))
-+ : istate->timestamp.sec <= sd->sd_mtime.sec
- #else
- istate->timestamp.sec <= sd->sd_mtime.sec
- #endif
++ (repo_config_values(istate->repo)->use_nanosec
++ ? (istate->timestamp.sec < sd->sd_mtime.sec ||
++ (istate->timestamp.sec == sd->sd_mtime.sec &&
++ istate->timestamp.nsec <= sd->sd_mtime.nsec))
++ : istate->timestamp.sec <= sd->sd_mtime.sec));
+ }
+
+ int is_racy_timestamp(const struct index_state *istate,
## statinfo.c ##
@@ statinfo.c: int match_stat_data(const struct stat_data *sd, struct stat *st)
@@ statinfo.c: int match_stat_data(const struct stat_data *sd, struct stat *st)
- if (cfg->trust_ctime && cfg->check_stat &&
- sd->sd_ctime.nsec != ST_CTIME_NSEC(*st))
- changed |= CTIME_CHANGED;
-+#ifndef NO_NSEC
+-#endif
+ if (cfg->use_nanosec) {
+ if (cfg->check_stat && sd->sd_mtime.nsec != ST_MTIME_NSEC(*st))
+ changed |= MTIME_CHANGED;
@@ statinfo.c: int match_stat_data(const struct stat_data *sd, struct stat *st)
+ sd->sd_ctime.nsec != ST_CTIME_NSEC(*st))
+ changed |= CTIME_CHANGED;
+ }
- #endif
if (cfg->check_stat) {
+ if (sd->sd_uid != (unsigned int) st->st_uid ||
base-commit: 2c78326f810173a4f3aefd8021f1e07575412481
Signature · 1 line
2.55.0.860.g4b6b3295ed.dirty
PATCH v4 1/3 meson: expose knob for xmlto relative links in manuals
Makefile-based builds have had this knob for most of the project's life, since a479a564dc (Documentation/Makefile: allow man.base.url.for.relative.link to be set from Make, 2009-12-03).
Meson, however, hard-codes the equivalent of $prefix/$mandir, which is not really where all the HTML docs are stored in most distro builds. Plus, this value is missing a trailing slash, so links come out broken, like this in git.1:
1. Git User’s Manual
/usr/share/manuser-manual.html
Of course we can do better:
1. Change the default to match Make: use file://$(htmldir)/ (with trailing slash!) to form a local URL pointing at the HTML docs. This is safe because all current uses of link:<relative> point at HTML docs:
git grep 'link:[[:alnum:]]' Documentation | grep -ve html -e http
produces only a single result (Documentation/howto/howto-index.sh) which can be ignored. Since nothing else [*] in the normal build sets MAN_BASE_URL, this seems like the right default.
2. Provide a configurable knob, just like the Makefile, so distributions that build with Meson (like Gentoo) can decide where to make the links if they need to. Those that set htmldir probably won't need to tweak this any further, though.
[*]: Well, Git's todo branch has a script dodoc.sh to build and archive
docs for kernel.org; these docs are pulled by Homebrew installations, for example. It sets MAN_BASE_URL to "git_htmldocs", so the equivalent note on macOS + Homebrew is
1. Git User’s Manual
git-htmldocs/user-manual.html
which is not functional either, but that's a problem for downstream. In any case, users can recover the right path with "git --html-path".
---
Notes (benknoble/commits):
This patch is mostly because I noticed the link I added in a later patch didn't come out right.
I did an internet search for "MAN_BASE_URL" and got no real hits, so I'm not sure if any distros today actually use it, but that's not a proper audit in that I didn't look at any distro _code_ besides Gentoo (which, as noted, uses Meson).
Documentation/meson.build | 7 ++++++- meson_options.txt | 2 ++ 2 files changed, 8 insertions(+), 1 deletion(-)
Documentation/meson.build +6 -1
@@ -379,13 +379,18 @@ foreach manpage, category : manpages output: fs.stem(manpage) + '.xml', ) + man_base_url = 'file://' + htmldir + '/' + if get_option('man_base_url') != '' + man_base_url = get_option('man_base_url') + endif + doc_targets += custom_target( command: [ xmlto, '-m', '@INPUT0@', '-m', '@INPUT1@', '--stringparam', - 'man.base.url.for.relative.links=' + get_option('prefix') / get_option('mandir'), + 'man.base.url.for.relative.links=' + man_base_url, 'man', manpage_xml_target, '-o',
meson_options.txt +2 -0
@@ -111,6 +111,8 @@ option('default_help_format', type: 'combo', choices: ['man', 'html', 'platform' description: 'Default format used when executing git-help(1).') option('docs_backend', type: 'combo', choices: ['asciidoc', 'asciidoctor', 'auto'], value: 'auto', description: 'Which backend to use to generate documentation.') +option('man_base_url', type: 'string', value: '', + description: 'The base URL to use for relative links in manuals') # Testing. option('benchmarks', type: 'feature', value: 'auto',
Signature · 1 line
2.55.0.860.g4b6b3295ed.dirty
PATCH v4 2/3 environment: align repo_config_values_init with struct declaration
In reply to D. Ben KnobleThe order of assignments in repo_config_values_init is chaotic and hard to follow, especially when comparing with the struct definition to ensure all members are initialized. As new members will be added in the future, make it easier to validate changes by aligning the two.
Refactor assignment order with no behavioral changes.
--- environment.c | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-)
environment.c +12 -7
@@ -745,6 +745,7 @@ int git_default_config(const char *var, const char *value,void repo_config_values_init (struct repo_config_values *cfg) { +/* section "core" config values */ cfg->attributes_file =NULL ; cfg->excludes_file =NULL ; cfg->editor_program =NULL ;
@@ -756,20 +757,24 @@ void repo_config_values_init(struct repo_config_values *cfg) cfg->autorebase =AUTOREBASE_NEVER ; cfg->object_creation_mode =OBJECT_CREATION_MODE ; cfg->apply_sparse_checkout =0 ; - cfg->protect_hfs =PROTECT_HFS_DEFAULT ; - cfg->protect_ntfs =PROTECT_NTFS_DEFAULT ; - cfg->ignore_case =0 ; - cfg->trust_executable_bit =1 ; - cfg->has_symlinks =platform_has_symlinks (); - cfg->branch_track =BRANCH_TRACK_REMOTE ; cfg->trust_ctime =1 ; cfg->check_stat =1 ; cfg->zlib_compression_level =Z_BEST_SPEED ; cfg->pack_compression_level =Z_DEFAULT_COMPRESSION ; cfg->precomposed_unicode =-1 ;/* see probe_utf8_pathname_composition() */ cfg->core_sparse_checkout_cone =0 ; - cfg->sparse_expect_files_outside_of_patterns =0 ; cfg->warn_on_object_refname_ambiguity =1 ; + cfg->protect_hfs =PROTECT_HFS_DEFAULT ; + cfg->protect_ntfs =PROTECT_NTFS_DEFAULT ; + cfg->ignore_case =0 ; + cfg->trust_executable_bit =1 ; + cfg->has_symlinks =platform_has_symlinks (); + +/* section "sparse" config values */ + cfg->sparse_expect_files_outside_of_patterns =0 ; + +/* section "branch" config values */ + cfg->branch_track =BRANCH_TRACK_REMOTE ; }void repo_config_values_clear (struct repo_config_values *cfg)
Signature · 1 line
2.55.0.860.g4b6b3295ed.dirty
PATCH v4 3/3 core: convert build-time USE_NSEC into runtime core.useNanosec
In reply to D. Ben KnobleRacy Git problems persist today, manifesting themselves in the performance of commands like "git diff" in new worktrees [1]. We have long had a build knob "USE_NSEC" to tell Git to use in-core nanosecond precision when available, which mitigates most if not all racy issues, but most builds we know about don't use it. In part, that's because someone distributing Git can't safely enable it at compile-time if they don't know exactly what platforms their distribution will be used on.
[1]: https://lore.kernel.org/git/CALnO6CADMJSixqYvL1Yo8qKX5rWhKQ+2OoSEuPUh-yoeK9TseQ@mail.gmail.com
These days, most platforms are likely to be safe for the USE_NSEC code. Regardless, we want to give users the ability to benefit from it. This requires exposing the compile-time gated code as a runtime option.
In addition, update the Racy Git documentation and other mentions of USE_NSEC in the code.
Due to the conversion from #ifdef to runtime check, using the flag "--ignore-space-change" may be particularly helpful when viewing changes from this patch.
---
Notes (benknoble/commits):
Related benchmarks: <https://lore.kernel.org/git/CALnO6CBm4g27mWBvD9m6yL0e5YZu3M9_zcUeLZk7QwTgnxMLQA@mail.gmail.com/>
CI: <https://github.com/benknoble/git/actions/runs/32365602564>
Documentation/config/core.adoc | 7 +++++++ Documentation/technical/racy-git.adoc | 11 ++++++----- Makefile | 12 +----------- builtin/update-index.c | 2 +- compat/posix.h | 1 - configure.ac | 6 ------ environment.c | 8 ++++++++ environment.h | 1 + read-cache.c | 15 ++++++--------- statinfo.c | 14 +++++++------- 10 files changed, 37 insertions(+), 40 deletions(-)
Documentation/config/core.adoc +7 -0
@@ -118,6 +118,13 @@ core.trustctime:: crawlers and some backup systems). See linkgit:git-update-index[1]. True by default. +core.useNanosec:: + If true, use nanosecond precision for ctime and mtime + comparisions between the index and the working tree (if Git + was compiled to respect this option). + This is unsafe on some platforms; + see link:technical/racy-git.html[Racy Git]. False by default. + core.splitIndex:: If true, the split-index feature of the index will be used. See linkgit:git-update-index[1]. False by default.
Documentation/technical/racy-git.adoc +6 -5
@@ -39,8 +39,8 @@ files) from `st_mode` member, `st_mtime` and `st_ctime` timestamps, `st_uid`, `st_gid`, `st_ino`, and `st_size` members. With a `USE_STDEV` compile-time option, `st_dev` is also compared, but this is not enabled by default because this member -is not stable on network filesystems. With `USE_NSEC` -compile-time option, `st_mtim.tv_nsec` and `st_ctim.tv_nsec` +is not stable on network filesystems. With 'core.useNanosec' +config setting, `st_mtim.tv_nsec` and `st_ctim.tv_nsec` members are also compared. On Linux, this is not enabled by default because in-core timestamps can have finer granularity than on-disk timestamps, resulting in meaningless changes when an
@@ -49,9 +49,10 @@ of git://git.kernel.org/pub/scm/linux/kernel/git/tglx/history.git ([PATCH] Sync in core time granularity with filesystems, 2005-01-04). This patch is included in kernel 2.6.11 and newer, but only fixes the issue for file systems with exactly 1 ns or 1 s -resolution. Other file systems are still broken in current Linux -kernels (e.g. CEPH, CIFS, NTFS, UDF), see -https://lore.kernel.org/lkml/[email protected]/ +resolution. As of kernel 4.3, other file systems (CEPH, CIFS, NTFS, UFS, FUSE) +were fixed; see https://public-inbox.org/git/5605D88A.20104%40gmail.com/. FAT +has been fixed since 2015. The usual suspects (ext2, ext4, XFS) are known to +work, too. Racy Git --------
Makefile +1 -11
@@ -197,18 +197,11 @@ include shared.mak # Define NO_NORETURN if using buggy versions of gcc 4.6+ and profile feedback, # as the compiler can crash (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=49299) # -# Define USE_NSEC below if you want git to care about sub-second file mtimes -# and ctimes. Note that you need recent glibc (at least 2.2.4) for this. On -# Linux, kernel 2.6.11 or newer is required for reliable sub-second file times -# on file systems with exactly 1 ns or 1 s resolution. If you intend to use Git -# on other file systems (e.g. CEPH, CIFS, NTFS, UDF), don't enable USE_NSEC. See -# Documentation/technical/racy-git.adoc for details. -# # Define USE_ST_TIMESPEC if your "struct stat" uses "st_ctimespec" instead of # "st_ctim" # # Define NO_NSEC if your "struct stat" does not have "st_ctim.tv_nsec" -# available. This automatically turns USE_NSEC off. +# available. # # Define USE_STDEV below if you want git to care about the underlying device # change being considered an inode change from the update-index perspective.
@@ -1935,9 +1928,6 @@ endif ifdef NO_ST_BLOCKS_IN_STRUCT_STAT BASIC_CFLAGS += -DNO_ST_BLOCKS_IN_STRUCT_STAT endif -ifdef USE_NSEC - BASIC_CFLAGS += -DUSE_NSEC -endif ifdef USE_ST_TIMESPEC BASIC_CFLAGS += -DUSE_ST_TIMESPEC endif
builtin/update-index.c +1 -1
@@ -130,7 +130,7 @@ static void xrmdir(const char *path)static void avoid_racy (void ) {/* -* not use if we could usleep(10) if USE_NSEC is defined. The +* not use if we could usleep(10) if core.useNanosec is defined. The * field nsec could be there, but the OS could choose to * ignore it? */
compat/posix.h +0 -1
@@ -487,7 +487,6 @@ int git_qsort_s(void *base, size_t nmemb, size_t size, } while (0)#ifdef NO_NSEC -#undef USE_NSEC#define ST_CTIME_NSEC (st) 0#define ST_MTIME_NSEC (st) 0#else
configure.ac +0 -6
@@ -351,12 +351,6 @@ GIT_PARSE_WITH(iconv)) ## --enable-FEATURE[=ARG] and --disable-FEATURE # -# Define USE_NSEC below if you want git to care about sub-second file mtimes -# and ctimes. Note that you need recent glibc (at least 2.2.4) for this, and -# it will BREAK YOUR LOCAL DIFFS! show-diff and anything using it will likely -# randomly break unless your underlying filesystem supports those sub-second -# times (my ext3 doesn't). -# # Define USE_STDEV below if you want git to care about the underlying device # change being considered an inode change from the update-index perspective.
environment.c +8 -0
@@ -571,6 +571,13 @@ int git_default_core_config(const char *var, const char *value,return 0 ; } +#ifndef NO_NSEC +if (!strcmp (var,"core.usenanosec" )) { + cfg->use_nanosec =git_config_bool (var, value); +return 0 ; + } +#endif +/* Add other config variables here and to Documentation/config.adoc. */ return platform_core_config (var, value, ctx, cb); }
@@ -769,6 +776,7 @@ void repo_config_values_init(struct repo_config_values *cfg) cfg->ignore_case =0 ; cfg->trust_executable_bit =1 ; cfg->has_symlinks =platform_has_symlinks (); + cfg->use_nanosec =0 ;/* section "sparse" config values */ cfg->sparse_expect_files_outside_of_patterns =0 ;
environment.h +1 -0
@@ -139,6 +139,7 @@ struct repo_config_values {int ignore_case;int trust_executable_bit;int has_symlinks; +int use_nanosec;/* section "sparse" config values */ int sparse_expect_files_outside_of_patterns;
read-cache.c +6 -9
@@ -354,15 +354,12 @@ static int is_racy_stat(const struct index_state *istate, const struct stat_data *sd) {return (istate->timestamp.sec && -#ifdef USE_NSEC - /* nanosecond timestamped files can also be racy! */ - (istate->timestamp.sec < sd->sd_mtime.sec || - (istate->timestamp.sec == sd->sd_mtime.sec && - istate->timestamp.nsec <= sd->sd_mtime.nsec)) -#else - istate->timestamp.sec <= sd->sd_mtime.sec -#endif - ); +/* nanosecond timestamped files can also be racy! */ + (repo_config_values (istate->repo)->use_nanosec + ? (istate->timestamp.sec < sd->sd_mtime.sec || + (istate->timestamp.sec == sd->sd_mtime.sec && + istate->timestamp.nsec <= sd->sd_mtime.nsec)) + : istate->timestamp.sec <= sd->sd_mtime.sec)); } int is_racy_timestamp(const struct index_state *istate,
statinfo.c +7 -7
@@ -72,13 +72,13 @@ int match_stat_data(const struct stat_data *sd, struct stat *st) sd->sd_ctime.sec != (unsigned int)st->st_ctime) changed |=CTIME_CHANGED ; -#ifdef USE_NSEC -if (cfg->check_stat && sd->sd_mtime.nsec !=ST_MTIME_NSEC (*st)) - changed |=MTIME_CHANGED ; -if (cfg->trust_ctime && cfg->check_stat && - sd->sd_ctime.nsec !=ST_CTIME_NSEC (*st)) - changed |=CTIME_CHANGED ; -#endif +if (cfg->use_nanosec) { +if (cfg->check_stat && sd->sd_mtime.nsec !=ST_MTIME_NSEC (*st)) + changed |=MTIME_CHANGED ; +if (cfg->trust_ctime && cfg->check_stat && + sd->sd_ctime.nsec !=ST_CTIME_NSEC (*st)) + changed |=CTIME_CHANGED ; + }if (cfg->check_stat) { if (sd->sd_uid != (unsigned int) st->st_uid ||
Signature · 1 line
2.55.0.860.g4b6b3295ed.dirty
Re: [PATCH v4 2/3] environment: align repo_config_values_init with struct declaration
In reply to D. Ben KnobleQuoted 6 lines · D. Ben Knoble
The order of assignments in
repo_config_values_initis chaotic and hard to follow, especially when comparing with the struct definition to ensure all members are initialized. As new members will be added in the future, make it easier to validate changes by aligning the two.Refactor assignment order with no behavioral changes.
After reading the above three times, I am tempted to slightly tweak the above:
... comparing with the definition of 'struct repo_config_values' to ensure ...
Other than that, great improvement.
Thanks.
Quoted 50 lines
Signed-off-by D. Ben Knoble <[email protected]>--- environment.c | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-)environment.c +12
-7@@ -745,6 +745,7 @@ int git_default_config(const char *var, const char *value,void repo_config_values_init (struct repo_config_values *cfg) { +/* section "core" config values */ cfg->attributes_file =NULL ; cfg->excludes_file =NULL ; cfg->editor_program =NULL ;@@ -756,20 +757,24 @@ void repo_config_values_init(struct repo_config_values *cfg) cfg->autorebase =AUTOREBASE_NEVER ; cfg->object_creation_mode =OBJECT_CREATION_MODE ; cfg->apply_sparse_checkout =0 ; - cfg->protect_hfs =PROTECT_HFS_DEFAULT ; - cfg->protect_ntfs =PROTECT_NTFS_DEFAULT ; - cfg->ignore_case =0 ; - cfg->trust_executable_bit =1 ; - cfg->has_symlinks =platform_has_symlinks (); - cfg->branch_track =BRANCH_TRACK_REMOTE ; cfg->trust_ctime =1 ; cfg->check_stat =1 ; cfg->zlib_compression_level =Z_BEST_SPEED ; cfg->pack_compression_level =Z_DEFAULT_COMPRESSION ; cfg->precomposed_unicode =-1 ;/* see probe_utf8_pathname_composition() */ cfg->core_sparse_checkout_cone =0 ; - cfg->sparse_expect_files_outside_of_patterns =0 ; cfg->warn_on_object_refname_ambiguity =1 ; + cfg->protect_hfs =PROTECT_HFS_DEFAULT ; + cfg->protect_ntfs =PROTECT_NTFS_DEFAULT ; + cfg->ignore_case =0 ; + cfg->trust_executable_bit =1 ; + cfg->has_symlinks =platform_has_symlinks (); + +/* section "sparse" config values */ + cfg->sparse_expect_files_outside_of_patterns =0 ; + +/* section "branch" config values */ + cfg->branch_track =BRANCH_TRACK_REMOTE ; }void repo_config_values_clear (struct repo_config_values *cfg)
Re: [PATCH v4 2/3] environment: align repo_config_values_init with struct declaration
Quoted 13 lines · Junio C Hamano
Quoted 6 lines · D. Ben Knoble
The order of assignments in
repo_config_values_initis chaotic and hard to follow, especially when comparing with the struct definition to ensure all members are initialized. As new members will be added in the future, make it easier to validate changes by aligning the two.Refactor assignment order with no behavioral changes.
After reading the above three times, I am tempted to slightly tweak the above:
Quoted 6 lines
... comparing with the definition of 'struct
repo_config_values' to ensure ...Other than that, great improvement.
Thanks.
Yep, that flows much better. Amended locally.
Signature · 1 line
D. Ben Knoble
v5 cover Convert USE_NSEC to runtime config
In reply to D. Ben KnobleTopic name: dk/use-nsec-runtime (applied)
Topic summary: Expose USE_NSEC as a runtime configuration, since build-time is too early for distributing Git [1]. As a result, common index-related options, like git-diff, are less likely to hit "racy git" problems on supported filesystems.
[1]: https://git.github.io/rev_news/2026/07/31/edition-137/
Built on master (2c78326f81 (The 11th batch, 2026-08-05)).
Changes in v5:
- improve message flow in patch 2
Junio: my apologies. I missed that the "What's cooking" changed from "Needs review" (#10) to "expecting a small (hopefully final) reroll" (#11, #12), since I was expecting to see Patrick's review. This just tweaks the commit message as you suggested.
Changes in v4:
- fix message typo
- change #ifdef strategy: only ignore the config variable. Otherwise, use the use_nanosec member unconditionally. Also clarify that config might be ignore depending on build options in the docs.
- mention potential platform unsafety directly in config doc in addition to the link to Racy Git
Changes in v3:
- #ifdef out use_nanosec when NO_NSEC is requested
As I have heard no comments about the "Todo" lines below, which perhaps could more clearly be marked "RFC"/"RFH", I've added this line to call them out ;) and renamed them "Comments welcome"
Changes in v2:
- move Best-viewed-with trailer into message body as descriptive text.
- read core.useNanosec through struct repo instead of parsing config strings. The test suite passes locally this way, though that skipped 151 tests.
- CI run: https://github.com/benknoble/git/actions/runs/31701945211
Original cover letter:
Hi all, this series follows up on the previous racy Git/USE_NSEC conversations.
- The first patch is a mostly-unrelated documentation fix for Meson, but it came out of something I spotted while reviewing the outputs of the final (main) patch.
- The second patch is a preliminary no-op reorganization of repo_config_values_init.
- The third patch is the meat, converting USE_NSEC into core.useNanosec.
There is a small textual and semantic conflict with 'ty/repo-config-cleanups' in 'seen', since that branch removes the comments in 'struct repo_config_values' which this series adds to. (The semantic conflict is that, if we drop those comments, we should probably not add them to repo_config_values_init like I do in patch 2.)
Comments welcome: I haven't touched any tests; I saw a bunch of hits for "git grep racy t" but wasn't sure how to fit this particular change in, especially since it won't be equally valid on all systems? Advice welcome.
Comments welcome: I wonder if "useNanosec" paints us into too much of a corner; that is (slightly more abstractly), we are using *extended precision* in the index. Maybe the name and documentation should reflect that, so we aren't too committed to "nanoseconds"?
- Some platforms could offer extended precision that is not as precise as nanoseconds
- Some could offer precision _beyond_ nanoseconds
idk.
v1: <[email protected]>
v2: <[email protected]>
v3: <[email protected]>
v4: <[email protected]>
[1/3] meson: expose knob for xmlto relative links in manuals [2/3] environment: align repo_config_values_init with struct declaration [3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
Documentation/config/core.adoc | 7 +++++++ Documentation/meson.build | 7 ++++++- Documentation/technical/racy-git.adoc | 11 ++++++----- Makefile | 12 +----------- builtin/update-index.c | 2 +- compat/posix.h | 1 - configure.ac | 6 ------ environment.c | 27 ++++++++++++++++++++------- environment.h | 1 + meson_options.txt | 2 ++ read-cache.c | 15 ++++++--------- statinfo.c | 14 +++++++------- 12 files changed, 57 insertions(+), 48 deletions(-)
Diff-intervalle contre v4 :
1: d612de6c2d = 1: d612de6c2d meson: expose knob for xmlto relative links in manuals 2: 5693baa992 ! 2: 12974e07d0 environment: align repo_config_values_init with struct declaration
@@ Commit message
environment: align repo_config_values_init with struct declaration
The order of assignments in repo_config_values_init is chaotic and hard
- to follow, especially when comparing with the struct definition to
- ensure all members are initialized. As new members will be added in the
- future, make it easier to validate changes by aligning the two.
+ to follow, especially with the definition of 'struct repo_config_values'
+ to ensure all members are initialized. As new members will be added in
+ the future, make it easier to validate changes by aligning the two.
Refactor assignment order with no behavioral changes.
3: 0aa0e9fc17 = 3: 01cd487cd2 core: convert build-time USE_NSEC into runtime core.useNanosec
base-commit: 2c78326f810173a4f3aefd8021f1e07575412481
Signature · 1 line
2.55.0.860.g4b6b3295ed.dirty
PATCH v5 1/3 meson: expose knob for xmlto relative links in manuals
Makefile-based builds have had this knob for most of the project's life, since a479a564dc (Documentation/Makefile: allow man.base.url.for.relative.link to be set from Make, 2009-12-03).
Meson, however, hard-codes the equivalent of $prefix/$mandir, which is not really where all the HTML docs are stored in most distro builds. Plus, this value is missing a trailing slash, so links come out broken, like this in git.1:
1. Git User’s Manual
/usr/share/manuser-manual.html
Of course we can do better:
1. Change the default to match Make: use file://$(htmldir)/ (with trailing slash!) to form a local URL pointing at the HTML docs. This is safe because all current uses of link:<relative> point at HTML docs:
git grep 'link:[[:alnum:]]' Documentation | grep -ve html -e http
produces only a single result (Documentation/howto/howto-index.sh) which can be ignored. Since nothing else [*] in the normal build sets MAN_BASE_URL, this seems like the right default.
2. Provide a configurable knob, just like the Makefile, so distributions that build with Meson (like Gentoo) can decide where to make the links if they need to. Those that set htmldir probably won't need to tweak this any further, though.
[*]: Well, Git's todo branch has a script dodoc.sh to build and archive
docs for kernel.org; these docs are pulled by Homebrew installations, for example. It sets MAN_BASE_URL to "git_htmldocs", so the equivalent note on macOS + Homebrew is
1. Git User’s Manual
git-htmldocs/user-manual.html
which is not functional either, but that's a problem for downstream. In any case, users can recover the right path with "git --html-path".
---
Notes (benknoble/commits):
This patch is mostly because I noticed the link I added in a later patch didn't come out right.
I did an internet search for "MAN_BASE_URL" and got no real hits, so I'm not sure if any distros today actually use it, but that's not a proper audit in that I didn't look at any distro _code_ besides Gentoo (which, as noted, uses Meson).
Documentation/meson.build | 7 ++++++- meson_options.txt | 2 ++ 2 files changed, 8 insertions(+), 1 deletion(-)
Documentation/meson.build +6 -1
@@ -379,13 +379,18 @@ foreach manpage, category : manpages output: fs.stem(manpage) + '.xml', ) + man_base_url = 'file://' + htmldir + '/' + if get_option('man_base_url') != '' + man_base_url = get_option('man_base_url') + endif + doc_targets += custom_target( command: [ xmlto, '-m', '@INPUT0@', '-m', '@INPUT1@', '--stringparam', - 'man.base.url.for.relative.links=' + get_option('prefix') / get_option('mandir'), + 'man.base.url.for.relative.links=' + man_base_url, 'man', manpage_xml_target, '-o',
meson_options.txt +2 -0
@@ -111,6 +111,8 @@ option('default_help_format', type: 'combo', choices: ['man', 'html', 'platform' description: 'Default format used when executing git-help(1).') option('docs_backend', type: 'combo', choices: ['asciidoc', 'asciidoctor', 'auto'], value: 'auto', description: 'Which backend to use to generate documentation.') +option('man_base_url', type: 'string', value: '', + description: 'The base URL to use for relative links in manuals') # Testing. option('benchmarks', type: 'feature', value: 'auto',
Signature · 1 line
2.55.0.860.g4b6b3295ed.dirty
PATCH v5 2/3 environment: align repo_config_values_init with struct declaration
In reply to D. Ben KnobleThe order of assignments in repo_config_values_init is chaotic and hard to follow, especially with the definition of 'struct repo_config_values' to ensure all members are initialized. As new members will be added in the future, make it easier to validate changes by aligning the two.
Refactor assignment order with no behavioral changes.
--- environment.c | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-)
environment.c +12 -7
@@ -745,6 +745,7 @@ int git_default_config(const char *var, const char *value,void repo_config_values_init (struct repo_config_values *cfg) { +/* section "core" config values */ cfg->attributes_file =NULL ; cfg->excludes_file =NULL ; cfg->editor_program =NULL ;
@@ -756,20 +757,24 @@ void repo_config_values_init(struct repo_config_values *cfg) cfg->autorebase =AUTOREBASE_NEVER ; cfg->object_creation_mode =OBJECT_CREATION_MODE ; cfg->apply_sparse_checkout =0 ; - cfg->protect_hfs =PROTECT_HFS_DEFAULT ; - cfg->protect_ntfs =PROTECT_NTFS_DEFAULT ; - cfg->ignore_case =0 ; - cfg->trust_executable_bit =1 ; - cfg->has_symlinks =platform_has_symlinks (); - cfg->branch_track =BRANCH_TRACK_REMOTE ; cfg->trust_ctime =1 ; cfg->check_stat =1 ; cfg->zlib_compression_level =Z_BEST_SPEED ; cfg->pack_compression_level =Z_DEFAULT_COMPRESSION ; cfg->precomposed_unicode =-1 ;/* see probe_utf8_pathname_composition() */ cfg->core_sparse_checkout_cone =0 ; - cfg->sparse_expect_files_outside_of_patterns =0 ; cfg->warn_on_object_refname_ambiguity =1 ; + cfg->protect_hfs =PROTECT_HFS_DEFAULT ; + cfg->protect_ntfs =PROTECT_NTFS_DEFAULT ; + cfg->ignore_case =0 ; + cfg->trust_executable_bit =1 ; + cfg->has_symlinks =platform_has_symlinks (); + +/* section "sparse" config values */ + cfg->sparse_expect_files_outside_of_patterns =0 ; + +/* section "branch" config values */ + cfg->branch_track =BRANCH_TRACK_REMOTE ; }void repo_config_values_clear (struct repo_config_values *cfg)
Signature · 1 line
2.55.0.860.g4b6b3295ed.dirty
PATCH v5 3/3 core: convert build-time USE_NSEC into runtime core.useNanosec
In reply to D. Ben KnobleRacy Git problems persist today, manifesting themselves in the performance of commands like "git diff" in new worktrees [1]. We have long had a build knob "USE_NSEC" to tell Git to use in-core nanosecond precision when available, which mitigates most if not all racy issues, but most builds we know about don't use it. In part, that's because someone distributing Git can't safely enable it at compile-time if they don't know exactly what platforms their distribution will be used on.
[1]: https://lore.kernel.org/git/CALnO6CADMJSixqYvL1Yo8qKX5rWhKQ+2OoSEuPUh-yoeK9TseQ@mail.gmail.com
These days, most platforms are likely to be safe for the USE_NSEC code. Regardless, we want to give users the ability to benefit from it. This requires exposing the compile-time gated code as a runtime option.
In addition, update the Racy Git documentation and other mentions of USE_NSEC in the code.
Due to the conversion from #ifdef to runtime check, using the flag "--ignore-space-change" may be particularly helpful when viewing changes from this patch.
---
Notes (benknoble/commits):
Related benchmarks: <https://lore.kernel.org/git/CALnO6CBm4g27mWBvD9m6yL0e5YZu3M9_zcUeLZk7QwTgnxMLQA@mail.gmail.com/>
CI: <https://github.com/benknoble/git/actions/runs/32365602564>
Documentation/config/core.adoc | 7 +++++++ Documentation/technical/racy-git.adoc | 11 ++++++----- Makefile | 12 +----------- builtin/update-index.c | 2 +- compat/posix.h | 1 - configure.ac | 6 ------ environment.c | 8 ++++++++ environment.h | 1 + read-cache.c | 15 ++++++--------- statinfo.c | 14 +++++++------- 10 files changed, 37 insertions(+), 40 deletions(-)
Documentation/config/core.adoc +7 -0
@@ -118,6 +118,13 @@ core.trustctime:: crawlers and some backup systems). See linkgit:git-update-index[1]. True by default. +core.useNanosec:: + If true, use nanosecond precision for ctime and mtime + comparisions between the index and the working tree (if Git + was compiled to respect this option). + This is unsafe on some platforms; + see link:technical/racy-git.html[Racy Git]. False by default. + core.splitIndex:: If true, the split-index feature of the index will be used. See linkgit:git-update-index[1]. False by default.
Documentation/technical/racy-git.adoc +6 -5
@@ -39,8 +39,8 @@ files) from `st_mode` member, `st_mtime` and `st_ctime` timestamps, `st_uid`, `st_gid`, `st_ino`, and `st_size` members. With a `USE_STDEV` compile-time option, `st_dev` is also compared, but this is not enabled by default because this member -is not stable on network filesystems. With `USE_NSEC` -compile-time option, `st_mtim.tv_nsec` and `st_ctim.tv_nsec` +is not stable on network filesystems. With 'core.useNanosec' +config setting, `st_mtim.tv_nsec` and `st_ctim.tv_nsec` members are also compared. On Linux, this is not enabled by default because in-core timestamps can have finer granularity than on-disk timestamps, resulting in meaningless changes when an
@@ -49,9 +49,10 @@ of git://git.kernel.org/pub/scm/linux/kernel/git/tglx/history.git ([PATCH] Sync in core time granularity with filesystems, 2005-01-04). This patch is included in kernel 2.6.11 and newer, but only fixes the issue for file systems with exactly 1 ns or 1 s -resolution. Other file systems are still broken in current Linux -kernels (e.g. CEPH, CIFS, NTFS, UDF), see -https://lore.kernel.org/lkml/[email protected]/ +resolution. As of kernel 4.3, other file systems (CEPH, CIFS, NTFS, UFS, FUSE) +were fixed; see https://public-inbox.org/git/5605D88A.20104%40gmail.com/. FAT +has been fixed since 2015. The usual suspects (ext2, ext4, XFS) are known to +work, too. Racy Git --------
Makefile +1 -11
@@ -197,18 +197,11 @@ include shared.mak # Define NO_NORETURN if using buggy versions of gcc 4.6+ and profile feedback, # as the compiler can crash (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=49299) # -# Define USE_NSEC below if you want git to care about sub-second file mtimes -# and ctimes. Note that you need recent glibc (at least 2.2.4) for this. On -# Linux, kernel 2.6.11 or newer is required for reliable sub-second file times -# on file systems with exactly 1 ns or 1 s resolution. If you intend to use Git -# on other file systems (e.g. CEPH, CIFS, NTFS, UDF), don't enable USE_NSEC. See -# Documentation/technical/racy-git.adoc for details. -# # Define USE_ST_TIMESPEC if your "struct stat" uses "st_ctimespec" instead of # "st_ctim" # # Define NO_NSEC if your "struct stat" does not have "st_ctim.tv_nsec" -# available. This automatically turns USE_NSEC off. +# available. # # Define USE_STDEV below if you want git to care about the underlying device # change being considered an inode change from the update-index perspective.
@@ -1935,9 +1928,6 @@ endif ifdef NO_ST_BLOCKS_IN_STRUCT_STAT BASIC_CFLAGS += -DNO_ST_BLOCKS_IN_STRUCT_STAT endif -ifdef USE_NSEC - BASIC_CFLAGS += -DUSE_NSEC -endif ifdef USE_ST_TIMESPEC BASIC_CFLAGS += -DUSE_ST_TIMESPEC endif
builtin/update-index.c +1 -1
@@ -130,7 +130,7 @@ static void xrmdir(const char *path)static void avoid_racy (void ) {/* -* not use if we could usleep(10) if USE_NSEC is defined. The +* not use if we could usleep(10) if core.useNanosec is defined. The * field nsec could be there, but the OS could choose to * ignore it? */
compat/posix.h +0 -1
@@ -487,7 +487,6 @@ int git_qsort_s(void *base, size_t nmemb, size_t size, } while (0)#ifdef NO_NSEC -#undef USE_NSEC#define ST_CTIME_NSEC (st) 0#define ST_MTIME_NSEC (st) 0#else
configure.ac +0 -6
@@ -351,12 +351,6 @@ GIT_PARSE_WITH(iconv)) ## --enable-FEATURE[=ARG] and --disable-FEATURE # -# Define USE_NSEC below if you want git to care about sub-second file mtimes -# and ctimes. Note that you need recent glibc (at least 2.2.4) for this, and -# it will BREAK YOUR LOCAL DIFFS! show-diff and anything using it will likely -# randomly break unless your underlying filesystem supports those sub-second -# times (my ext3 doesn't). -# # Define USE_STDEV below if you want git to care about the underlying device # change being considered an inode change from the update-index perspective.
environment.c +8 -0
@@ -571,6 +571,13 @@ int git_default_core_config(const char *var, const char *value,return 0 ; } +#ifndef NO_NSEC +if (!strcmp (var,"core.usenanosec" )) { + cfg->use_nanosec =git_config_bool (var, value); +return 0 ; + } +#endif +/* Add other config variables here and to Documentation/config.adoc. */ return platform_core_config (var, value, ctx, cb); }
@@ -769,6 +776,7 @@ void repo_config_values_init(struct repo_config_values *cfg) cfg->ignore_case =0 ; cfg->trust_executable_bit =1 ; cfg->has_symlinks =platform_has_symlinks (); + cfg->use_nanosec =0 ;/* section "sparse" config values */ cfg->sparse_expect_files_outside_of_patterns =0 ;
environment.h +1 -0
@@ -139,6 +139,7 @@ struct repo_config_values {int ignore_case;int trust_executable_bit;int has_symlinks; +int use_nanosec;/* section "sparse" config values */ int sparse_expect_files_outside_of_patterns;
read-cache.c +6 -9
@@ -354,15 +354,12 @@ static int is_racy_stat(const struct index_state *istate, const struct stat_data *sd) {return (istate->timestamp.sec && -#ifdef USE_NSEC - /* nanosecond timestamped files can also be racy! */ - (istate->timestamp.sec < sd->sd_mtime.sec || - (istate->timestamp.sec == sd->sd_mtime.sec && - istate->timestamp.nsec <= sd->sd_mtime.nsec)) -#else - istate->timestamp.sec <= sd->sd_mtime.sec -#endif - ); +/* nanosecond timestamped files can also be racy! */ + (repo_config_values (istate->repo)->use_nanosec + ? (istate->timestamp.sec < sd->sd_mtime.sec || + (istate->timestamp.sec == sd->sd_mtime.sec && + istate->timestamp.nsec <= sd->sd_mtime.nsec)) + : istate->timestamp.sec <= sd->sd_mtime.sec)); } int is_racy_timestamp(const struct index_state *istate,
statinfo.c +7 -7
@@ -72,13 +72,13 @@ int match_stat_data(const struct stat_data *sd, struct stat *st) sd->sd_ctime.sec != (unsigned int)st->st_ctime) changed |=CTIME_CHANGED ; -#ifdef USE_NSEC -if (cfg->check_stat && sd->sd_mtime.nsec !=ST_MTIME_NSEC (*st)) - changed |=MTIME_CHANGED ; -if (cfg->trust_ctime && cfg->check_stat && - sd->sd_ctime.nsec !=ST_CTIME_NSEC (*st)) - changed |=CTIME_CHANGED ; -#endif +if (cfg->use_nanosec) { +if (cfg->check_stat && sd->sd_mtime.nsec !=ST_MTIME_NSEC (*st)) + changed |=MTIME_CHANGED ; +if (cfg->trust_ctime && cfg->check_stat && + sd->sd_ctime.nsec !=ST_CTIME_NSEC (*st)) + changed |=CTIME_CHANGED ; + }if (cfg->check_stat) { if (sd->sd_uid != (unsigned int) st->st_uid ||
Signature · 1 line
2.55.0.860.g4b6b3295ed.dirty
Re: [PATCH v5 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
read-cache.c static int is_racy_stat(const struct index_state *istate, · line 363
Quoted 7 lines · D. Ben Knoble
read-cache.c +6
-0+/* nanosecond timestamped files can also be racy! */ + (repo_config_values (istate->repo)->use_nanosec + ? (istate->timestamp.sec < sd->sd_mtime.sec || + (istate->timestamp.sec == sd->sd_mtime.sec && + istate->timestamp.nsec <= sd->sd_mtime.nsec)) + : istate->timestamp.sec <= sd->sd_mtime.sec)); }
Currently this is probably fine, but the use of repo_config_values() here means that the order in which we can transition/libify two unrelated things are forced on us:
* We'd first need to make sure repo_config_values() can work on an instance of repository that is not the_repository,
* And until the above happens, we cannot do a --recurse-submodule option that loads the index in a submodule and operate on it in the same process (e.g., "git diff --resurse-submodules"), because immediately at this step, istate taken from a submodule would have its .repo member pointing at something that is not the_repository and we will hit a BUG().
And after writing all of the above, I realized that I am mostly repeating what Patric already said in the upstream, e.g.,
https://lore.kernel.org/git/[email protected]/
Other than that, this looks good to me.
Re: [PATCH v5 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
Quoted 25 lines · Junio C Hamano
read-cache.c static int is_racy_stat(const struct index_state *istate, · line 363
Quoted 7 lines · D. Ben Knoble
read-cache.c +6
-0+/* nanosecond timestamped files can also be racy! */ + (repo_config_values (istate->repo)->use_nanosec + ? (istate->timestamp.sec < sd->sd_mtime.sec || + (istate->timestamp.sec == sd->sd_mtime.sec && + istate->timestamp.nsec <= sd->sd_mtime.nsec)) + : istate->timestamp.sec <= sd->sd_mtime.sec)); }Currently this is probably fine, but the use of
repo_config_values()here means that the order in which we can transition/libify two unrelated things are forced on us:* We'd first need to make sure
repo_config_values()can work on an instance of repository that is notthe_repository,* And until the above happens, we cannot do a --recurse-submodule option that loads the index in a submodule and operate on it in the same process (e.g., "git diff --resurse-submodules"), because immediately at this step, istate taken from a submodule would have its .repo member pointing at something that is not
the_repositoryand we will hit aBUG().
Quoted 4 lines
And after writing all of the above, I realized that I am mostly repeating what Patric already said in the upstream, e.g.,
https://lore.kernel.org/git/[email protected]/
Yep---just so I'm clear, we don't currently have such an option, right? I mean, there is no --recurse-submodules for git-diff(1), and I tweaked t4060 to run "git -c core.useNanosec=true diff --submodule=diff" without any issue.
I would happily prove that at least none of our existing tests fail with core.useNanosec=true, but I'm not really sure how to shove configuration into every test invocation of git. Even if we could, I'm not sure we necessarily want to add another CI job for that (though that's a separate matter).
In particular, (among others) I have not received any concrete comments for
Quoted 4 lines
Comments welcome: I haven't touched any tests; I saw a bunch of hits for "git grep racy t" but wasn't sure how to fit this particular change in, especially since it won't be equally valid on all systems? Advice welcome.
so if there's at least a way to exercise this path on all the tests on my system (which should support it), that would probably be a good thing.
Quoted 1 line
Other than that, this looks good to me.
Thank
--
D. Ben Knoble
Re: [PATCH v5 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
Quoted 29 lines · D. Ben Knoble
Quoted 28 lines · Junio C Hamano
read-cache.c static int is_racy_stat(const struct index_state *istate, · line 363
Quoted 7 lines · D. Ben Knoble
read-cache.c +6
-0+/* nanosecond timestamped files can also be racy! */ + (repo_config_values (istate->repo)->use_nanosec + ? (istate->timestamp.sec < sd->sd_mtime.sec || + (istate->timestamp.sec == sd->sd_mtime.sec && + istate->timestamp.nsec <= sd->sd_mtime.nsec)) + : istate->timestamp.sec <= sd->sd_mtime.sec)); }Currently this is probably fine, but the use of
repo_config_values()here means that the order in which we can transition/libify two unrelated things are forced on us:* We'd first need to make sure
repo_config_values()can work on an instance of repository that is notthe_repository,* And until the above happens, we cannot do a --recurse-submodule option that loads the index in a submodule and operate on it in the same process (e.g., "git diff --resurse-submodules"), because immediately at this step, istate taken from a submodule would have its .repo member pointing at something that is not
the_repositoryand we will hit aBUG().And after writing all of the above, I realized that I am mostly repeating what Patric already said in the upstream, e.g.,
Quoted 6 lines
Quoted 1 line
https://lore.kernel.org/git/[email protected]/Yep---just so I'm clear, we don't currently have such an option, right? I mean, there is no --recurse-submodules for git-diff(1), and I tweaked t4060 to run "git -c core.useNanosec=true diff --submodule=diff" without any issue.
I do have a patch series coming up where we start to rely more on sub-repositories when recursing. The motivation behind that series is that it allows us to get rid of registering submodule object databases with the main ODB. But I just double-checked, and your series luckily doesn't break it.
Quoted 8 lines
I would happily prove that at least none of our existing tests fail with core.useNanosec=true, but I'm not really sure how to shove configuration into every test invocation of git. Even if we could, I'm not sure we necessarily want to add another CI job for that (though that's a separate matter).
In particular, (among others) I have not received any concrete comments for
Quoted 8 lines
Quoted 4 lines
Comments welcome: I haven't touched any tests; I saw a bunch of hits for "git grep racy t" but wasn't sure how to fit this particular change in, especially since it won't be equally valid on all systems? Advice welcome.
so if there's at least a way to exercise this path on all the tests on my system (which should support it), that would probably be a good thing.
Yeah, I simply don't have a good answer here. It's messy, and I'm not a fan of the current direction of repo_config_values() because nobody has yet stepped up to untangle it from the_repository. I gave it a quick shot at one point in time, but the result was messy at best because of how we populate it via repo_config(git_default_config).
In any case, if we see that your changes interact badly with some edge cases that we don't currently have on our radar then we can still refactor the series and move the value into struct repo_settings instead, as that structure works alright with different repositories.
Thanks!
Patrick
Re: [PATCH v5 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
In reply to D. Ben Knoblebuiltin/update-index.c static void xrmdir(const char *path) · line 133
Quoted 10 lines · D. Ben Knoble
builtin/update-index.c +1
-1@@ -130,7 +130,7 @@ static void xrmdir(const char *path)static void avoid_racy (void ) {/* -* not use if we could usleep(10) if USE_NSEC is defined. The +* not use if we could usleep(10) if core.useNanosec is defined. The
Micronit: s/defined/enabled/
Otherwise I'm happy with this patch. It looks a lot better now that we use less preprocessor directives. Thanks!
Patrick
Re: [PATCH v5 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
In reply to Patrick Steinhardt[Apologies for duplicates; Gmail switched out of plain-text mode without permission. I've yet to finish setting up aerc…]
Hi Patrick,
Quoted 38 lines · Patrick Steinhardt
Quoted 29 lines · D. Ben Knoble
Quoted 28 lines · Junio C Hamano
read-cache.c static int is_racy_stat(const struct index_state *istate, · line 363
Quoted 7 lines · D. Ben Knoble
read-cache.c +6
-0+/* nanosecond timestamped files can also be racy! */ + (repo_config_values (istate->repo)->use_nanosec + ? (istate->timestamp.sec < sd->sd_mtime.sec || + (istate->timestamp.sec == sd->sd_mtime.sec && + istate->timestamp.nsec <= sd->sd_mtime.nsec)) + : istate->timestamp.sec <= sd->sd_mtime.sec)); }Currently this is probably fine, but the use of
repo_config_values()here means that the order in which we can transition/libify two unrelated things are forced on us:* We'd first need to make sure
repo_config_values()can work on an instance of repository that is notthe_repository,* And until the above happens, we cannot do a --recurse-submodule option that loads the index in a submodule and operate on it in the same process (e.g., "git diff --resurse-submodules"), because immediately at this step, istate taken from a submodule would have its .repo member pointing at something that is not
the_repositoryand we will hit aBUG().And after writing all of the above, I realized that I am mostly repeating what Patric already said in the upstream, e.g.,
Quoted 6 lines
Quoted 1 line
https://lore.kernel.org/git/[email protected]/Yep---just so I'm clear, we don't currently have such an option, right? I mean, there is no --recurse-submodules for git-diff(1), and I tweaked t4060 to run "git -c core.useNanosec=true diff --submodule=diff" without any issue.
Quoted 5 lines
I do have a patch series coming up where we start to rely more on sub-repositories when recursing. The motivation behind that series is that it allows us to get rid of registering submodule object databases with the main ODB. But I just double-checked, and your series luckily doesn't break it.
Glad it worked out ;)
Quoted 17 lines
Quoted 8 lines
I would happily prove that at least none of our existing tests fail with core.useNanosec=true, but I'm not really sure how to shove configuration into every test invocation of git. Even if we could, I'm not sure we necessarily want to add another CI job for that (though that's a separate matter).
In particular, (among others) I have not received any concrete comments for
Quoted 8 lines
Quoted 4 lines
Comments welcome: I haven't touched any tests; I saw a bunch of hits for "git grep racy t" but wasn't sure how to fit this particular change in, especially since it won't be equally valid on all systems? Advice welcome.
so if there's at least a way to exercise this path on all the tests on my system (which should support it), that would probably be a good thing.
Quoted 5 lines
Yeah, I simply don't have a good answer here. It's messy, and I'm not a fan of the current direction of
repo_config_values()because nobody has yet stepped up to untangle it fromthe_repository. I gave it a quick shot at one point in time, but the result was messy at best because of how we populate it viarepo_config(git_default_config).
I took a quick look (being unfamiliar), and yeah, it does seem pretty tangled. I suppose one way to go about it would be to have repo_config() forward the repository argument through configset_iter to the config_fn_t callback? I'm a bit surprised (leaving aside how pervasive the_repository is otherwise) to see it doesn't already do that :)
Is that the approach you took? Or, where else did you feel hung up about the resulting code? Just wondering.
Quoted 8 lines
In any case, if we see that your changes interact badly with some edge cases that we don't currently have on our radar then we can still refactor the series and move the value into
struct repo_settingsinstead, as that structure works alright with different repositories.Thanks!
Patrick
This sounds reasonable to me. If nothing else, this series might become good motivation to untangle repo_config_values…
Sounds to me like we might be ready for 'next'?
Signature · 1 line
D. Ben Knoble
Re: [PATCH v5 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
In reply to a message not in the archiveQuoted 3 lines · D. Ben Knoble
Quoted 2 lines · Patrick Steinhardt
Quoted 1 line · D. Ben Knoble
On Sun, Aug 30, 2026 at 5:15 PM Junio C Hamano <[email protected]> wrote:
[snip]
Quoted 26 lines
Quoted 19 lines
Quoted 7 lines
I would happily prove that at least none of our existing tests fail with core.useNanosec=true, but I'm not really sure how to shove configuration into every test invocation of git. Even if we could, I'm not sure we necessarily want to add another CI job for that (though that's a separate matter).
In particular, (among others) I have not received any concrete comments
for
Quoted 2 lines
Quoted 2 lines
Comments welcome: I haven't touched any tests; I saw a bunch of hits
for
Quoted 7 lines
Quoted 3 lines
"git grep racy t" but wasn't sure how to fit this particular change in, especially since it won't be equally valid on all systems? Advice welcome.
so if there's at least a way to exercise this path on all the tests on my system (which should support it), that would probably be a good thing.
Quoted 6 lines
Yeah, I simply don't have a good answer here. It's messy, and I'm not a fan of the current direction of
repo_config_values()because nobody has yet stepped up to untangle it fromthe_repository. I gave it a quick shot at one point in time, but the result was messy at best because of how we populate it viarepo_config(git_default_config).
Quoted 8 lines
I took a quick look (being unfamiliar), and yeah, it does seem pretty tangled. I suppose one way to go about it would be to have
repo_config()forward the repository argument throughconfigset_iterto theconfig_fn_tcallback? I'm a bit surprised (leaving aside how pervasivethe_repositoryis otherwise) to see it doesn't already do that :)Is that the approach you took? Or, where else did you feel hung up about the resulting code? Just wondering.
Yeah, that's what I did. I don't quite remember what was awkward about it though. It might've been that callers have to be aware whether a repo is initialized, and whether it has all info to be able to read its own configuration? Or I was trying to make it auto-lazy-load or something like that, but because our config subsystem is so fragile that led to lots of weird edge cases.
Sometimes I really wonder whether that whole caching layer is even worth it. We already store the configuration as part of the configset, so caching the parsed values probably does not buy us a lot. For some very central aspects like the bareness of a repository or the location of the worktree it probably even makes sense, but for everything else... I dunno. By now I feel like it would make more sense there to find localized solutions specific to subsystems instead of having that one big global struct that has weird semantics.
Quoted 9 lines
Quoted 4 lines
In any case, if we see that your changes interact badly with some edge cases that we don't currently have on our radar then we can still refactor the series and move the value into
struct repo_settingsinstead, as that structure works alright with different repositories.This sounds reasonable to me. If nothing else, this series might become good motivation to untangle repo_config_values…
Sounds to me like we might be ready for 'next'?
Works for me.
Patrick
Re: [PATCH v5 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
In reply to Patrick SteinhardtQuoted 17 lines · Patrick Steinhardt
builtin/update-index.c static void xrmdir(const char *path) · line 133
Quoted 10 lines · D. Ben Knoble
builtin/update-index.c +1
-1@@ -130,7 +130,7 @@ static void xrmdir(const char *path)static void avoid_racy (void ) {/* -* not use if we could usleep(10) if USE_NSEC is defined. The +* not use if we could usleep(10) if core.useNanosec is defined. The Micronit: s/defined/enabled/
Quoted 4 lines
Otherwise I'm happy with this patch. It looks a lot better now that we use less preprocessor directives. Thanks!
Patrick
Thanks, queued locally. Will send out as v6 this evening (~5–6h from now) unless I hear differently from anyone. (That comment is hard to read to begin with!)
PATCH v6 1/3 meson: expose knob for xmlto relative links in manuals
In reply to D. Ben KnobleMakefile-based builds have had this knob for most of the project's life, since a479a564dc (Documentation/Makefile: allow man.base.url.for.relative.link to be set from Make, 2009-12-03).
Meson, however, hard-codes the equivalent of $prefix/$mandir, which is not really where all the HTML docs are stored in most distro builds. Plus, this value is missing a trailing slash, so links come out broken, like this in git.1:
1. Git User’s Manual
/usr/share/manuser-manual.html
Of course we can do better:
1. Change the default to match Make: use file://$(htmldir)/ (with trailing slash!) to form a local URL pointing at the HTML docs. This is safe because all current uses of link:<relative> point at HTML docs:
git grep 'link:[[:alnum:]]' Documentation | grep -ve html -e http
produces only a single result (Documentation/howto/howto-index.sh) which can be ignored. Since nothing else [*] in the normal build sets MAN_BASE_URL, this seems like the right default.
2. Provide a configurable knob, just like the Makefile, so distributions that build with Meson (like Gentoo) can decide where to make the links if they need to. Those that set htmldir probably won't need to tweak this any further, though.
[*]: Well, Git's todo branch has a script dodoc.sh to build and archive
docs for kernel.org; these docs are pulled by Homebrew installations, for example. It sets MAN_BASE_URL to "git_htmldocs", so the equivalent note on macOS + Homebrew is
1. Git User’s Manual
git-htmldocs/user-manual.html
which is not functional either, but that's a problem for downstream. In any case, users can recover the right path with "git --html-path".
---
Notes (benknoble/commits):
This patch is mostly because I noticed the link I added in a later patch didn't come out right.
I did an internet search for "MAN_BASE_URL" and got no real hits, so I'm not sure if any distros today actually use it, but that's not a proper audit in that I didn't look at any distro _code_ besides Gentoo (which, as noted, uses Meson).
Documentation/meson.build | 7 ++++++- meson_options.txt | 2 ++ 2 files changed, 8 insertions(+), 1 deletion(-)
Documentation/meson.build +6 -1
@@ -379,13 +379,18 @@ foreach manpage, category : manpages output: fs.stem(manpage) + '.xml', ) + man_base_url = 'file://' + htmldir + '/' + if get_option('man_base_url') != '' + man_base_url = get_option('man_base_url') + endif + doc_targets += custom_target( command: [ xmlto, '-m', '@INPUT0@', '-m', '@INPUT1@', '--stringparam', - 'man.base.url.for.relative.links=' + get_option('prefix') / get_option('mandir'), + 'man.base.url.for.relative.links=' + man_base_url, 'man', manpage_xml_target, '-o',
meson_options.txt +2 -0
@@ -111,6 +111,8 @@ option('default_help_format', type: 'combo', choices: ['man', 'html', 'platform' description: 'Default format used when executing git-help(1).') option('docs_backend', type: 'combo', choices: ['asciidoc', 'asciidoctor', 'auto'], value: 'auto', description: 'Which backend to use to generate documentation.') +option('man_base_url', type: 'string', value: '', + description: 'The base URL to use for relative links in manuals') # Testing. option('benchmarks', type: 'feature', value: 'auto',
Signature · 1 line
2.55.0.860.g4b6b3295ed.dirty
PATCH v6 2/3 environment: align repo_config_values_init with struct declaration
In reply to D. Ben KnobleThe order of assignments in repo_config_values_init is chaotic and hard to follow, especially with the definition of 'struct repo_config_values' to ensure all members are initialized. As new members will be added in the future, make it easier to validate changes by aligning the two.
Refactor assignment order with no behavioral changes.
--- environment.c | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-)
environment.c +12 -7
@@ -745,6 +745,7 @@ int git_default_config(const char *var, const char *value,void repo_config_values_init (struct repo_config_values *cfg) { +/* section "core" config values */ cfg->attributes_file =NULL ; cfg->excludes_file =NULL ; cfg->editor_program =NULL ;
@@ -756,20 +757,24 @@ void repo_config_values_init(struct repo_config_values *cfg) cfg->autorebase =AUTOREBASE_NEVER ; cfg->object_creation_mode =OBJECT_CREATION_MODE ; cfg->apply_sparse_checkout =0 ; - cfg->protect_hfs =PROTECT_HFS_DEFAULT ; - cfg->protect_ntfs =PROTECT_NTFS_DEFAULT ; - cfg->ignore_case =0 ; - cfg->trust_executable_bit =1 ; - cfg->has_symlinks =platform_has_symlinks (); - cfg->branch_track =BRANCH_TRACK_REMOTE ; cfg->trust_ctime =1 ; cfg->check_stat =1 ; cfg->zlib_compression_level =Z_BEST_SPEED ; cfg->pack_compression_level =Z_DEFAULT_COMPRESSION ; cfg->precomposed_unicode =-1 ;/* see probe_utf8_pathname_composition() */ cfg->core_sparse_checkout_cone =0 ; - cfg->sparse_expect_files_outside_of_patterns =0 ; cfg->warn_on_object_refname_ambiguity =1 ; + cfg->protect_hfs =PROTECT_HFS_DEFAULT ; + cfg->protect_ntfs =PROTECT_NTFS_DEFAULT ; + cfg->ignore_case =0 ; + cfg->trust_executable_bit =1 ; + cfg->has_symlinks =platform_has_symlinks (); + +/* section "sparse" config values */ + cfg->sparse_expect_files_outside_of_patterns =0 ; + +/* section "branch" config values */ + cfg->branch_track =BRANCH_TRACK_REMOTE ; }void repo_config_values_clear (struct repo_config_values *cfg)
Signature · 1 line
2.55.0.860.g4b6b3295ed.dirty
PATCH v6 3/3 core: convert build-time USE_NSEC into runtime core.useNanosec
In reply to D. Ben KnobleRacy Git problems persist today, manifesting themselves in the performance of commands like "git diff" in new worktrees [1]. We have long had a build knob "USE_NSEC" to tell Git to use in-core nanosecond precision when available, which mitigates most if not all racy issues, but most builds we know about don't use it. In part, that's because someone distributing Git can't safely enable it at compile-time if they don't know exactly what platforms their distribution will be used on.
[1]: https://lore.kernel.org/git/CALnO6CADMJSixqYvL1Yo8qKX5rWhKQ+2OoSEuPUh-yoeK9TseQ@mail.gmail.com
These days, most platforms are likely to be safe for the USE_NSEC code. Regardless, we want to give users the ability to benefit from it. This requires exposing the compile-time gated code as a runtime option.
In addition, update the Racy Git documentation and other mentions of USE_NSEC in the code.
Due to the conversion from #ifdef to runtime check, using the flag "--ignore-space-change" may be particularly helpful when viewing changes from this patch.
---
Notes (benknoble/commits):
Related benchmarks: <https://lore.kernel.org/git/CALnO6CBm4g27mWBvD9m6yL0e5YZu3M9_zcUeLZk7QwTgnxMLQA@mail.gmail.com/>
CI: <https://github.com/benknoble/git/actions/runs/32365602564>
Documentation/config/core.adoc | 7 +++++++ Documentation/technical/racy-git.adoc | 11 ++++++----- Makefile | 12 +----------- builtin/update-index.c | 2 +- compat/posix.h | 1 - configure.ac | 6 ------ environment.c | 8 ++++++++ environment.h | 1 + read-cache.c | 15 ++++++--------- statinfo.c | 14 +++++++------- 10 files changed, 37 insertions(+), 40 deletions(-)
Documentation/config/core.adoc +7 -0
@@ -118,6 +118,13 @@ core.trustctime:: crawlers and some backup systems). See linkgit:git-update-index[1]. True by default. +core.useNanosec:: + If true, use nanosecond precision for ctime and mtime + comparisions between the index and the working tree (if Git + was compiled to respect this option). + This is unsafe on some platforms; + see link:technical/racy-git.html[Racy Git]. False by default. + core.splitIndex:: If true, the split-index feature of the index will be used. See linkgit:git-update-index[1]. False by default.
Documentation/technical/racy-git.adoc +6 -5
@@ -39,8 +39,8 @@ files) from `st_mode` member, `st_mtime` and `st_ctime` timestamps, `st_uid`, `st_gid`, `st_ino`, and `st_size` members. With a `USE_STDEV` compile-time option, `st_dev` is also compared, but this is not enabled by default because this member -is not stable on network filesystems. With `USE_NSEC` -compile-time option, `st_mtim.tv_nsec` and `st_ctim.tv_nsec` +is not stable on network filesystems. With 'core.useNanosec' +config setting, `st_mtim.tv_nsec` and `st_ctim.tv_nsec` members are also compared. On Linux, this is not enabled by default because in-core timestamps can have finer granularity than on-disk timestamps, resulting in meaningless changes when an
@@ -49,9 +49,10 @@ of git://git.kernel.org/pub/scm/linux/kernel/git/tglx/history.git ([PATCH] Sync in core time granularity with filesystems, 2005-01-04). This patch is included in kernel 2.6.11 and newer, but only fixes the issue for file systems with exactly 1 ns or 1 s -resolution. Other file systems are still broken in current Linux -kernels (e.g. CEPH, CIFS, NTFS, UDF), see -https://lore.kernel.org/lkml/[email protected]/ +resolution. As of kernel 4.3, other file systems (CEPH, CIFS, NTFS, UFS, FUSE) +were fixed; see https://public-inbox.org/git/5605D88A.20104%40gmail.com/. FAT +has been fixed since 2015. The usual suspects (ext2, ext4, XFS) are known to +work, too. Racy Git --------
Makefile +1 -11
@@ -197,18 +197,11 @@ include shared.mak # Define NO_NORETURN if using buggy versions of gcc 4.6+ and profile feedback, # as the compiler can crash (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=49299) # -# Define USE_NSEC below if you want git to care about sub-second file mtimes -# and ctimes. Note that you need recent glibc (at least 2.2.4) for this. On -# Linux, kernel 2.6.11 or newer is required for reliable sub-second file times -# on file systems with exactly 1 ns or 1 s resolution. If you intend to use Git -# on other file systems (e.g. CEPH, CIFS, NTFS, UDF), don't enable USE_NSEC. See -# Documentation/technical/racy-git.adoc for details. -# # Define USE_ST_TIMESPEC if your "struct stat" uses "st_ctimespec" instead of # "st_ctim" # # Define NO_NSEC if your "struct stat" does not have "st_ctim.tv_nsec" -# available. This automatically turns USE_NSEC off. +# available. # # Define USE_STDEV below if you want git to care about the underlying device # change being considered an inode change from the update-index perspective.
@@ -1935,9 +1928,6 @@ endif ifdef NO_ST_BLOCKS_IN_STRUCT_STAT BASIC_CFLAGS += -DNO_ST_BLOCKS_IN_STRUCT_STAT endif -ifdef USE_NSEC - BASIC_CFLAGS += -DUSE_NSEC -endif ifdef USE_ST_TIMESPEC BASIC_CFLAGS += -DUSE_ST_TIMESPEC endif
builtin/update-index.c +1 -1
@@ -130,7 +130,7 @@ static void xrmdir(const char *path)static void avoid_racy (void ) {/* -* not use if we could usleep(10) if USE_NSEC is defined. The +* not use if we could usleep(10) if core.useNanosec is enabled. The * field nsec could be there, but the OS could choose to * ignore it? */
compat/posix.h +0 -1
@@ -487,7 +487,6 @@ int git_qsort_s(void *base, size_t nmemb, size_t size, } while (0)#ifdef NO_NSEC -#undef USE_NSEC#define ST_CTIME_NSEC (st) 0#define ST_MTIME_NSEC (st) 0#else
configure.ac +0 -6
@@ -351,12 +351,6 @@ GIT_PARSE_WITH(iconv)) ## --enable-FEATURE[=ARG] and --disable-FEATURE # -# Define USE_NSEC below if you want git to care about sub-second file mtimes -# and ctimes. Note that you need recent glibc (at least 2.2.4) for this, and -# it will BREAK YOUR LOCAL DIFFS! show-diff and anything using it will likely -# randomly break unless your underlying filesystem supports those sub-second -# times (my ext3 doesn't). -# # Define USE_STDEV below if you want git to care about the underlying device # change being considered an inode change from the update-index perspective.
environment.c +8 -0
@@ -571,6 +571,13 @@ int git_default_core_config(const char *var, const char *value,return 0 ; } +#ifndef NO_NSEC +if (!strcmp (var,"core.usenanosec" )) { + cfg->use_nanosec =git_config_bool (var, value); +return 0 ; + } +#endif +/* Add other config variables here and to Documentation/config.adoc. */ return platform_core_config (var, value, ctx, cb); }
@@ -769,6 +776,7 @@ void repo_config_values_init(struct repo_config_values *cfg) cfg->ignore_case =0 ; cfg->trust_executable_bit =1 ; cfg->has_symlinks =platform_has_symlinks (); + cfg->use_nanosec =0 ;/* section "sparse" config values */ cfg->sparse_expect_files_outside_of_patterns =0 ;
environment.h +1 -0
@@ -139,6 +139,7 @@ struct repo_config_values {int ignore_case;int trust_executable_bit;int has_symlinks; +int use_nanosec;/* section "sparse" config values */ int sparse_expect_files_outside_of_patterns;
read-cache.c +6 -9
@@ -354,15 +354,12 @@ static int is_racy_stat(const struct index_state *istate, const struct stat_data *sd) {return (istate->timestamp.sec && -#ifdef USE_NSEC - /* nanosecond timestamped files can also be racy! */ - (istate->timestamp.sec < sd->sd_mtime.sec || - (istate->timestamp.sec == sd->sd_mtime.sec && - istate->timestamp.nsec <= sd->sd_mtime.nsec)) -#else - istate->timestamp.sec <= sd->sd_mtime.sec -#endif - ); +/* nanosecond timestamped files can also be racy! */ + (repo_config_values (istate->repo)->use_nanosec + ? (istate->timestamp.sec < sd->sd_mtime.sec || + (istate->timestamp.sec == sd->sd_mtime.sec && + istate->timestamp.nsec <= sd->sd_mtime.nsec)) + : istate->timestamp.sec <= sd->sd_mtime.sec)); } int is_racy_timestamp(const struct index_state *istate,
statinfo.c +7 -7
@@ -72,13 +72,13 @@ int match_stat_data(const struct stat_data *sd, struct stat *st) sd->sd_ctime.sec != (unsigned int)st->st_ctime) changed |=CTIME_CHANGED ; -#ifdef USE_NSEC -if (cfg->check_stat && sd->sd_mtime.nsec !=ST_MTIME_NSEC (*st)) - changed |=MTIME_CHANGED ; -if (cfg->trust_ctime && cfg->check_stat && - sd->sd_ctime.nsec !=ST_CTIME_NSEC (*st)) - changed |=CTIME_CHANGED ; -#endif +if (cfg->use_nanosec) { +if (cfg->check_stat && sd->sd_mtime.nsec !=ST_MTIME_NSEC (*st)) + changed |=MTIME_CHANGED ; +if (cfg->trust_ctime && cfg->check_stat && + sd->sd_ctime.nsec !=ST_CTIME_NSEC (*st)) + changed |=CTIME_CHANGED ; + }if (cfg->check_stat) { if (sd->sd_uid != (unsigned int) st->st_uid ||
Signature · 1 line
2.55.0.860.g4b6b3295ed.dirty
v6 cover Convert USE_NSEC to runtime config
In reply to D. Ben KnobleTopic name: dk/use-nsec-runtime (applied)
Topic summary: Expose USE_NSEC as a runtime configuration, since build-time is too early for distributing Git [1]. As a result, common index-related options, like git-diff, are less likely to hit "racy git" problems on supported filesystems.
[1]: https://git.github.io/rev_news/2026/07/31/edition-137/
Built on master (2c78326f81 (The 11th batch, 2026-08-05)).
Changes in v5:
- improve message flow in patch 2
Changes in v4:
- fix message typo
- change #ifdef strategy: only ignore the config variable. Otherwise, use the use_nanosec member unconditionally. Also clarify that config might be ignore depending on build options in the docs.
- mention potential platform unsafety directly in config doc in addition to the link to Racy Git
Changes in v3:
- #ifdef out use_nanosec when NO_NSEC is requested
As I have heard no comments about the "Todo" lines below, which perhaps could more clearly be marked "RFC"/"RFH", I've added this line to call them out ;) and renamed them "Comments welcome"
Changes in v2:
- move Best-viewed-with trailer into message body as descriptive text.
- read core.useNanosec through struct repo instead of parsing config strings. The test suite passes locally this way, though that skipped 151 tests.
- CI run: https://github.com/benknoble/git/actions/runs/31701945211
Original cover letter:
Hi all, this series follows up on the previous racy Git/USE_NSEC conversations.
- The first patch is a mostly-unrelated documentation fix for Meson, but it came out of something I spotted while reviewing the outputs of the final (main) patch.
- The second patch is a preliminary no-op reorganization of repo_config_values_init.
- The third patch is the meat, converting USE_NSEC into core.useNanosec.
There is a small textual and semantic conflict with 'ty/repo-config-cleanups' in 'seen', since that branch removes the comments in 'struct repo_config_values' which this series adds to. (The semantic conflict is that, if we drop those comments, we should probably not add them to repo_config_values_init like I do in patch 2.)
Comments welcome: I haven't touched any tests; I saw a bunch of hits for "git grep racy t" but wasn't sure how to fit this particular change in, especially since it won't be equally valid on all systems? Advice welcome.
Comments welcome: I wonder if "useNanosec" paints us into too much of a corner; that is (slightly more abstractly), we are using *extended precision* in the index. Maybe the name and documentation should reflect that, so we aren't too committed to "nanoseconds"?
- Some platforms could offer extended precision that is not as precise as nanoseconds
- Some could offer precision _beyond_ nanoseconds
idk.
v1: <[email protected]>
v2: <[email protected]>
v3: <[email protected]>
v4: <[email protected]>
v5: <[email protected]>
[1/3] meson: expose knob for xmlto relative links in manuals [2/3] environment: align repo_config_values_init with struct declaration [3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
Documentation/config/core.adoc | 7 +++++++ Documentation/meson.build | 7 ++++++- Documentation/technical/racy-git.adoc | 11 ++++++----- Makefile | 12 +----------- builtin/update-index.c | 2 +- compat/posix.h | 1 - configure.ac | 6 ------ environment.c | 27 ++++++++++++++++++++------- environment.h | 1 + meson_options.txt | 2 ++ read-cache.c | 15 ++++++--------- statinfo.c | 14 +++++++------- 12 files changed, 57 insertions(+), 48 deletions(-)
Diff-intervalle contre v5 :
1: d612de6c2d = 1: d612de6c2d meson: expose knob for xmlto relative links in manuals 2: 12974e07d0 = 2: 12974e07d0 environment: align repo_config_values_init with struct declaration 3: 01cd487cd2 ! 3: 0a611f6140 core: convert build-time USE_NSEC into runtime core.useNanosec
@@ builtin/update-index.c: static void xrmdir(const char *path)
{
/*
- * not use if we could usleep(10) if USE_NSEC is defined. The
-+ * not use if we could usleep(10) if core.useNanosec is defined. The
++ * not use if we could usleep(10) if core.useNanosec is enabled. The
* field nsec could be there, but the OS could choose to
* ignore it?
*/
base-commit: 2c78326f810173a4f3aefd8021f1e07575412481
Signature · 1 line
2.55.0.860.g4b6b3295ed.dirty
Erm, woops
Quoted 11 lines · D. Ben Knoble
Topic name: dk/use-nsec-runtime (applied)
Topic summary: Expose
USE_NSECas a runtime configuration, since build-time is too early for distributing Git [1]. As a result, common index-related options, like git-diff, are less likely to hit "racy git" problems on supported filesystems.[1]: https://git.github.io/rev_news/2026/07/31/edition-137/
Built on master (
2c78326f81(The 11th batch, 2026-08-05)).
Changes in v6: slight comment tweak
Quoted 3 lines
Changes in v5:
- improve message flow in patch 2
[snip]
Quoted 14 lines
Diff-intervalle contre v5 :
1: d612de6c2d = 1: d612de6c2d meson: expose knob for xmlto relative links in manuals 2: 12974e07d0 = 2: 12974e07d0 environment: align repo_config_values_init with struct declaration 3: 01cd487cd2 ! 3: 0a611f6140 core: convert build-time USE_NSEC into runtime core.useNanosec@@
builtin/update-index.c: static voidxrmdir(const char *path)
{
/*
- * not use if we could usleep(10) ifUSE_NSECis defined. The
-+ * not use if we could usleep(10) if core.useNanosec is defined. The
++ * not use if we could usleep(10) if core.useNanosec is enabled. The
* field nsec could be there, but the OS could choose to
* ignore it?
*/
Quoted 3 lines
base-commit:
2c78326f810173a4f3aefd8021f1e07575412481--
2.55.0.860.g4b6b3295ed.dirty
Thanks
Signature · 1 line
D. Ben Knoble
Re: [PATCH v5 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
In reply to Patrick SteinhardtQuoted 50 lines · Patrick Steinhardt
Quoted 3 lines · D. Ben Knoble
Quoted 3 lines · Patrick Steinhardt
Quoted 1 line · D. Ben Knoble
On Sun, Aug 30, 2026 at 5:15 PM Junio C Hamano <[email protected]> wrote:
[snip]
Quoted 26 lines
Quoted 19 lines
Quoted 7 lines
I would happily prove that at least none of our existing tests fail with core.useNanosec=true, but I'm not really sure how to shove configuration into every test invocation of git. Even if we could, I'm not sure we necessarily want to add another CI job for that (though that's a separate matter).
In particular, (among others) I have not received any concrete comments
for
Quoted 2 lines
Quoted 2 lines
Comments welcome: I haven't touched any tests; I saw a bunch of hits
for
Quoted 7 lines
Quoted 3 lines
"git grep racy t" but wasn't sure how to fit this particular change in, especially since it won't be equally valid on all systems? Advice welcome.
so if there's at least a way to exercise this path on all the tests on my system (which should support it), that would probably be a good thing.
Quoted 6 lines
Yeah, I simply don't have a good answer here. It's messy, and I'm not a fan of the current direction of
repo_config_values()because nobody has yet stepped up to untangle it fromthe_repository. I gave it a quick shot at one point in time, but the result was messy at best because of how we populate it viarepo_config(git_default_config).Quoted 8 lines
I took a quick look (being unfamiliar), and yeah, it does seem pretty tangled. I suppose one way to go about it would be to have
repo_config()forward the repository argument throughconfigset_iterto theconfig_fn_tcallback? I'm a bit surprised (leaving aside how pervasivethe_repositoryis otherwise) to see it doesn't already do that :)Is that the approach you took? Or, where else did you feel hung up about the resulting code? Just wondering.
Yeah, that's what I did. I don't quite remember what was awkward about it though. It might've been that callers have to be aware whether a repo is initialized, and whether it has all info to be able to read its own configuration? Or I was trying to make it auto-lazy-load or something like that, but because our config subsystem is so fragile that led to lots of weird edge cases.
Quoted 8 lines
Sometimes I really wonder whether that whole caching layer is even worth it. We already store the configuration as part of the configset, so caching the parsed values probably does not buy us a lot. For some very central aspects like the bareness of a repository or the location of the worktree it probably even makes sense, but for everything else... I dunno. By now I feel like it would make more sense there to find localized solutions specific to subsystems instead of having that one big global struct that has weird semantics.
Interesting, yeah. I can’t say I’m too motivated to look into this further, personally, but the config system seems fairly complex…
Maybe I’ll take a tour of it one day though, depending on the next itch I scratch ;)
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
In any case, if we see that your changes interact badly with some edge cases that we don't currently have on our radar then we can still refactor the series and move the value into
struct repo_settingsinstead, as that structure works alright with different repositories.
Quoted 8 lines
Quoted 4 lines
This sounds reasonable to me. If nothing else, this series might become good motivation to untangle repo_config_values…
Sounds to me like we might be ready for 'next'?
Works for me.
Patrick
Thanks!
Re: [PATCH v6 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
In reply to D. Ben KnobleDocumentation/config/core.adoc core.trustctime:: · line 123
Quoted 3 lines · D. Ben Knoble
Documentation/config/core.adoc +3
-0+core.useNanosec:: + If true, use nanosecond precision for ctime and mtime + comparisions between the index and the working tree (if Git
comparisions?
Quoted 3 lines
Documentation/config/core.adoc +3
-0+ was compiled to respect this option). + This is unsafe on some platforms; + see link:technical/racy-git.html[Racy Git]. False by default.
Re: [PATCH v6 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
In reply to D. Ben KnobleQuoted 5 lines · D. Ben Knoble
environment.c +0
-0
environment.c int git_default_core_config(const char *var, const char *value, · line 579
Quoted 9 lines
environment.c +6
-0@@ -571,6 +571,13 @@ int git_default_core_config(const char *var, const char *value,return 0 ; } +#ifndef NO_NSEC +if (!strcmp (var,"core.usenanosec" )) { + cfg->use_nanosec =git_config_bool (var, value); +return 0 ; + } +#endif
This hunk made me wonder if we even need to do any build-time magic here at all. If your platform doesn't support nanosecond stat entries, then you're probably not going to ask for core.usenanosec in the first place. But if you do, I think the code still works; we fake the entries as "0", so they'd always yield a racy tie, just as if core.usenanosec was disabled.
I guess you might be able to get into a funny state, though, if you build two versions of Git, one with NO_NSEC and one without, on a system that actually does support nanosecond timestamps. Because IIRC even if we aren't _using_ the values, we still store them in the index. So an index generated with the regular build would store the actual nanosec stamps, which would then get a false comparison using the NO_NSEC version.
That seems quite unlikely to happen in practice, and there is a certain amount of "if it hurts, don't do that". But it's not like by dropping this #ifndef we could get rid of NO_NSEC. So it would not simplify the code overall, nor the number of build knobs that we expose to the user. So it probably is reasonable to keep it.
I haven't been following the topic closely, but from my cursory read everything else looked as I'd expect it to.
-Peff
Re: [PATCH v6 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
Quoted 11 lines · Jeff King
Quoted 8 lines · D. Ben Knoble
environment.c +0
-0@@ -571,6 +571,13 @@ int git_default_core_config(const char *var, const char *value,return 0 ; }
Quoted 13 lines
environment.c int git_default_core_config(const char *var, const char *value, · line 579
Quoted 6 lines
environment.c +6
-0+#ifndef NO_NSEC +if (!strcmp (var,"core.usenanosec" )) { + cfg->use_nanosec =git_config_bool (var, value); +return 0 ; + } +#endif This hunk made me wonder if we even need to do any build-time magic here at all. If your platform doesn't support nanosecond stat entries, then you're probably not going to ask for core.usenanosec in the first place. But if you do, I think the code still works; we fake the entries as "0", so they'd always yield a racy tie, just as if core.usenanosec was disabled.
At first I thought you meant we fake the cfg->use_nanosec as 0; it took me a moment to realize you mean that we fake the index entries as 0ns. (That is what you mean, right?)
In that case, yes, I suppose it would work. Might be confusing in a debugger to see use_nanosec set and checked, though?
Quoted 10 lines
I guess you might be able to get into a funny state, though, if you build two versions of Git, one with
NO_NSECand one without, on a system that actually does support nanosecond timestamps. Because IIRC even if we aren't _using_ the values, we still store them in the index. So an index generated with the regular build would store the actual nanosec stamps, which would then get a false comparison using theNO_NSECversion.That seems quite unlikely to happen in practice, and there is a certain amount of "if it hurts, don't do that".
Hm, yeah. I haven't thought too hard either about the interactions where you toggle core.usenanosec on and off, but giving it an initial think they seem fine. Unlike this hypothetical case, when it's off we don't look at the ns fields, so I don't think we end up with any false negatives.
And in this hypothetical, by restricting the option parsing we avoid reading the ns values on unsupported platforms, I think?
The build-time conditional _does_ mean that if your distro (e.g.) provides a NO_NSEC build, you can't access the core.usenanosec feature without compiling yourself, even if your platform supports it. But I haven't thought too hard either about what it looks like to get rid of NO_NSEC entirely, and I'm not totally sure if that's a good idea.
Quoted 9 lines
But it's not like by dropping
this #ifndef we could get rid ofNO_NSEC. So it would not simplify the code overall, nor the number of build knobs that we expose to the user. So it probably is reasonable to keep it.I haven't been following the topic closely, but from my cursory read everything else looked as I'd expect it to.
-Peff
Sounds good, thanks!
Signature · 1 line
D. Ben Knoble
Re: [PATCH v6 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
In reply to Junio C HamanoQuoted 12 lines · Junio C Hamano
Documentation/config/core.adoc core.trustctime:: · line 123
Quoted 3 lines · D. Ben Knoble
Documentation/config/core.adoc +3
-0+core.useNanosec:: + If true, use nanosecond precision for ctime and mtime + comparisions between the index and the working tree (if Gitcomparisions?
Quoted 3 lines
Documentation/config/core.adoc +3
-0+ was compiled to respect this option). + This is unsafe on some platforms; + see link:technical/racy-git.html[Racy Git]. False by default.
Ouch, good eyes. Obviously should be "comparisons"---I've amended locally but will hold onto the new version for a bit.
As this topic is not in next yet, I presume that sending a new version with the typofix is the correct thing to do. I'll wait a while today to see if any other comments trickle in.
Signature · 1 line
D. Ben Knoble
Re: [PATCH v6 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
Quoted 14 lines · D. Ben Knoble
Quoted 12 lines · Junio C Hamano
Documentation/config/core.adoc core.trustctime:: · line 123
Quoted 3 lines · D. Ben Knoble
Documentation/config/core.adoc +3
-0+core.useNanosec:: + If true, use nanosecond precision for ctime and mtime + comparisions between the index and the working tree (if Gitcomparisions?
Quoted 3 lines
Documentation/config/core.adoc +3
-0+ was compiled to respect this option). + This is unsafe on some platforms; + see link:technical/racy-git.html[Racy Git]. False by default.
Quoted 6 lines
Ouch, good eyes. Obviously should be "comparisons"---I've amended locally but will hold onto the new version for a bit.
As this topic is not in next yet, I presume that sending a new version with the typofix is the correct thing to do. I'll wait a while today to see if any other comments trickle in.
Yeah, and in the meantime I'll locallly amend what I have. If we do not hear any other issues in a few days perhaps we can do without the final reroll that way.
Thanks.
Re: [PATCH v6 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
In reply to D. Ben KnobleQuoted 10 lines · D. Ben Knoble
Quoted 6 lines
This hunk made me wonder if we even need to do any build-time magic here at all. If your platform doesn't support nanosecond stat entries, then you're probably not going to ask for core.usenanosec in the first place. But if you do, I think the code still works; we fake the entries as "0", so they'd always yield a racy tie, just as if core.usenanosec was disabled.
At first I thought you meant we fake the
cfg->use_nanosecas 0; it took me a moment to realize you mean that we fake the index entries as 0ns. (That is what you mean, right?)
Yeah, sorry to be unclear. I meant that we still have this code:
#ifdef NO_NSEC #define ST_CTIME_NSEC(st) 0 #define ST_MTIME_NSEC(st) 0
So we are free to pretend that stat nsecs exist and compare them.
Quoted 2 lines
In that case, yes, I suppose it would work. Might be confusing in a debugger to see
use_nanosecset and checked, though?
Maybe. Looking at the list of NO_NSEC flags in config.mak.uname, I suspect it's a pretty small population in the first place.
Quoted 8 lines
Hm, yeah. I haven't thought too hard either about the interactions where you toggle core.usenanosec on and off, but giving it an initial think they seem fine. Unlike this hypothetical case, when it's off we don't look at the ns fields, so I don't think we end up with any false negatives.
And in this hypothetical, by restricting the option parsing we avoid reading the ns values on unsupported platforms, I think?
I'd have to double check, but I thought that even without USE_NSEC (and thus even with your new core.usenanosec off) we still read and store the nanosecond values in the index, as long as the platform supports it (and if not, then we use those "0" fallback values).
So they are always there in the index. I guess the same odd sequence applies even today. If you:
1. Build with NO_NSEC and get "fake" 0 values in your index.
2. Re-build without NO_NSEC, and also enable USE_NSEC. Now we get _real_ values when we stat(), and compare them to the fake values in the index.
Now the index values appear up to 1-second older than they actually are. Which could maybe yield a racy miss of an update? Probably not for stat-freshness (where we want an exact match), but maybe for some index vs entry racy-git comparison. I didn't think that hard about it, because at some point this sequence is just kind of insane.
Quoted 5 lines
The build-time conditional _does_ mean that if your distro (e.g.) provides a
NO_NSECbuild, you can't access the core.usenanosec feature without compiling yourself, even if your platform supports it. But I haven't thought too hard either about what it looks like to get rid ofNO_NSECentirely, and I'm not totally sure if that's a good idea.
You couldn't access it even if core.usenanosec is supported in the build, because your fake nsec values would all be "0" and it's effectively a noop. ;)
My suggestion wasn't really about supporting more cases, but just about making the code simpler by having one less #ifdef. But like I said earlier, we can't get rid of the NO_NSEC knob entirely, so it's probably not worth worrying about the one #ifdef either way.
-Peff
Re: [PATCH v6 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
Quoted 69 lines · Jeff King
Quoted 10 lines · D. Ben Knoble
Quoted 6 lines
This hunk made me wonder if we even need to do any build-time magic here at all. If your platform doesn't support nanosecond stat entries, then you're probably not going to ask for core.usenanosec in the first place. But if you do, I think the code still works; we fake the entries as "0", so they'd always yield a racy tie, just as if core.usenanosec was disabled.
At first I thought you meant we fake the
cfg->use_nanosecas 0; it took me a moment to realize you mean that we fake the index entries as 0ns. (That is what you mean, right?)Yeah, sorry to be unclear. I meant that we still have this code:
#ifdef
NO_NSEC
#defineST_CTIME_NSEC(st)0
#defineST_MTIME_NSEC(st)0So we are free to pretend that stat nsecs exist and compare them.
Quoted 2 lines
In that case, yes, I suppose it would work. Might be confusing in a debugger to see
use_nanosecset and checked, though?Maybe. Looking at the list of
NO_NSECflags in config.mak.uname, I suspect it's a pretty small population in the first place.Quoted 8 lines
Hm, yeah. I haven't thought too hard either about the interactions where you toggle core.usenanosec on and off, but giving it an initial think they seem fine. Unlike this hypothetical case, when it's off we don't look at the ns fields, so I don't think we end up with any false negatives.
And in this hypothetical, by restricting the option parsing we avoid reading the ns values on unsupported platforms, I think?
I'd have to double check, but I thought that even without
USE_NSEC(and thus even with your new core.usenanosec off) we still read and store the nanosecond values in the index, as long as the platform supports it (and if not, then we use those "0" fallback values).So they are always there in the index. I guess the same odd sequence applies even today. If you:
1. Build with
NO_NSECand get "fake" 0 values in your index.2. Re-build without
NO_NSEC, and also enableUSE_NSEC. Now we get _real_ values when westat(), and compare them to the fake values in the index.Now the index values appear up to 1-second older than they actually are. Which could maybe yield a racy miss of an update? Probably not for stat-freshness (where we want an exact match), but maybe for some index vs entry racy-git comparison. I didn't think that hard about it, because at some point this sequence is just kind of insane.
Quoted 5 lines
The build-time conditional _does_ mean that if your distro (e.g.) provides a
NO_NSECbuild, you can't access the core.usenanosec feature without compiling yourself, even if your platform supports it. But I haven't thought too hard either about what it looks like to get rid ofNO_NSECentirely, and I'm not totally sure if that's a good idea.You couldn't access it even if core.usenanosec is supported in the build, because your fake nsec values would all be "0" and it's effectively a noop. ;)
Quoted 6 lines
My suggestion wasn't really about supporting more cases, but just about making the code simpler by having one less #ifdef. But like I said earlier, we can't get rid of the
NO_NSECknob entirely, so it's probably not worth worrying about the one #ifdef either way.-Peff
Right on. Always good to find myself nodding along with your explanations :)
Re: [PATCH v6 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
Quoted 8 lines · Ben Knoble
Quoted 6 lines
My suggestion wasn't really about supporting more cases, but just about making the code simpler by having one less #ifdef. But like I said earlier, we can't get rid of the
NO_NSECknob entirely, so it's probably not worth worrying about the one #ifdef either way.-Peff
Right on. Always good to find myself nodding along with your explanations :)
OK. So will we see a hopefully small and final reroll that takes advantage of the fact that ST_XTIME_NSEC(st) would usefully hide the NO_NSEC build-time differences?
I still am worried that something that sits this deep in the callchain can easily BUG() when working on a repository that is not the_repository due to the use of repo_config_values(), and we might be better off adopting safe default when istate->repo is different from the_repository, but other than that, I think the series is in great shape.
Thanks.
Re: [PATCH v6 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
Quoted 10 lines · Junio C Hamano
Quoted 5 lines · Ben Knoble
Quoted 5 lines
My suggestion wasn't really about supporting more cases, but just about making the code simpler by having one less #ifdef. But like I said earlier, we can't get rid of the
NO_NSECknob entirely, so it's probably not worth worrying about the one #ifdef either way.
Quoted 7 lines
Quoted 3 lines
Quoted 1 line
-Peff
Right on. Always good to find myself nodding along with your explanations :)
OK. So will we see a hopefully small and final reroll that takes advantage of the fact that
ST_XTIME_NSEC(st)would usefully hide theNO_NSECbuild-time differences?
Ah, no: I wasn’t planning on removing this ifdef, as I think Peff and I agree that it’s not worth the hassle (at least for now).
Quoted 8 lines
I still am worried that something that sits this deep in the callchain can easily
BUG()when working on a repository that is notthe_repositorydue to the use ofrepo_config_values(), and we might be better off adopting safe default whenistate->repois different fromthe_repository, but other than that, I think the series is in great shape.Thanks.
Yea. See previous messages re: convincing the test apparatus to set this globally. If I could run it that way at least locally, it would go a little ways towards scaring those BUGs out into the light.
Absent suggestions, though, I’m afraid my time is limited to explore the guts of yet another subsystem ;)
Re: [PATCH v6 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
Quoted 8 lines · Ben Knoble
Quoted 8 lines
I still am worried that something that sits this deep in the callchain can easily
BUG()when working on a repository that is notthe_repositorydue to the use ofrepo_config_values(), and we might be better off adopting safe default whenistate->repois different fromthe_repository, but other than that, I think the series is in great shape.Thanks.
[administrivia: wrap overly long lines]
Quoted 4 lines
Yea. See previous messages re: convincing the test apparatus to set this globally. If I could run it that way at least locally, it would go a little ways towards scaring those BUGs out into the light.
I am not worried too much about the current code. I am more worried about how much this will hinder future development of new features, e.g., diff or status recursively going into submodules without spawning subprocesses, which is done for grep already. Testing and seeing 'git grep --recurse-submodule' not hitting a BUG() does not assure us all that much, as I do not think it needs to deal with racily clean entries any specially.
Thanks.
Re: [PATCH v6 3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
Quoted 19 lines · Junio C Hamano
Quoted 7 lines · Ben Knoble
Quoted 7 lines
I still am worried that something that sits this deep in the callchain can easily
BUG()when working on a repository that is notthe_repositorydue to the use ofrepo_config_values(), and we might be better off adopting safe default whenistate->repois different fromthe_repository, but other than that, I think the series is in great shape.Quoted 6 lines
Quoted 1 line
Thanks.
Yea. See previous messages re: convincing the test apparatus to set this globally. If I could run it that way at least locally, it would go a little ways towards scaring those BUGs out into the light.
Quoted 4 lines
I am not worried too much about the current code. I am more worried about how much this will hinder future development of new features, e.g., diff or status recursively going into submodules without spawning subprocesses, which is done for grep already.
Sure. Some kind of safe default could alleviate that. But seeing recent work in these areas convinces me that we should use this as impetus to lift the restriction, and I worry that papering over it will remove that impetus. Still, if a later series needs such a band-aid, I suppose it can add the safe fallback. And that’s where testing would be nice for automatic feedback on new such interactions.
Quoted 4 lines
Testing and
seeing 'git grep --recurse-submodule' not hitting aBUG()does not assure us all that much, as I do not think it needs to deal with racily clean entries any specially.
A prior reply of mine to Patrick specifically mentioned diff with submodules, I believe. But I agree that positive evidence is probably better than negative evidence.
All-in-all, I’m not inclined to change the shape of this series at the present point in this discussion, but if you (or others) feel strongly about this « safe default » being a requirement, I will find some time eventually.
v7 cover Convert USE_NSEC to runtime config
In reply to D. Ben KnobleTopic name: dk/use-nsec-runtime (applied)
Topic summary: Expose USE_NSEC as a runtime configuration, since build-time is too early for distributing Git [1]. As a result, common index-related options, like git-diff, are less likely to hit "racy git" problems on supported filesystems.
[1]: https://git.github.io/rev_news/2026/07/31/edition-137/
Built on master (2c78326f81 (The 11th batch, 2026-08-05)).
Changes in v7:
• documentation typofix
• I opted not to finagle #ifdefs more [2] nor to add a "safe default when istate->repo is different from the_repository" (replies to [2])
[2]: https://lore.kernel.org/git/[email protected]/
Changes in v6:
• comment wording tweak
Changes in v5:
• improve message flow in patch 2
Changes in v4:
• fix message typo
• change #ifdef strategy: only ignore the config variable. Otherwise, use the use_nanosec member unconditionally. Also clarify that config might be ignore depending on build options in the docs.
• mention potential platform unsafety directly in config doc in addition to the link to Racy Git
Changes in v3:
• #ifdef out use_nanosec when NO_NSEC is requested
As I have heard no comments about the "Todo" lines below, which perhaps could more clearly be marked "RFC"/"RFH", I've added this line to call them out ;) and renamed them "Comments welcome"
Changes in v2:
• move Best-viewed-with trailer into message body as descriptive text.
• read core.useNanosec through struct repo instead of parsing config strings. The test suite passes locally this way, though that skipped 151 tests.
• CI run: https://github.com/benknoble/git/actions/runs/31701945211
Original cover letter:
Hi all, this series follows up on the previous racy Git/USE_NSEC conversations.
• The first patch is a mostly-unrelated documentation fix for Meson, but it came out of something I spotted while reviewing the outputs of the final (main) patch.
• The second patch is a preliminary no-op reorganization of repo_config_values_init.
• The third patch is the meat, converting USE_NSEC into core.useNanosec.
There is a small textual and semantic conflict with 'ty/repo-config-cleanups' in 'seen', since that branch removes the comments in 'struct repo_config_values' which this series adds to. (The semantic conflict is that, if we drop those comments, we should probably not add them to repo_config_values_init like I do in patch 2.)
Comments welcome: I haven't touched any tests; I saw a bunch of hits for "git grep racy t" but wasn't sure how to fit this particular change in, especially since it won't be equally valid on all systems? Advice welcome.
Comments welcome: I wonder if "useNanosec" paints us into too much of a corner; that is (slightly more abstractly), we are using *extended precision* in the index. Maybe the name and documentation should reflect that, so we aren't too committed to "nanoseconds"?
• Some platforms could offer extended precision that is not as precise as nanoseconds
• Some could offer precision _beyond_ nanoseconds
idk.
v1: <[email protected]>
v2: <[email protected]>
v3: <[email protected]>
v4: <[email protected]>
v5: <[email protected]>
v6: <[email protected]>
[1/3] meson: expose knob for xmlto relative links in manuals [2/3] environment: align repo_config_values_init with struct declaration [3/3] core: convert build-time USE_NSEC into runtime core.useNanosec
Documentation/config/core.adoc | 7 +++++++ Documentation/meson.build | 7 ++++++- Documentation/technical/racy-git.adoc | 11 ++++++----- Makefile | 12 +----------- builtin/update-index.c | 2 +- compat/posix.h | 1 - configure.ac | 6 ------ environment.c | 27 ++++++++++++++++++++------- environment.h | 1 + meson_options.txt | 2 ++ read-cache.c | 15 ++++++--------- statinfo.c | 14 +++++++------- 12 files changed, 57 insertions(+), 48 deletions(-)
Diff-intervalle contre v6 :
1: d612de6c2d = 1: d612de6c2d meson: expose knob for xmlto relative links in manuals 2: 12974e07d0 = 2: 12974e07d0 environment: align repo_config_values_init with struct declaration 3: 0a611f6140 ! 3: d983e2f0a5 core: convert build-time USE_NSEC into runtime core.useNanosec
@@ Documentation/config/core.adoc: core.trustctime::
+core.useNanosec::
+ If true, use nanosecond precision for ctime and mtime
-+ comparisions between the index and the working tree (if Git
++ comparisons between the index and the working tree (if Git
+ was compiled to respect this option).
+ This is unsafe on some platforms;
+ see link:technical/racy-git.html[Racy Git]. False by default.
base-commit: 2c78326f810173a4f3aefd8021f1e07575412481
Signature · 1 line
2.55.0.1003.g10538fe699.dirty
PATCH v7 1/3 meson: expose knob for xmlto relative links in manuals
Makefile-based builds have had this knob for most of the project's life, since a479a564dc (Documentation/Makefile: allow man.base.url.for.relative.link to be set from Make, 2009-12-03).
Meson, however, hard-codes the equivalent of $prefix/$mandir, which is not really where all the HTML docs are stored in most distro builds. Plus, this value is missing a trailing slash, so links come out broken, like this in git.1:
1. Git User’s Manual
/usr/share/manuser-manual.html
Of course we can do better:
1. Change the default to match Make: use file://$(htmldir)/ (with trailing slash!) to form a local URL pointing at the HTML docs. This is safe because all current uses of link:<relative> point at HTML docs:
git grep 'link:[[:alnum:]]' Documentation | grep -ve html -e http
produces only a single result (Documentation/howto/howto-index.sh) which can be ignored. Since nothing else [*] in the normal build sets MAN_BASE_URL, this seems like the right default.
2. Provide a configurable knob, just like the Makefile, so distributions that build with Meson (like Gentoo) can decide where to make the links if they need to. Those that set htmldir probably won't need to tweak this any further, though.
[*]: Well, Git's todo branch has a script dodoc.sh to build and archive
docs for kernel.org; these docs are pulled by Homebrew installations, for example. It sets MAN_BASE_URL to "git_htmldocs", so the equivalent note on macOS + Homebrew is
1. Git User’s Manual
git-htmldocs/user-manual.html
which is not functional either, but that's a problem for downstream. In any case, users can recover the right path with "git --html-path".
---
Notes (benknoble/commits):
This patch is mostly because I noticed the link I added in a later patch didn't come out right.
I did an internet search for "MAN_BASE_URL" and got no real hits, so I'm not sure if any distros today actually use it, but that's not a proper audit in that I didn't look at any distro _code_ besides Gentoo (which, as noted, uses Meson).
Documentation/meson.build | 7 ++++++- meson_options.txt | 2 ++ 2 files changed, 8 insertions(+), 1 deletion(-)
Documentation/meson.build +6 -1
@@ -379,13 +379,18 @@ foreach manpage, category : manpages output: fs.stem(manpage) + '.xml', ) + man_base_url = 'file://' + htmldir + '/' + if get_option('man_base_url') != '' + man_base_url = get_option('man_base_url') + endif + doc_targets += custom_target( command: [ xmlto, '-m', '@INPUT0@', '-m', '@INPUT1@', '--stringparam', - 'man.base.url.for.relative.links=' + get_option('prefix') / get_option('mandir'), + 'man.base.url.for.relative.links=' + man_base_url, 'man', manpage_xml_target, '-o',
meson_options.txt +2 -0
@@ -111,6 +111,8 @@ option('default_help_format', type: 'combo', choices: ['man', 'html', 'platform' description: 'Default format used when executing git-help(1).') option('docs_backend', type: 'combo', choices: ['asciidoc', 'asciidoctor', 'auto'], value: 'auto', description: 'Which backend to use to generate documentation.') +option('man_base_url', type: 'string', value: '', + description: 'The base URL to use for relative links in manuals') # Testing. option('benchmarks', type: 'feature', value: 'auto',
Signature · 1 line
2.55.0.1003.g10538fe699.dirty
PATCH v7 2/3 environment: align repo_config_values_init with struct declaration
In reply to D. Ben KnobleThe order of assignments in repo_config_values_init is chaotic and hard to follow, especially with the definition of 'struct repo_config_values' to ensure all members are initialized. As new members will be added in the future, make it easier to validate changes by aligning the two.
Refactor assignment order with no behavioral changes.
--- environment.c | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-)
environment.c +12 -7
@@ -745,6 +745,7 @@ int git_default_config(const char *var, const char *value,void repo_config_values_init (struct repo_config_values *cfg) { +/* section "core" config values */ cfg->attributes_file =NULL ; cfg->excludes_file =NULL ; cfg->editor_program =NULL ;
@@ -756,20 +757,24 @@ void repo_config_values_init(struct repo_config_values *cfg) cfg->autorebase =AUTOREBASE_NEVER ; cfg->object_creation_mode =OBJECT_CREATION_MODE ; cfg->apply_sparse_checkout =0 ; - cfg->protect_hfs =PROTECT_HFS_DEFAULT ; - cfg->protect_ntfs =PROTECT_NTFS_DEFAULT ; - cfg->ignore_case =0 ; - cfg->trust_executable_bit =1 ; - cfg->has_symlinks =platform_has_symlinks (); - cfg->branch_track =BRANCH_TRACK_REMOTE ; cfg->trust_ctime =1 ; cfg->check_stat =1 ; cfg->zlib_compression_level =Z_BEST_SPEED ; cfg->pack_compression_level =Z_DEFAULT_COMPRESSION ; cfg->precomposed_unicode =-1 ;/* see probe_utf8_pathname_composition() */ cfg->core_sparse_checkout_cone =0 ; - cfg->sparse_expect_files_outside_of_patterns =0 ; cfg->warn_on_object_refname_ambiguity =1 ; + cfg->protect_hfs =PROTECT_HFS_DEFAULT ; + cfg->protect_ntfs =PROTECT_NTFS_DEFAULT ; + cfg->ignore_case =0 ; + cfg->trust_executable_bit =1 ; + cfg->has_symlinks =platform_has_symlinks (); + +/* section "sparse" config values */ + cfg->sparse_expect_files_outside_of_patterns =0 ; + +/* section "branch" config values */ + cfg->branch_track =BRANCH_TRACK_REMOTE ; }void repo_config_values_clear (struct repo_config_values *cfg)
Signature · 1 line
2.55.0.1003.g10538fe699.dirty
PATCH v7 3/3 core: convert build-time USE_NSEC into runtime core.useNanosec
In reply to D. Ben KnobleRacy Git problems persist today, manifesting themselves in the performance of commands like "git diff" in new worktrees [1]. We have long had a build knob "USE_NSEC" to tell Git to use in-core nanosecond precision when available, which mitigates most if not all racy issues, but most builds we know about don't use it. In part, that's because someone distributing Git can't safely enable it at compile-time if they don't know exactly what platforms their distribution will be used on.
[1]: https://lore.kernel.org/git/CALnO6CADMJSixqYvL1Yo8qKX5rWhKQ+2OoSEuPUh-yoeK9TseQ@mail.gmail.com
These days, most platforms are likely to be safe for the USE_NSEC code. Regardless, we want to give users the ability to benefit from it. This requires exposing the compile-time gated code as a runtime option.
In addition, update the Racy Git documentation and other mentions of USE_NSEC in the code.
Due to the conversion from #ifdef to runtime check, using the flag "--ignore-space-change" may be particularly helpful when viewing changes from this patch.
---
Notes (benknoble/commits):
Related benchmarks: <https://lore.kernel.org/git/CALnO6CBm4g27mWBvD9m6yL0e5YZu3M9_zcUeLZk7QwTgnxMLQA@mail.gmail.com/>
CI: <https://github.com/benknoble/git/actions/runs/32365602564>
Documentation/config/core.adoc | 7 +++++++ Documentation/technical/racy-git.adoc | 11 ++++++----- Makefile | 12 +----------- builtin/update-index.c | 2 +- compat/posix.h | 1 - configure.ac | 6 ------ environment.c | 8 ++++++++ environment.h | 1 + read-cache.c | 15 ++++++--------- statinfo.c | 14 +++++++------- 10 files changed, 37 insertions(+), 40 deletions(-)
Documentation/config/core.adoc +7 -0
@@ -118,6 +118,13 @@ core.trustctime:: crawlers and some backup systems). See linkgit:git-update-index[1]. True by default. +core.useNanosec:: + If true, use nanosecond precision for ctime and mtime + comparisons between the index and the working tree (if Git + was compiled to respect this option). + This is unsafe on some platforms; + see link:technical/racy-git.html[Racy Git]. False by default. + core.splitIndex:: If true, the split-index feature of the index will be used. See linkgit:git-update-index[1]. False by default.
Documentation/technical/racy-git.adoc +6 -5
@@ -39,8 +39,8 @@ files) from `st_mode` member, `st_mtime` and `st_ctime` timestamps, `st_uid`, `st_gid`, `st_ino`, and `st_size` members. With a `USE_STDEV` compile-time option, `st_dev` is also compared, but this is not enabled by default because this member -is not stable on network filesystems. With `USE_NSEC` -compile-time option, `st_mtim.tv_nsec` and `st_ctim.tv_nsec` +is not stable on network filesystems. With 'core.useNanosec' +config setting, `st_mtim.tv_nsec` and `st_ctim.tv_nsec` members are also compared. On Linux, this is not enabled by default because in-core timestamps can have finer granularity than on-disk timestamps, resulting in meaningless changes when an
@@ -49,9 +49,10 @@ of git://git.kernel.org/pub/scm/linux/kernel/git/tglx/history.git ([PATCH] Sync in core time granularity with filesystems, 2005-01-04). This patch is included in kernel 2.6.11 and newer, but only fixes the issue for file systems with exactly 1 ns or 1 s -resolution. Other file systems are still broken in current Linux -kernels (e.g. CEPH, CIFS, NTFS, UDF), see -https://lore.kernel.org/lkml/[email protected]/ +resolution. As of kernel 4.3, other file systems (CEPH, CIFS, NTFS, UFS, FUSE) +were fixed; see https://public-inbox.org/git/5605D88A.20104%40gmail.com/. FAT +has been fixed since 2015. The usual suspects (ext2, ext4, XFS) are known to +work, too. Racy Git --------
Makefile +1 -11
@@ -197,18 +197,11 @@ include shared.mak # Define NO_NORETURN if using buggy versions of gcc 4.6+ and profile feedback, # as the compiler can crash (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=49299) # -# Define USE_NSEC below if you want git to care about sub-second file mtimes -# and ctimes. Note that you need recent glibc (at least 2.2.4) for this. On -# Linux, kernel 2.6.11 or newer is required for reliable sub-second file times -# on file systems with exactly 1 ns or 1 s resolution. If you intend to use Git -# on other file systems (e.g. CEPH, CIFS, NTFS, UDF), don't enable USE_NSEC. See -# Documentation/technical/racy-git.adoc for details. -# # Define USE_ST_TIMESPEC if your "struct stat" uses "st_ctimespec" instead of # "st_ctim" # # Define NO_NSEC if your "struct stat" does not have "st_ctim.tv_nsec" -# available. This automatically turns USE_NSEC off. +# available. # # Define USE_STDEV below if you want git to care about the underlying device # change being considered an inode change from the update-index perspective.
@@ -1935,9 +1928,6 @@ endif ifdef NO_ST_BLOCKS_IN_STRUCT_STAT BASIC_CFLAGS += -DNO_ST_BLOCKS_IN_STRUCT_STAT endif -ifdef USE_NSEC - BASIC_CFLAGS += -DUSE_NSEC -endif ifdef USE_ST_TIMESPEC BASIC_CFLAGS += -DUSE_ST_TIMESPEC endif
builtin/update-index.c +1 -1
@@ -130,7 +130,7 @@ static void xrmdir(const char *path)static void avoid_racy (void ) {/* -* not use if we could usleep(10) if USE_NSEC is defined. The +* not use if we could usleep(10) if core.useNanosec is enabled. The * field nsec could be there, but the OS could choose to * ignore it? */
compat/posix.h +0 -1
@@ -487,7 +487,6 @@ int git_qsort_s(void *base, size_t nmemb, size_t size, } while (0)#ifdef NO_NSEC -#undef USE_NSEC#define ST_CTIME_NSEC (st) 0#define ST_MTIME_NSEC (st) 0#else
configure.ac +0 -6
@@ -351,12 +351,6 @@ GIT_PARSE_WITH(iconv)) ## --enable-FEATURE[=ARG] and --disable-FEATURE # -# Define USE_NSEC below if you want git to care about sub-second file mtimes -# and ctimes. Note that you need recent glibc (at least 2.2.4) for this, and -# it will BREAK YOUR LOCAL DIFFS! show-diff and anything using it will likely -# randomly break unless your underlying filesystem supports those sub-second -# times (my ext3 doesn't). -# # Define USE_STDEV below if you want git to care about the underlying device # change being considered an inode change from the update-index perspective.
environment.c +8 -0
@@ -571,6 +571,13 @@ int git_default_core_config(const char *var, const char *value,return 0 ; } +#ifndef NO_NSEC +if (!strcmp (var,"core.usenanosec" )) { + cfg->use_nanosec =git_config_bool (var, value); +return 0 ; + } +#endif +/* Add other config variables here and to Documentation/config.adoc. */ return platform_core_config (var, value, ctx, cb); }
@@ -769,6 +776,7 @@ void repo_config_values_init(struct repo_config_values *cfg) cfg->ignore_case =0 ; cfg->trust_executable_bit =1 ; cfg->has_symlinks =platform_has_symlinks (); + cfg->use_nanosec =0 ;/* section "sparse" config values */ cfg->sparse_expect_files_outside_of_patterns =0 ;
environment.h +1 -0
@@ -139,6 +139,7 @@ struct repo_config_values {int ignore_case;int trust_executable_bit;int has_symlinks; +int use_nanosec;/* section "sparse" config values */ int sparse_expect_files_outside_of_patterns;
read-cache.c +6 -9
@@ -354,15 +354,12 @@ static int is_racy_stat(const struct index_state *istate, const struct stat_data *sd) {return (istate->timestamp.sec && -#ifdef USE_NSEC - /* nanosecond timestamped files can also be racy! */ - (istate->timestamp.sec < sd->sd_mtime.sec || - (istate->timestamp.sec == sd->sd_mtime.sec && - istate->timestamp.nsec <= sd->sd_mtime.nsec)) -#else - istate->timestamp.sec <= sd->sd_mtime.sec -#endif - ); +/* nanosecond timestamped files can also be racy! */ + (repo_config_values (istate->repo)->use_nanosec + ? (istate->timestamp.sec < sd->sd_mtime.sec || + (istate->timestamp.sec == sd->sd_mtime.sec && + istate->timestamp.nsec <= sd->sd_mtime.nsec)) + : istate->timestamp.sec <= sd->sd_mtime.sec)); } int is_racy_timestamp(const struct index_state *istate,
statinfo.c +7 -7
@@ -72,13 +72,13 @@ int match_stat_data(const struct stat_data *sd, struct stat *st) sd->sd_ctime.sec != (unsigned int)st->st_ctime) changed |=CTIME_CHANGED ; -#ifdef USE_NSEC -if (cfg->check_stat && sd->sd_mtime.nsec !=ST_MTIME_NSEC (*st)) - changed |=MTIME_CHANGED ; -if (cfg->trust_ctime && cfg->check_stat && - sd->sd_ctime.nsec !=ST_CTIME_NSEC (*st)) - changed |=CTIME_CHANGED ; -#endif +if (cfg->use_nanosec) { +if (cfg->check_stat && sd->sd_mtime.nsec !=ST_MTIME_NSEC (*st)) + changed |=MTIME_CHANGED ; +if (cfg->trust_ctime && cfg->check_stat && + sd->sd_ctime.nsec !=ST_CTIME_NSEC (*st)) + changed |=CTIME_CHANGED ; + }if (cfg->check_stat) { if (sd->sd_uid != (unsigned int) st->st_uid ||
Signature · 1 line
2.55.0.1003.g10538fe699.dirty
Junio C Hamano
AppliedQuoted 9 lines · D. Ben Knoble
Topic name: dk/use-nsec-runtime (applied)
Topic summary: Expose
USE_NSECas a runtime configuration, since build-time is too early for distributing Git [1]. As a result, common index-related options, like git-diff, are less likely to hit "racy git" problems on supported filesystems.
Quoted 7 lines
Built on master (
2c78326f81(The 11th batch, 2026-08-05)).Changes in v7:
• documentation typofix
• I opted not to finagle #ifdefs more [2] nor to add a "safe default whenistate->repois different fromthe_repository" (replies to [2])
These three match exactly what I had when I queued v6, which confused me quite a lot. It turns out that I have already locally reworded the log message of [v6 3/3] "comparisions" ;-).
Queued. Thanks.
Quoted 22 lines · Junio C Hamano
Quoted 9 lines · "D. Ben Knoble
Topic name: dk/use-nsec-runtime (applied)
Topic summary: Expose
USE_NSECas a runtime configuration, since build-time is too early for distributing Git [1]. As a result, common index-related options, like git-diff, are less likely to hit "racy git" problems on supported filesystems.Quoted 7 lines
Built on master (
2c78326f81(The 11th batch, 2026-08-05)).Changes in v7:
• documentation typofix
• I opted not to finagle #ifdefs more [2] nor to add a "safe default whenistate->repois different fromthe_repository" (replies to [2])
Quoted 5 lines
These three match exactly what I had when I queued v6, which confused me quite a lot. It turns out that I have already locally reworded the log message of [v6 3/3] "comparisions" ;-).
Queued. Thanks.
Right, I should have mentioned you already had this as far as I knew. This was primarily a nudge for the (in my view?) stale What’s Cooking status :)