some few more 'git checkout' improvements
cover some few more 'git checkout' improvements
- one bugfix (I hope)
- and one patch to make life easier when using the USE_NSEC define
These 2 patches is based on 'next' because patch 2/2 is based on one previous patch, which is only in 'next' for the moment.
Kjetil Barvik (2):
stat_tracking_info(): only count real commits better introduction of GIT with USE_NSEC defined
read-cache.c | 23 +---------------------- remote.c | 3 ++- unpack-trees.c | 2 -- 3 files changed, 3 insertions(+), 25 deletions(-)
PATCH 1/2 stat_tracking_info(): only count real commits
stat_tracking_info() in remote.c is used to collect the statistics to be able to say (for instance) from the output of "git checkout':
Your branch and 'foo' have diverged,
and have X and Y different commit(s) each, respectively.
Currently X and Y also includes the count of merges. This patch excludes the merges from being counted.
---
I hope this is a correct fix, and that it is realy a bugfix.
~~
By the way, I have noticed that when the Y number above is large (for instance for a branch I have where Y is ~ 600), then the function get_merge_bases_many() and in particular merge_bases_many() in commit.c will take a noticable amount of user time (aprox 0.4 seconds). So if the chekcout results in that few files need to be updated (< 10), this will sometimes acount for much of the total time needed for the 'git checkout' command.
It seems that even though only max 4000 or so unique commits is touched (when Y ~ 600), each commit is touched over 250 times, for instancce by the insert_by_date() function inside the while-loop in merge_bases_many().
Do someone think it is possible to have a better algorithm here? Maybe O(Nlog(n)) or better? Does someone has a hint about how to make it better?
remote.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-)
remote.c +2 -1
@@ -1310,9 +1310,10 @@ int stat_tracking_info(struct branch *branch, int *num_ours, int *num_theirs)if (theirs == ours)return 0 ; -/* Run "rev-list --left-right ours...theirs" internally... */ +/* Run "rev-list --no-merges --left-right ours...theirs" internally... */ rev_argc =0 ; rev_argv[rev_argc++] =NULL ; + rev_argv[rev_argc++] ="--no-merges" ; rev_argv[rev_argc++] ="--left-right" ; rev_argv[rev_argc++] = symmetric; rev_argv[rev_argc++] ="--" ;
Signature · 1 line
1.6.1.GIT
PATCH 2/2 better introduction of GIT with USE_NSEC defined
In reply to Kjetil BarvikChange the source code such that when USE_NSEC is not defined, possible nanosecond timestamps will still be saved in the index file, but not used inside if-test's, and will therefore not affect the outcome of GIT commands, other than the saved nanosecond timestamps in the index file.
This will make it easier to use a system with 2 versions of GIT, one with and one without USE_NSEC defined.
--- read-cache.c | 23 +---------------------- unpack-trees.c | 2 -- 2 files changed, 1 insertions(+), 24 deletions(-)
read-cache.c +1 -22
@@ -69,13 +69,8 @@ void fill_stat_cache_info(struct cache_entry *ce, struct stat *st) { ce->ce_ctime.sec = (unsigned int )st->st_ctime; ce->ce_mtime.sec = (unsigned int )st->st_mtime; -#ifdef USE_NSEC ce->ce_ctime.nsec = (unsigned int )st->st_ctim.tv_nsec; ce->ce_mtime.nsec = (unsigned int )st->st_mtim.tv_nsec; -#else - ce->ce_ctime.nsec =0 ; - ce->ce_mtime.nsec =0 ; -#endif ce->ce_dev = st->st_dev; ce->ce_ino = st->st_ino; ce->ce_uid = st->st_uid;
@@ -1183,13 +1178,8 @@ static void convert_from_disk(struct ondisk_cache_entry *ondisk, struct cache_en ce->ce_ctime.sec =ntohl (ondisk->ctime.sec); ce->ce_mtime.sec =ntohl (ondisk->mtime.sec); -#ifdef USE_NSEC ce->ce_ctime.nsec =ntohl (ondisk->ctime.nsec); ce->ce_mtime.nsec =ntohl (ondisk->mtime.nsec); -#else - ce->ce_ctime.nsec =0 ; - ce->ce_mtime.nsec =0 ; -#endif ce->ce_dev =ntohl (ondisk->dev); ce->ce_ino =ntohl (ondisk->ino); ce->ce_mode =ntohl (ondisk->mode);
@@ -1308,12 +1298,8 @@ int read_index_from(struct index_state *istate, const char *path) src_offset +=ondisk_ce_size (ce); dst_offset +=ce_size (ce); } - istate->timestamp.sec = st.st_mtime; -#ifdef USE_NSEC + istate->timestamp.sec = (unsigned int )st.st_mtime; istate->timestamp.nsec = (unsigned int )st.st_mtim.tv_nsec; -#else - istate->timestamp.nsec =0 ; -#endif while (src_offset <= mmap_size -20 -8 ) {/* After an array of active_nr index entries,
@@ -1500,13 +1486,8 @@ static int ce_write_entry(git_SHA_CTX *c, int fd, struct cache_entry *ce) ondisk->ctime.sec =htonl (ce->ce_ctime.sec); ondisk->mtime.sec =htonl (ce->ce_mtime.sec); -#ifdef USE_NSEC ondisk->ctime.nsec =htonl (ce->ce_ctime.nsec); ondisk->mtime.nsec =htonl (ce->ce_mtime.nsec); -#else - ondisk->ctime.nsec =0 ; - ondisk->mtime.nsec =0 ; -#endif ondisk->dev =htonl (ce->ce_dev); ondisk->ino =htonl (ce->ce_ino); ondisk->mode =htonl (ce->ce_mode);
@@ -1583,9 +1564,7 @@ int write_index(struct index_state *istate, int newfd)if (ce_flush (&c, newfd) ||fstat (newfd, &st))return -1 ; istate->timestamp.sec = (unsigned int )st.st_ctime; -#ifdef USE_NSEC istate->timestamp.nsec = (unsigned int )st.st_ctim.tv_nsec; -#endif return 0 ; }
unpack-trees.c +0 -2
@@ -362,9 +362,7 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options o->result.initialized =1 ;if (o->src_index) { o->result.timestamp.sec = o->src_index->timestamp.sec; -#ifdef USE_NSEC o->result.timestamp.nsec = o->src_index->timestamp.nsec; -#endif } o->merge_size = len;
Signature · 1 line
1.6.1.GIT
Re: [PATCH 2/2] better introduction of GIT with USE_NSEC defined
read-cache.c int read_index_from(struct index_state *istate, const char *path) · line 1316
Quoted 7 lines · Kjetil Barvik
read-cache.c +1
-5- istate->timestamp.sec = st.st_mtime; -#ifdef USE_NSEC + istate->timestamp.sec = (unsigned int )st.st_mtime; istate->timestamp.nsec = (unsigned int )st.st_mtim.tv_nsec; -#else - istate->timestamp.nsec =0 ; -#endif
Doesn't this break on systems where st_mtime is time_t and st_mtim does not exist?
-- Hannes
Re: [PATCH 2/2] better introduction of GIT with USE_NSEC defined
Quoted 11 lines · Johannes Sixt
Review
Quoted 7 lines · Kjetil Barvik
(quoted diff) +1
-5- istate->timestamp.sec = st.st_mtime; -#ifdef USE_NSEC + istate->timestamp.sec = (unsigned int)st.st_mtime; istate->timestamp.nsec = (unsigned int)st.st_mtim.tv_nsec; -#else - istate->timestamp.nsec = 0; -#endifDoesn't this break on systems where
st_mtimeistime_tandst_mtimdoes not exist?
Yes, it breaks. You can test this on Linux by commenting out these two lines in git-compat-util.h:
git-compat-util.h +2 -2
@@ -44,8 +44,8 @@#define _XOPEN_SOURCE_EXTENDED 1/* AIX 5.3L needs this */ #endif #define _ALL_SOURCE 1 -#define _GNU_SOURCE 1 -#define _BSD_SOURCE 1 +/*#define _GNU_SOURCE 1*/ +/*#define _BSD_SOURCE 1*/ #include <unistd.h> #include <stdio.h>
The result even passes the test suite (as long as I don't merge kb/checkout-optim, of course).
-- Hannes
Re: [PATCH 2/2] better introduction of GIT with USE_NSEC defined
Quoted 19 lines · Johannes Sixt
Yes, it breaks. You can test this on Linux by commenting out these two lines in
git-compat-util.h:git-compat-util.h +2
-2@@ -44,8 +44,8 @@#define _XOPEN_SOURCE_EXTENDED 1/* AIX 5.3L needs this */ #endif #define _ALL_SOURCE 1 -#define _GNU_SOURCE 1 -#define _BSD_SOURCE 1 +/*#define _GNU_SOURCE 1*/ +/*#define _BSD_SOURCE 1*/ #include <unistd.h> #include <stdio.h>
Quoted 2 lines
The result even passes the test suite (as long as I don't merge kb/checkout-optim, of course).
Isn't it better to separate "do we want to use that information if we can nanosecocond timestamps" and "is the API available for reading nanosecond timestamps" into separete switches, like this partial patch on top of the patch we are discussing, then? It is partial because it shows the way to convert only one call site as an example and of course you need to update the Makefile and autoconf.
git-compat-util.h +8 -0
@@ -384,4 +384,12 @@ void git_qsort(void *base, size_t nmemb, size_t size,# define FORCE_DIR_SET_GID 0#endif +#ifdef HAS_NSEC +#define ST_CTIME_NSEC (st) ((unsigned ing)((st).st_ctim.tv_nsec)) +#define ST_MTIME_NSEC (st) ((unsigned ing)((st).st_mtim.tv_nsec)) +#else +#define ST_CTIME_NSEC (st) 0 +#define ST_MTIME_NSEC (st) 0 +#endif +#endif
read-cache.c +2 -7
@@ -69,13 +69,8 @@ void fill_stat_cache_info(struct cache_entry *ce, struct stat *st) { ce->ce_ctime.sec = (unsigned int )st->st_ctime; ce->ce_mtime.sec = (unsigned int )st->st_mtime; -#ifdef USE_NSEC - ce->ce_ctime.nsec = (unsigned int )st->st_ctim.tv_nsec; - ce->ce_mtime.nsec = (unsigned int )st->st_mtim.tv_nsec; -#else - ce->ce_ctime.nsec =0 ; - ce->ce_mtime.nsec =0 ; -#endif + ce->ce_ctime.nsec =ST_CTIME_NSEC (*st); + ce->ce_mtime.nsec =ST_MTIME_NSEC (*st); ce->ce_dev = st->st_dev; ce->ce_ino = st->st_ino; ce->ce_uid = st->st_uid;
Re: [PATCH 2/2] better introduction of GIT with USE_NSEC defined
Quoted 20 lines · Junio C Hamano
Quoted 18 lines · Johannes Sixt
Yes, it breaks. You can test this on Linux by commenting out these two lines in
git-compat-util.h:git-compat-util.h +2
-2@@ -44,8 +44,8 @@#define _XOPEN_SOURCE_EXTENDED 1/* AIX 5.3L needs this */ #endif #define _ALL_SOURCE 1 -#define _GNU_SOURCE 1 -#define _BSD_SOURCE 1 +/*#define _GNU_SOURCE 1*/ +/*#define _BSD_SOURCE 1*/ #include <unistd.h> #include <stdio.h>
Quoted 2 lines
Quoted 2 lines
The result even passes the test suite (as long as I don't merge kb/checkout-optim, of course).
This actually will trigger more compilation breakages than that. For example, it hides memmem() and strchrnul() from everybody, and makes the compilation error out with this subtract as the compiler has to assume you are subtracting a pointer from an integer:
linelen = strchrnul(msg_p, '\n') - msg_p;
I cannot test the result myself to see if it still works this way.
Quoted 6 lines
Isn't it better to separate "do we want to use that information if we can nanosecocond timestamps" and "is the API available for reading nanosecond timestamps" into separete switches, like this partial patch on top of the patch we are discussing, then? It is partial because it shows the way to convert only one call site as an example and of course you need to update the
Makefileand autoconf.
And here is what I did. It is meant to replace the patch in discussion.
-- >8 --
Record ns-timestamps if possible, but do not use it without USE_NSEC
Traditionally, the lack of USE_NSEC meant "do not record nor use the nanosecond resolution part of the file timestamps". To avoid problems on filesystems that lose the ns part when the metadata is flushed to the disk and then later read back in, disabling USE_NSEC has been a good idea in general.
If you are on a filesystem without such an issue, it does not hurt to read and store them in the cached stat data in the index entries even if your git is compiled without USE_NSEC. The index left with such a version of git can be read by git compiled with USE_NSEC and it can make use of the nanosecond part to optimize the check to see if the path on the filesystem hsa been modified since we last looked at.
--- Makefile | 7 +++++++ builtin-fetch-pack.c | 4 +--- git-compat-util.h | 9 +++++++++ read-cache.c | 29 ++++------------------------- unpack-trees.c | 2 -- 5 files changed, 21 insertions(+), 30 deletions(-)
Makefile +7 -0
@@ -126,6 +126,9 @@ all:: # randomly break unless your underlying filesystem supports those sub-second # times (my ext3 doesn't). # +# Define NO_NSEC if your "struct stat" does not have "st_ctim.tv_nsec" +# available. This automatically turns USE_NSEC off. +# # Define USE_STDEV below if you want git to care about the underlying device # change being considered an inode change from the update-index perspective. #
@@ -802,6 +805,7 @@ ifneq (,$(findstring MINGW,$(uname_S))) RUNTIME_PREFIX = YesPlease NO_POSIX_ONLY_PROGRAMS = YesPlease NO_ST_BLOCKS_IN_STRUCT_STAT = YesPlease + NO_NSEC = YesPlease COMPAT_CFLAGS += -D__USE_MINGW_ACCESS -DNOGDI -Icompat -Icompat/regex -Icompat/fnmatch COMPAT_CFLAGS += -DSNPRINTF_SIZE_CORR=1 COMPAT_CFLAGS += -DSTRIP_EXTENSION=\".exe\"
@@ -923,6 +927,9 @@ endif ifdef NO_ST_BLOCKS_IN_STRUCT_STAT BASIC_CFLAGS += -DNO_ST_BLOCKS_IN_STRUCT_STAT endif +ifdef NO_NSEC + BASIC_CFLAGS += -DNO_NSEC +endif ifdef NO_C99_FORMAT BASIC_CFLAGS += -DNO_C99_FORMAT endif
builtin-fetch-pack.c +1 -3
@@ -801,9 +801,7 @@ struct ref *fetch_pack(struct fetch_pack_args *my_args,int fd; mtime.sec = st.st_mtime; -#ifdef USE_NSEC - mtime.nsec = st.st_mtim.tv_nsec; -#endif + mtime.nsec =ST_MTIME_NSEC (st);if (stat (shallow, &st)) {if (mtime.sec)die ("shallow file was removed during fetch" );
git-compat-util.h +9 -0
@@ -384,4 +384,13 @@ void git_qsort(void *base, size_t nmemb, size_t size,# define FORCE_DIR_SET_GID 0#endif +#ifdef NO_NSEC +#undef USE_NSEC +#define ST_CTIME_NSEC (st) 0 +#define ST_MTIME_NSEC (st) 0 +#else +#define ST_CTIME_NSEC (st) ((unsigned int)((st).st_ctim.tv_nsec)) +#define ST_MTIME_NSEC (st) ((unsigned int)((st).st_mtim.tv_nsec)) +#endif +#endif
read-cache.c +4 -25
@@ -69,13 +69,8 @@ void fill_stat_cache_info(struct cache_entry *ce, struct stat *st) { ce->ce_ctime.sec = (unsigned int )st->st_ctime; ce->ce_mtime.sec = (unsigned int )st->st_mtime; -#ifdef USE_NSEC - ce->ce_ctime.nsec = (unsigned int )st->st_ctim.tv_nsec; - ce->ce_mtime.nsec = (unsigned int )st->st_mtim.tv_nsec; -#else - ce->ce_ctime.nsec =0 ; - ce->ce_mtime.nsec =0 ; -#endif + ce->ce_ctime.nsec =ST_CTIME_NSEC (*st); + ce->ce_mtime.nsec =ST_MTIME_NSEC (*st); ce->ce_dev = st->st_dev; ce->ce_ino = st->st_ino; ce->ce_uid = st->st_uid;
@@ -1183,13 +1178,8 @@ static void convert_from_disk(struct ondisk_cache_entry *ondisk, struct cache_en ce->ce_ctime.sec =ntohl (ondisk->ctime.sec); ce->ce_mtime.sec =ntohl (ondisk->mtime.sec); -#ifdef USE_NSEC ce->ce_ctime.nsec =ntohl (ondisk->ctime.nsec); ce->ce_mtime.nsec =ntohl (ondisk->mtime.nsec); -#else - ce->ce_ctime.nsec =0 ; - ce->ce_mtime.nsec =0 ; -#endif ce->ce_dev =ntohl (ondisk->dev); ce->ce_ino =ntohl (ondisk->ino); ce->ce_mode =ntohl (ondisk->mode);
@@ -1309,11 +1299,7 @@ int read_index_from(struct index_state *istate, const char *path) dst_offset +=ce_size (ce); } istate->timestamp.sec = st.st_mtime; -#ifdef USE_NSEC - istate->timestamp.nsec = (unsigned int )st.st_mtim.tv_nsec; -#else - istate->timestamp.nsec =0 ; -#endif + istate->timestamp.nsec =ST_MTIME_NSEC (st);while (src_offset <= mmap_size -20 -8 ) {/* After an array of active_nr index entries,
@@ -1500,13 +1486,8 @@ static int ce_write_entry(git_SHA_CTX *c, int fd, struct cache_entry *ce) ondisk->ctime.sec =htonl (ce->ce_ctime.sec); ondisk->mtime.sec =htonl (ce->ce_mtime.sec); -#ifdef USE_NSEC ondisk->ctime.nsec =htonl (ce->ce_ctime.nsec); ondisk->mtime.nsec =htonl (ce->ce_mtime.nsec); -#else - ondisk->ctime.nsec =0 ; - ondisk->mtime.nsec =0 ; -#endif ondisk->dev =htonl (ce->ce_dev); ondisk->ino =htonl (ce->ce_ino); ondisk->mode =htonl (ce->ce_mode);
@@ -1583,9 +1564,7 @@ int write_index(struct index_state *istate, int newfd)if (ce_flush (&c, newfd) ||fstat (newfd, &st))return -1 ; istate->timestamp.sec = (unsigned int )st.st_ctime; -#ifdef USE_NSEC - istate->timestamp.nsec = (unsigned int )st.st_ctim.tv_nsec; -#endif + istate->timestamp.nsec =ST_CTIME_NSEC (st);return 0 ; }
unpack-trees.c +0 -2
@@ -362,9 +362,7 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options o->result.initialized =1 ;if (o->src_index) { o->result.timestamp.sec = o->src_index->timestamp.sec; -#ifdef USE_NSEC o->result.timestamp.nsec = o->src_index->timestamp.nsec; -#endif } o->merge_size = len;
Re: [PATCH 2/2] better introduction of GIT with USE_NSEC defined
Quoted 17 lines · Junio C Hamano
Record ns-timestamps if possible, but do not use it without
USE_NSECTraditionally, the lack of
USE_NSECmeant "do not record nor use the nanosecond resolution part of the file timestamps". To avoid problems on filesystems that lose the ns part when the metadata is flushed to the disk and then later read back in, disablingUSE_NSEChas been a good idea in general.If you are on a filesystem without such an issue, it does not hurt to read and store them in the cached stat data in the index entries even if your git is compiled without
USE_NSEC. The index left with such a version of git can be read by git compiled withUSE_NSECand it can make use of the nanosecond part to optimize the check to see if the path on the filesystem hsa been modified since we last looked at.Signed-off-by Junio C Hamano <[email protected]>---
...
Quoted 3 lines
(quoted diff) +3
-0+# Define NO_NSEC if your "struct stat" does not have "st_ctim.tv_nsec" +# available. This automatically turns USE_NSEC off. +#
...
Review ifneq (,$(findstring MINGW,$(uname_S))) · line 811
Quoted 8 lines
(quoted diff) +1
-0@@ -802,6 +805,7 @@ ifneq (,$(findstring MINGW,$(uname_S))) RUNTIME_PREFIX = YesPlease NO_POSIX_ONLY_PROGRAMS = YesPlease NO_ST_BLOCKS_IN_STRUCT_STAT = YesPlease + NO_NSEC = YesPlease COMPAT_CFLAGS += -D__USE_MINGW_ACCESS -DNOGDI -Icompat -Icompat/regex -Icompat/fnmatch COMPAT_CFLAGS += -DSNPRINTF_SIZE_CORR=1 COMPAT_CFLAGS += -DSTRIP_EXTENSION=\".exe\"
Thanks, with this the build no longer breaks on Windows. But I need this option on AIX as well:
Makefile +1 -0
@@ -737,6 +737,7 @@ ifeq ($(uname_S),AIX) NO_MEMMEM = YesPlease NO_MKDTEMP = YesPlease NO_STRLCPY = YesPlease + NO_NSEC = YesPlease FREAD_READS_DIRECTORIES = UnfortunatelyYes INTERNAL_QSORT = UnfortunatelyYes NEEDS_LIBICONV=YesPlease
-- Hannes
Re: [PATCH 2/2] better introduction of GIT with USE_NSEC defined
In reply to Kjetil BarvikQuoted 8 lines · Kjetil Barvik
Change the source code such that when
USE_NSECis not defined, possible nanosecond timestamps will still be saved in the index file, but not used inside if-test's, and will therefore not affect the outcome of GIT commands, other than the saved nanosecond timestamps in the index file.This will make it easier to use a system with 2 versions of GIT, one with and one without
USE_NSECdefined.
I take it that you are responding to my earlier question with this patch?
From: Junio C Hamano <[email protected]> Subject: Re: [PATCH/RFC v2 2/3] make USE_NSEC work as expected Date: Fri, 20 Feb 2009 00:35:35 -0800 Message-ID: <[email protected]> Kjetil Barvik <[email protected]> writes: > diff --git a/read-cache.c b/read-cache.c > index 940ec76..ca4bec2 100644 > --- a/read-cache.c > +++ b/read-cache.c > @@ -67,8 +67,15 @@ void rename_index_entry_at(struct index_state *istate,.. > */ > void fill_stat_cache_info(struct cache_entry *ce, struct stat *st) > { > - ce->ce_ctime = st->st_ctime; > - ce->ce_mtime = st->st_mtime; > + ce->ce_ctime.sec = (unsigned int)st->st_ctime; > + ce->ce_mtime.sec = (unsigned int)st->st_mtime; > +#ifdef USE_NSEC > + ce->ce_ctime.nsec = (unsigned int)st->st_ctim.tv_nsec; > + ce->ce_mtime.nsec = (unsigned int)st->st_mtim.tv_nsec; > +#else > + ce->ce_ctime.nsec = 0; > + ce->ce_mtime.nsec = 0; > +#endif How does this affect a use case where the same index file used with two instances of git (one compiled with and another without USE_NSEC)?
Re: [PATCH 2/2] better introduction of GIT with USE_NSEC defined
Quoted 8 lines · Junio C Hamano
Quoted 6 lines · Kjetil Barvik
Change the source code such that when
USE_NSECis not defined, possible nanosecond timestamps will still be saved in the index file, but not used inside if-test's, and will therefore not affect the outcome of GIT commands, other than the saved nanosecond timestamps in the index file.
Quoted 5 lines
Quoted 2 lines
This will make it easier to use a system with 2 versions of GIT, one with and one without
USE_NSECdefined.I take it that you are responding to my earlier question with this patch?
Yes, you are correct.
[....]
Quoted 2 lines
How does this affect a use case where the same index file used with two instances of git (one compiled with and another without
USE_NSEC)?
If both persons in this use case use this patch, the one with USE_NSEC defined will now be able to take full advantage of the nanosecond timestamps at all times.
The one without USE_NSEC defined should not be able to tell the difference (without looking into to details of the index file).
-- kjetil
Re: [PATCH 2/2] better introduction of GIT with USE_NSEC defined
Quoted 11 lines · Kjetil Barvik
Junio C Hamano <[email protected]> writes:
...Quoted 2 lines
How does this affect a use case where the same index file used with two instances of git (one compiled with and another without
USE_NSEC)?If both persons in this use case use this patch, the one with
USE_NSECdefined will now be able to take full advantage of the nanosecond timestamps at all times.The one without
USE_NSECdefined should not be able to tell the difference (without looking into to details of the index file).
As long as the implementation does not give false cleanliness it is perfectly fine; false dirtinesss is just a bit of wasted cycle.
Thanks.