Skip to contentSkip to conversation

The ext4 filesystem and racy git

PATCH Superseded Kjetil Barvik 1 participant 3 messages Jun 15, 2016

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover The ext4 filesystem and racy git

Ext4 is marked stable in Linux v2.6.28, and I have done a very simple test on one disk-partition of my slow laptop, and the numbers look very promising.

With default created ext3 disk-partition on my harddisk the 'git checkout -q my-v2.6.25/27' test takes around 20 seconds real time for the best cases. The same test run on a ext4 partition takes around 14 seconds for the best cases.

And, since ext4 supports nanosecond timestamps, when I added patch 2/2, the checkout time is much more stable and more close to 14-15 seconds most of the time.

Conclusion: for GIT on my laptop the ext4 filesystem is a speedup!

Quoted 1 line

From '/proc/mounts' here is the mount options:

/dev/hda10 /home ext3 rw,noatime,errors=continue,data=ordered 0 0 /dev/hda12 /opt2 ext4 rw,noatime,barrier=1,journal_async_commit,data=ordered 0 0

/dev/hda10 is formatted with default (gentoo) ext3 parameters, and the /dev/hda12 is formatted with:

/sbin/mkfs -t ext4 -I 256 -G 64 -Oflex_bg,uninit_bg /dev/hda12

Kjetil Barvik (2):
fix compile error when USE_NSEC is defined
make USE_NSEC work as expected

 builtin-fetch-pack.c |    4 +-
 cache.h              |    6 ++--
 read-cache.c         |   70 ++++++++++++++++++++++++++++++++++++++++----------
 3 files changed, 61 insertions(+), 19 deletions(-)
#1 lore raw

PATCH 1/2 fix compile error when USE_NSEC is defined

'struct cache' does not have a 'usec' member, but a 'unsigned int nsec' member. Simmilar 'struct stat' does not have a 'st_mtim.usec' member, and we should instead use 'st_mtim.tv_nsec'.

---
 builtin-fetch-pack.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)
builtin-fetch-pack.c +2 -2
index 67fb80e..3b210c7 100644
@@ -802,14 +802,14 @@ struct ref *fetch_pack(struct fetch_pack_args *my_args,
 
 		mtime.sec = st.st_mtime;
 #ifdef USE_NSEC
-		mtime.usec = st.st_mtim.usec;
+		mtime.nsec = st.st_mtim.tv_nsec;
 #endif
 		if (stat(shallow, &st)) {
 			if (mtime.sec)
 				die("shallow file was removed during fetch");
 		} else if (st.st_mtime != mtime.sec
 #ifdef USE_NSEC
-				|| st.st_mtim.usec != mtime.usec
+				|| st.st_mtim.tv_nsec != mtime.nsec
 #endif
 			  )
 			die("shallow file was changed during fetch");
Signature · 1 line
1.6.1.349.g99fa5
#2 lore raw

PATCH 2/2 make USE_NSEC work as expected

In reply to Kjetil Barvik

Since the filesystem ext4 is now defined as stable in Linux v2.6.28, and ext4 supports nanonsecond resolution timestamps natively, it is time to make USE_NSEC work as expected.

This will make racy git situations less likely to happen. For 'git checkout' this means it will be less likely that we have to open, read the contents of the file into RAM, and check if file is really modified or not. The result sould be a litle less used CPU time, less pagefaults and a litle faster program, at least for 'git checkout'.

Since the number of possible racy git situations would increase when disks gets faster, this patch would be more and more helpfull as times go by. For a fast Solid State Disk, this patch should be helpfull.

Note that, when file operations starts to take less than 1 nanosecond, one would again start to get more racy git situations.

For more info on racy git, see Documentation/technical/racy-git.txt For more info on ext4, see http://kernelnewbies.org/Ext4

---
 cache.h      |    6 ++--
 read-cache.c |   70 ++++++++++++++++++++++++++++++++++++++++++++++-----------
 2 files changed, 59 insertions(+), 17 deletions(-)
cache.h +3 -3
index 37dfb1c..309053d 100644
@@ -140,8 +140,8 @@ struct ondisk_cache_entry_extended {
 };
 
 struct cache_entry {
-	unsigned int ce_ctime;
-	unsigned int ce_mtime;
+	struct cache_time ce_ctime;
+	struct cache_time ce_mtime;
 	unsigned int ce_dev;
 	unsigned int ce_ino;
 	unsigned int ce_mode;
@@ -282,7 +282,7 @@ struct index_state {
 	struct cache_entry **cache;
 	unsigned int cache_nr, cache_alloc, cache_changed;
 	struct cache_tree *cache_tree;
-	time_t timestamp;
+	struct cache_time timestamp;
 	void *alloc;
 	unsigned name_hash_initialized : 1,
 		 initialized : 1;
read-cache.c +56 -14

129 lines not on this page · raw message

Signature · 1 line
1.6.1.349.g99fa5