Fix for mh/packed-ref-store-prep

When preparing a new patch series, I noticed a bug that crept into mh/packed-ref-store-prep while I was shuffling patches around. It introduces a race condition if another process rewrites the packed-refs file just before we lock it. See the commit message for more information.

This patch applies on top of mh/packed-ref-store-prep and merges to master without problems.

Michael

Michael Haggerty (1):
lock_packed_refs(): fix cache validity check

 refs/files-backend.c | 32 +++++++++++++++++++++++---------
 1 file changed, 23 insertions(+), 9 deletions(-)

2.11.0

Commit 28ed9830b1 (get_packed_ref_cache(): assume "packed-refs" won't change while locked, 2017-05-22) assumes that the "packed-refs" file cannot change while we hold the lock. That assumption is justified *if* the lock has been held the whole time since the "packed-refs" file was last read.

But in lock_packed_refs(), we ourselves lock the "packed-refs" file and then call get_packed_ref_cache() to ensure that the cache agrees with the file. The intent is to guard against the possibility that another process changed the "packed-refs" file the moment before we locked it.

This check was defeated because get_packed_ref_cache() saw that the file was locked, and therefore didn't do the stat_validity_check() that we want.

The mistake was compounded with a misleading comment in lock_packed_refs() claiming that it was doing the right thing. That comment came from an earlier draft of the mh/packed-ref-store-prep patch series when the commits were in a different order.

So instead:

* Extract a function validate_packed_ref_cache() that does the validity check independent of whether the lock is held.

* Change get_packed_ref_cache() to call the new function, but only if the lock *isn't* held.

* Change lock_packed_refs() to call the new function in any case before calling get_packed_ref_cache().

* Fix the comment in lock_packed_refs().

---
If anybody is curious, patch 28ed9830b1 originally appeared before 00d174489e (files_ref_store: put the packed files lock directly in this struct, 2017-05-22), when refs->packed_refs_lock could be left NULL until after get_packed_ref_cache() was called. In that version of the patch series, the breakage occurred when the the lock was moved and the check in get_packed_ref_cache() was changed from refs->packed_refs_lock to is_lock_file_locked(&refs->packed_refs_lock). So either way, it was the combination of the two patches that caused the regression.

 refs/files-backend.c | 32 +++++++++++++++++++++++---------
 1 file changed, 23 insertions(+), 9 deletions(-)

index d8b3f73147..b040bb3b0a 100644
@@ -369,6 +369,18 @@ static void files_ref_path(struct files_ref_store *refs,
 	}
 }
 
+/*
+ * Check that the packed refs cache (if any) still reflects the
+ * contents of the file. If not, clear the cache.
+ */
+static void validate_packed_ref_cache(struct files_ref_store *refs)
+{
+	if (refs->packed &&
+	    !stat_validity_check(&refs->packed->validity,
+				 files_packed_refs_path(refs)))
+		clear_packed_ref_cache(refs);
+}
+
 /*
  * Get the packed_ref_cache for the specified files_ref_store,
  * creating and populating it if it hasn't been read before or if the

@@ -381,10 +393,8 @@ static struct packed_ref_cache *get_packed_ref_cache(struct files_ref_store *ref
 {
 	const char *packed_refs_file = files_packed_refs_path(refs);
 
-	if (refs->packed &&
-	    !is_lock_file_locked(&refs->packed_refs_lock) &&
-	    !stat_validity_check(&refs->packed->validity, packed_refs_file))
-		clear_packed_ref_cache(refs);
+	if (!is_lock_file_locked(&refs->packed_refs_lock))
+		validate_packed_ref_cache(refs);
 
 	if (!refs->packed)
 		refs->packed = read_packed_refs(packed_refs_file);

@@ -1311,13 +1321,17 @@ static int lock_packed_refs(struct files_ref_store *refs, int flags)
 			    &refs->packed_refs_lock, files_packed_refs_path(refs),
 			    flags, timeout_value) < 0)
 		return -1;
+
 	/*
-	 * Get the current packed-refs while holding the lock. It is
-	 * important that we call `get_packed_ref_cache()` before
-	 * setting `packed_ref_cache->lock`, because otherwise the
-	 * former will see that the file is locked and assume that the
-	 * cache can't be stale.
+	 * Now that we hold the `packed-refs` lock, make sure that our
+	 * cache matches the current version of the file. Normally
+	 * `get_packed_ref_cache()` does that for us, but that
+	 * function assumes that when the file is locked, any existing
+	 * cache is still valid. We've just locked the file, but it
+	 * might have changed the moment *before* we locked it.
 	 */
+	validate_packed_ref_cache(refs);
+
 	packed_ref_cache = get_packed_ref_cache(refs);
 	/* Increment the reference count to prevent it from being freed: */
 	acquire_packed_ref_cache(packed_ref_cache);

2.11.0

Your explanation makes perfect sense, and I agree this should fix the problem. Sorry to have missed it in review.

-Peff