Skip to contentSkip to conversation

RevWalk fixes for UNINTERESTING

PATCH In review Shawn O. Pearce 2 participants 13 messages Jun 15, 2016

lore
Share & tools
lore
#0 lore raw

cover RevWalk fixes for UNINTERESTING

Today I uncovered some ugly cases with "jgit rev-list B ^A", where some commits reachable from A were still being output, even though we asked that they be excluded.

This series attempts to fix it by forcing date ordering, and delaying output a little to try and work over any clock skew discovered near the end of the traversal, just before we give up.

Shawn O. Pearce (5):
Show critical flags in debug toString() descriptions of rev queues Make RevObject.getType implementations final
Remove the horribly stupid RevSort.START_ORDER
Fix RevWalk with Linus Torvald's occasional bad commit date hack Avoid incorrect output of UNINTERESTING commits when clock skew
occurs

 .../org/spearce/jgit/revwalk/AbstractRevQueue.java |    5 +
 .../src/org/spearce/jgit/revwalk/DateRevQueue.java |   10 +--
 .../org/spearce/jgit/revwalk/DelayRevQueue.java    |   92 ++++++++++++++++++++
 .../src/org/spearce/jgit/revwalk/FIFORevQueue.java |   10 +--
 .../jgit/revwalk/FixUninterestingGenerator.java    |   77 ++++++++++++++++
 .../src/org/spearce/jgit/revwalk/LIFORevQueue.java |   10 +--
 .../org/spearce/jgit/revwalk/PendingGenerator.java |   59 +++++++++++--
 .../src/org/spearce/jgit/revwalk/RevBlob.java      |    2 +-
 .../src/org/spearce/jgit/revwalk/RevCommit.java    |   15 +++-
 .../src/org/spearce/jgit/revwalk/RevObject.java    |   24 +++++
 .../src/org/spearce/jgit/revwalk/RevSort.java      |   11 ---
 .../src/org/spearce/jgit/revwalk/RevTag.java       |    2 +-
 .../src/org/spearce/jgit/revwalk/RevTree.java      |    2 +-
 .../src/org/spearce/jgit/revwalk/RevWalk.java      |    6 +-
 .../org/spearce/jgit/revwalk/StartGenerator.java   |   27 ++++--
 15 files changed, 296 insertions(+), 56 deletions(-)
 create mode 100644 org.spearce.jgit/src/org/spearce/jgit/revwalk/DelayRevQueue.java
 create mode 100644 org.spearce.jgit/src/org/spearce/jgit/revwalk/FixUninterestingGenerator.java
#1 lore raw

PATCH 1/5 Show critical flags in debug toString() descriptions of rev queues

These can help identify the state of each object, especially the important UNINTERESTING flag being present (or not).

---
 .../org/spearce/jgit/revwalk/AbstractRevQueue.java |    5 ++++
 .../src/org/spearce/jgit/revwalk/DateRevQueue.java |   10 ++------
 .../src/org/spearce/jgit/revwalk/FIFORevQueue.java |   10 ++------
 .../src/org/spearce/jgit/revwalk/LIFORevQueue.java |   10 ++------
 .../src/org/spearce/jgit/revwalk/RevCommit.java    |   12 ++++++++++
 .../src/org/spearce/jgit/revwalk/RevObject.java    |   23 ++++++++++++++++++++
 6 files changed, 49 insertions(+), 21 deletions(-)
org.spearce.jgit/src/org/spearce/jgit/revwalk/AbstractRevQueue.java +5 -0
index 4cf7dae..5bb969d 100644
@@ -115,6 +115,11 @@ int outputType() {
 		return outputType;
 	}
 
+	protected static void describe(final StringBuilder s, final RevCommit c) {
+		s.append(c.toString());
+		s.append('\n');
+	}
+
 	private static class AlwaysEmptyQueue extends AbstractRevQueue {
 		@Override
 		public void add(RevCommit c) {
org.spearce.jgit/src/org/spearce/jgit/revwalk/DateRevQueue.java +3 -7
index f797477..210f985 100644
@@ -126,13 +126,9 @@ int outputType() {
 	}
 
 	public String toString() {
-		final StringBuffer s = new StringBuffer();
-		for (Entry q = head; q != null; q = q.next) {
-			s.append(q.commit.name());
-			s.append(' ');
-			s.append(q.commit.commitTime);
-			s.append('\n');
-		}
+		final StringBuilder s = new StringBuilder();
+		for (Entry q = head; q != null; q = q.next)
+			describe(s, q.commit);
 		return s.toString();
 	}
 
org.spearce.jgit/src/org/spearce/jgit/revwalk/FIFORevQueue.java +3 -7
index 2c4c003..f086928 100644
@@ -149,14 +149,10 @@ void removeFlag(final int f) {
 	}
 
 	public String toString() {
-		final StringBuffer s = new StringBuffer();
+		final StringBuilder s = new StringBuilder();
 		for (Block q = head; q != null; q = q.next) {
-			for (int i = q.headIndex; i < q.tailIndex; i++) {
-				s.append(q.commits[i].name());
-				s.append(' ');
-				s.append(q.commits[i].commitTime);
-				s.append('\n');
-			}
+			for (int i = q.headIndex; i < q.tailIndex; i++)
+				describe(s, q.commits[i]);
 		}
 		return s.toString();
 	}
org.spearce.jgit/src/org/spearce/jgit/revwalk/LIFORevQueue.java +3 -7
index 5e885c0..045f7f1 100644
@@ -104,14 +104,10 @@ boolean anybodyHasFlag(final int f) {
 	}
 
 	public String toString() {
-		final StringBuffer s = new StringBuffer();
+		final StringBuilder s = new StringBuilder();
 		for (Block q = head; q != null; q = q.next) {
-			for (int i = q.headIndex; i < q.tailIndex; i++) {
-				s.append(q.commits[i].name());
-				s.append(' ');
-				s.append(q.commits[i].commitTime);
-				s.append('\n');
-			}
+			for (int i = q.headIndex; i < q.tailIndex; i++)
+				describe(s, q.commits[i]);
 		}
 		return s.toString();
 	}
org.spearce.jgit/src/org/spearce/jgit/revwalk/RevCommit.java +12 -0
index de11c39..1b25fce 100644
@@ -388,4 +388,16 @@ public void dispose() {
 		flags &= ~PARSED;
 		buffer = null;
 	}
+
+	@Override
+	public String toString() {
+		final StringBuilder s = new StringBuilder();
+		s.append(Constants.typeString(getType()));
+		s.append(name());
+		s.append(' ');
+		s.append(commitTime);
+		s.append(' ');
+		appendCoreFlags(s);
+		return s.toString();
+	}
 }
org.spearce.jgit/src/org/spearce/jgit/revwalk/RevObject.java +23 -0
index 451205c..7dadb7b 100644
@@ -167,4 +167,27 @@ public void remove(final RevFlagSet set) {
 	public void dispose() {
 		// Nothing needs to be done for most objects.
 	}
+
+	@Override
+	public String toString() {
+		final StringBuilder s = new StringBuilder();
+		s.append(Constants.typeString(getType()));
+		s.append(name());
+		s.append(' ');
+		appendCoreFlags(s);
+		return s.toString();
+	}
+
+	/**
+	 * @param s
+	 *            buffer to append a debug description of core RevFlags onto.
+	 */
+	protected void appendCoreFlags(final StringBuilder s) {
+		s.append((flags & RevWalk.TOPO_DELAY) != 0 ? 'o' : '-');
+		s.append((flags & RevWalk.TEMP_MARK) != 0 ? 't' : '-');
+		s.append((flags & RevWalk.REWRITE) != 0 ? 'r' : '-');
+		s.append((flags & RevWalk.UNINTERESTING) != 0 ? 'u' : '-');
+		s.append((flags & RevWalk.SEEN) != 0 ? 's' : '-');
+		s.append((flags & RevWalk.PARSED) != 0 ? 'p' : '-');
+	}
 }
Signature · 1 line
1.6.2.288.gc3f22
#2 lore raw

PATCH 2/5 Make RevObject.getType implementations final

These methods should never be overridden once defined by the base class of RevCommit, RevTree, RevBlob or RevTag. An override is only going to provide confusion to calls who rely upon the return value to know if a downcast is safe.

---
 .../src/org/spearce/jgit/revwalk/RevBlob.java      |    2 +-
 .../src/org/spearce/jgit/revwalk/RevCommit.java    |    3 ++-
 .../src/org/spearce/jgit/revwalk/RevObject.java    |    1 +
 .../src/org/spearce/jgit/revwalk/RevTag.java       |    2 +-
 .../src/org/spearce/jgit/revwalk/RevTree.java      |    2 +-
 5 files changed, 6 insertions(+), 4 deletions(-)
org.spearce.jgit/src/org/spearce/jgit/revwalk/RevBlob.java +1 -1
index 66cdc02..cf241cf 100644
@@ -58,7 +58,7 @@ void parse(final RevWalk walk) {
 	}
 	
 	@Override
-	public int getType() {
+	public final int getType() {
 		return Constants.OBJ_BLOB;
 	}
 }
org.spearce.jgit/src/org/spearce/jgit/revwalk/RevCommit.java +2 -1
index 1b25fce..2a59ec4 100644
@@ -139,7 +139,7 @@ else if (nParents == 1) {
 	}
 	
 	@Override
-	public int getType() {
+	public final int getType() {
 		return Constants.OBJ_COMMIT;
 	}
 
@@ -393,6 +393,7 @@ public void dispose() {
 	public String toString() {
 		final StringBuilder s = new StringBuilder();
 		s.append(Constants.typeString(getType()));
+		s.append(' ');
 		s.append(name());
 		s.append(' ');
 		s.append(commitTime);
org.spearce.jgit/src/org/spearce/jgit/revwalk/RevObject.java +1 -0
index 7dadb7b..8c7cc23 100644
@@ -172,6 +172,7 @@ public void dispose() {
 	public String toString() {
 		final StringBuilder s = new StringBuilder();
 		s.append(Constants.typeString(getType()));
+		s.append(' ');
 		s.append(name());
 		s.append(' ');
 		appendCoreFlags(s);
org.spearce.jgit/src/org/spearce/jgit/revwalk/RevTag.java +1 -1
index aba8744..cace82d 100644
@@ -99,7 +99,7 @@ void parseCanonical(final RevWalk walk, final byte[] rawTag)
 	}
 
 	@Override
-	public int getType() {
+	public final int getType() {
 		return Constants.OBJ_TAG;
 	}
 
org.spearce.jgit/src/org/spearce/jgit/revwalk/RevTree.java +1 -1
index e1cd4b5..4d767e4 100644
@@ -58,7 +58,7 @@ void parse(final RevWalk walk) {
 	}
 	
 	@Override
-	public int getType() {
+	public final int getType() {
 		return Constants.OBJ_TREE;
 	}
 }
Signature · 1 line
1.6.2.288.gc3f22
#3 lore raw

PATCH 3/5 Remove the horribly stupid RevSort.START_ORDER

I must have been on crack the day I wrote 3b27268f49 ("Allow RevWalk users to ask for FIFO style ordering"). Its a really bad idea.

Applications can get themselves into a situation where they process one branch long before another, and then abort too early, before all commits have been correctly flagged UNINTERESTING.

For example, given the graph:

Z-A------------B
| /
| ---------
\ /
Q-R-S-T-U-V

If B is "interesting" and A,V are UNINTERESTING, without forcing the commit timestamp ordering in the pending queue we wind up processing all of B-Z, producing R,Q as "interesting" output in the process, and terminate before S can even be parsed. Consequently we never push the UNINTERESTING flag down onto R, and R,Q produced when they shouldn't.

We now require that the pending queue use a DateRevQueue instead of the FIFORevQueue, so that in the above graph S must be parsed before we can even consider R or A, even though R,A were reached earlier. This of course still assumes there is no clock skew between S and R. The next commit will address some limited clock skew issues.

---
 .../org/spearce/jgit/revwalk/PendingGenerator.java |    8 +++-----
 .../src/org/spearce/jgit/revwalk/RevSort.java      |   11 -----------
 .../src/org/spearce/jgit/revwalk/RevWalk.java      |    6 +++---
 .../org/spearce/jgit/revwalk/StartGenerator.java   |    8 ++------
 4 files changed, 8 insertions(+), 25 deletions(-)
org.spearce.jgit/src/org/spearce/jgit/revwalk/PendingGenerator.java +3 -5
index 25b2966..144e909 100644
@@ -62,7 +62,7 @@
 
 	private final RevWalk walker;
 
-	private final AbstractRevQueue pending;
+	private final DateRevQueue pending;
 
 	private final RevFilter filter;
 
@@ -70,7 +70,7 @@
 
 	boolean canDispose;
 
-	PendingGenerator(final RevWalk w, final AbstractRevQueue p,
+	PendingGenerator(final RevWalk w, final DateRevQueue p,
 			final RevFilter f, final int out) {
 		walker = w;
 		pending = p;
@@ -81,9 +81,7 @@ PendingGenerator(final RevWalk w, final AbstractRevQueue p,
 
 	@Override
 	int outputType() {
-		if (pending instanceof DateRevQueue)
-			return output | SORT_COMMIT_TIME_DESC;
-		return output;
+		return output | SORT_COMMIT_TIME_DESC;
 	}
 
 	@Override
org.spearce.jgit/src/org/spearce/jgit/revwalk/RevSort.java +0 -11
index b0a03ad..8c3eaed 100644
@@ -49,17 +49,6 @@
 	NONE,
 
 	/**
-	 * Maintain the order commits were marked as starting points.
-	 * <p>
-	 * This strategy is largely a FIFO strategy; commits are enumerated in the
-	 * order they were added to the RevWalk by the application. Parents not yet
-	 * visited are added behind all commits already enqueued for visiting.
-	 * <p>
-	 * This strategy should not be combined with {@link #COMMIT_TIME_DESC}.
-	 */
-	START_ORDER,
-
-	/**
 	 * Sort by commit time, descending (newest first, oldest last).
 	 * <p>
 	 * This strategy can be combined with {@link #TOPO}.
org.spearce.jgit/src/org/spearce/jgit/revwalk/RevWalk.java +3 -3
index c8ec458..316f722 100644
@@ -187,7 +187,7 @@ public RevWalk(final Repository repo) {
 		idBuffer = new MutableObjectId();
 		objects = new ObjectIdSubclassMap<RevObject>();
 		roots = new ArrayList<RevCommit>();
-		queue = new FIFORevQueue();
+		queue = new DateRevQueue();
 		pending = new StartGenerator(this);
 		sorting = EnumSet.of(RevSort.NONE);
 		filter = RevFilter.ALL;
@@ -915,7 +915,7 @@ protected void reset(int retainFlags) {
 
 		curs.release();
 		roots.clear();
-		queue = new FIFORevQueue();
+		queue = new DateRevQueue();
 		pending = new StartGenerator(this);
 	}
 
@@ -934,7 +934,7 @@ public void dispose() {
 		objects.clear();
 		curs.release();
 		roots.clear();
-		queue = new FIFORevQueue();
+		queue = new DateRevQueue();
 		pending = new StartGenerator(this);
 	}
 
org.spearce.jgit/src/org/spearce/jgit/revwalk/StartGenerator.java +2 -6
index 1b7947f..bf7067a 100644
@@ -108,11 +108,7 @@ RevCommit next() throws MissingObjectException,
 		}
 
 		int pendingOutputType = 0;
-		if (walker.hasRevSort(RevSort.START_ORDER)
-				&& !(q instanceof FIFORevQueue))
-			q = new FIFORevQueue(q);
-		if (walker.hasRevSort(RevSort.COMMIT_TIME_DESC)
-				&& !(q instanceof DateRevQueue))
+		if (!(q instanceof DateRevQueue))
 			q = new DateRevQueue(q);
 		if (tf != TreeFilter.ALL) {
 			rf = AndRevFilter.create(rf, new RewriteTreeFilter(w, tf));
@@ -120,7 +116,7 @@ RevCommit next() throws MissingObjectException,
 		}
 
 		walker.queue = q;
-		g = new PendingGenerator(w, q, rf, pendingOutputType);
+		g = new PendingGenerator(w, (DateRevQueue) q, rf, pendingOutputType);
 
 		if (boundary) {
 			// Because the boundary generator may produce uninteresting
Signature · 1 line
1.6.2.288.gc3f22
#4 lore raw

PATCH 4/5 Fix RevWalk with Linus Torvald's occasional bad commit date hack

In git-core commit 7d004199d134c9d465e013064f72dbc04507f6c0 Linus describes a hack he created to improve the handling of cases where commit dates are out of order, such as a child commit having a date older than its parent. These cases can show up when there is bad imported data, or significant clock skew between distributed peers.

The original git-core bug report identified a failure in:

git rev-list A..B C

where commits contained in both A and B were reported, due to out of order commit dates.

We keep running until the most recent commit in the pending queue is more recent than the last commit sent to the caller. If the pending queue suddenly goes "backwards" due to one of our parent's having a more recent commit date, this new check ensures we will keep processing the graph until we see a more consistent cut.

We process an extra OVER_SCAN commits after we decide that all remaining commits are UNINTERESTING. Processing these extra commits ensures flags are carried back further in the graph, increasing the chances that we correctly mark relevant nodes.

As a result of this hack, we may produce a commit to our caller, but then later mark it UNINTERESTING if we discover date skew. To handle such cases, callers could buffer produced commits and filter out those that are UNINTERESTING, but this somewhat costly and may not always be necessary.

---
 .../org/spearce/jgit/revwalk/PendingGenerator.java |   51 ++++++++++++++++++--
 1 files changed, 47 insertions(+), 4 deletions(-)
org.spearce.jgit/src/org/spearce/jgit/revwalk/PendingGenerator.java +47 -4
index 144e909..cd24e8f 100644
@@ -42,6 +42,7 @@
 import org.spearce.jgit.errors.IncorrectObjectTypeException;
 import org.spearce.jgit.errors.MissingObjectException;
 import org.spearce.jgit.errors.StopWalkException;
+import org.spearce.jgit.lib.ObjectId;
 import org.spearce.jgit.revwalk.filter.RevFilter;
 
 /**
@@ -60,6 +61,24 @@
 
 	private static final int UNINTERESTING = RevWalk.UNINTERESTING;
 
+	/**
+	 * Number of additional commits to scan after we think we are done.
+	 * <p>
+	 * This small buffer of commits is scanned to ensure we didn't miss anything
+	 * as a result of clock skew when the commits were made. We need to set our
+	 * constant to 1 additional commit due to the use of a pre-increment
+	 * operator when accessing the value.
+	 */
+	private static final int OVER_SCAN = 5 + 1;
+
+	/** A commit near the end of time, to initialize {@link #last} with. */
+	private static final RevCommit INIT_LAST;
+
+	static {
+		INIT_LAST = new RevCommit(ObjectId.zeroId());
+		INIT_LAST.commitTime = Integer.MAX_VALUE;
+	}
+
 	private final RevWalk walker;
 
 	private final DateRevQueue pending;
@@ -68,6 +87,17 @@
 
 	private final int output;
 
+	/** Last commit produced to the caller from {@link #next()}. */
+	private RevCommit last = INIT_LAST;
+
+	/**
+	 * Number of commits we have remaining in our over-scan allotment.
+	 * <p>
+	 * Only relevant if there are {@link #UNINTERESTING} commits in the
+	 * {@link #pending} queue.
+	 */
+	private int overScan = OVER_SCAN;
+
 	boolean canDispose;
 
 	PendingGenerator(final RevWalk w, final DateRevQueue p,
@@ -112,14 +142,27 @@ RevCommit next() throws MissingObjectException,
 				walker.carryFlagsImpl(c);
 
 				if ((c.flags & UNINTERESTING) != 0) {
-					if (pending.everbodyHasFlag(UNINTERESTING))
-						throw StopWalkException.INSTANCE;
-					c.dispose();
+					if (pending.everbodyHasFlag(UNINTERESTING)) {
+						final RevCommit n = pending.peek();
+						if (n != null && n.commitTime <= last.commitTime) {
+							// This is too close to call. The next commit we
+							// would pop is before the last one we produced.
+							// We have to keep going to ensure that we carry
+							// flags as much as necessary.
+							//
+							overScan = OVER_SCAN;
+						} else if (--overScan == 0)
+							throw StopWalkException.INSTANCE;
+					} else {
+						overScan = OVER_SCAN;
+					}
+					if (canDispose)
+						c.dispose();
 					continue;
 				}
 
 				if (produce)
-					return c;
+					return last = c;
 				else if (canDispose)
 					c.dispose();
 			}
Signature · 1 line
1.6.2.288.gc3f22
#5 lore raw

PATCH 5/5 Avoid incorrect output of UNINTERESTING commits when clock skew occurs

The prior commit added functionality to scan a few extra commits when we otherwise would have aborted due to everything left being colored UNINTERESTING. When that happens we may wind up coloring a commit that we already produced to the caller, giving the caller an incorrect result set.

If we insert a fully buffered generator in the pipeline, such as that used for RevSort.TOPO or RevSort.REVERSE, we can easily filter these late-colored commits out before we show them to the application. But otherwise we delay the output by 6 commits, just long enough to give PendingGenerator a reasonable chance at getting the coloring right.

This is only an approximation. It is still possible for commits to produce when they should be UNINTERESTING, such as if more than the OVER_SCAN limit had clock skew between two branches and the common merge base, even if we are fully buffering our output.

---
 .../org/spearce/jgit/revwalk/DelayRevQueue.java    |   92 ++++++++++++++++++++
 .../jgit/revwalk/FixUninterestingGenerator.java    |   77 ++++++++++++++++
 .../org/spearce/jgit/revwalk/PendingGenerator.java |    2 +-
 .../org/spearce/jgit/revwalk/StartGenerator.java   |   23 ++++-
 4 files changed, 189 insertions(+), 5 deletions(-)
 create mode 100644 org.spearce.jgit/src/org/spearce/jgit/revwalk/DelayRevQueue.java
 create mode 100644 org.spearce.jgit/src/org/spearce/jgit/revwalk/FixUninterestingGenerator.java
org.spearce.jgit/src/org/spearce/jgit/revwalk/DelayRevQueue.java new file +92 -0
new file mode 100644
index 0000000..1eb7064
@@ -0,0 +1,92 @@
+/*
+ * Copyright (C) 2009, Google Inc.
+ *
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above copyright
+ *   notice, this list of conditions and the following disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ *   copyright notice, this list of conditions and the following
+ *   disclaimer in the documentation and/or other materials provided
+ *   with the distribution.
+ *
+ * - Neither the name of the Git Development Community nor the
+ *   names of its contributors may be used to endorse or promote
+ *   products derived from this software without specific prior
+ *   written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
+ * CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
+ * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
+ * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
+ * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+ * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
+ * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+package org.spearce.jgit.revwalk;
+
+import java.io.IOException;
+
+import org.spearce.jgit.errors.IncorrectObjectTypeException;
+import org.spearce.jgit.errors.MissingObjectException;
+
+/**
+ * Delays commits to be at least {@link PendingGenerator#OVER_SCAN} late.
+ * <p>
+ * This helps to "fix up" weird corner cases resulting from clock skew, by
+ * slowing down what we produce to the caller we get a better chance to ensure
+ * PendingGenerator reached back far enough in the graph to correctly mark
+ * commits {@link RevWalk#UNINTERESTING} if necessary.
+ * <p>
+ * This generator should appear before {@link FixUninterestingGenerator} if the
+ * lower level {@link #pending} isn't already fully buffered.
+ */
+final class DelayRevQueue extends Generator {
+	private static final int OVER_SCAN = PendingGenerator.OVER_SCAN;
+
+	private final Generator pending;
+
+	private final FIFORevQueue delay;
+
+	private int size;
+
+	DelayRevQueue(final Generator g) {
+		pending = g;
+		delay = new FIFORevQueue();
+	}
+
+	@Override
+	int outputType() {
+		return pending.outputType();
+	}
+
+	@Override
+	RevCommit next() throws MissingObjectException,
+			IncorrectObjectTypeException, IOException {
+		while (size < OVER_SCAN) {
+			final RevCommit c = pending.next();
+			if (c == null)
+				break;
+			delay.add(c);
+			size++;
+		}
+
+		final RevCommit c = delay.next();
+		if (c == null)
+			return null;
+		size--;
+		return c;
+	}
+}
org.spearce.jgit/src/org/spearce/jgit/revwalk/FixUninterestingGenerator.java new file +77 -0

77 lines not on this page · raw message

org.spearce.jgit/src/org/spearce/jgit/revwalk/PendingGenerator.java +1 -1
index cd24e8f..ebbb39f 100644
@@ -69,7 +69,7 @@
 	 * constant to 1 additional commit due to the use of a pre-increment
 	 * operator when accessing the value.
 	 */
-	private static final int OVER_SCAN = 5 + 1;
+	static final int OVER_SCAN = 5 + 1;
 
 	/** A commit near the end of time, to initialize {@link #last} with. */
 	private static final RevCommit INIT_LAST;
org.spearce.jgit/src/org/spearce/jgit/revwalk/StartGenerator.java +19 -4
index bf7067a..3535250 100644
@@ -90,6 +90,7 @@ RevCommit next() throws MissingObjectException,
 			return mbg.next();
 		}
 
+		final boolean uninteresting = q.anybodyHasFlag(RevWalk.UNINTERESTING);
 		boolean boundary = walker.hasRevSort(RevSort.BOUNDARY);
 
 		if (!boundary && walker instanceof ObjectWalk) {
@@ -99,7 +100,7 @@ RevCommit next() throws MissingObjectException,
 			//
 			boundary = true;
 		}
-		if (boundary && !q.anybodyHasFlag(RevWalk.UNINTERESTING)) {
+		if (boundary && !uninteresting) {
 			// If we were not fed uninteresting commits we will never
 			// construct a boundary. There is no reason to include the
 			// extra overhead associated with that in our pipeline.
@@ -107,16 +108,19 @@ RevCommit next() throws MissingObjectException,
 			boundary = false;
 		}
 
+		final DateRevQueue pending;
 		int pendingOutputType = 0;
-		if (!(q instanceof DateRevQueue))
-			q = new DateRevQueue(q);
+		if (q instanceof DateRevQueue)
+			pending = (DateRevQueue)q;
+		else
+			pending = new DateRevQueue(q);
 		if (tf != TreeFilter.ALL) {
 			rf = AndRevFilter.create(rf, new RewriteTreeFilter(w, tf));
 			pendingOutputType |= HAS_REWRITE | NEEDS_REWRITE;
 		}
 
 		walker.queue = q;
-		g = new PendingGenerator(w, (DateRevQueue) q, rf, pendingOutputType);
+		g = new PendingGenerator(w, pending, rf, pendingOutputType);
 
 		if (boundary) {
 			// Because the boundary generator may produce uninteresting
@@ -143,6 +147,17 @@ RevCommit next() throws MissingObjectException,
 			g = new LIFORevQueue(g);
 		if (boundary)
 			g = new BoundaryGenerator(w, g);
+		else if (uninteresting) {
+			// Try to protect ourselves from uninteresting commits producing
+			// due to clock skew in the commit time stamps. Delay such that
+			// we have a chance at coloring enough of the graph correctly,
+			// and then strip any UNINTERESTING nodes that may have leaked
+			// through early.
+			//
+			if (pending.peek() != null)
+				g = new DelayRevQueue(g);
+			g = new FixUninterestingGenerator(g);
+		}
 
 		w.pending = g;
 		return g.next();
Signature · 1 line
1.6.2.288.gc3f22
#6 lore raw

PATCH v2 4/5 Fix RevWalk with Linus Torvald's occasional bad commit date hack

In reply to Shawn O. Pearce

In git-core commit 7d004199d134c9d465e013064f72dbc04507f6c0 Linus describes a hack he created to improve the handling of cases where commit dates are out of order, such as a child commit having a date older than its parent. These cases can show up when there is bad imported data, or significant clock skew between distributed peers.

The original git-core bug report identified a failure in:

git rev-list A..B C

where commits contained in both A and B were reported, due to out of order commit dates.

We keep running until the most recent commit in the pending queue is more recent than the last commit sent to the caller. If the pending queue suddenly goes "backwards" due to one of our parent's having a more recent commit date, this new check ensures we will keep processing the graph until we see a more consistent cut.

We process an extra OVER_SCAN commits after we decide that all remaining commits are UNINTERESTING. Processing these extra commits ensures flags are carried back further in the graph, increasing the chances that we correctly mark relevant nodes.

As a result of this hack, we may produce a commit to our caller, but then later mark it UNINTERESTING if we discover date skew. To handle such cases, callers could buffer produced commits and filter out those that are UNINTERESTING, but this somewhat costly and may not always be necessary.

---

This version inverts the "n.commitTime <= last.commitTime", I confused myself and wrote it backwards.

The next patch I'm going to send with the tests has a test for this condition, to verify its correct.

 .../org/spearce/jgit/revwalk/PendingGenerator.java |   51 ++++++++++++++++++--
 1 files changed, 47 insertions(+), 4 deletions(-)
org.spearce.jgit/src/org/spearce/jgit/revwalk/PendingGenerator.java +47 -4
index 144e909..4e24431 100644
@@ -42,6 +42,7 @@
 import org.spearce.jgit.errors.IncorrectObjectTypeException;
 import org.spearce.jgit.errors.MissingObjectException;
 import org.spearce.jgit.errors.StopWalkException;
+import org.spearce.jgit.lib.ObjectId;
 import org.spearce.jgit.revwalk.filter.RevFilter;
 
 /**
@@ -60,6 +61,24 @@
 
 	private static final int UNINTERESTING = RevWalk.UNINTERESTING;
 
+	/**
+	 * Number of additional commits to scan after we think we are done.
+	 * <p>
+	 * This small buffer of commits is scanned to ensure we didn't miss anything
+	 * as a result of clock skew when the commits were made. We need to set our
+	 * constant to 1 additional commit due to the use of a pre-increment
+	 * operator when accessing the value.
+	 */
+	private static final int OVER_SCAN = 5 + 1;
+
+	/** A commit near the end of time, to initialize {@link #last} with. */
+	private static final RevCommit INIT_LAST;
+
+	static {
+		INIT_LAST = new RevCommit(ObjectId.zeroId());
+		INIT_LAST.commitTime = Integer.MAX_VALUE;
+	}
+
 	private final RevWalk walker;
 
 	private final DateRevQueue pending;
@@ -68,6 +87,17 @@
 
 	private final int output;
 
+	/** Last commit produced to the caller from {@link #next()}. */
+	private RevCommit last = INIT_LAST;
+
+	/**
+	 * Number of commits we have remaining in our over-scan allotment.
+	 * <p>
+	 * Only relevant if there are {@link #UNINTERESTING} commits in the
+	 * {@link #pending} queue.
+	 */
+	private int overScan = OVER_SCAN;
+
 	boolean canDispose;
 
 	PendingGenerator(final RevWalk w, final DateRevQueue p,
@@ -112,14 +142,27 @@ RevCommit next() throws MissingObjectException,
 				walker.carryFlagsImpl(c);
 
 				if ((c.flags & UNINTERESTING) != 0) {
-					if (pending.everbodyHasFlag(UNINTERESTING))
-						throw StopWalkException.INSTANCE;
-					c.dispose();
+					if (pending.everbodyHasFlag(UNINTERESTING)) {
+						final RevCommit n = pending.peek();
+						if (n != null && n.commitTime >= last.commitTime) {
+							// This is too close to call. The next commit we
+							// would pop is dated after the last one produced.
+							// We have to keep going to ensure that we carry
+							// flags as much as necessary.
+							//
+							overScan = OVER_SCAN;
+						} else if (--overScan == 0)
+							throw StopWalkException.INSTANCE;
+					} else {
+						overScan = OVER_SCAN;
+					}
+					if (canDispose)
+						c.dispose();
 					continue;
 				}
 
 				if (produce)
-					return c;
+					return last = c;
 				else if (canDispose)
 					c.dispose();
 			}
Signature · 1 line
1.6.2.288.gc3f22
#7 lore raw
In reply to Shawn O. Pearce

fredag 13 mars 2009 03:07:37 skrev "Shawn O. Pearce" <[email protected]>:

Quoted 3 lines

Today I uncovered some ugly cases with "jgit rev-list B ^A", where some commits reachable from A were still being output, even though we asked that they be excluded.

How about a test suite to prove this is better than before?

-- robin

#8 lore raw

cover Add tests for RevWalk and its supporting code

These new tests cover some of the common cases we see with using a RevWalk, and increase our code coverage in this critical area of the JGit library.

---

Quoted 6 lines · Robin Rosenberg

fredag 13 mars 2009 03:07:37 skrev "Shawn O. Pearce" <[email protected]>:

Quoted 3 lines

Today I uncovered some ugly cases with "jgit rev-list B ^A", where some commits reachable from A were still being output, even though we asked that they be excluded.

How about a test suite to prove this is better than before?

Its better. :-)

These tests don't pass before the series. They pass after.

 .../org/spearce/jgit/revwalk/RevFlagSetTest.java   |  131 +++++++++++
 .../org/spearce/jgit/revwalk/RevWalkCullTest.java  |   75 +++++++
 .../spearce/jgit/revwalk/RevWalkFilterTest.java    |  233 ++++++++++++++++++++
 .../spearce/jgit/revwalk/RevWalkMergeBaseTest.java |  117 ++++++++++
 .../org/spearce/jgit/revwalk/RevWalkSortTest.java  |  164 ++++++++++++++
 .../org/spearce/jgit/revwalk/RevWalkTestCase.java  |  102 +++++++++
 6 files changed, 822 insertions(+), 0 deletions(-)
 create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/revwalk/RevFlagSetTest.java
 create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/revwalk/RevWalkCullTest.java
 create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/revwalk/RevWalkFilterTest.java
 create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/revwalk/RevWalkMergeBaseTest.java
 create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/revwalk/RevWalkSortTest.java
 create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/revwalk/RevWalkTestCase.java
org.spearce.jgit.test/tst/org/spearce/jgit/revwalk/RevFlagSetTest.java new file +131 -0
new file mode 100644
index 0000000..76f3cbb
@@ -0,0 +1,131 @@
+/*
+ * Copyright (C) 2008, Google Inc.
+ *
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above copyright
+ *   notice, this list of conditions and the following disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ *   copyright notice, this list of conditions and the following
+ *   disclaimer in the documentation and/or other materials provided
+ *   with the distribution.
+ *
+ * - Neither the name of the Git Development Community nor the
+ *   names of its contributors may be used to endorse or promote
+ *   products derived from this software without specific prior
+ *   written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
+ * CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
+ * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
+ * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
+ * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+ * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
+ * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+package org.spearce.jgit.revwalk;
+
+import java.util.Arrays;
+import java.util.Iterator;
+
+public class RevFlagSetTest extends RevWalkTestCase {
+	public void testEmpty() {
+		final RevFlagSet set = new RevFlagSet();
+		assertEquals(0, set.mask);
+		assertEquals(0, set.size());
+		assertNotNull(set.iterator());
+		assertFalse(set.iterator().hasNext());
+	}
+
+	public void testAddOne() {
+		final String flagName = "flag";
+		final RevFlag flag = rw.newFlag(flagName);
+		assertTrue(0 != flag.mask);
+		assertSame(flagName, flag.name);
+
+		final RevFlagSet set = new RevFlagSet();
+		assertTrue(set.add(flag));
+		assertFalse(set.add(flag));
+		assertEquals(flag.mask, set.mask);
+		assertEquals(1, set.size());
+		final Iterator<RevFlag> i = set.iterator();
+		assertTrue(i.hasNext());
+		assertSame(flag, i.next());
+		assertFalse(i.hasNext());
+	}
+
+	public void testAddTwo() {
+		final RevFlag flag1 = rw.newFlag("flag_1");
+		final RevFlag flag2 = rw.newFlag("flag_2");
+		assertTrue((flag1.mask & flag2.mask) == 0);
+
+		final RevFlagSet set = new RevFlagSet();
+		assertTrue(set.add(flag1));
+		assertTrue(set.add(flag2));
+		assertEquals(flag1.mask | flag2.mask, set.mask);
+		assertEquals(2, set.size());
+	}
+
+	public void testContainsAll() {
+		final RevFlag flag1 = rw.newFlag("flag_1");
+		final RevFlag flag2 = rw.newFlag("flag_2");
+		final RevFlagSet set1 = new RevFlagSet();
+		assertTrue(set1.add(flag1));
+		assertTrue(set1.add(flag2));
+
+		assertTrue(set1.containsAll(set1));
+		assertTrue(set1.containsAll(Arrays
+				.asList(new RevFlag[] { flag1, flag2 })));
+
+		final RevFlagSet set2 = new RevFlagSet();
+		set2.add(rw.newFlag("flag_3"));
+		assertFalse(set1.containsAll(set2));
+	}
+
+	public void testEquals() {
+		final RevFlag flag1 = rw.newFlag("flag_1");
+		final RevFlag flag2 = rw.newFlag("flag_2");
+		final RevFlagSet set = new RevFlagSet();
+		assertTrue(set.add(flag1));
+		assertTrue(set.add(flag2));
+
+		assertTrue(new RevFlagSet(set).equals(set));
+		assertTrue(new RevFlagSet(Arrays.asList(new RevFlag[] { flag1, flag2 }))
+				.equals(set));
+	}
+
+	public void testRemove() {
+		final RevFlag flag1 = rw.newFlag("flag_1");
+		final RevFlag flag2 = rw.newFlag("flag_2");
+		final RevFlagSet set = new RevFlagSet();
+		assertTrue(set.add(flag1));
+		assertTrue(set.add(flag2));
+
+		assertTrue(set.remove(flag1));
+		assertFalse(set.remove(flag1));
+		assertEquals(flag2.mask, set.mask);
+		assertFalse(set.contains(flag1));
+	}
+
+	public void testContains() {
+		final RevFlag flag1 = rw.newFlag("flag_1");
+		final RevFlag flag2 = rw.newFlag("flag_2");
+		final RevFlagSet set = new RevFlagSet();
+		set.add(flag1);
+		assertTrue(set.contains(flag1));
+		assertFalse(set.contains(flag2));
+		assertFalse(set.contains("bob"));
+	}
+}
org.spearce.jgit.test/tst/org/spearce/jgit/revwalk/RevWalkCullTest.java new file +75 -0

75 lines not on this page · raw message

org.spearce.jgit.test/tst/org/spearce/jgit/revwalk/RevWalkFilterTest.java new file +233 -0

233 lines not on this page · raw message

org.spearce.jgit.test/tst/org/spearce/jgit/revwalk/RevWalkMergeBaseTest.java new file +117 -0

117 lines not on this page · raw message

org.spearce.jgit.test/tst/org/spearce/jgit/revwalk/RevWalkSortTest.java new file +164 -0

164 lines not on this page · raw message

org.spearce.jgit.test/tst/org/spearce/jgit/revwalk/RevWalkTestCase.java new file +102 -0

102 lines not on this page · raw message

Signature · 1 line
1.6.2.288.gc3f22
#9 lore raw

Re: [JGIT PATCH 0/6] Add tests for RevWalk and its supporting code

Quoted 7 lines · Shawn O. Pearce

These new tests cover some of the common cases we see with using a RevWalk, and increase our code coverage in this critical area of the JGit library.

---
Robin Rosenberg <[email protected]> wrote:

Quoted 8 lines
Quoted 6 lines

fredag 13 mars 2009 03:07:37 skrev "Shawn O. Pearce" <[email protected]>:

Quoted 3 lines

Today I uncovered some ugly cases with "jgit rev-list B ^A", where some commits reachable from A were still being output, even though we asked that they be excluded.

How about a test suite to prove this is better than before?

Its better. :-)

Its actually not better. Although the tests I'm replying to are right, there's another bug introduced by this series that they don't detect, but that shows up on the Linux kernel repository.

I have yet to track down the full thing... but I know its busted.

Signature · 1 line
Shawn.
#10 lore raw

[JGIT PATCH 6/5 v2] Add tests for RevWalk and its supporting code

These new tests cover some of the common cases we see with using a RevWalk, and increase our code coverage in this critical area of the JGit library.

---

Quoted 1 line · Shawn O. Pearce

I have yet to track down the full thing... but I know its busted.

This updated test patch includes a test for the bug I was talking about, and fixed with 4/5 v2.

 .../org/spearce/jgit/revwalk/RevFlagSetTest.java   |  131 +++++++++++
 .../org/spearce/jgit/revwalk/RevWalkCullTest.java  |   96 ++++++++
 .../spearce/jgit/revwalk/RevWalkFilterTest.java    |  233 ++++++++++++++++++++
 .../spearce/jgit/revwalk/RevWalkMergeBaseTest.java |  117 ++++++++++
 .../org/spearce/jgit/revwalk/RevWalkSortTest.java  |  164 ++++++++++++++
 .../org/spearce/jgit/revwalk/RevWalkTestCase.java  |  102 +++++++++
 6 files changed, 843 insertions(+), 0 deletions(-)
 create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/revwalk/RevFlagSetTest.java
 create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/revwalk/RevWalkCullTest.java
 create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/revwalk/RevWalkFilterTest.java
 create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/revwalk/RevWalkMergeBaseTest.java
 create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/revwalk/RevWalkSortTest.java
 create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/revwalk/RevWalkTestCase.java
org.spearce.jgit.test/tst/org/spearce/jgit/revwalk/RevFlagSetTest.java new file +131 -0
new file mode 100644
index 0000000..76f3cbb
@@ -0,0 +1,131 @@
+/*
+ * Copyright (C) 2008, Google Inc.
+ *
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above copyright
+ *   notice, this list of conditions and the following disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ *   copyright notice, this list of conditions and the following
+ *   disclaimer in the documentation and/or other materials provided
+ *   with the distribution.
+ *
+ * - Neither the name of the Git Development Community nor the
+ *   names of its contributors may be used to endorse or promote
+ *   products derived from this software without specific prior
+ *   written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
+ * CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
+ * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
+ * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
+ * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+ * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
+ * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+package org.spearce.jgit.revwalk;
+
+import java.util.Arrays;
+import java.util.Iterator;
+
+public class RevFlagSetTest extends RevWalkTestCase {
+	public void testEmpty() {
+		final RevFlagSet set = new RevFlagSet();
+		assertEquals(0, set.mask);
+		assertEquals(0, set.size());
+		assertNotNull(set.iterator());
+		assertFalse(set.iterator().hasNext());
+	}
+
+	public void testAddOne() {
+		final String flagName = "flag";
+		final RevFlag flag = rw.newFlag(flagName);
+		assertTrue(0 != flag.mask);
+		assertSame(flagName, flag.name);
+
+		final RevFlagSet set = new RevFlagSet();
+		assertTrue(set.add(flag));
+		assertFalse(set.add(flag));
+		assertEquals(flag.mask, set.mask);
+		assertEquals(1, set.size());
+		final Iterator<RevFlag> i = set.iterator();
+		assertTrue(i.hasNext());
+		assertSame(flag, i.next());
+		assertFalse(i.hasNext());
+	}
+
+	public void testAddTwo() {
+		final RevFlag flag1 = rw.newFlag("flag_1");
+		final RevFlag flag2 = rw.newFlag("flag_2");
+		assertTrue((flag1.mask & flag2.mask) == 0);
+
+		final RevFlagSet set = new RevFlagSet();
+		assertTrue(set.add(flag1));
+		assertTrue(set.add(flag2));
+		assertEquals(flag1.mask | flag2.mask, set.mask);
+		assertEquals(2, set.size());
+	}
+
+	public void testContainsAll() {
+		final RevFlag flag1 = rw.newFlag("flag_1");
+		final RevFlag flag2 = rw.newFlag("flag_2");
+		final RevFlagSet set1 = new RevFlagSet();
+		assertTrue(set1.add(flag1));
+		assertTrue(set1.add(flag2));
+
+		assertTrue(set1.containsAll(set1));
+		assertTrue(set1.containsAll(Arrays
+				.asList(new RevFlag[] { flag1, flag2 })));
+
+		final RevFlagSet set2 = new RevFlagSet();
+		set2.add(rw.newFlag("flag_3"));
+		assertFalse(set1.containsAll(set2));
+	}
+
+	public void testEquals() {
+		final RevFlag flag1 = rw.newFlag("flag_1");
+		final RevFlag flag2 = rw.newFlag("flag_2");
+		final RevFlagSet set = new RevFlagSet();
+		assertTrue(set.add(flag1));
+		assertTrue(set.add(flag2));
+
+		assertTrue(new RevFlagSet(set).equals(set));
+		assertTrue(new RevFlagSet(Arrays.asList(new RevFlag[] { flag1, flag2 }))
+				.equals(set));
+	}
+
+	public void testRemove() {
+		final RevFlag flag1 = rw.newFlag("flag_1");
+		final RevFlag flag2 = rw.newFlag("flag_2");
+		final RevFlagSet set = new RevFlagSet();
+		assertTrue(set.add(flag1));
+		assertTrue(set.add(flag2));
+
+		assertTrue(set.remove(flag1));
+		assertFalse(set.remove(flag1));
+		assertEquals(flag2.mask, set.mask);
+		assertFalse(set.contains(flag1));
+	}
+
+	public void testContains() {
+		final RevFlag flag1 = rw.newFlag("flag_1");
+		final RevFlag flag2 = rw.newFlag("flag_2");
+		final RevFlagSet set = new RevFlagSet();
+		set.add(flag1);
+		assertTrue(set.contains(flag1));
+		assertFalse(set.contains(flag2));
+		assertFalse(set.contains("bob"));
+	}
+}
org.spearce.jgit.test/tst/org/spearce/jgit/revwalk/RevWalkCullTest.java new file +96 -0

96 lines not on this page · raw message

org.spearce.jgit.test/tst/org/spearce/jgit/revwalk/RevWalkFilterTest.java new file +233 -0

233 lines not on this page · raw message

org.spearce.jgit.test/tst/org/spearce/jgit/revwalk/RevWalkMergeBaseTest.java new file +117 -0

117 lines not on this page · raw message

org.spearce.jgit.test/tst/org/spearce/jgit/revwalk/RevWalkSortTest.java new file +164 -0

164 lines not on this page · raw message

org.spearce.jgit.test/tst/org/spearce/jgit/revwalk/RevWalkTestCase.java new file +102 -0

102 lines not on this page · raw message

Signature · 1 line
1.6.2.288.gc3f22
#11 lore raw

Re: [JGIT PATCH 6/5 v2] Add tests for RevWalk and its supporting code

A few /09 interesting/ places in StartGenerator are not covered by the tests.

		if (q instanceof DateRevQueue)
			pending = (DateRevQueue)q;
		else

--> pending = new DateRevQueue(q);

and

		if (tf != TreeFilter.ALL) {
-->			rf = AndRevFilter.create(rf, new RewriteTreeFilter(w, tf));
-->			pendingOutputType |= HAS_REWRITE | NEEDS_REWRITE;
		}

PendingGenerator

						if (n != null && n.commitTime >= last.commitTime) {
							// This is too close to call. The next commit we
							// would pop is dated after the last one produced.
							// We have to keep going to ensure that we carry
							// flags as much as necessary.
							//

--> overScan = OVER_SCAN;

I'll merge it anyway since this is still a huge improvement compared to the previous state.

-- robin

#12 lore raw

Re: [JGIT PATCH 6/5 v2] Add tests for RevWalk and its supporting code

Quoted 7 lines · Robin Rosenberg

A few /09 interesting/ places in StartGenerator are not covered by the tests.

		if (q instanceof DateRevQueue)
			pending = (DateRevQueue)q;
		else

--> pending = new DateRevQueue(q);

This one is difficult to test. Near as I can tell from the code in the revwalk package, it never happens. But if we did get here with a non date rev queue, DateRevQueue()'s constructor can copy the data from q over to one with no risk.

Perhaps just test this constructor in isolation?

Quoted 6 lines

and

		if (tf != TreeFilter.ALL) {
-->			rf = AndRevFilter.create(rf, new RewriteTreeFilter(w, tf));
-->			pendingOutputType |= HAS_REWRITE | NEEDS_REWRITE;
		}

That says we have no tests on the path filter code. Adding one is likely to identify at least one bug. Clearly we need additional tests on the path filter sections. And it isn't just for this one little conditional; the entire RewriteTreeFilter and RewriteGenerator are untested right now.

I can work on adding more test coverage here, but its not in the critical path for me at day-job right now. The other problems I tried to address in this series were. So its lower in my queue so-to-speak. But I'll see what I can do in the near future to write additional tests for this area of the revwalk package.

Quoted 8 lines

PendingGenerator

						if (n != null && n.commitTime >= last.commitTime) {
							// This is too close to call. The next commit we
							// would pop is dated after the last one produced.
							// We have to keep going to ensure that we carry
							// flags as much as necessary.
							//

--> overScan = OVER_SCAN;

*sigh* We never get here in the test suite? I missed that. I'll try to work out a test case today and send an additional patch to get coverage here. I thought RevWalkCullTest testProperlyCullAllAncestors1() would be hitting here. It doesn't.

Quoted 2 lines

I'll merge it anyway since this is still a huge improvement compared to the previous state.

Thanks.

A co-worker and I are also currently trying to track down a deadlock between C git-fetch and JGit upload-pack. Both get stuck waiting for the other to answer. Clearly its JGit's fault.

Signature · 1 line
Shawn.