Skip to contentSkip to conversation

Support receive.fsckobjects

PATCH In review Shawn O. Pearce 4 participants 11 messages Jun 15, 2016

lore
Share & tools
lore
#0 lore raw

cover Support receive.fsckobjects

This series adds support for receive.fsckobjects, but on the fetch side of the connection. Perhaps it should be transfer.fsckobjects or fetch.fsckobjects, but git.git doesn't support either of those right now.

I mainly need this series because I'm fetching out of untrusted bundles. The content of the bundle has to pass git-fsck for it to be considered safe.

The ObjectChecker class covers the same rules as git-fsck does, and is perhaps even stricter on some of the things git-fsck lets slide. I think git-fsck is too lenient in some areas, and I'd like to try and improve the rules more in git.git, but I don't have time for it right now.

Shawn O. Pearce (5):
Expose RawParseUtils.match to application callers Fix UnpackedObjectLoader.getBytes to return a copy Object validation tests for "jgit fsck"
Expose the critical receive configuration options to JGit Honor receive.fsckobjects during any fetch connection

 .../org/spearce/jgit/lib/ObjectCheckerTest.java    | 1294 ++++++++++++++++++++
 .../src/org/spearce/jgit/lib/ObjectChecker.java    |  352 ++++++
 .../src/org/spearce/jgit/lib/ObjectLoader.java     |    7 +-
 .../org/spearce/jgit/lib/PackedObjectLoader.java   |    7 -
 .../src/org/spearce/jgit/lib/RepositoryConfig.java |   10 +
 .../src/org/spearce/jgit/lib/TransferConfig.java   |   56 +
 .../org/spearce/jgit/lib/UnpackedObjectLoader.java |    4 -
 .../jgit/transport/BasePackFetchConnection.java    |    1 +
 .../src/org/spearce/jgit/transport/IndexPack.java  |   60 +-
 .../src/org/spearce/jgit/transport/Transport.java  |   24 +
 .../spearce/jgit/transport/TransportBundle.java    |   10 +-
 .../jgit/transport/WalkFetchConnection.java        |   26 +-
 .../src/org/spearce/jgit/util/RawParseUtils.java   |   23 +-
 13 files changed, 1842 insertions(+), 32 deletions(-)
 create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/lib/ObjectCheckerTest.java
 create mode 100644 org.spearce.jgit/src/org/spearce/jgit/lib/ObjectChecker.java
 create mode 100644 org.spearce.jgit/src/org/spearce/jgit/lib/TransferConfig.java
#1 lore raw

PATCH 1/5 Expose RawParseUtils.match to application callers

This utility can be useful when parsing a buffer directly, such as for a commit or tag object's header lines.

---
 .../src/org/spearce/jgit/util/RawParseUtils.java   |   13 ++++++++++++-
 1 files changed, 12 insertions(+), 1 deletions(-)
org.spearce.jgit/src/org/spearce/jgit/util/RawParseUtils.java +12 -1
index dbc2e83..2ab3bfe 100644
@@ -61,7 +61,18 @@
 			digits[i] = (byte) (i - '0');
 	}
 
-	private static final int match(final byte[] b, int ptr, final byte[] src) {
+	/**
+	 * Determine if b[ptr] matches src.
+	 * 
+	 * @param b
+	 *            the buffer to scan.
+	 * @param ptr
+	 *            first position within b, this should match src[0].
+	 * @param src
+	 *            the buffer to test for equality with b.
+	 * @return ptr += src.length if b[ptr..src.length] == src; else -1.
+	 */
+	public static final int match(final byte[] b, int ptr, final byte[] src) {
 		if (ptr + src.length >= b.length)
 			return -1;
 		for (int i = 0; i < src.length; i++, ptr++)
Signature · 1 line
1.6.0.2.569.g798a2a
#2 lore raw

PATCH 2/5 Fix UnpackedObjectLoader.getBytes to return a copy

The contract for ObjectLoader.getBytes() says the caller can modify the returned array. UnpackedObjectLoader must copy the data and not return its internal cached byte array.

---
 .../src/org/spearce/jgit/lib/ObjectLoader.java     |    7 ++++++-
 .../org/spearce/jgit/lib/PackedObjectLoader.java   |    7 -------
 .../org/spearce/jgit/lib/UnpackedObjectLoader.java |    4 ----
 3 files changed, 6 insertions(+), 12 deletions(-)
org.spearce.jgit/src/org/spearce/jgit/lib/ObjectLoader.java +6 -1
index 5282491..87e861f 100644
@@ -105,7 +105,12 @@ protected void setId(final ObjectId id) {
 	 * @throws IOException
 	 *             the object cannot be read.
 	 */
-	public abstract byte[] getBytes() throws IOException;
+	public final byte[] getBytes() throws IOException {
+		final byte[] data = getCachedBytes();
+		final byte[] copy = new byte[data.length];
+		System.arraycopy(data, 0, copy, 0, data.length);
+		return data;
+	}
 
 	/**
 	 * Obtain a reference to the (possibly cached) bytes of this object.
org.spearce.jgit/src/org/spearce/jgit/lib/PackedObjectLoader.java +0 -7
index fa414d6..35983fe 100644
@@ -80,13 +80,6 @@ public long getDataOffset() {
 		return dataOffset;
 	}
 
-	public final byte[] getBytes() throws IOException {
-		final byte[] data = getCachedBytes();
-		final byte[] copy = new byte[data.length];
-		System.arraycopy(data, 0, copy, 0, data.length);
-		return data;
-	}
-
 	/**
 	 * Copy raw object representation from storage to provided output stream.
 	 * <p>
org.spearce.jgit/src/org/spearce/jgit/lib/UnpackedObjectLoader.java +0 -4
index 3c61254..3ad273f 100644
@@ -200,10 +200,6 @@ public long getSize() {
 		return objectSize;
 	}
 
-	public byte[] getBytes() {
-		return bytes;
-	}
-
 	@Override
 	public byte[] getCachedBytes() throws IOException {
 		return bytes;
Signature · 1 line
1.6.0.2.569.g798a2a
#3 lore raw

PATCH 3/5 Object validation tests for "jgit fsck"

The ObjectChecker provides strict validation rules for objects. If any commit, tag or tree object is malformed in a way that might cause a Git implementation to misinterpret the data CorruptObjectException is thrown back to the caller for error handling.

Due to the shear size of the validation code this change provides only the validation code and its unit tests. It is at least as paranoid as git.git's "git-fsck" is on the same object types, but is actually a bit stricter about the commit and tag objects having the well known header fields populated correctly.

---
 .../org/spearce/jgit/lib/ObjectCheckerTest.java    | 1294 ++++++++++++++++++++
 .../src/org/spearce/jgit/lib/ObjectChecker.java    |  352 ++++++
 .../src/org/spearce/jgit/util/RawParseUtils.java   |   10 +-
 3 files changed, 1650 insertions(+), 6 deletions(-)
 create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/lib/ObjectCheckerTest.java
 create mode 100644 org.spearce.jgit/src/org/spearce/jgit/lib/ObjectChecker.java
org.spearce.jgit.test/tst/org/spearce/jgit/lib/ObjectCheckerTest.java new file +1294 -0
new file mode 100644
index 0000000..fa37fb5
@@ -0,0 +1,1294 @@
+/*
+ * Copyright (C) 2008, Shawn O. Pearce <[email protected]>
+ * Copyright (C) 2008, Google Inc.
+ *
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above copyright
+ *   notice, this list of conditions and the following disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ *   copyright notice, this list of conditions and the following
+ *   disclaimer in the documentation and/or other materials provided
+ *   with the distribution.
+ *
+ * - Neither the name of the Git Development Community nor the
+ *   names of its contributors may be used to endorse or promote
+ *   products derived from this software without specific prior
+ *   written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
+ * CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
+ * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
+ * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
+ * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+ * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
+ * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+package org.spearce.jgit.lib;
+
+import junit.framework.TestCase;
+
+import org.spearce.jgit.errors.CorruptObjectException;
+
+public class ObjectCheckerTest extends TestCase {
+	private ObjectChecker checker;
+
+	protected void setUp() throws Exception {
+		super.setUp();
+		checker = new ObjectChecker();
+	}
+
+	public void testInvalidType() {
+		try {
+			checker.check(Constants.OBJ_BAD, new byte[0]);
+			fail("Did not throw CorruptObjectException");
+		} catch (CorruptObjectException e) {
+			final String m = e.getMessage();
+			assertEquals("Invalid object type: " + Constants.OBJ_BAD, m);
+		}
+	}
+
+	public void testCheckBlob() throws CorruptObjectException {
+		// Any blob should pass...
+		checker.checkBlob(new byte[0]);
+		checker.checkBlob(new byte[1]);
+
+		checker.check(Constants.OBJ_BLOB, new byte[0]);
+		checker.check(Constants.OBJ_BLOB, new byte[1]);
+	}
+
+	public void testValidCommitNoParent() throws CorruptObjectException {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("tree ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		b.append("author A. U. Thor <author@localhost> 1 +0000\n");
+		b.append("committer A. U. Thor <author@localhost> 1 +0000\n");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		checker.checkCommit(data);
+		checker.check(Constants.OBJ_COMMIT, data);
+	}
+
+	public void testValidCommitBlankAuthor() throws CorruptObjectException {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("tree ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		b.append("author <> 0 +0000\n");
+		b.append("committer <> 0 +0000\n");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		checker.checkCommit(data);
+		checker.check(Constants.OBJ_COMMIT, data);
+	}
+
+	public void testValidCommit1Parent() throws CorruptObjectException {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("tree ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		b.append("parent ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		b.append("author A. U. Thor <author@localhost> 1 +0000\n");
+		b.append("committer A. U. Thor <author@localhost> 1 +0000\n");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		checker.checkCommit(data);
+		checker.check(Constants.OBJ_COMMIT, data);
+	}
+
+	public void testValidCommit2Parent() throws CorruptObjectException {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("tree ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		b.append("parent ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		b.append("parent ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		b.append("author A. U. Thor <author@localhost> 1 +0000\n");
+		b.append("committer A. U. Thor <author@localhost> 1 +0000\n");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		checker.checkCommit(data);
+		checker.check(Constants.OBJ_COMMIT, data);
+	}
+
+	public void testValidCommit128Parent() throws CorruptObjectException {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("tree ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		for (int i = 0; i < 128; i++) {
+			b.append("parent ");
+			b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+			b.append('\n');
+		}
+
+		b.append("author A. U. Thor <author@localhost> 1 +0000\n");
+		b.append("committer A. U. Thor <author@localhost> 1 +0000\n");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		checker.checkCommit(data);
+		checker.check(Constants.OBJ_COMMIT, data);
+	}
+
+	public void testValidCommitNormalTime() throws CorruptObjectException {
+		final StringBuilder b = new StringBuilder();
+		final String when = "1222757360 -0730";
+
+		b.append("tree ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		b.append("author A. U. Thor <author@localhost> " + when + "\n");
+		b.append("committer A. U. Thor <author@localhost> " + when + "\n");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		checker.checkCommit(data);
+		checker.check(Constants.OBJ_COMMIT, data);
+	}
+
+	public void testInvalidCommitNoTree1() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("parent ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkCommit(data);
+			fail("Did not catch corrupt object");
+		} catch (CorruptObjectException e) {
+			assertEquals("no tree header", e.getMessage());
+		}
+	}
+
+	public void testInvalidCommitNoTree2() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("trie ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkCommit(data);
+			fail("Did not catch corrupt object");
+		} catch (CorruptObjectException e) {
+			assertEquals("no tree header", e.getMessage());
+		}
+	}
+
+	public void testInvalidCommitNoTree3() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("tree");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkCommit(data);
+			fail("Did not catch corrupt object");
+		} catch (CorruptObjectException e) {
+			assertEquals("no tree header", e.getMessage());
+		}
+	}
+
+	public void testInvalidCommitNoTree4() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("tree\t");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkCommit(data);
+			fail("Did not catch corrupt object");
+		} catch (CorruptObjectException e) {
+			assertEquals("no tree header", e.getMessage());
+		}
+	}
+
+	public void testInvalidCommitInvalidTree1() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("tree ");
+		b.append("zzzzfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkCommit(data);
+			fail("Did not catch corrupt object");
+		} catch (CorruptObjectException e) {
+			assertEquals("invalid tree", e.getMessage());
+		}
+	}
+
+	public void testInvalidCommitInvalidTree2() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("tree ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append("z\n");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkCommit(data);
+			fail("Did not catch corrupt object");
+		} catch (CorruptObjectException e) {
+			assertEquals("invalid tree", e.getMessage());
+		}
+	}
+
+	public void testInvalidCommitInvalidTree3() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("tree ");
+		b.append("be9b");
+		b.append("\n");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkCommit(data);
+			fail("Did not catch corrupt object");
+		} catch (CorruptObjectException e) {
+			assertEquals("invalid tree", e.getMessage());
+		}
+	}
+
+	public void testInvalidCommitInvalidTree4() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("tree  ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkCommit(data);
+			fail("Did not catch corrupt object");
+		} catch (CorruptObjectException e) {
+			assertEquals("invalid tree", e.getMessage());
+		}
+	}
+
+	public void testInvalidCommitInvalidParent1() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("tree ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		b.append("parent ");
+		b.append("\n");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkCommit(data);
+			fail("Did not catch corrupt object");
+		} catch (CorruptObjectException e) {
+			assertEquals("invalid parent", e.getMessage());
+		}
+	}
+
+	public void testInvalidCommitInvalidParent2() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("tree ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		b.append("parent ");
+		b.append("zzzzfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append("\n");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkCommit(data);
+			fail("Did not catch corrupt object");
+		} catch (CorruptObjectException e) {
+			assertEquals("invalid parent", e.getMessage());
+		}
+	}
+
+	public void testInvalidCommitInvalidParent3() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("tree ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		b.append("parent  ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append("\n");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkCommit(data);
+			fail("Did not catch corrupt object");
+		} catch (CorruptObjectException e) {
+			assertEquals("invalid parent", e.getMessage());
+		}
+	}
+
+	public void testInvalidCommitInvalidParent4() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("tree ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		b.append("parent  ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append("z\n");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkCommit(data);
+			fail("Did not catch corrupt object");
+		} catch (CorruptObjectException e) {
+			assertEquals("invalid parent", e.getMessage());
+		}
+	}
+
+	public void testInvalidCommitInvalidParent5() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("tree ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		b.append("parent\t");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append("\n");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkCommit(data);
+			fail("Did not catch corrupt object");
+		} catch (CorruptObjectException e) {
+			// Yes, really, we complain about author not being
+			// found as the invalid parent line wasn't consumed.
+			assertEquals("no author", e.getMessage());
+		}
+	}
+
+	public void testInvalidCommitNoAuthor() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("tree ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		b.append("committer A. U. Thor <author@localhost> 1 +0000\n");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkCommit(data);
+			fail("Did not catch corrupt object");
+		} catch (CorruptObjectException e) {
+			// Yes, really, we complain about author not being
+			// found as the invalid parent line wasn't consumed.
+			assertEquals("no author", e.getMessage());
+		}
+	}
+
+	public void testInvalidCommitNoCommitter1() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("tree ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		b.append("author A. U. Thor <author@localhost> 1 +0000\n");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkCommit(data);
+			fail("Did not catch corrupt object");
+		} catch (CorruptObjectException e) {
+			// Yes, really, we complain about author not being
+			// found as the invalid parent line wasn't consumed.
+			assertEquals("no committer", e.getMessage());
+		}
+	}
+
+	public void testInvalidCommitNoCommitter2() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("tree ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		b.append("author A. U. Thor <author@localhost> 1 +0000\n");
+		b.append("\n");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkCommit(data);
+			fail("Did not catch corrupt object");
+		} catch (CorruptObjectException e) {
+			// Yes, really, we complain about author not being
+			// found as the invalid parent line wasn't consumed.
+			assertEquals("no committer", e.getMessage());
+		}
+	}
+
+	public void testInvalidCommitInvalidAuthor1() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("tree ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		b.append("author A. U. Thor <foo 1 +0000\n");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkCommit(data);
+			fail("Did not catch corrupt object");
+		} catch (CorruptObjectException e) {
+			// Yes, really, we complain about author not being
+			// found as the invalid parent line wasn't consumed.
+			assertEquals("invalid author", e.getMessage());
+		}
+	}
+
+	public void testInvalidCommitInvalidAuthor2() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("tree ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		b.append("author A. U. Thor foo> 1 +0000\n");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkCommit(data);
+			fail("Did not catch corrupt object");
+		} catch (CorruptObjectException e) {
+			// Yes, really, we complain about author not being
+			// found as the invalid parent line wasn't consumed.
+			assertEquals("invalid author", e.getMessage());
+		}
+	}
+
+	public void testInvalidCommitInvalidAuthor3() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("tree ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		b.append("author 1 +0000\n");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkCommit(data);
+			fail("Did not catch corrupt object");
+		} catch (CorruptObjectException e) {
+			// Yes, really, we complain about author not being
+			// found as the invalid parent line wasn't consumed.
+			assertEquals("invalid author", e.getMessage());
+		}
+	}
+
+	public void testInvalidCommitInvalidAuthor4() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("tree ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		b.append("author a <b> +0000\n");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkCommit(data);
+			fail("Did not catch corrupt object");
+		} catch (CorruptObjectException e) {
+			// Yes, really, we complain about author not being
+			// found as the invalid parent line wasn't consumed.
+			assertEquals("invalid author", e.getMessage());
+		}
+	}
+
+	public void testInvalidCommitInvalidAuthor5() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("tree ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		b.append("author a <b>\n");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkCommit(data);
+			fail("Did not catch corrupt object");
+		} catch (CorruptObjectException e) {
+			// Yes, really, we complain about author not being
+			// found as the invalid parent line wasn't consumed.
+			assertEquals("invalid author", e.getMessage());
+		}
+	}
+
+	public void testInvalidCommitInvalidAuthor6() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("tree ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		b.append("author a <b> z");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkCommit(data);
+			fail("Did not catch corrupt object");
+		} catch (CorruptObjectException e) {
+			// Yes, really, we complain about author not being
+			// found as the invalid parent line wasn't consumed.
+			assertEquals("invalid author", e.getMessage());
+		}
+	}
+
+	public void testInvalidCommitInvalidAuthor7() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("tree ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		b.append("author a <b> 1 z");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkCommit(data);
+			fail("Did not catch corrupt object");
+		} catch (CorruptObjectException e) {
+			// Yes, really, we complain about author not being
+			// found as the invalid parent line wasn't consumed.
+			assertEquals("invalid author", e.getMessage());
+		}
+	}
+
+	public void testInvalidCommitInvalidCommitter() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("tree ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		b.append("author a <b> 1 +0000\n");
+		b.append("committer a <");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkCommit(data);
+			fail("Did not catch corrupt object");
+		} catch (CorruptObjectException e) {
+			// Yes, really, we complain about author not being
+			// found as the invalid parent line wasn't consumed.
+			assertEquals("invalid committer", e.getMessage());
+		}
+	}
+
+	public void testValidTag() throws CorruptObjectException {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("object ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		b.append("type commit\n");
+		b.append("tag test-tag\n");
+		b.append("tagger A. U. Thor <author@localhost> 1 +0000\n");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		checker.checkTag(data);
+		checker.check(Constants.OBJ_TAG, data);
+	}
+
+	public void testInvalidTagNoObject1() {
+		final StringBuilder b = new StringBuilder();
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkTag(data);
+			fail("incorrectly accepted invalid tag");
+		} catch (CorruptObjectException e) {
+			assertEquals("no object header", e.getMessage());
+		}
+	}
+
+	public void testInvalidTagNoObject2() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("object\t");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkTag(data);
+			fail("incorrectly accepted invalid tag");
+		} catch (CorruptObjectException e) {
+			assertEquals("no object header", e.getMessage());
+		}
+	}
+
+	public void testInvalidTagNoObject3() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("obejct ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkTag(data);
+			fail("incorrectly accepted invalid tag");
+		} catch (CorruptObjectException e) {
+			assertEquals("no object header", e.getMessage());
+		}
+	}
+
+	public void testInvalidTagNoObject4() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("object ");
+		b.append("zz9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkTag(data);
+			fail("incorrectly accepted invalid tag");
+		} catch (CorruptObjectException e) {
+			assertEquals("invalid object", e.getMessage());
+		}
+	}
+
+	public void testInvalidTagNoObject5() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("object ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append(" \n");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkTag(data);
+			fail("incorrectly accepted invalid tag");
+		} catch (CorruptObjectException e) {
+			assertEquals("invalid object", e.getMessage());
+		}
+	}
+
+	public void testInvalidTagNoObject6() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("object ");
+		b.append("be9");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkTag(data);
+			fail("incorrectly accepted invalid tag");
+		} catch (CorruptObjectException e) {
+			assertEquals("invalid object", e.getMessage());
+		}
+	}
+
+	public void testInvalidTagNoType1() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("object ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkTag(data);
+			fail("incorrectly accepted invalid tag");
+		} catch (CorruptObjectException e) {
+			assertEquals("no type header", e.getMessage());
+		}
+	}
+
+	public void testInvalidTagNoType2() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("object ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		b.append("type\tcommit\n");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkTag(data);
+			fail("incorrectly accepted invalid tag");
+		} catch (CorruptObjectException e) {
+			assertEquals("no type header", e.getMessage());
+		}
+	}
+
+	public void testInvalidTagNoType3() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("object ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		b.append("tpye commit\n");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkTag(data);
+			fail("incorrectly accepted invalid tag");
+		} catch (CorruptObjectException e) {
+			assertEquals("no type header", e.getMessage());
+		}
+	}
+
+	public void testInvalidTagNoType4() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("object ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		b.append("type commit");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkTag(data);
+			fail("incorrectly accepted invalid tag");
+		} catch (CorruptObjectException e) {
+			assertEquals("no tag header", e.getMessage());
+		}
+	}
+
+	public void testInvalidTagNoTagHeader1() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("object ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		b.append("type commit\n");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkTag(data);
+			fail("incorrectly accepted invalid tag");
+		} catch (CorruptObjectException e) {
+			assertEquals("no tag header", e.getMessage());
+		}
+	}
+
+	public void testInvalidTagNoTagHeader2() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("object ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		b.append("type commit\n");
+		b.append("tag\tfoo\n");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkTag(data);
+			fail("incorrectly accepted invalid tag");
+		} catch (CorruptObjectException e) {
+			assertEquals("no tag header", e.getMessage());
+		}
+	}
+
+	public void testInvalidTagNoTagHeader3() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("object ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		b.append("type commit\n");
+		b.append("tga foo\n");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkTag(data);
+			fail("incorrectly accepted invalid tag");
+		} catch (CorruptObjectException e) {
+			assertEquals("no tag header", e.getMessage());
+		}
+	}
+
+	public void testInvalidTagNoTagHeader4() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("object ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		b.append("type commit\n");
+		b.append("tag foo");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkTag(data);
+			fail("incorrectly accepted invalid tag");
+		} catch (CorruptObjectException e) {
+			assertEquals("no tagger header", e.getMessage());
+		}
+	}
+
+	public void testInvalidTagNoTaggerHeader1() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("object ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		b.append("type commit\n");
+		b.append("tag foo\n");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkTag(data);
+			fail("incorrectly accepted invalid tag");
+		} catch (CorruptObjectException e) {
+			assertEquals("no tagger header", e.getMessage());
+		}
+	}
+
+	public void testInvalidTagInvalidTaggerHeader1() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("object ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		b.append("type commit\n");
+		b.append("tag foo\n");
+		b.append("tagger \n");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkTag(data);
+			fail("incorrectly accepted invalid tag");
+		} catch (CorruptObjectException e) {
+			assertEquals("invalid tagger", e.getMessage());
+		}
+	}
+
+	public void testInvalidTagInvalidTaggerHeader3() {
+		final StringBuilder b = new StringBuilder();
+
+		b.append("object ");
+		b.append("be9bfa841874ccc9f2ef7c48d0c76226f89b7189");
+		b.append('\n');
+
+		b.append("type commit\n");
+		b.append("tag foo\n");
+		b.append("tagger a < 1 +000\n");
+
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkTag(data);
+			fail("incorrectly accepted invalid tag");
+		} catch (CorruptObjectException e) {
+			assertEquals("invalid tagger", e.getMessage());
+		}
+	}
+
+	public void testValidEmptyTree() throws CorruptObjectException {
+		checker.checkTree(new byte[0]);
+		checker.check(Constants.OBJ_TREE, new byte[0]);
+	}
+
+	public void testValidTree1() throws CorruptObjectException {
+		final StringBuilder b = new StringBuilder();
+		entry(b, "100644 regular-file");
+		final byte[] data = Constants.encodeASCII(b.toString());
+		checker.checkTree(data);
+	}
+
+	public void testValidTree2() throws CorruptObjectException {
+		final StringBuilder b = new StringBuilder();
+		entry(b, "100755 executable");
+		final byte[] data = Constants.encodeASCII(b.toString());
+		checker.checkTree(data);
+	}
+
+	public void testValidTree3() throws CorruptObjectException {
+		final StringBuilder b = new StringBuilder();
+		entry(b, "40000 tree");
+		final byte[] data = Constants.encodeASCII(b.toString());
+		checker.checkTree(data);
+	}
+
+	public void testValidTree4() throws CorruptObjectException {
+		final StringBuilder b = new StringBuilder();
+		entry(b, "120000 symlink");
+		final byte[] data = Constants.encodeASCII(b.toString());
+		checker.checkTree(data);
+	}
+
+	public void testValidTree5() throws CorruptObjectException {
+		final StringBuilder b = new StringBuilder();
+		entry(b, "160000 git link");
+		final byte[] data = Constants.encodeASCII(b.toString());
+		checker.checkTree(data);
+	}
+
+	public void testValidTreeSorting1() throws CorruptObjectException {
+		final StringBuilder b = new StringBuilder();
+		entry(b, "100644 fooaaa");
+		entry(b, "100755 foobar");
+		final byte[] data = Constants.encodeASCII(b.toString());
+		checker.checkTree(data);
+	}
+
+	public void testValidTreeSorting2() throws CorruptObjectException {
+		final StringBuilder b = new StringBuilder();
+		entry(b, "100755 fooaaa");
+		entry(b, "100644 foobar");
+		final byte[] data = Constants.encodeASCII(b.toString());
+		checker.checkTree(data);
+	}
+
+	public void testValidTreeSorting3() throws CorruptObjectException {
+		final StringBuilder b = new StringBuilder();
+		entry(b, "40000 a");
+		entry(b, "100644 b");
+		final byte[] data = Constants.encodeASCII(b.toString());
+		checker.checkTree(data);
+	}
+
+	public void testValidTreeSorting4() throws CorruptObjectException {
+		final StringBuilder b = new StringBuilder();
+		entry(b, "100644 a");
+		entry(b, "40000 b");
+		final byte[] data = Constants.encodeASCII(b.toString());
+		checker.checkTree(data);
+	}
+
+	public void testValidTreeSorting5() throws CorruptObjectException {
+		final StringBuilder b = new StringBuilder();
+		entry(b, "100644 a.c");
+		entry(b, "40000 a");
+		entry(b, "100644 a0c");
+		final byte[] data = Constants.encodeASCII(b.toString());
+		checker.checkTree(data);
+	}
+
+	public void testValidTreeSorting6() throws CorruptObjectException {
+		final StringBuilder b = new StringBuilder();
+		entry(b, "40000 a");
+		entry(b, "100644 apple");
+		final byte[] data = Constants.encodeASCII(b.toString());
+		checker.checkTree(data);
+	}
+
+	public void testValidTreeSorting7() throws CorruptObjectException {
+		final StringBuilder b = new StringBuilder();
+		entry(b, "40000 an orang");
+		entry(b, "40000 an orange");
+		final byte[] data = Constants.encodeASCII(b.toString());
+		checker.checkTree(data);
+	}
+
+	public void testValidTreeSorting8() throws CorruptObjectException {
+		final StringBuilder b = new StringBuilder();
+		entry(b, "100644 a");
+		entry(b, "100644 a0c");
+		entry(b, "100644 b");
+		final byte[] data = Constants.encodeASCII(b.toString());
+		checker.checkTree(data);
+	}
+
+	public void testInvalidTreeModeStartsWithZero1() {
+		final StringBuilder b = new StringBuilder();
+		entry(b, "0 a");
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkTree(data);
+			fail("incorrectly accepted an invalid tree");
+		} catch (CorruptObjectException e) {
+			assertEquals("mode starts with '0'", e.getMessage());
+		}
+	}
+
+	public void testInvalidTreeModeStartsWithZero2() {
+		final StringBuilder b = new StringBuilder();
+		entry(b, "0100644 a");
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkTree(data);
+			fail("incorrectly accepted an invalid tree");
+		} catch (CorruptObjectException e) {
+			assertEquals("mode starts with '0'", e.getMessage());
+		}
+	}
+
+	public void testInvalidTreeModeStartsWithZero3() {
+		final StringBuilder b = new StringBuilder();
+		entry(b, "040000 a");
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkTree(data);
+			fail("incorrectly accepted an invalid tree");
+		} catch (CorruptObjectException e) {
+			assertEquals("mode starts with '0'", e.getMessage());
+		}
+	}
+
+	public void testInvalidTreeModeNotOctal1() {
+		final StringBuilder b = new StringBuilder();
+		entry(b, "8 a");
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkTree(data);
+			fail("incorrectly accepted an invalid tree");
+		} catch (CorruptObjectException e) {
+			assertEquals("invalid mode character", e.getMessage());
+		}
+	}
+
+	public void testInvalidTreeModeNotOctal2() {
+		final StringBuilder b = new StringBuilder();
+		entry(b, "Z a");
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkTree(data);
+			fail("incorrectly accepted an invalid tree");
+		} catch (CorruptObjectException e) {
+			assertEquals("invalid mode character", e.getMessage());
+		}
+	}
+
+	public void testInvalidTreeModeNotSupportedMode1() {
+		final StringBuilder b = new StringBuilder();
+		entry(b, "1 a");
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkTree(data);
+			fail("incorrectly accepted an invalid tree");
+		} catch (CorruptObjectException e) {
+			assertEquals("invalid mode 1", e.getMessage());
+		}
+	}
+
+	public void testInvalidTreeModeNotSupportedMode2() {
+		final StringBuilder b = new StringBuilder();
+		entry(b, "170000 a");
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkTree(data);
+			fail("incorrectly accepted an invalid tree");
+		} catch (CorruptObjectException e) {
+			assertEquals("invalid mode " + 0170000, e.getMessage());
+		}
+	}
+
+	public void testInvalidTreeModeMissingName() {
+		final StringBuilder b = new StringBuilder();
+		b.append("100644");
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkTree(data);
+			fail("incorrectly accepted an invalid tree");
+		} catch (CorruptObjectException e) {
+			assertEquals("truncated in mode", e.getMessage());
+		}
+	}
+
+	public void testInvalidTreeNameContainsSlash() {
+		final StringBuilder b = new StringBuilder();
+		entry(b, "100644 a/b");
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkTree(data);
+			fail("incorrectly accepted an invalid tree");
+		} catch (CorruptObjectException e) {
+			assertEquals("name contains '/'", e.getMessage());
+		}
+	}
+
+	public void testInvalidTreeNameIsEmpty() {
+		final StringBuilder b = new StringBuilder();
+		entry(b, "100644 ");
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkTree(data);
+			fail("incorrectly accepted an invalid tree");
+		} catch (CorruptObjectException e) {
+			assertEquals("zero length name", e.getMessage());
+		}
+	}
+
+	public void testInvalidTreeTruncatedInName() {
+		final StringBuilder b = new StringBuilder();
+		b.append("100644 b");
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkTree(data);
+			fail("incorrectly accepted an invalid tree");
+		} catch (CorruptObjectException e) {
+			assertEquals("truncated in name", e.getMessage());
+		}
+	}
+
+	public void testInvalidTreeTruncatedInObjectId() {
+		final StringBuilder b = new StringBuilder();
+		b.append("100644 b\0\1\2");
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkTree(data);
+			fail("incorrectly accepted an invalid tree");
+		} catch (CorruptObjectException e) {
+			assertEquals("truncated in object id", e.getMessage());
+		}
+	}
+
+	public void testInvalidTreeBadSorting1() {
+		final StringBuilder b = new StringBuilder();
+		entry(b, "100644 foobar");
+		entry(b, "100644 fooaaa");
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkTree(data);
+			fail("incorrectly accepted an invalid tree");
+		} catch (CorruptObjectException e) {
+			assertEquals("incorrectly sorted", e.getMessage());
+		}
+	}
+
+	public void testInvalidTreeBadSorting2() {
+		final StringBuilder b = new StringBuilder();
+		entry(b, "40000 a");
+		entry(b, "100644 a.c");
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkTree(data);
+			fail("incorrectly accepted an invalid tree");
+		} catch (CorruptObjectException e) {
+			assertEquals("incorrectly sorted", e.getMessage());
+		}
+	}
+
+	public void testInvalidTreeBadSorting3() {
+		final StringBuilder b = new StringBuilder();
+		entry(b, "100644 a0c");
+		entry(b, "40000 a");
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkTree(data);
+			fail("incorrectly accepted an invalid tree");
+		} catch (CorruptObjectException e) {
+			assertEquals("incorrectly sorted", e.getMessage());
+		}
+	}
+
+	public void testInvalidTreeDuplicateNames1() {
+		final StringBuilder b = new StringBuilder();
+		entry(b, "100644 a");
+		entry(b, "100644 a");
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkTree(data);
+			fail("incorrectly accepted an invalid tree");
+		} catch (CorruptObjectException e) {
+			assertEquals("duplicate entry names", e.getMessage());
+		}
+	}
+
+	public void testInvalidTreeDuplicateNames2() {
+		final StringBuilder b = new StringBuilder();
+		entry(b, "100644 a");
+		entry(b, "100755 a");
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkTree(data);
+			fail("incorrectly accepted an invalid tree");
+		} catch (CorruptObjectException e) {
+			assertEquals("duplicate entry names", e.getMessage());
+		}
+	}
+
+	public void testInvalidTreeDuplicateNames3() {
+		final StringBuilder b = new StringBuilder();
+		entry(b, "100644 a");
+		entry(b, "40000 a");
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkTree(data);
+			fail("incorrectly accepted an invalid tree");
+		} catch (CorruptObjectException e) {
+			assertEquals("duplicate entry names", e.getMessage());
+		}
+	}
+
+	public void testInvalidTreeDuplicateNames4() {
+		final StringBuilder b = new StringBuilder();
+		entry(b, "100644 a");
+		entry(b, "100644 a.c");
+		entry(b, "100644 a.d");
+		entry(b, "100644 a.e");
+		entry(b, "40000 a");
+		entry(b, "100644 zoo");
+		final byte[] data = Constants.encodeASCII(b.toString());
+		try {
+			checker.checkTree(data);
+			fail("incorrectly accepted an invalid tree");
+		} catch (CorruptObjectException e) {
+			assertEquals("duplicate entry names", e.getMessage());
+		}
+	}
+
+	private static void entry(final StringBuilder b, final String modeName) {
+		b.append(modeName);
+		b.append('\0');
+		for (int i = 0; i < Constants.OBJECT_ID_LENGTH; i++)
+			b.append((char) i);
+	}
+}
org.spearce.jgit/src/org/spearce/jgit/lib/ObjectChecker.java new file +352 -0

352 lines not on this page · raw message

org.spearce.jgit/src/org/spearce/jgit/util/RawParseUtils.java +4 -6
index 2ab3bfe..c2c9b00 100644
@@ -37,6 +37,10 @@
 
 package org.spearce.jgit.util;
 
+import static org.spearce.jgit.lib.ObjectChecker.author;
+import static org.spearce.jgit.lib.ObjectChecker.committer;
+import static org.spearce.jgit.lib.ObjectChecker.encoding;
+
 import java.nio.ByteBuffer;
 import java.nio.charset.Charset;
 import java.util.Arrays;
@@ -46,12 +50,6 @@
 
 /** Handy utility functions to parse raw object contents. */
 public final class RawParseUtils {
-	private static final byte[] author = Constants.encodeASCII("author ");
-
-	private static final byte[] committer = Constants.encodeASCII("committer ");
-
-	private static final byte[] encoding = Constants.encodeASCII("encoding ");
-
 	private static final byte[] digits;
 
 	static {
Signature · 1 line
1.6.0.2.569.g798a2a
#4 lore raw

PATCH 4/5 Expose the critical receive configuration options to JGit

The TransferConfig object keeps track of basic transfer related options from the transfer, receive and fetch groups within a git config file.

Right now we only care about receive.fsckobjects.

---
 .../src/org/spearce/jgit/lib/RepositoryConfig.java |   10 ++++
 .../src/org/spearce/jgit/lib/TransferConfig.java   |   56 ++++++++++++++++++++
 2 files changed, 66 insertions(+), 0 deletions(-)
 create mode 100644 org.spearce.jgit/src/org/spearce/jgit/lib/TransferConfig.java
org.spearce.jgit/src/org/spearce/jgit/lib/RepositoryConfig.java +10 -0
index d3c29ac..d8fd2fa 100644
@@ -92,6 +92,8 @@ public static RepositoryConfig openUserConfig() {
 
 	private CoreConfig core;
 
+	private TransferConfig transfer;
+
 	private List<Entry> entries;
 
 	private Map<String, Object> byName;
@@ -126,6 +128,13 @@ public CoreConfig getCore() {
 	}
 
 	/**
+	 * @return transfer, fetch and receive configuration values
+	 */
+	public TransferConfig getTransfer() {
+		return transfer;
+	}
+
+	/**
 	 * Obtain an integer value from the configuration.
 	 *
 	 * @param section
@@ -667,6 +676,7 @@ public void load() throws IOException {
 		}
 
 		core = new CoreConfig(this);
+		transfer = new TransferConfig(this);
 	}
 
 	private void clear() {
org.spearce.jgit/src/org/spearce/jgit/lib/TransferConfig.java new file +56 -0

56 lines not on this page · raw message

Signature · 1 line
1.6.0.2.569.g798a2a
#5 lore raw

PATCH 5/5 Honor receive.fsckobjects during any fetch connection

If the configuration option receive.fsckobjects is true or if the application requests it on the Transport each object received over the wire is validated to pass "git fsck" style rules. This can be useful when fetching data from an untrusted source, to ensure that the incoming objects comply with parsing standards.

The optional checking does require extra CPU on the client side. A test against git.git (69601 objects 49719 deltas) showed:

    receive.fsckobjects       average time
    --------------------------------------
    false (default)           0m17.588s
    true                      0m18.465s

So the additional checking costs about 5% more in client side time.

---
 .../jgit/transport/BasePackFetchConnection.java    |    1 +
 .../src/org/spearce/jgit/transport/IndexPack.java  |   60 ++++++++++++++++++--
 .../src/org/spearce/jgit/transport/Transport.java  |   24 ++++++++
 .../spearce/jgit/transport/TransportBundle.java    |   10 +++-
 .../jgit/transport/WalkFetchConnection.java        |   26 +++++++--
 5 files changed, 108 insertions(+), 13 deletions(-)
org.spearce.jgit/src/org/spearce/jgit/transport/BasePackFetchConnection.java +1 -0
index a22b33d..a542eb7 100644
@@ -452,6 +452,7 @@ private void receivePack(final ProgressMonitor monitor) throws IOException {
 
 		ip = IndexPack.create(local, sideband ? pckIn.sideband(monitor) : in);
 		ip.setFixThin(thinPack);
+		ip.setObjectChecking(transport.isCheckFetchedObjects());
 		ip.index(monitor);
 		ip.renameAndOpenPack();
 	}
org.spearce.jgit/src/org/spearce/jgit/transport/IndexPack.java +54 -6

101 lines not on this page · raw message

org.spearce.jgit/src/org/spearce/jgit/transport/Transport.java +24 -0
index 7284b28..28700b7 100644
@@ -55,6 +55,7 @@
 import org.spearce.jgit.lib.ProgressMonitor;
 import org.spearce.jgit.lib.Ref;
 import org.spearce.jgit.lib.Repository;
+import org.spearce.jgit.lib.TransferConfig;
 
 /**
  * Connects two Git repositories together and copies objects between them.
@@ -351,6 +352,9 @@ private static String findTrackingRefName(final String remoteName,
 	/** Should push just check for operation result, not really push. */
 	private boolean dryRun;
 
+	/** Should an incoming (fetch) transfer validate objects? */
+	private boolean checkFetchedObjects;
+
 	/**
 	 * Create a new transport instance.
 	 * 
@@ -363,8 +367,10 @@ private static String findTrackingRefName(final String remoteName,
 	 *            URI passed to {@link #open(Repository, URIish)}.
 	 */
 	protected Transport(final Repository local, final URIish uri) {
+		final TransferConfig tc = local.getConfig().getTransfer();
 		this.local = local;
 		this.uri = uri;
+		this.checkFetchedObjects = tc.isFsckObjects();
 	}
 
 	/**
@@ -444,6 +450,24 @@ public void setFetchThin(final boolean fetchThin) {
 	}
 
 	/**
+	 * @return true if fetch will verify received objects are formatted
+	 *         correctly. Validating objects requires more CPU time on the
+	 *         client side of the connection.
+	 */
+	public boolean isCheckFetchedObjects() {
+		return checkFetchedObjects;
+	}
+
+	/**
+	 * @param check
+	 *            true to enable checking received objects; false to assume all
+	 *            received objects are valid.
+	 */
+	public void setCheckFetchedObjects(final boolean check) {
+		checkFetchedObjects = check;
+	}
+
+	/**
 	 * Default setting is: {@value RemoteConfig#DEFAULT_RECEIVE_PACK}
 	 *
 	 * @return remote executable providing receive-pack service for pack
org.spearce.jgit/src/org/spearce/jgit/transport/TransportBundle.java +8 -2
index d43a18f..5b321a0 100644
@@ -170,8 +170,7 @@ protected void doFetch(final ProgressMonitor monitor,
 				final Collection<Ref> want) throws TransportException {
 			verifyPrerequisites();
 			try {
-				final IndexPack ip = IndexPack.create(local, bin);
-				ip.setFixThin(true);
+				final IndexPack ip = newIndexPack();
 				ip.index(monitor);
 				ip.renameAndOpenPack();
 			} catch (IOException err) {
@@ -183,6 +182,13 @@ protected void doFetch(final ProgressMonitor monitor,
 			}
 		}
 
+		private IndexPack newIndexPack() throws IOException {
+			final IndexPack ip = IndexPack.create(local, bin);
+			ip.setFixThin(true);
+			ip.setObjectChecking(TransportBundle.this.isCheckFetchedObjects());
+			return ip;
+		}
+
 		private void verifyPrerequisites() throws TransportException {
 			if (prereqs.isEmpty())
 				return;
org.spearce.jgit/src/org/spearce/jgit/transport/WalkFetchConnection.java +21 -5
index f309fe1..5638454 100644
@@ -59,6 +59,7 @@
 import org.spearce.jgit.lib.AnyObjectId;
 import org.spearce.jgit.lib.Constants;
 import org.spearce.jgit.lib.FileMode;
+import org.spearce.jgit.lib.ObjectChecker;
 import org.spearce.jgit.lib.ObjectId;
 import org.spearce.jgit.lib.PackIndex;
 import org.spearce.jgit.lib.ProgressMonitor;
@@ -99,6 +100,9 @@
 	/** The repository this transport fetches into, or pushes out of. */
 	private final Repository local;
 
+	/** If not null the validator for received objects. */
+	private final ObjectChecker objCheck;
+
 	/**
 	 * List of all remote repositories we may need to get objects out of.
 	 * <p>
@@ -157,9 +161,9 @@
 	 */
 	private final HashMap<ObjectId, List<Throwable>> fetchErrors;
 
-	WalkFetchConnection(final WalkTransport walkTransport,
-			final WalkRemoteObjectDatabase w) {
-		local = walkTransport.local;
+	WalkFetchConnection(final WalkTransport wt, final WalkRemoteObjectDatabase w) {
+		local = wt.local;
+		objCheck = wt.isCheckFetchedObjects() ? new ObjectChecker() : null;
 
 		remotes = new ArrayList<WalkRemoteObjectDatabase>();
 		remotes.add(w);
@@ -560,10 +564,21 @@ private void verifyLooseObject(final AnyObjectId id, final byte[] compressed)
 			throw e;
 		}
 
-		if (!AnyObjectId.equals(id, uol.getId()))
+		if (!AnyObjectId.equals(id, uol.getId())) {
 			throw new TransportException("Incorrect hash for " + id.name()
 					+ "; computed " + uol.getId().name() + " as a "
-					+ uol.getType() + " from " + compressed.length + " bytes.");
+					+ Constants.encodedTypeString(uol.getType()) + " from "
+					+ compressed.length + " bytes.");
+		}
+		if (objCheck != null) {
+			try {
+				objCheck.check(uol.getType(), uol.getCachedBytes());
+			} catch (CorruptObjectException e) {
+				throw new TransportException("Invalid "
+						+ Constants.encodedTypeString(uol.getType()) + " "
+						+ id.name() + ":" + e.getMessage());
+			}
+		}
 	}
 
 	private void saveLooseObject(final AnyObjectId id, final byte[] compressed)
@@ -811,6 +826,7 @@ void downloadPack(final ProgressMonitor monitor) throws IOException {
 			s = connection.open("pack/" + packName);
 			ip = IndexPack.create(local, s.in);
 			ip.setFixThin(false);
+			ip.setObjectChecker(objCheck);
 			ip.index(monitor);
 			ip.renameAndOpenPack();
 		}
Signature · 1 line
1.6.0.2.569.g798a2a
#6 lore raw

Re: [JGIT PATCH 4/5] Expose the critical receive configuration options to JGit

In reply to Shawn O. Pearce

The repo configuration setup fails. I'll squash this in

org.spearce.jgit/src/org/spearce/jgit/lib/RepositoryConfig.java +1 -0
@@ -534,6 +534,7 @@ public void create() {
                add(e);
 
                core = new CoreConfig(this);
+               transfer = new TransferConfig(this);
        }
 
        /**

i.e. initialize the transfer object when creating a new repo in junit tests.

I also noted we try to read ~/.gitconfig which may give us som headaches later on.

-- robin

#7 lore raw

Re: [JGIT PATCH 4/5] Expose the critical receive configuration options to JGit

Quoted 2 lines · Robin Rosenberg

The repo configuration setup fails. I'll squash this in

Whoops. Thanks.

Quoted 2 lines

I also noted we try to read ~/.gitconfig which may give us som headaches later on.

Oy. Issue #42 on the issue tracker for it. We shouldn't be reading that during the tests.

Signature · 1 line
Shawn.
#8 lore raw

Re: [JGIT PATCH 2/5] Fix UnpackedObjectLoader.getBytes to return a copy

In reply to Shawn O. Pearce
Quoted 5 lines · Shawn O. Pearce
org.spearce.jgit/src/org/spearce/jgit/lib/ObjectLoader.java +0 -0
index 5282491..87e861f 100644

org.spearce.jgit/src/org/spearce/jgit/lib/ObjectLoader.java protected void setId(final ObjectId id) { · line 113

Quoted 10 lines
org.spearce.jgit/src/org/spearce/jgit/lib/ObjectLoader.java +6 -1
@@ -105,7 +105,12 @@ protected void setId(final ObjectId id) {
        * @throws IOException
        *             the object cannot be read.
        */
-       public abstract byte[] getBytes() throws IOException;
+       public final byte[] getBytes() throws IOException {
+               final byte[] data = getCachedBytes();
+               final byte[] copy = new byte[data.length];
+               System.arraycopy(data, 0, copy, 0, data.length);
+               return data;
+       }

If I understand correctly, shouldn't this return the copy variable?

Signature · 1 line
Jonas Fonseca
#9 lore raw

Re: [JGIT PATCH 2/5] Fix UnpackedObjectLoader.getBytes to return a copy

Quoted 10 lines · Jonas Fonseca
Quoted 9 lines · Shawn O. Pearce
org.spearce.jgit/src/org/spearce/jgit/lib/ObjectLoader.java +0 -1
index 5282491..87e861f 100644
@@ -105,7 +105,12 @@ protected void setId(final ObjectId id) {
        * @throws IOException
        *             the object cannot be read.
        */
-       public abstract byte[] getBytes() throws IOException;
Quoted 8 lines

org.spearce.jgit/src/org/spearce/jgit/lib/ObjectLoader.java protected void setId(final ObjectId id) { · line 113

Quoted 6 lines
org.spearce.jgit/src/org/spearce/jgit/lib/ObjectLoader.java +6 -0
+       public final byte[] getBytes() throws IOException {
+               final byte[] data = getCachedBytes();
+               final byte[] copy = new byte[data.length];
+               System.arraycopy(data, 0, copy, 0, data.length);
+               return data;
+       }

If I understand correctly, shouldn't this return the copy variable?

F&@!#*@#&@!#*@!@!&#@#@*@!

Yes. :-)

I think its already committed to master too. Can you send a patch along to fix, and point out how stupid I am?

Signature · 1 line
Shawn.
#10 lore raw

PATCH 1/1 Fix ObjectLoader.getBytes to really return the created copy

The bug was carried over from the original code in PackedObjectLoader by commit 4c49ab5a4ec8555681ceabf17142a15bf90c2c24.

---
 .../src/org/spearce/jgit/lib/ObjectLoader.java     |    2 +-

Shawn O. Pearce <[email protected]> wrote Thu, Oct 09, 2008:

Quoted 2 lines

I think its already committed to master too. Can you send a patch along to fix, and point out how stupid I am?

Yes, I noticed it has already been applied. I am lagging behind a bit. Anyway, here is a trivial patch.

org.spearce.jgit/src/org/spearce/jgit/lib/ObjectLoader.java +1 -1
index 87e861f..8d745dd 100644
@@ -109,7 +109,7 @@ protected void setId(final ObjectId id) {
 		final byte[] data = getCachedBytes();
 		final byte[] copy = new byte[data.length];
 		System.arraycopy(data, 0, copy, 0, data.length);
-		return data;
+		return copy;
 	}
 
 	/**
Signature · 5 lines
1.6.0.2.665.g48ddf


-- 
Jonas Fonseca