Patch parsing API
cover Patch parsing API
This is an API to parse a Git style patch file and extract the critical metadata from the header lines, including the hunk headers and what lines they correspond to in the pre and post image files.
It requires the two other series I already sent out today for QuotedString and AbbreviatedObjectId.
There's TODO markers left where we still need to insert code to create some sort of warning object, and then hang the warnings off the Patch class. Given the size of the code I'm inclined to do that as yet an additional patch, rather than squash it into this series.
My short-term roadmap related to this part of JGit:
* Compute and collect warnings from malformed git-style patches
* Correctly parse "git diff --cc" style output
* Get it into egit.git
I'm stopping development once I have the diff --cc output parsing correctly. My rationale is right now I need the patch metadata parsing in Gerrit 2, so that's what I'm teaching JGit to do. Maybe later in the month or early next I'll add patch application support, because I also want that in Gerrit 2. Patch application is not currently a blocking item for me; but reading the patch metadata is.
Traditional patch support is really stubbed out too; there's a very small subset of traditional (non-git) style patches this code can scan the metadata from, but no tests to verify it. Gerrit 2 gets all of its data from a "git diff" process, so I only need support for git diffs right now. Yes, I'd like to add traditional patch support too, but it won't be until later in 2009 that I would even think about working on that myself.
Shawn O. Pearce (5):
Add toByteArray() to TemporaryBuffer
Add copy(InputStream) to TemporaryBuffer
Define FileHeader to parse the header block of a git diff Define Patch to parse a sequence of patch FileHeaders Add HunkHeader to represent a single hunk of a file within a patch
.../tst/org/spearce/jgit/patch/FileHeaderTest.java | 395 ++++++++++++++++ .../tst/org/spearce/jgit/patch/PatchTest.java | 155 +++++++ .../patch/testParse_ConfigCaseInsensitive.patch | 67 +++ .../src/org/spearce/jgit/patch/FileHeader.java | 480 ++++++++++++++++++++ .../src/org/spearce/jgit/patch/HunkHeader.java | 185 ++++++++ .../src/org/spearce/jgit/patch/Patch.java | 267 +++++++++++ .../src/org/spearce/jgit/util/TemporaryBuffer.java | 51 ++ 7 files changed, 1600 insertions(+), 0 deletions(-) create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/patch/FileHeaderTest.java create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/patch/PatchTest.java create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/patch/testParse_ConfigCaseInsensitive.patch create mode 100644 org.spearce.jgit/src/org/spearce/jgit/patch/FileHeader.java create mode 100644 org.spearce.jgit/src/org/spearce/jgit/patch/HunkHeader.java create mode 100644 org.spearce.jgit/src/org/spearce/jgit/patch/Patch.java
PATCH 1/5 Add toByteArray() to TemporaryBuffer
It can be more useful to convert a buffered output stream into a single byte array, without paying the penalties associated with ByteArrayOutputStream to do the same action.
--- .../src/org/spearce/jgit/util/TemporaryBuffer.java | 34 ++++++++++++++++++++ 1 files changed, 34 insertions(+), 0 deletions(-)
org.spearce.jgit/src/org/spearce/jgit/util/TemporaryBuffer.java +34 -0
@@ -182,6 +182,40 @@ public long length() { } /** + * Convert this buffer's contents into a contiguous byte array. + * <p> + * The buffer is only complete after {@link #close()} has been invoked. + * + * @return the complete byte array; length matches {@link #length()}. + * @throws IOException + * an error occurred reading from a local temporary file + * @throws OutOfMemoryError + * the buffer cannot fit in memory + */ + public byte[] toByteArray() throws IOException { + final long len = length(); + if (Integer.MAX_VALUE < len) + throw new OutOfMemoryError("Length exceeds maximum array size"); + + final byte[] out = new byte[(int) len]; + if (blocks != null) { + int outPtr = 0; + for (final Block b : blocks) { + System.arraycopy(b.buffer, 0, out, outPtr, b.count); + outPtr += b.count; + } + } else { + final FileInputStream in = new FileInputStream(onDiskFile); + try { + NB.readFully(in, out, 0, (int) len); + } finally { + in.close(); + } + } + return out; + } + + /** * Send this buffer to an output stream. * <p> * This method may only be invoked after {@link #close()} has completed
Signature · 1 line
1.6.1.rc2.299.gead4c
PATCH 2/5 Add copy(InputStream) to TemporaryBuffer
In some places we may find it ourselves with an InputStream we need to copy into a TemporaryBuffer, so we can flatten out the entire stream to a single byte[]. Putting the copy loop here is more useful then duplicating it in application level code.
--- .../src/org/spearce/jgit/util/TemporaryBuffer.java | 17 +++++++++++++++++ 1 files changed, 17 insertions(+), 0 deletions(-)
org.spearce.jgit/src/org/spearce/jgit/util/TemporaryBuffer.java +17 -0
@@ -42,6 +42,7 @@ import java.io.FileInputStream; import java.io.FileOutputStream; import java.io.IOException; +import java.io.InputStream; import java.io.OutputStream; import java.util.ArrayList;
@@ -135,6 +136,22 @@ public void write(final byte[] b, int off, int len) throws IOException { diskOut.write(b, off, len); } + /** + * Copy all bytes remaining on the input stream into this buffer. + * + * @param in + * the stream to read from, until EOF is reached. + * @throws IOException + * an error occurred reading from the input stream, or while + * writing to a local temporary file. + */ + public void copy(final InputStream in) throws IOException { + final byte[] b = new byte[2048]; + int n; + while ((n = in.read(b)) > 0) + write(b, 0, n); + } + private Block last() { return blocks.get(blocks.size() - 1); }
Signature · 1 line
1.6.1.rc2.299.gead4c
PATCH 3/5 Define FileHeader to parse the header block of a git diff
This class parses the top header lines of a git style diff, such as:
diff --git a/SUBMITTING_PATCHES b/Q similarity index 100% copy from SUBMITTING_PATCHES copy to Q
or:
diff --git a/Q b/Q new file mode 100644 index 0000000..e4a135e --- /dev/null +++ b/Q
and makes the information available in an object form. Unit tests cover the different styles of headers that are commonly created by C git, including both rename formats.
The hunk header information is not handled by this class, and it does not have a public API. It is my intention to wrap this into a larger container class that handles multiple FileHeaders at once, with the base case of course being a single FileHeader describing a patch that impacts only one file.
--- .../tst/org/spearce/jgit/patch/FileHeaderTest.java | 391 ++++++++++++++++++ .../src/org/spearce/jgit/patch/FileHeader.java | 430 ++++++++++++++++++++ 2 files changed, 821 insertions(+), 0 deletions(-) create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/patch/FileHeaderTest.java create mode 100644 org.spearce.jgit/src/org/spearce/jgit/patch/FileHeader.java
org.spearce.jgit.test/tst/org/spearce/jgit/patch/FileHeaderTest.java new file +391 -0
@@ -0,0 +1,391 @@ +/* + * Copyright (C) 2008, Google Inc. + * + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials provided + * with the distribution. + * + * - Neither the name of the Git Development Community nor the + * names of its contributors may be used to endorse or promote + * products derived from this software without specific prior + * written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND + * CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, + * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR + * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT + * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; + * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER + * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, + * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) + * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF + * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ +package org.spearce.jgit.patch; + +import junit.framework.TestCase; + +import org.spearce.jgit.lib.Constants; +import org.spearce.jgit.lib.FileMode; +import org.spearce.jgit.lib.ObjectId; + +public class FileHeaderTest extends TestCase { + public void testParseGitFileName_Empty() { + assertEquals(-1, data("").parseGitFileName(0)); + } + + public void testParseGitFileName_NoLF() { + assertEquals(-1, data("a/ b/").parseGitFileName(0)); + } + + public void testParseGitFileName_NoSecondLine() { + assertEquals(-1, data("\n").parseGitFileName(0)); + } + + public void testParseGitFileName_EmptyHeader() { + assertEquals(1, data("\n\n").parseGitFileName(0)); + } + + public void testParseGitFileName_Foo() { + final String name = "foo"; + final FileHeader fh = header(name); + assertEquals(gitLine(name).length(), fh.parseGitFileName(0)); + assertEquals(name, fh.getOldName()); + assertSame(fh.getOldName(), fh.getNewName()); + } + + public void testParseGitFileName_FailFooBar() { + final FileHeader fh = data("a/foo b/bar\n-"); + assertTrue(fh.parseGitFileName(0) > 0); + assertNull(fh.getOldName()); + assertNull(fh.getNewName()); + } + + public void testParseGitFileName_FooSpBar() { + final String name = "foo bar"; + final FileHeader fh = header(name); + assertEquals(gitLine(name).length(), fh.parseGitFileName(0)); + assertEquals(name, fh.getOldName()); + assertSame(fh.getOldName(), fh.getNewName()); + } + + public void testParseGitFileName_DqFooTabBar() { + final String name = "foo\tbar"; + final String dqName = "foo\\tbar"; + final FileHeader fh = dqHeader(dqName); + assertEquals(dqGitLine(dqName).length(), fh.parseGitFileName(0)); + assertEquals(name, fh.getOldName()); + assertSame(fh.getOldName(), fh.getNewName()); + } + + public void testParseGitFileName_DqFooSpLfNulBar() { + final String name = "foo \n\0bar"; + final String dqName = "foo \\n\\0bar"; + final FileHeader fh = dqHeader(dqName); + assertEquals(dqGitLine(dqName).length(), fh.parseGitFileName(0)); + assertEquals(name, fh.getOldName()); + assertSame(fh.getOldName(), fh.getNewName()); + } + + public void testParseGitFileName_SrcFooC() { + final String name = "src/foo/bar/argh/code.c"; + final FileHeader fh = header(name); + assertEquals(gitLine(name).length(), fh.parseGitFileName(0)); + assertEquals(name, fh.getOldName()); + assertSame(fh.getOldName(), fh.getNewName()); + } + + public void testParseGitFileName_SrcFooCNonStandardPrefix() { + final String name = "src/foo/bar/argh/code.c"; + final String header = "project-v-1.0/" + name + " mydev/" + name + "\n"; + final FileHeader fh = data(header + "-"); + assertEquals(header.length(), fh.parseGitFileName(0)); + assertEquals(name, fh.getOldName()); + assertSame(fh.getOldName(), fh.getNewName()); + } + + public void testParseUnicodeName_NewFile() { + final FileHeader fh = data("diff --git \"a/\\303\\205ngstr\\303\\266m\" \"b/\\303\\205ngstr\\303\\266m\"\n" + + "new file mode 100644\n" + + "index 0000000..7898192\n" + + "--- /dev/null\n" + + "+++ \"b/\\303\\205ngstr\\303\\266m\"\n" + + "@@ -0,0 +1 @@\n" + "+a\n"); + assertParse(fh); + + assertEquals("/dev/null", fh.getOldName()); + assertSame(FileHeader.DEV_NULL, fh.getOldName()); + assertEquals("\u00c5ngstr\u00f6m", fh.getNewName()); + + assertSame(FileHeader.ChangeType.ADD, fh.getChangeType()); + + assertNull(fh.getOldMode()); + assertSame(FileMode.REGULAR_FILE, fh.getNewMode()); + + assertEquals("0000000", fh.getOldId().name()); + assertEquals("7898192", fh.getNewId().name()); + assertEquals(0, fh.getScore()); + } + + public void testParseUnicodeName_DeleteFile() { + final FileHeader fh = data("diff --git \"a/\\303\\205ngstr\\303\\266m\" \"b/\\303\\205ngstr\\303\\266m\"\n" + + "deleted file mode 100644\n" + + "index 7898192..0000000\n" + + "--- \"a/\\303\\205ngstr\\303\\266m\"\n" + + "+++ /dev/null\n" + + "@@ -1 +0,0 @@\n" + "-a\n"); + assertParse(fh); + + assertEquals("\u00c5ngstr\u00f6m", fh.getOldName()); + assertEquals("/dev/null", fh.getNewName()); + assertSame(FileHeader.DEV_NULL, fh.getNewName()); + + assertSame(FileHeader.ChangeType.DELETE, fh.getChangeType()); + + assertSame(FileMode.REGULAR_FILE, fh.getOldMode()); + assertNull(fh.getNewMode()); + + assertEquals("7898192", fh.getOldId().name()); + assertEquals("0000000", fh.getNewId().name()); + assertEquals(0, fh.getScore()); + } + + public void testParseModeChange() { + final FileHeader fh = data("diff --git a/a b b/a b\n" + + "old mode 100644\n" + "new mode 100755\n"); + assertParse(fh); + assertEquals("a b", fh.getOldName()); + assertEquals("a b", fh.getNewName()); + + assertSame(FileHeader.ChangeType.MODIFY, fh.getChangeType()); + + assertNull(fh.getOldId()); + assertNull(fh.getNewId()); + + assertSame(FileMode.REGULAR_FILE, fh.getOldMode()); + assertSame(FileMode.EXECUTABLE_FILE, fh.getNewMode()); + assertEquals(0, fh.getScore()); + } + + public void testParseRename100_NewStyle() { + final FileHeader fh = data("diff --git a/a b/ c/\\303\\205ngstr\\303\\266m\n" + + "similarity index 100%\n" + + "rename from a\n" + + "rename to \" c/\\303\\205ngstr\\303\\266m\"\n"); + int ptr = fh.parseGitFileName(0); + assertTrue(ptr > 0); + assertNull(fh.getOldName()); // can't parse names on a rename + assertNull(fh.getNewName()); + + ptr = fh.parseGitHeaders(ptr); + assertTrue(ptr > 0); + + assertEquals("a", fh.getOldName()); + assertEquals(" c/\u00c5ngstr\u00f6m", fh.getNewName()); + + assertSame(FileHeader.ChangeType.RENAME, fh.getChangeType()); + + assertNull(fh.getOldId()); + assertNull(fh.getNewId()); + + assertNull(fh.getOldMode()); + assertNull(fh.getNewMode()); + + assertEquals(100, fh.getScore()); + } + + public void testParseRename100_OldStyle() { + final FileHeader fh = data("diff --git a/a b/ c/\\303\\205ngstr\\303\\266m\n" + + "similarity index 100%\n" + + "rename old a\n" + + "rename new \" c/\\303\\205ngstr\\303\\266m\"\n"); + int ptr = fh.parseGitFileName(0); + assertTrue(ptr > 0); + assertNull(fh.getOldName()); // can't parse names on a rename + assertNull(fh.getNewName()); + + ptr = fh.parseGitHeaders(ptr); + assertTrue(ptr > 0); + + assertEquals("a", fh.getOldName()); + assertEquals(" c/\u00c5ngstr\u00f6m", fh.getNewName()); + + assertSame(FileHeader.ChangeType.RENAME, fh.getChangeType()); + + assertNull(fh.getOldId()); + assertNull(fh.getNewId()); + + assertNull(fh.getOldMode()); + assertNull(fh.getNewMode()); + + assertEquals(100, fh.getScore()); + } + public void testParseCopy100() { + final FileHeader fh = data("diff --git a/a b/ c/\\303\\205ngstr\\303\\266m\n" + + "similarity index 100%\n" + + "copy from a\n" + + "copy to \" c/\\303\\205ngstr\\303\\266m\"\n"); + int ptr = fh.parseGitFileName(0); + assertTrue(ptr > 0); + assertNull(fh.getOldName()); // can't parse names on a copy + assertNull(fh.getNewName()); + + ptr = fh.parseGitHeaders(ptr); + assertTrue(ptr > 0); + + assertEquals("a", fh.getOldName()); + assertEquals(" c/\u00c5ngstr\u00f6m", fh.getNewName()); + + assertSame(FileHeader.ChangeType.COPY, fh.getChangeType()); + + assertNull(fh.getOldId()); + assertNull(fh.getNewId()); + + assertNull(fh.getOldMode()); + assertNull(fh.getNewMode()); + + assertEquals(100, fh.getScore()); + } + + public void testParseFullIndexLine_WithMode() { + final String oid = "78981922613b2afb6025042ff6bd878ac1994e85"; + final String nid = "61780798228d17af2d34fce4cfbdf35556832472"; + final FileHeader fh = data("diff --git a/a b/a\n" + "index " + oid + + ".." + nid + " 100644\n" + "--- a/a\n" + "+++ b/a\n"); + assertParse(fh); + + assertEquals("a", fh.getOldName()); + assertEquals("a", fh.getNewName()); + + assertSame(FileMode.REGULAR_FILE, fh.getOldMode()); + assertSame(FileMode.REGULAR_FILE, fh.getNewMode()); + + assertNotNull(fh.getOldId()); + assertNotNull(fh.getNewId()); + + assertTrue(fh.getOldId().isComplete()); + assertTrue(fh.getNewId().isComplete()); + + assertEquals(ObjectId.fromString(oid), fh.getOldId().toObjectId()); + assertEquals(ObjectId.fromString(nid), fh.getNewId().toObjectId()); + } + + public void testParseFullIndexLine_NoMode() { + final String oid = "78981922613b2afb6025042ff6bd878ac1994e85"; + final String nid = "61780798228d17af2d34fce4cfbdf35556832472"; + final FileHeader fh = data("diff --git a/a b/a\n" + "index " + oid + + ".." + nid + "\n" + "--- a/a\n" + "+++ b/a\n"); + assertParse(fh); + + assertEquals("a", fh.getOldName()); + assertEquals("a", fh.getNewName()); + + assertNull(fh.getOldMode()); + assertNull(fh.getNewMode()); + + assertNotNull(fh.getOldId()); + assertNotNull(fh.getNewId()); + + assertTrue(fh.getOldId().isComplete()); + assertTrue(fh.getNewId().isComplete()); + + assertEquals(ObjectId.fromString(oid), fh.getOldId().toObjectId()); + assertEquals(ObjectId.fromString(nid), fh.getNewId().toObjectId()); + } + + public void testParseAbbrIndexLine_WithMode() { + final int a = 7; + final String oid = "78981922613b2afb6025042ff6bd878ac1994e85"; + final String nid = "61780798228d17af2d34fce4cfbdf35556832472"; + final FileHeader fh = data("diff --git a/a b/a\n" + "index " + + oid.substring(0, a - 1) + ".." + nid.substring(0, a - 1) + + " 100644\n" + "--- a/a\n" + "+++ b/a\n"); + assertParse(fh); + + assertEquals("a", fh.getOldName()); + assertEquals("a", fh.getNewName()); + + assertSame(FileMode.REGULAR_FILE, fh.getOldMode()); + assertSame(FileMode.REGULAR_FILE, fh.getNewMode()); + + assertNotNull(fh.getOldId()); + assertNotNull(fh.getNewId()); + + assertFalse(fh.getOldId().isComplete()); + assertFalse(fh.getNewId().isComplete()); + + assertEquals(oid.substring(0, a - 1), fh.getOldId().name()); + assertEquals(nid.substring(0, a - 1), fh.getNewId().name()); + + assertTrue(ObjectId.fromString(oid).startsWith(fh.getOldId())); + assertTrue(ObjectId.fromString(nid).startsWith(fh.getNewId())); + } + + public void testParseAbbrIndexLine_NoMode() { + final int a = 7; + final String oid = "78981922613b2afb6025042ff6bd878ac1994e85"; + final String nid = "61780798228d17af2d34fce4cfbdf35556832472"; + final FileHeader fh = data("diff --git a/a b/a\n" + "index " + + oid.substring(0, a - 1) + ".." + nid.substring(0, a - 1) + + "\n" + "--- a/a\n" + "+++ b/a\n"); + assertParse(fh); + + assertEquals("a", fh.getOldName()); + assertEquals("a", fh.getNewName()); + + assertNull(fh.getOldMode()); + assertNull(fh.getNewMode()); + + assertNotNull(fh.getOldId()); + assertNotNull(fh.getNewId()); + + assertFalse(fh.getOldId().isComplete()); + assertFalse(fh.getNewId().isComplete()); + + assertEquals(oid.substring(0, a - 1), fh.getOldId().name()); + assertEquals(nid.substring(0, a - 1), fh.getNewId().name()); + + assertTrue(ObjectId.fromString(oid).startsWith(fh.getOldId())); + assertTrue(ObjectId.fromString(nid).startsWith(fh.getNewId())); + } + + private static void assertParse(final FileHeader fh) { + int ptr = fh.parseGitFileName(0); + assertTrue(ptr > 0); + ptr = fh.parseGitHeaders(ptr); + assertTrue(ptr > 0); + } + + private static FileHeader data(final String in) { + return new FileHeader(Constants.encodeASCII(in), 0); + } + + private static FileHeader header(final String path) { + return data(gitLine(path) + "--- " + path + "\n"); + } + + private static String gitLine(final String path) { + return "a/" + path + " b/" + path + "\n"; + } + + private static FileHeader dqHeader(final String path) { + return data(dqGitLine(path) + "--- " + path + "\n"); + } + + private static String dqGitLine(final String path) { + return "\"a/" + path + "\" \"b/" + path + "\"\n"; + } +}
org.spearce.jgit/src/org/spearce/jgit/patch/FileHeader.java new file +430 -0
430 lines not on this page · raw message
Signature · 1 line
1.6.1.rc2.299.gead4c
PATCH 4/5 Define Patch to parse a sequence of patch FileHeaders
Most patch scripts impact more than one file at a time, so we need to support parsing multiple FileHeaders from the same input stream and collect them into a larger entity representing the entire script.
--- .../tst/org/spearce/jgit/patch/PatchTest.java | 97 +++++++++ .../patch/testParse_ConfigCaseInsensitive.patch | 67 ++++++ .../src/org/spearce/jgit/patch/FileHeader.java | 28 +++ .../src/org/spearce/jgit/patch/Patch.java | 217 ++++++++++++++++++++ 4 files changed, 409 insertions(+), 0 deletions(-) create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/patch/PatchTest.java create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/patch/testParse_ConfigCaseInsensitive.patch create mode 100644 org.spearce.jgit/src/org/spearce/jgit/patch/Patch.java
org.spearce.jgit.test/tst/org/spearce/jgit/patch/PatchTest.java new file +97 -0
@@ -0,0 +1,97 @@ +/* + * Copyright (C) 2008, Google Inc. + * + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or + * without modification, are permitted provided that the following + * conditions are met: + * + * - Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * - Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials provided + * with the distribution. + * + * - Neither the name of the Git Development Community nor the + * names of its contributors may be used to endorse or promote + * products derived from this software without specific prior + * written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND + * CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, + * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES + * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR + * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT + * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; + * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER + * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, + * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) + * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF + * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +package org.spearce.jgit.patch; + +import java.io.IOException; +import java.io.InputStream; + +import junit.framework.TestCase; + +import org.spearce.jgit.lib.FileMode; + +public class PatchTest extends TestCase { + public void testEmpty() { + final Patch p = new Patch(); + assertTrue(p.getFiles().isEmpty()); + } + + public void testParse_ConfigCaseInsensitive() throws IOException { + final Patch p = parseTestPatchFile(); + assertEquals(2, p.getFiles().size()); + + final FileHeader fRepositoryConfigTest = p.getFiles().get(0); + final FileHeader fRepositoryConfig = p.getFiles().get(1); + + assertEquals( + "org.spearce.jgit.test/tst/org/spearce/jgit/lib/RepositoryConfigTest.java", + fRepositoryConfigTest.getNewName()); + + assertEquals( + "org.spearce.jgit/src/org/spearce/jgit/lib/RepositoryConfig.java", + fRepositoryConfig.getNewName()); + + assertEquals(572, fRepositoryConfigTest.startOffset); + assertEquals(1490, fRepositoryConfig.startOffset); + + assertEquals("da7e704", fRepositoryConfigTest.getOldId().name()); + assertEquals("34ce04a", fRepositoryConfigTest.getNewId().name()); + assertSame(FileMode.REGULAR_FILE, fRepositoryConfigTest.getOldMode()); + assertSame(FileMode.REGULAR_FILE, fRepositoryConfigTest.getNewMode()); + + assertEquals("45c2f8a", fRepositoryConfig.getOldId().name()); + assertEquals("3291bba", fRepositoryConfig.getNewId().name()); + assertSame(FileMode.REGULAR_FILE, fRepositoryConfig.getOldMode()); + assertSame(FileMode.REGULAR_FILE, fRepositoryConfig.getNewMode()); + } + + private Patch parseTestPatchFile() throws IOException { + final String patchFile = getName() + ".patch"; + final InputStream in = getClass().getResourceAsStream(patchFile); + if (in == null) { + fail("No " + patchFile + " test vector"); + return null; // Never happens + } + try { + final Patch p = new Patch(); + p.parse(in); + return p; + } finally { + in.close(); + } + } +}
org.spearce.jgit.test/tst/org/spearce/jgit/patch/testParse_ConfigCaseInsensitive.patch new file +67 -0
67 lines not on this page · raw message
org.spearce.jgit/src/org/spearce/jgit/patch/FileHeader.java +28 -0
@@ -364,6 +364,34 @@ int parseGitHeaders(int ptr) { return ptr; } + int parseTraditionalHeaders(int ptr) { + final int sz = buf.length; + while (ptr < sz) { + final int eol = nextLF(buf, ptr); + if (match(buf, ptr, HUNK_HDR) >= 0) { + // First hunk header; break out and parse them later. + break; + + } else if (match(buf, ptr, OLD_NAME) >= 0) { + oldName = p1(parseName(oldName, ptr + OLD_NAME.length, eol)); + if (oldName == DEV_NULL) + changeType = ChangeType.ADD; + + } else if (match(buf, ptr, NEW_NAME) >= 0) { + newName = p1(parseName(newName, ptr + NEW_NAME.length, eol)); + if (newName == DEV_NULL) + changeType = ChangeType.DELETE; + + } else { + // Possibly an empty patch. + break; + } + + ptr = eol; + } + return ptr; + } + private String parseName(final String expect, int ptr, final int end) { if (ptr == end) return expect;
org.spearce.jgit/src/org/spearce/jgit/patch/Patch.java new file +217 -0
217 lines not on this page · raw message
Signature · 1 line
1.6.1.rc2.299.gead4c
PATCH 5/5 Add HunkHeader to represent a single hunk of a file within a patch
The hunk header parses the lines of the hunk, counting up the body to ensure it matches with the header line. We store the hunks on on the FileHeader, providing public access to them after a patch has been parsed.
If the line counts in the body differ from the header we don't make that an immediate error. This permits a future implementation of something like "git apply --recount", where the hunk headers can be recomputed based on the actual hunk contents after a human has done manual editing. We may also need to parse a corrupt patch and show it in a visual tool for a human to repair, so aborting with any sort of exception isn't useful.
--- .../tst/org/spearce/jgit/patch/FileHeaderTest.java | 6 +- .../tst/org/spearce/jgit/patch/PatchTest.java | 60 ++++++- .../src/org/spearce/jgit/patch/FileHeader.java | 22 +++ .../src/org/spearce/jgit/patch/HunkHeader.java | 185 ++++++++++++++++++++ .../src/org/spearce/jgit/patch/Patch.java | 54 ++++++- 5 files changed, 323 insertions(+), 4 deletions(-) create mode 100644 org.spearce.jgit/src/org/spearce/jgit/patch/HunkHeader.java
org.spearce.jgit.test/tst/org/spearce/jgit/patch/FileHeaderTest.java +5 -1
@@ -44,7 +44,10 @@ public class FileHeaderTest extends TestCase { public void testParseGitFileName_Empty() { - assertEquals(-1, data("").parseGitFileName(0)); + final FileHeader fh = data(""); + assertEquals(-1, fh.parseGitFileName(0)); + assertNotNull(fh.getHunks()); + assertTrue(fh.getHunks().isEmpty()); } public void testParseGitFileName_NoLF() {
@@ -233,6 +236,7 @@ public void testParseRename100_OldStyle() { assertEquals(100, fh.getScore()); } + public void testParseCopy100() { final FileHeader fh = data("diff --git a/a b/ c/\\303\\205ngstr\\303\\266m\n" + "similarity index 100%\n"
org.spearce.jgit.test/tst/org/spearce/jgit/patch/PatchTest.java +59 -1
77 lines not on this page · raw message
org.spearce.jgit/src/org/spearce/jgit/patch/FileHeader.java +22 -0
@@ -43,6 +43,10 @@ import static org.spearce.jgit.util.RawParseUtils.nextLF; import static org.spearce.jgit.util.RawParseUtils.parseBase10; +import java.util.ArrayList; +import java.util.Collections; +import java.util.List; + import org.spearce.jgit.lib.AbbreviatedObjectId; import org.spearce.jgit.lib.Constants; import org.spearce.jgit.lib.FileMode;
@@ -136,6 +140,9 @@ /** ObjectId listed on the index line for the new (post-image) */ private AbbreviatedObjectId newId; + /** The hunks of this file */ + private List<HunkHeader> hunks; + FileHeader(final byte[] b, final int offset) { buf = b; startOffset = offset;
@@ -222,6 +229,21 @@ public AbbreviatedObjectId getNewId() { return newId; } + /** @return hunks altering this file; in order of appearance in patch */ + public List<HunkHeader> getHunks() { + if (hunks == null) + return Collections.emptyList(); + return hunks; + } + + void addHunk(final HunkHeader h) { + if (h.getFileHeader() != this) + throw new IllegalArgumentException("Hunk belongs to another file"); + if (hunks == null) + hunks = new ArrayList<HunkHeader>(); + hunks.add(h); + } + /** * Parse a "diff --git" or "diff --cc" line. *
org.spearce.jgit/src/org/spearce/jgit/patch/HunkHeader.java new file +185 -0
185 lines not on this page · raw message
org.spearce.jgit/src/org/spearce/jgit/patch/Patch.java +52 -2
82 lines not on this page · raw message
Signature · 1 line
1.6.1.rc2.299.gead4c
Re: [JGIT PATCH 4/5] Define Patch to parse a sequence of patch FileHeaders
In reply to Shawn O. Pearcetorsdag 11 december 2008 05:58:41 skrev Shawn O. Pearce:
Quoted 3 lines
Most patch scripts impact more than one file at a time, so we need to support parsing multiple FileHeaders from the same input stream and collect them into a larger entity representing the entire script.
...
Quoted 7 lines
org.spearce.jgit.test/tst/org/spearce/jgit/patch/PatchTest.java new file +7
-0+ public void testParse_ConfigCaseInsensitive() throws IOException { + final Patch p = parseTestPatchFile(); + assertEquals(2, p.getFiles().size()); + + final FileHeader fRepositoryConfigTest = p.getFiles().get(0); + final FileHeader fRepositoryConfig = p.getFiles().get(1); +
org.spearce.jgit.test/tst/org/spearce/jgit/patch/PatchTest.java line 69
Quoted 10 lines
org.spearce.jgit.test/tst/org/spearce/jgit/patch/PatchTest.java new file +10
-0+ assertEquals( + "org.spearce.jgit.test/tst/org/spearce/jgit/lib/RepositoryConfigTest.java", + fRepositoryConfigTest.getNewName()); + + assertEquals( + "org.spearce.jgit/src/org/spearce/jgit/lib/RepositoryConfig.java", + fRepositoryConfig.getNewName()); + + assertEquals(572, fRepositoryConfigTest.startOffset); + assertEquals(1490, fRepositoryConfig.startOffset);
1487 here
-- robin
Re: [JGIT PATCH 4/5] Define Patch to parse a sequence of patch FileHeaders
Quoted 4 lines · Robin Rosenberg
org.spearce.jgit.test/tst/org/spearce/jgit/patch/PatchTest.java line 69
Quoted 2 lines
org.spearce.jgit.test/tst/org/spearce/jgit/patch/PatchTest.java new file +2
-0+ assertEquals(572, fRepositoryConfigTest.startOffset); + assertEquals(1490, fRepositoryConfig.startOffset);1487 here
Really? 1490 is the only value that the test vector passes with. What's the 3 bytes you think I'm off by?
Signature · 1 line
Shawn.
Re: [JGIT PATCH 4/5] Define Patch to parse a sequence of patch FileHeaders
torsdag 11 december 2008 19:39:54 skrev Shawn O. Pearce:
Quoted 8 lines
Quoted 4 lines · Robin Rosenberg
org.spearce.jgit.test/tst/org/spearce/jgit/patch/PatchTest.java line 69
Quoted 2 lines
org.spearce.jgit.test/tst/org/spearce/jgit/patch/PatchTest.java new file +2
-0+ assertEquals(572, fRepositoryConfigTest.startOffset); + assertEquals(1490, fRepositoryConfig.startOffset);1487 here
Really? 1490 is the only value that the test vector passes with. What's the 3 bytes you think I'm off by?
Ah, --whitespace=fix did that.
-- robin
Re: [JGIT PATCH 4/5] Define Patch to parse a sequence of patch FileHeaders
Quoted 11 lines · Robin Rosenberg
torsdag 11 december 2008 19:39:54 skrev Shawn O. Pearce:
Quoted 8 lines
Quoted 4 lines · Robin Rosenberg
org.spearce.jgit.test/tst/org/spearce/jgit/patch/PatchTest.java line 69
Quoted 2 lines
org.spearce.jgit.test/tst/org/spearce/jgit/patch/PatchTest.java new file +2
-0+ assertEquals(572, fRepositoryConfigTest.startOffset); + assertEquals(1490, fRepositoryConfig.startOffset);1487 here
Really? 1490 is the only value that the test vector passes with. What's the 3 bytes you think I'm off by?
Ah, --whitespace=fix did that.
Ok. I know you like to apply with --whitespace=fix, but I would prefer to leave these *.patch test input files[*1*] exactly as they were created by git format-patch or git diff, so I'm sure we are parsing the same thing git would have produced and sent to us.
*1*: I have more patches coming which add 2 more test inputs
to the same PatchTest suite.
Signature · 1 line
Shawn.
Re: [JGIT PATCH 4/5] Define Patch to parse a sequence of patch FileHeaders
In reply to Shawn O. Pearcetorsdag 11 december 2008 19:39:54 skrev Shawn O. Pearce:
Quoted 8 lines
Quoted 4 lines · Robin Rosenberg
Review
Quoted 2 lines
(quoted diff) +2
-0+ assertEquals(572, fRepositoryConfigTest.startOffset); + assertEquals(1490, fRepositoryConfig.startOffset);1487 here
Really? 1490 is the only value that the test vector passes with. What's the 3 bytes you think I'm off by?
...Quick, quick, find something else to pick on.. :->
Yes. Very little of the code in TemporaryBuffer is covered by the unit tests and number of conditionals in there are rather large. I tried messing with the constants in there to improve that and then PatchTest started to fail.
Here are the changes I tried with. I think it should still work with thes changes. Rather than changing the other tests, we might want to create a special test for only the buffer class.
-- robin
org.spearce.jgit/src/org/spearce/jgit/util/TemporaryBuffer.java +2 -2
@@ -60,7 +60,7 @@ * after this stream has been properly closed by {@link #close()}. */ public class TemporaryBuffer extends OutputStream { - private static final int DEFAULT_IN_CORE_LIMIT = 1024 * 1024; + private static final int DEFAULT_IN_CORE_LIMIT = 1024; /** Chain of data, if we are still completely in-core; otherwise null. */ private ArrayList<Block> blocks;
@@ -315,7 +315,7 @@ public void destroy() { } private static class Block { - static final int SZ = 8 * 1024; + static final int SZ = 512; final byte[] buffer = new byte[SZ];
Re: [JGIT PATCH 4/5] Define Patch to parse a sequence of patch FileHeaders
Quoted 1 line · Robin Rosenberg
...Quick, quick, find something else to pick on.. :->
Heh, you'll get more patches again from me today, so there'll still be more to pick on. :)
Quoted 7 lines
Yes. Very little of the code in TemporaryBuffer is covered by the unit tests and number of conditionals in there are rather large. I tried messing with the constants in there to improve that and then PatchTest started to fail.
Here are the changes I tried with. I think it should still work with thes changes. Rather than changing the other tests, we might want to create a special test for only the buffer class.
Ok. I was thinking the same thing actually, that I should spend a bit of time today and try to get coverage on TemporaryBuffer. I'll write a unit test for it.
Signature · 1 line
Shawn.
Re: [JGIT PATCH 2/5] Add copy(InputStream) to TemporaryBuffer
In reply to Shawn O. Pearcetorsdag 11 december 2008 05:58:39 skrev Shawn O. Pearce:
org.spearce.jgit/src/org/spearce/jgit/util/TemporaryBuffer.java public void write(final byte[] b, int off, int len) throws IOException { · line 149
Quoted 6 lines
In some places we may find it ourselves with an InputStream we need to copy into a TemporaryBuffer, so we can flatten out the entire stream to a single byte[]. Putting the copy loop here is more useful then duplicating it in application level code.
org.spearce.jgit/src/org/spearce/jgit/util/TemporaryBuffer.java +2
-0+ public void copy(final InputStream in) throws IOException { + final byte[] b = new byte[2048];
Why not 8192 here too?
-- robin
Re: [JGIT PATCH 2/5] Add copy(InputStream) to TemporaryBuffer
Quoted 9 lines · Robin Rosenberg
torsdag 11 december 2008 05:58:39 skrev Shawn O. Pearce:
org.spearce.jgit/src/org/spearce/jgit/util/TemporaryBuffer.java public void write(final byte[] b, int off, int len) throws IOException { · line 149
Quoted 6 lines
In some places we may find it ourselves with an InputStream we need to copy into a TemporaryBuffer, so we can flatten out the entire stream to a single byte[]. Putting the copy loop here is more useful then duplicating it in application level code.
org.spearce.jgit/src/org/spearce/jgit/util/TemporaryBuffer.java +2
-0+ public void copy(final InputStream in) throws IOException { + final byte[] b = new byte[2048];Why not 8192 here too?
Blargh, you're right. Actually what I should do is look to see if blocks != null, in which case I should alloc a block and read directly into it. That would avoid one copy of the data.
Signature · 1 line
Shawn.
PATCH v2 2/5 Add copy(InputStream) to TemporaryBuffer
In some places we may find ourselves with an InputStream we need to copy into a TemporaryBuffer, so we can flatten out the entire stream to a single byte[]. Putting the copy loop here is more useful then duplicating it in application level code.
---
Quoted 9 lines · Shawn O. Pearce
Quoted 4 lines · Robin Rosenberg
torsdag 11 december 2008 05:58:39 skrev Shawn O. Pearce:
Review
Quoted 1 line
(quoted diff) +1
-0+ final byte[] b = new byte[2048];Why not 8192 here too?
Blargh, you're right. Actually what I should do is look to see if blocks != null, in which case I should alloc a block and read directly into it. That would avoid one copy of the data.
And now we do that...
org.spearce.jgit/src/org/spearce/jgit/util/TemporaryBuffer.java +34 -0
@@ -42,6 +42,7 @@ import java.io.FileInputStream; import java.io.FileOutputStream; import java.io.IOException; +import java.io.InputStream; import java.io.OutputStream; import java.util.ArrayList;
@@ -135,6 +136,39 @@ public void write(final byte[] b, int off, int len) throws IOException { diskOut.write(b, off, len); } + /** + * Copy all bytes remaining on the input stream into this buffer. + * + * @param in + * the stream to read from, until EOF is reached. + * @throws IOException + * an error occurred reading from the input stream, or while + * writing to a local temporary file. + */ + public void copy(final InputStream in) throws IOException { + if (blocks != null) { + for (;;) { + Block s = last(); + if (s.isFull()) { + if (reachedInCoreLimit()) + break; + s = new Block(); + blocks.add(s); + } + + final int n = in.read(s.buffer, s.count, Block.SZ - s.count); + if (n < 1) + return; + s.count += n; + } + } + + final byte[] tmp = new byte[Block.SZ]; + int n; + while ((n = in.read(tmp)) > 0) + diskOut.write(tmp, 0, n); + } + private Block last() { return blocks.get(blocks.size() - 1); }
Signature · 5 lines
1.6.1.rc2.306.ge5d5e -- Shawn.