Detection of directory renames
v4 cover Detection of directory renames
At last a reroll of that old series. The most prominent change is a split of the old --factorize-renames flag into --detect-dir-renames and --hide-dir-rename-details - hopefully following Junio's comments dated nearly 2 years ago. While doing this, a couple of testcases got added (as well as a list of more testcases to be written), and some minor optimizations were done, but no heavy surgery.
I hope we can turn the simple detection feature to something acceptable for inclusion - that is, once unified diff output shows the detected renames as annotations, and once the known major holes are plugged (both from FIXME and testcases). For the "hiding" one OTOH, a more precise description of what we want must probably be done before progressing, but I suggest we keep that for later.
Also, maybe the --detect-dir-renames would get a good alias as "-M -M".
Does all of that look reasonable ?
Yann Dirson (4):
Introduce wholesame directory move detection in diffcore. Add testcases for the --detect-dir-renames diffcore flag. Allow hiding renames of individual files involved in a directory
rename.
Add testcases for the --hide-dir-rename-details diffcore flag.
diff-lib.c | 6 +- diff.c | 12 + diff.h | 6 + diffcore-rename.c | 316 +++++++++++++++++++++++++++- diffcore.h | 1 + t/t4046-diff-rename-factorize.sh | 433 ++++++++++++++++++++++++++++++++++++++ tree-diff.c | 4 +- 7 files changed, 766 insertions(+), 12 deletions(-) create mode 100755 t/t4046-diff-rename-factorize.sh
Signature · 1 line
1.7.2.3
PATCH v4 1/4 Introduce wholesame directory move detection in diffcore.
This feature tries to group together files moving from and to identical directories - the most common case being directory renames.
It is activated by the new --detect-dir-renames diffcore flag.
This is only the first step, adding the basic functionnality and adding support to raw diff output (and it breaks unified-diff output which does not know how to handle that stuff yet).
Even the output format may not be kept as is. For now both the result of "mv a b" and "mv a/* b/" are displayed as "Rnnn a/ b/", which is probably not what we want. "Rnnn a/* b/" could be a good choice for the latter if we want them to be distinguished, and even if we want them to look the same.
Other future developements to be made on top of this include:
* extension of unified-diff format to express this
* application of such new diffs
* teach git-svn (and others ?) to make use of that flag
* merge correctly in case of addition into a moved dir
* detect "directory splits" so merge can flag a conflict on file adds
* use inexact dir renames to bump score of below-threshold renames from/to same locations
* add yours here
--- diff-lib.c | 6 +- diff.c | 5 + diff.h | 3 + diffcore-rename.c | 253 +++++++++++++++++++++++++++++++++++++++++++++++++++-- tree-diff.c | 4 +- 5 files changed, 260 insertions(+), 11 deletions(-)
diff-lib.c +4 -2
@@ -208,7 +208,8 @@ int run_diff_files(struct rev_info *revs, unsigned int option) ce_option, &dirty_submodule);if (!changed && !dirty_submodule) {ce_mark_uptodate (ce); -if (!DIFF_OPT_TST (&revs->diffopt,FIND_COPIES_HARDER )) +if (!DIFF_OPT_TST (&revs->diffopt,FIND_COPIES_HARDER ) && + !DIFF_OPT_TST (&revs->diffopt,DETECT_DIR_RENAMES ))continue ; } oldmode = ce->ce_mode;
@@ -338,7 +339,8 @@ static int show_modified(struct rev_info *revs, oldmode = old->ce_mode;if (mode == oldmode && !hashcmp (sha1, old->sha1) && !dirty_submodule && - !DIFF_OPT_TST (&revs->diffopt,FIND_COPIES_HARDER )) + !DIFF_OPT_TST (&revs->diffopt,FIND_COPIES_HARDER ) && + !DIFF_OPT_TST (&revs->diffopt,DETECT_DIR_RENAMES ))return 0 ; diff_change(&revs->diffopt, oldmode, mode,
diff.c +5 -0
@@ -3188,6 +3188,11 @@ int diff_opt_parse(struct diff_options *options, const char **av, int ac)DIFF_OPT_SET (options,REVERSE_DIFF ); else if (!strcmp(arg, "--find-copies-harder"))DIFF_OPT_SET (options,FIND_COPIES_HARDER ); + else if (!strcmp(arg, "--detect-dir-renames")) { +DIFF_OPT_SET (options,DETECT_DIR_RENAMES ); +if (!options->detect_rename) + options->detect_rename =DIFF_DETECT_RENAME ; + } else if (!strcmp(arg, "--follow"))DIFF_OPT_SET (options,FOLLOW_RENAMES ); else if (!strcmp(arg, "--color"))
diff.h +3 -0
@@ -78,6 +78,7 @@ typedef struct strbuf *(*diff_prefix_fn_t)(struct diff_options *opt, void *data)#define DIFF_OPT_IGNORE_UNTRACKED_IN_SUBMODULES (1 << 25)#define DIFF_OPT_IGNORE_DIRTY_SUBMODULES (1 << 26)#define DIFF_OPT_OVERRIDE_SUBMODULE_CONFIG (1 << 27) +#define DIFF_OPT_DETECT_DIR_RENAMES (1 << 28)#define DIFF_OPT_TST (opts, flag) ((opts)->flags & DIFF_OPT_##flag)#define DIFF_OPT_SET (opts, flag) ((opts)->flags |= DIFF_OPT_##flag)
@@ -265,6 +266,8 @@ extern void diffcore_fix_diff_index(struct diff_options *);" -C detect copies.\n" \" --find-copies-harder\n" \" try unchanged files as candidate for copy detection.\n" \ +" --detect-dir-renames\n" \ +" detect wholesale directory renames.\n" \" -l<n> limit rename attempts up to <n> paths.\n" \" -O<file> reorder diffs according to the <file>.\n" \" -S<string> find filepair whose only one side contains the string.\n" \
diffcore-rename.c +245 -8
300 lines not on this page · raw message
tree-diff.c +3 -1
@@ -49,7 +49,9 @@ static int compare_tree_entry(struct tree_desc *t1, struct tree_desc *t2, constshow_entry (opt,"+" , t2, base, baselen);return 1 ; } -if (!DIFF_OPT_TST (opt,FIND_COPIES_HARDER ) && !hashcmp (sha1, sha2) && mode1 == mode2) +if (!DIFF_OPT_TST (opt,FIND_COPIES_HARDER ) && + !DIFF_OPT_TST (opt,DETECT_DIR_RENAMES ) && + !hashcmp (sha1, sha2) && mode1 == mode2)return 0 ;/*
Signature · 1 line
1.7.2.3
PATCH v4 2/4 Add testcases for the --detect-dir-renames diffcore flag.
From: Yann Dirson <[email protected]>
This notably includes a couple of tests for cases known not to be working correctly yet.
--- t/t4046-diff-rename-factorize.sh | 326 ++++++++++++++++++++++++++++++++++++++ 1 files changed, 326 insertions(+), 0 deletions(-) create mode 100755 t/t4046-diff-rename-factorize.sh
t/t4046-diff-rename-factorize.sh new file +326 -0
@@ -0,0 +1,326 @@ +#!/bin/sh +# +# Copyright (c) 2008,2010 Yann Dirson +# Copyright (c) 2005 Junio C Hamano +# + +# TODO - missing tests: +# * two dirs or more moving all their files to a single dir +# * simultaneous bulkmove and rename +# * add a new file under a dir that was moved in the same commit + +test_description='Test rename factorization in diff engine. + +' +. ./test-lib.sh +. "$TEST_DIRECTORY"/diff-lib.sh + +test_expect_success \ + 'commit the index.' \ + 'git update-ref HEAD $(echo "original empty commit" | git commit-tree $(git write-tree))' + +mkdir a +echo >a/path0 'Line 1 +Line 2 +Line 3 +Line 4 +Line 5 +Line 6 +Line 7 +Line 8 +Line 9 +Line 10 +line 11 +Line 12 +Line 13 +Line 14 +Line 15 +' +sed <a/path0 >a/path1 s/Line/Record/ +sed <a/path0 >a/path2 s/Line/Stuff/ +sed <a/path0 >a/path3 s/Line/Blurb/ + +test_expect_success \ + 'update-index --add file inside a directory.' \ + 'git update-index --add a/path*' + +test_expect_success \ + 'commit the index.' \ + 'git update-ref HEAD $(echo "original set of files" | git commit-tree $(git write-tree))' + +mv a b +test_expect_success \ + 'renamed the directory.' \ + 'git update-index --add --remove a/path0 a/path1 a/path2 a/path3 b/path*' + +test_expect_success \ + 'git diff-index --detect-dir-renames after directory move.' \ + 'git diff-index --detect-dir-renames HEAD >current' +grep -v "^\[DBG\] " <current >current.filtered +cat >expected <<\EOF +:040000 040000 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 R100 a/ b/ +:100644 100644 fdbec444a77953b1bcc899d9fabfa202e5e68f08 fdbec444a77953b1bcc899d9fabfa202e5e68f08 R100 a/path0 b/path0 +:100644 100644 2f1f8d70c0fdad990819dfe37a31deb010805161 2f1f8d70c0fdad990819dfe37a31deb010805161 R100 a/path1 b/path1 +:100644 100644 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 R100 a/path2 b/path2 +:100644 100644 c6971ab9f08a6cd9c89a0f87d94ae347aad6144a c6971ab9f08a6cd9c89a0f87d94ae347aad6144a R100 a/path3 b/path3 +EOF + +test_expect_success \ + 'validate the output for directory move.' \ + 'compare_diff_patch expected current.filtered' + +# now test non-100% renames + +echo 'Line 16' >> b/path0 +mv b/path2 b/2path +rm b/path3 +echo anything > b/path100 +test_expect_success \ + 'edited dir contents.' \ + 'git update-index --add --remove b/* b/path2 b/path3' + +test_expect_success \ + 'git diff-index --detect-dir-renames after directory move and content changes.' \ + 'git diff-index --detect-dir-renames HEAD >current' +grep -v "^\[DBG\] " <current >current.filtered +cat >expected <<\EOF +:040000 040000 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 R100 a/ b/ +:100644 000000 c6971ab9f08a6cd9c89a0f87d94ae347aad6144a 0000000000000000000000000000000000000000 D a/path3 +:100644 100644 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 R100 a/path2 b/2path +:100644 100644 fdbec444a77953b1bcc899d9fabfa202e5e68f08 4db595d12886f90e36765fc1732c17bccb836663 R093 a/path0 b/path0 +:100644 100644 2f1f8d70c0fdad990819dfe37a31deb010805161 2f1f8d70c0fdad990819dfe37a31deb010805161 R100 a/path1 b/path1 +:000000 100644 0000000000000000000000000000000000000000 1ba4650885513e62386fd3e23aeb45beeb67d3bb A b/path100 +EOF + +test_expect_success \ + 'validate the output for directory move and content changes.' \ + 'compare_diff_patch expected current.filtered' + +git reset -q --hard + +# now test bulk moves that are not directory moves (get consensus before going further ?) + +mkdir c +for i in 0 1 2; do cp a/path$i c/apath$i; done +test_expect_success \ + 'add files into a new directory.' \ + 'git update-index --add c/apath*' + +test_expect_success \ + 'commit all this.' \ + 'git commit -m "first set of changes"' + +mv c/* a/ +test_expect_success \ + 'move all of the new dir contents into a preexisting dir.' \ + 'git update-index --add --remove a/* c/apath0 c/apath1 c/apath2' + +test_expect_success \ + 'git diff-index --detect-dir-renames without full-dir rename.' \ + 'git diff-index --detect-dir-renames HEAD >current' +grep -v "^\[DBG\] " <current >current.filtered +cat >expected <<\EOF +:040000 040000 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 R100 c/* a/ +:100644 100644 fdbec444a77953b1bcc899d9fabfa202e5e68f08 fdbec444a77953b1bcc899d9fabfa202e5e68f08 R100 c/apath0 a/apath0 +:100644 100644 2f1f8d70c0fdad990819dfe37a31deb010805161 2f1f8d70c0fdad990819dfe37a31deb010805161 R100 c/apath1 a/apath1 +:100644 100644 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 R100 c/apath2 a/apath2 +EOF + +test_expect_failure \ + 'validate the output for bulk rename without full-dir rename.' \ + 'compare_diff_patch expected current.filtered' + +git reset -q --hard + +# now test moves to toplevel + +mv c/* . +test_expect_success \ + 'move all of the new dir contents into toplevel.' \ + 'git update-index --add --remove apath* c/apath0 c/apath1 c/apath2' + +test_expect_success \ + 'git diff-index --detect-dir-renames files bulk-moved to toplevel.' \ + 'git diff-index --detect-dir-renames HEAD >current' +grep -v "^\[DBG\] " <current >current.filtered +cat >expected <<\EOF +:040000 040000 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 R100 c/* ./ +:100644 100644 fdbec444a77953b1bcc899d9fabfa202e5e68f08 fdbec444a77953b1bcc899d9fabfa202e5e68f08 R100 c/apath0 apath0 +:100644 100644 2f1f8d70c0fdad990819dfe37a31deb010805161 2f1f8d70c0fdad990819dfe37a31deb010805161 R100 c/apath1 apath1 +:100644 100644 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 R100 c/apath2 apath2 +EOF + +test_expect_failure \ + 'validate the output for files bulk-moved to toplevel.' \ + 'compare_diff_patch expected current.filtered' + +git reset -q --hard + +# now test renaming with subdirs (does not take subdirs into account) + +mv c a/ +test_expect_success \ + 'move the new dir as subdir of another.' \ + 'git update-index --add --remove a/c/* c/apath0 c/apath1 c/apath2' + +test_expect_success \ + 'commit all this.' \ + 'git commit -m "move as subdir"' + +mv a b +echo foo >> b/c/apath0 +test_expect_success \ + 'rename the directory with some changes.' \ + 'git update-index --add --remove a/path0 a/path1 a/path2 a/path3 a/c/apath0 a/c/apath1 a/c/apath2 b/path* b/c/apath*' + +test_expect_success \ + 'git diff-index --detect-dir-renames on a move including a subdir.' \ + 'git diff-index --detect-dir-renames HEAD >current' +grep -v "^\[DBG\] " <current >current.filtered +cat >expected <<\EOF +:040000 040000 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 R100 a/ b/ +:100644 100644 fdbec444a77953b1bcc899d9fabfa202e5e68f08 00084e5ea68b5ae339b7c4b429e4a70fe25d069b R096 a/c/apath0 b/c/apath0 +:100644 100644 2f1f8d70c0fdad990819dfe37a31deb010805161 2f1f8d70c0fdad990819dfe37a31deb010805161 R100 a/c/apath1 b/c/apath1 +:100644 100644 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 R100 a/c/apath2 b/c/apath2 +:100644 100644 fdbec444a77953b1bcc899d9fabfa202e5e68f08 fdbec444a77953b1bcc899d9fabfa202e5e68f08 R100 a/path0 b/path0 +:100644 100644 2f1f8d70c0fdad990819dfe37a31deb010805161 2f1f8d70c0fdad990819dfe37a31deb010805161 R100 a/path1 b/path1 +:100644 100644 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 R100 a/path2 b/path2 +:100644 100644 c6971ab9f08a6cd9c89a0f87d94ae347aad6144a c6971ab9f08a6cd9c89a0f87d94ae347aad6144a R100 a/path3 b/path3 +EOF + +test_expect_failure \ + 'validate the output for a move including a subdir.' \ + 'compare_diff_patch expected current.filtered' + +git reset -q --hard + +# now check that moving all files but not subdirs is not mistaken for dir move + +mkdir b +mv a/path* b/ +test_expect_success \ + 'rename files in the directory but not subdir.' \ + 'git update-index --add --remove a/path0 a/path1 a/path2 a/path3 b/path*' + +test_expect_success \ + 'git diff-index --detect-dir-renames on a move without a subdir.' \ + 'git diff-index --detect-dir-renames HEAD >current' +grep -v "^\[DBG\] " <current >current.filtered +cat >expected <<\EOF +:100644 100644 fdbec444a77953b1bcc899d9fabfa202e5e68f08 fdbec444a77953b1bcc899d9fabfa202e5e68f08 R100 a/path0 b/path0 +:100644 100644 2f1f8d70c0fdad990819dfe37a31deb010805161 2f1f8d70c0fdad990819dfe37a31deb010805161 R100 a/path1 b/path1 +:100644 100644 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 R100 a/path2 b/path2 +:100644 100644 c6971ab9f08a6cd9c89a0f87d94ae347aad6144a c6971ab9f08a6cd9c89a0f87d94ae347aad6144a R100 a/path3 b/path3 +EOF + +test_expect_success \ + 'validate the output for a move without a subdir.' \ + 'compare_diff_patch expected current.filtered' + +git reset -q --hard + +# now check that moving subdirs into one dir and files into another is not mistaken for dir move +# (well, clearly it is ...) + +mv a/c b +mv a d +test_expect_success \ + 'rename subdir and files into different places.' \ + 'git update-index --add --remove a/path0 a/path1 a/path2 a/path3 a/c/apath0 a/c/apath1 a/c/apath2 d/path* b/apath*' + +test_expect_success \ + 'git diff-index --detect-dir-renames on a split of subdir and files into different places.' \ + 'git diff-index --detect-dir-renames HEAD >current' +grep -v "^\[DBG\] " <current >current.filtered +cat >expected <<\EOF +:040000 040000 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 R100 a/c/ b/ +:100644 100644 fdbec444a77953b1bcc899d9fabfa202e5e68f08 fdbec444a77953b1bcc899d9fabfa202e5e68f08 R100 a/c/apath0 b/apath0 +:100644 100644 2f1f8d70c0fdad990819dfe37a31deb010805161 2f1f8d70c0fdad990819dfe37a31deb010805161 R100 a/c/apath1 b/apath1 +:100644 100644 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 R100 a/c/apath2 b/apath2 +:100644 100644 fdbec444a77953b1bcc899d9fabfa202e5e68f08 fdbec444a77953b1bcc899d9fabfa202e5e68f08 R100 a/path0 d/path0 +:100644 100644 2f1f8d70c0fdad990819dfe37a31deb010805161 2f1f8d70c0fdad990819dfe37a31deb010805161 R100 a/path1 d/path1 +:100644 100644 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 R100 a/path2 d/path2 +:100644 100644 c6971ab9f08a6cd9c89a0f87d94ae347aad6144a c6971ab9f08a6cd9c89a0f87d94ae347aad6144a R100 a/path3 d/path3 +EOF + +test_expect_failure \ + 'validate the output for a split of subdir and files into different places.' \ + 'compare_diff_patch expected current.filtered' + +# now test moving a dir with no files but only subdirs +# (only factorizes lowest-level directories - not a big deal, just not perfect) + +git reset -q --hard +mkdir a/b +mv a/path* a/b/ +test_expect_success \ + 'setup the directory with only subdirs, no direct child files.' \ + 'git update-index --add --remove a/path0 a/path1 a/path2 a/path3 a/b/path*' +test_expect_success \ + 'commit all this.' \ + 'git commit -m "move all toplevel files down one level"' + +test_expect_success \ + 'move this dir.' \ + 'git mv a z' + +test_expect_success \ + 'git diff-index --detect-dir-renames with only subdirs' \ + 'git diff-index --detect-dir-renames HEAD >current' +grep -v "^\[DBG\] " <current >current.filtered +cat >expected <<\EOF +:040000 040000 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 R100 a/ z/ +:040000 040000 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 R100 a/b/ z/b/ +:040000 040000 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 R100 a/c/ z/c/ +:100644 100644 fdbec444a77953b1bcc899d9fabfa202e5e68f08 fdbec444a77953b1bcc899d9fabfa202e5e68f08 R100 a/b/path0 z/b/path0 +:100644 100644 2f1f8d70c0fdad990819dfe37a31deb010805161 2f1f8d70c0fdad990819dfe37a31deb010805161 R100 a/b/path1 z/b/path1 +:100644 100644 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 R100 a/b/path2 z/b/path2 +:100644 100644 c6971ab9f08a6cd9c89a0f87d94ae347aad6144a c6971ab9f08a6cd9c89a0f87d94ae347aad6144a R100 a/b/path3 z/b/path3 +:100644 100644 fdbec444a77953b1bcc899d9fabfa202e5e68f08 fdbec444a77953b1bcc899d9fabfa202e5e68f08 R100 a/c/apath0 z/c/apath0 +:100644 100644 2f1f8d70c0fdad990819dfe37a31deb010805161 2f1f8d70c0fdad990819dfe37a31deb010805161 R100 a/c/apath1 z/c/apath1 +:100644 100644 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 R100 a/c/apath2 z/c/apath2 +EOF + +test_expect_failure \ + 'validate the output for a move with only subdirs.' \ + 'compare_diff_patch expected current.filtered' + +# now test moving all files from toplevel into subdir (does not hides file moves) (needs consensus on syntax) +# Note: this is a special case of move of a dir into one of its own subdirs, which in +# turn is a variant of new files/dirs being added into a dir after all its contents +# are moved away + +git reset -q --hard HEAD~3 + +mv a/* . +test_expect_success \ + 'rename the directory with some changes.' \ + 'git update-index --add --remove a/path0 a/path1 a/path2 a/path3 path*' + +test_expect_success \ + 'commit all this.' \ + 'git commit -m "move all files to toplevel"' + +mkdir z +mv path* z/ +test_expect_success \ + 'rename the directory with some changes.' \ + 'git update-index --add --remove path0 path1 path2 path3 z/path*' + +test_expect_success \ + 'git diff-index --detect-dir-renames everything from toplevel.' \ + 'git diff-index --detect-dir-renames HEAD >current' +grep -v "^\[DBG\] " <current >current.filtered +cat >expected <<\EOF +:040000 040000 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 R100 ./* z/ +:100644 100644 fdbec444a77953b1bcc899d9fabfa202e5e68f08 fdbec444a77953b1bcc899d9fabfa202e5e68f08 R100 path0 z/path0 +:100644 100644 2f1f8d70c0fdad990819dfe37a31deb010805161 2f1f8d70c0fdad990819dfe37a31deb010805161 R100 path1 z/path1 +:100644 100644 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 R100 path2 z/path2 +:100644 100644 c6971ab9f08a6cd9c89a0f87d94ae347aad6144a c6971ab9f08a6cd9c89a0f87d94ae347aad6144a R100 path3 z/path3 +EOF + +test_expect_failure \ + 'validate the output for a move of everything from toplevel.' \ + 'compare_diff_patch expected current.filtered' + +test_done
Signature · 1 line
1.7.2.3
PATCH v4 3/4 Allow hiding renames of individual files involved in a directory rename.
From: Yann Dirson <[email protected]>
Once --detect-dir-renames has identified groups of bulk-moved files, and then the --hide-dir-rename-details flag hides those of the individual renames which carry no other information (further name change, or content changes).
This makes it much easier to a human reader to spot content changes in a commit that also moves a whole subtree.
--- diff.c | 7 +++++ diff.h | 3 ++ diffcore-rename.c | 67 ++++++++++++++++++++++++++++++++++++++++++++++++++-- diffcore.h | 1 + 4 files changed, 75 insertions(+), 3 deletions(-)
diff.c +7 -0
@@ -3193,6 +3193,13 @@ int diff_opt_parse(struct diff_options *options, const char **av, int ac)if (!options->detect_rename) options->detect_rename =DIFF_DETECT_RENAME ; } +else if (!strcmp (arg,"--hide-dir-rename-details" )) { +DIFF_OPT_SET (options,HIDE_DIR_RENAME_DETAILS ); +if (!DIFF_OPT_TST (options,DETECT_DIR_RENAMES )) +DIFF_OPT_SET (options,DETECT_DIR_RENAMES ); +if (!options->detect_rename) + options->detect_rename =DIFF_DETECT_RENAME ; + }else if (!strcmp (arg,"--follow" ))DIFF_OPT_SET (options,FOLLOW_RENAMES );else if (!strcmp (arg,"--color" ))
diff.h +3 -0
@@ -79,6 +79,7 @@ typedef struct strbuf *(*diff_prefix_fn_t)(struct diff_options *opt, void *data)#define DIFF_OPT_IGNORE_DIRTY_SUBMODULES (1 << 26)#define DIFF_OPT_OVERRIDE_SUBMODULE_CONFIG (1 << 27)#define DIFF_OPT_DETECT_DIR_RENAMES (1 << 28) +#define DIFF_OPT_HIDE_DIR_RENAME_DETAILS (1 << 29)#define DIFF_OPT_TST (opts, flag) ((opts)->flags & DIFF_OPT_##flag)#define DIFF_OPT_SET (opts, flag) ((opts)->flags |= DIFF_OPT_##flag)
@@ -268,6 +269,8 @@ extern void diffcore_fix_diff_index(struct diff_options *);" try unchanged files as candidate for copy detection.\n" \" --detect-dir-renames\n" \" detect wholesale directory renames.\n" \ +" --hide-dir-rename-details\n" \ +" hide renames of individual files in a directory rename.\n" \" -l<n> limit rename attempts up to <n> paths.\n" \" -O<file> reorder diffs according to the <file>.\n" \" -S<string> find filepair whose only one side contains the string.\n" \
diffcore-rename.c +64 -3
97 lines not on this page · raw message
diffcore.h +1 -0
@@ -69,6 +69,7 @@ struct diff_filepair { unsigned broken_pair : 1; unsigned renamed_pair : 1; unsigned is_unmerged : 1; + unsigned uninteresting_rename : 1; };#define DIFF_PAIR_UNMERGED (p) ((p)->is_unmerged)
Signature · 1 line
1.7.2.3
PATCH v4 4/4 Add testcases for the --hide-dir-rename-details diffcore flag.
From: Yann Dirson <[email protected]>
--- t/t4046-diff-rename-factorize.sh | 107 ++++++++++++++++++++++++++++++++++++++ 1 files changed, 107 insertions(+), 0 deletions(-)
t/t4046-diff-rename-factorize.sh +107 -0
@@ -69,6 +69,18 @@ test_expect_success \ 'validate the output for directory move.' \ 'compare_diff_patch expected current.filtered' +test_expect_success \ + 'git diff-index --hide-dir-rename-details after directory move.' \ + 'git diff-index --hide-dir-rename-details HEAD >current' +grep -v "^\[DBG\] " <current >current.filtered +cat >expected <<\EOF +:040000 040000 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 R100 a/ b/ +EOF + +test_expect_success \ + 'validate the output for directory move.' \ + 'compare_diff_patch expected current.filtered' + # now test non-100% renames echo 'Line 16' >> b/path0
@@ -96,6 +108,22 @@ test_expect_success \ 'validate the output for directory move and content changes.' \ 'compare_diff_patch expected current.filtered' +test_expect_success \ + 'git diff-index --hide-dir-rename-details after directory move and content changes.' \ + 'git diff-index --hide-dir-rename-details HEAD >current' +grep -v "^\[DBG\] " <current >current.filtered +cat >expected <<\EOF +:040000 040000 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 R100 a/ b/ +:100644 000000 c6971ab9f08a6cd9c89a0f87d94ae347aad6144a 0000000000000000000000000000000000000000 D a/path3 +:100644 100644 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 R100 a/path2 b/2path +:100644 100644 fdbec444a77953b1bcc899d9fabfa202e5e68f08 4db595d12886f90e36765fc1732c17bccb836663 R093 a/path0 b/path0 +:000000 100644 0000000000000000000000000000000000000000 1ba4650885513e62386fd3e23aeb45beeb67d3bb A b/path100 +EOF + +test_expect_success \ + 'validate the output for directory move and content changes.' \ + 'compare_diff_patch expected current.filtered' + git reset -q --hard # now test bulk moves that are not directory moves (get consensus before going further ?)
@@ -130,6 +158,18 @@ test_expect_failure \ 'validate the output for bulk rename without full-dir rename.' \ 'compare_diff_patch expected current.filtered' +test_expect_success \ + 'git diff-index --hide-dir-rename-details without full-dir rename.' \ + 'git diff-index --hide-dir-rename-details HEAD >current' +grep -v "^\[DBG\] " <current >current.filtered +cat >expected <<\EOF +:040000 040000 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 R100 c/* a/ +EOF + +test_expect_failure \ + 'validate the output for bulk rename without full-dir rename.' \ + 'compare_diff_patch expected current.filtered' + git reset -q --hard # now test moves to toplevel
@@ -154,6 +194,18 @@ test_expect_failure \ 'validate the output for files bulk-moved to toplevel.' \ 'compare_diff_patch expected current.filtered' +test_expect_success \ + 'git diff-index --hide-dir-rename-details files bulk-moved to toplevel.' \ + 'git diff-index --hide-dir-rename-details HEAD >current' +grep -v "^\[DBG\] " <current >current.filtered +cat >expected <<\EOF +:040000 040000 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 R100 c/* ./ +EOF + +test_expect_failure \ + 'validate the output for files bulk-moved to toplevel.' \ + 'compare_diff_patch expected current.filtered' + git reset -q --hard # now test renaming with subdirs (does not take subdirs into account)
@@ -192,6 +244,19 @@ test_expect_failure \ 'validate the output for a move including a subdir.' \ 'compare_diff_patch expected current.filtered' +test_expect_success \ + 'git diff-index --hide-dir-rename-details on a move including a subdir.' \ + 'git diff-index --hide-dir-rename-details HEAD >current' +grep -v "^\[DBG\] " <current >current.filtered +cat >expected <<\EOF +:040000 040000 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 R100 a/ b/ +:100644 100644 fdbec444a77953b1bcc899d9fabfa202e5e68f08 00084e5ea68b5ae339b7c4b429e4a70fe25d069b R096 a/c/apath0 b/c/apath0 +EOF + +test_expect_failure \ + 'validate the output for a move including a subdir.' \ + 'compare_diff_patch expected current.filtered' + git reset -q --hard # now check that moving all files but not subdirs is not mistaken for dir move
@@ -217,6 +282,8 @@ test_expect_success \ 'validate the output for a move without a subdir.' \ 'compare_diff_patch expected current.filtered' +# no need to test for --hide-dir-rename-details here + git reset -q --hard # now check that moving subdirs into one dir and files into another is not mistaken for dir move
@@ -247,6 +314,22 @@ test_expect_failure \ 'validate the output for a split of subdir and files into different places.' \ 'compare_diff_patch expected current.filtered' +test_expect_success \ + 'git diff-index --hide-dir-rename-details on a split of subdir and files into different places.' \ + 'git diff-index --hide-dir-rename-details HEAD >current' +grep -v "^\[DBG\] " <current >current.filtered +cat >expected <<\EOF +:040000 040000 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 R100 a/c/ b/ +:100644 100644 fdbec444a77953b1bcc899d9fabfa202e5e68f08 fdbec444a77953b1bcc899d9fabfa202e5e68f08 R100 a/path0 d/path0 +:100644 100644 2f1f8d70c0fdad990819dfe37a31deb010805161 2f1f8d70c0fdad990819dfe37a31deb010805161 R100 a/path1 d/path1 +:100644 100644 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 R100 a/path2 d/path2 +:100644 100644 c6971ab9f08a6cd9c89a0f87d94ae347aad6144a c6971ab9f08a6cd9c89a0f87d94ae347aad6144a R100 a/path3 d/path3 +EOF + +test_expect_failure \ + 'validate the output for a split of subdir and files into different places.' \ + 'compare_diff_patch expected current.filtered' + # now test moving a dir with no files but only subdirs # (only factorizes lowest-level directories - not a big deal, just not perfect)
@@ -285,6 +368,18 @@ test_expect_failure \ 'validate the output for a move with only subdirs.' \ 'compare_diff_patch expected current.filtered' +test_expect_success \ + 'git diff-index --hide-dir-rename-details with only subdirs' \ + 'git diff-index --hide-dir-rename-details HEAD >current' +grep -v "^\[DBG\] " <current >current.filtered +cat >expected <<\EOF +:040000 040000 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 R100 a/ z/ +EOF + +test_expect_failure \ + 'validate the output for a move with only subdirs.' \ + 'compare_diff_patch expected current.filtered' + # now test moving all files from toplevel into subdir (does not hides file moves) (needs consensus on syntax) # Note: this is a special case of move of a dir into one of its own subdirs, which in # turn is a variant of new files/dirs being added into a dir after all its contents
@@ -323,4 +418,16 @@ test_expect_failure \ 'validate the output for a move of everything from toplevel.' \ 'compare_diff_patch expected current.filtered' +test_expect_success \ + 'git diff-index --hide-dir-rename-details everything from toplevel.' \ + 'git diff-index --hide-dir-rename-details HEAD >current' +grep -v "^\[DBG\] " <current >current.filtered +cat >expected <<\EOF +:040000 040000 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 R100 ./* z/ +EOF + +test_expect_failure \ + 'validate the output for a move of everything from toplevel.' \ + 'compare_diff_patch expected current.filtered' + test_done
Signature · 1 line
1.7.2.3
Re: [PATCH v4 4/4] Add testcases for the --hide-dir-rename-details diffcore flag.
Heya,
Quoted 5 lines · Yann Dirson
+:040000 040000 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 R100 a/ b/ +:100644 000000
c6971ab9f08a6cd9c89a0f87d94ae347aad6144a0000000000000000000000000000000000000000 D a/path3 +:100644 100644dbde7141d737c8aa0003672c1bc21ded48c6c3b9dbde7141d737c8aa0003672c1bc21ded48c6c3b9R100 a/path2 b/2path +:100644 100644fdbec444a77953b1bcc899d9fabfa202e5e68f084db595d12886f90e36765fc1732c17bccb836663R093 a/path0 b/path0 +:000000 100644 00000000000000000000000000000000000000001ba4650885513e62386fd3e23aeb45beeb67d3bbA b/path100
There was a recent discussion that came to the conclusion that we don't want to be comparing against hashes outside t0000. So either don't use diff-index, or use 'git rev-parse' to get those objects first, and then compare against them symbolically (e.g., ":100644 000000 $(path3) $(nullref)", or such).
Signature · 3 lines
Cheers, Sverre Rabbelier
Re: [PATCH v4 3/4] Allow hiding renames of individual files involved in a directory rename.
In reply to Yann DirsonHeya,
Quoted 2 lines · Yann Dirson
This makes it much easier to a human reader to spot content changes in a commit that also moves a whole subtree.
I'd like to use this by default (but only for regular 'git diff', 'git log', etc., not for 'git format-patch') if/when it gets merged, can has config option?
Signature · 3 lines
Cheers, Sverre Rabbelier
Re: [PATCH v4 3/4] Allow hiding renames of individual files involved in a directory rename.
Quoted 3 lines · Sverre Rabbelier
I'd like to use this by default (but only for regular 'git diff', 'git log', etc., not for 'git format-patch') if/when it gets merged, can has config option?
Having a configuration variable might be a good idea, as long as there is a provision to avoid breaking scripted porcelains when the option is set.
Re: [PATCH v4 3/4] Allow hiding renames of individual files involved in a directory rename.
Heya,
Quoted 2 lines · Junio C Hamano
Having a configuration variable might be a good idea, as long as there is a provision to avoid breaking scripted porcelains when the option is set.
I agree... maybe we need a generic way to have a config that only affects porcelain?
Signature · 3 lines
Cheers, Sverre Rabbelier
Re: [PATCH v4 3/4] Allow hiding renames of individual files involved in a directory rename.
In reply to Sverre RabbelierQuoted 9 lines · Sverre Rabbelier
Heya,
Quoted 2 lines · Yann Dirson
This makes it much easier to a human reader to spot content changes in a commit that also moves a whole subtree.
I'd like to use this by default (but only for regular 'git diff', 'git log', etc., not for 'git format-patch') if/when it gets merged, can has config option?
Sure - I was leaving that sort of beels and whistles for after the core feature gets cold enough.
Re: [PATCH v4 2/4] Add testcases for the --detect-dir-renames diffcore flag.
In reply to Yann DirsonQuoted 25 lines · Yann Dirson
From: Yann Dirson <[email protected]>
This notably includes a couple of tests for cases known not to be working correctly yet.
---
t/t4046-diff-rename-factorize.sh | 326 ++++++++++++++++++++++++++++++++++++++1 files changed, 326 insertions(+), 0 deletions(-)create mode 100755
t/t4046-diff-rename-factorize.sht/t4046-diff-rename-factorize.sh new file +10
-0@@ -0,0 +1,326 @@ +#!/bin/sh +# +# Copyright (c) 2008,2010 Yann Dirson +# Copyright (c) 2005 Junio C Hamano +# + +# TODO - missing tests: +# * two dirs or more moving all their files to a single dir +# * simultaneous bulkmove and rename +# * add a new file under a dir that was moved in the same commit
t/t4046-diff-rename-factorize.sh line 20
Quoted 10 lines
t/t4046-diff-rename-factorize.sh new file +10
-0+ +test_description='Test rename factorization in diff engine. + +' +. ./test-lib.sh +. "$TEST_DIRECTORY"/diff-lib.sh + +test_expect_success \ + 'commit the index.' \ + 'git update-ref HEAD $(echo "original empty commit" | git commit-tree $(git write-tree))'
Since this is a new test it should use the nominal style:
test_expect_success 'commit the index.' '
git update-ref HEAD $(echo "original empty commit" | git
commit-tree $(git write-tree))
'
Quoted 10 lines
t/t4046-diff-rename-factorize.sh new file +10
-0+mkdir a +echo >a/path0 'Line 1 +Line 2 +Line 3 +Line 4 +Line 5 +Line 6 +Line 7 +Line 8 +Line 9
t/t4046-diff-rename-factorize.sh line 41
Quoted 10 lines
t/t4046-diff-rename-factorize.sh new file +10
-0+Line 10 +line 11 +Line 12 +Line 13 +Line 14 +Line 15 +' +sed <a/path0 >a/path1 s/Line/Record/ +sed <a/path0 >a/path2 s/Line/Stuff/ +sed <a/path0 >a/path3 s/Line/Blurb/
Should be in a test, see "Put all code inside test_expect_success and other assertions." in t/README.
t/t4046-diff-rename-factorize.sh line 51
Quoted 9 lines
t/t4046-diff-rename-factorize.sh new file +9
-0+test_expect_success \ + 'update-index --add file inside a directory.' \ + 'git update-index --add a/path*' + +test_expect_success \ + 'commit the index.' \ + 'git update-ref HEAD $(echo "original set of files" | git commit-tree $(git write-tree))' + +mv a b
likewise here, and throughout the test of the test.
Re: [PATCH v4 2/4] Add testcases for the --detect-dir-renames diffcore flag.
Quoted 1 line · ?var Arnfj?r? Bjarmason
Quoted 1 line
+mkdir a
[...]
Quoted 4 lines
Quoted 1 line
+sed <a/path0 >a/path3 s/Line/Blurb/
Should be in a test, see "Put all code inside
test_expect_successand other assertions." in t/README.
While I can see the value of having everything run from the test framework, I was thinking it would be even better to have a special "test_setup" clause, which could be as a first step just an alias to test_expect_success, but could afterwards gain features such as "not counted as success, and counted specially when failed", so we get better stats of the number of failures we get, and so that editor outlining/folding modes could be developped in a useful fashion.
Re: [PATCH v4 2/4] Add testcases for the --detect-dir-renames diffcore flag.
In reply to Yann DirsonQuoted 2 lines · Yann Dirson
From: Yann Dirson <[email protected]>
Subject: Add testcases for the --detect-dir-renames diffcore flag
Probably better for the test to be squashed with the patch that adds that option.
Quoted 3 lines
t/t4046-diff-rename-factorize.sh new file +0
-0@@ -0,0 +1,326 @@
[...]
t/t4046-diff-rename-factorize.sh
Quoted 3 lines
t/t4046-diff-rename-factorize.sh new file +3
-0+test_expect_success \ + 'commit the index.' \ + 'git update-ref HEAD $(echo "original empty commit" | git commit-tree $(git write-tree))'
Nit:
Now test assertions tend to be written as
test_expect_success '...' ' command && command && ... '
The tabs to indent and opening ' at the end of the first line means less fuss in lining things up. Making each test assertion include its setup means tests don't pass if something gets messed up in the setup, and using multiline test assertions with && means there is no confusion about what the tests were written to test.
As you mentioned, this has the unfortunate downside of messing with syntax highlighting. Maybe the common text editors need a new mode for git-style test scripts? Or maybe it would make sense to implement
test_begin_expecting_success '...' ... test_end
--- it would certainly make quoting easier.
It might make sense to compute the tree, commit, etc one at a time instead of this long one-liner.
Quoted 8 lines
t/t4046-diff-rename-factorize.sh new file +8
-0+ +mkdir a +echo >a/path0 'Line 1 +Line 2 +Line 3 +Line 4 +Line 5 +Line 6
t/t4046-diff-rename-factorize.sh
Quoted 10 lines
t/t4046-diff-rename-factorize.sh new file +10
-0+Line 7 +Line 8 +Line 9 +Line 10 +line 11 +Line 12 +Line 13 +Line 14 +Line 15 +'
To avoid quoting troubles and since these are innocuous commands, it could be nice to put this before the first test assertion. Or even better:
mkdir a && printf "Line %s\n" 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 >a/path0 && ...
[...]
t/t4046-diff-rename-factorize.sh
Quoted 11 lines
t/t4046-diff-rename-factorize.sh new file +11
-0+test_expect_success \ + 'git diff-index --detect-dir-renames after directory move.' \ + 'git diff-index --detect-dir-renames HEAD >current' +grep -v "^\[DBG\] " <current >current.filtered +cat >expected <<\EOF +:040000 040000 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 R100 a/ b/ +:100644 100644 fdbec444a77953b1bcc899d9fabfa202e5e68f08 fdbec444a77953b1bcc899d9fabfa202e5e68f08 R100 a/path0 b/path0 +:100644 100644 2f1f8d70c0fdad990819dfe37a31deb010805161 2f1f8d70c0fdad990819dfe37a31deb010805161 R100 a/path1 b/path1 +:100644 100644 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 R100 a/path2 b/path2 +:100644 100644 c6971ab9f08a6cd9c89a0f87d94ae347aad6144a c6971ab9f08a6cd9c89a0f87d94ae347aad6144a R100 a/path3 b/path3 +EOF
Nit: although compare_diff_patch ensures the result is not dependent on the hash function, these hard-coded hashes are still hard for a human to read. Could they be computed instead?
(Sorry, no thoughts about the actual content yet...)
---
t/t4046-diff-rename-factorize.sh +216 -282
@@ -15,312 +15,246 @@ test_description='Test rename factorization in diff engine. . ./test-lib.sh . "$TEST_DIRECTORY"/diff-lib.sh -test_expect_success \ - 'commit the index.' \ - 'git update-ref HEAD $(echo "original empty commit" | git commit-tree $(git write-tree))' - -mkdir a -echo >a/path0 'Line 1 -Line 2 -Line 3 -Line 4 -Line 5 -Line 6 -Line 7 -Line 8 -Line 9 -Line 10 -line 11 -Line 12 -Line 13 -Line 14 -Line 15 +test_expect_success 'setup' ' + zeroes=0000000000000000000000000000000000000000 && + + tree=$(git write-tree) && + commit=$(echo "original empty commit" | git commit-tree $tree) && + git update-ref HEAD $commit && + + mkdir a && + printf "Line %s\n" 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 >a/path0 && + sed <a/path0 >a/path1 s/Line/Record/ && + sed <a/path0 >a/path2 s/Line/Stuff/ && + sed <a/path0 >a/path3 s/Line/Blurb/ && + + git update-index --add a/path* && + tree=$(git write-tree) && + commit=$(echo "original set of files" | git commit-tree $tree) && + git update-ref HEAD $commit && + + path0_id=$(git rev-parse HEAD:a/path0) && + path1_id=$(git rev-parse HEAD:a/path1) && + path2_id=$(git rev-parse HEAD:a/path2) && + path3_id=$(git rev-parse HEAD:a/path3) && + + : rename the directory && + mv a b && + git update-index --add --remove a/path0 a/path1 a/path2 a/path3 b/path* ' -sed <a/path0 >a/path1 s/Line/Record/ -sed <a/path0 >a/path2 s/Line/Stuff/ -sed <a/path0 >a/path3 s/Line/Blurb/ - -test_expect_success \ - 'update-index --add file inside a directory.' \ - 'git update-index --add a/path*' - -test_expect_success \ - 'commit the index.' \ - 'git update-ref HEAD $(echo "original set of files" | git commit-tree $(git write-tree))' - -mv a b -test_expect_success \ - 'renamed the directory.' \ - 'git update-index --add --remove a/path0 a/path1 a/path2 a/path3 b/path*' - -test_expect_success \ - 'git diff-index --detect-dir-renames after directory move.' \ - 'git diff-index --detect-dir-renames HEAD >current' -grep -v "^\[DBG\] " <current >current.filtered -cat >expected <<\EOF -:040000 040000 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 R100 a/ b/ -:100644 100644 fdbec444a77953b1bcc899d9fabfa202e5e68f08 fdbec444a77953b1bcc899d9fabfa202e5e68f08 R100 a/path0 b/path0 -:100644 100644 2f1f8d70c0fdad990819dfe37a31deb010805161 2f1f8d70c0fdad990819dfe37a31deb010805161 R100 a/path1 b/path1 -:100644 100644 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 R100 a/path2 b/path2 -:100644 100644 c6971ab9f08a6cd9c89a0f87d94ae347aad6144a c6971ab9f08a6cd9c89a0f87d94ae347aad6144a R100 a/path3 b/path3 -EOF - -test_expect_success \ - 'validate the output for directory move.' \ - 'compare_diff_patch expected current.filtered' - -# now test non-100% renames - -echo 'Line 16' >> b/path0 -mv b/path2 b/2path -rm b/path3 -echo anything > b/path100 -test_expect_success \ - 'edited dir contents.' \ - 'git update-index --add --remove b/* b/path2 b/path3' - -test_expect_success \ - 'git diff-index --detect-dir-renames after directory move and content changes.' \ - 'git diff-index --detect-dir-renames HEAD >current' -grep -v "^\[DBG\] " <current >current.filtered -cat >expected <<\EOF -:040000 040000 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 R100 a/ b/ -:100644 000000 c6971ab9f08a6cd9c89a0f87d94ae347aad6144a 0000000000000000000000000000000000000000 D a/path3 -:100644 100644 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 R100 a/path2 b/2path -:100644 100644 fdbec444a77953b1bcc899d9fabfa202e5e68f08 4db595d12886f90e36765fc1732c17bccb836663 R093 a/path0 b/path0 -:100644 100644 2f1f8d70c0fdad990819dfe37a31deb010805161 2f1f8d70c0fdad990819dfe37a31deb010805161 R100 a/path1 b/path1 -:000000 100644 0000000000000000000000000000000000000000 1ba4650885513e62386fd3e23aeb45beeb67d3bb A b/path100 -EOF - -test_expect_success \ - 'validate the output for directory move and content changes.' \ - 'compare_diff_patch expected current.filtered' - -git reset -q --hard - -# now test bulk moves that are not directory moves (get consensus before going further ?) - -mkdir c -for i in 0 1 2; do cp a/path$i c/apath$i; done -test_expect_success \ - 'add files into a new directory.' \ - 'git update-index --add c/apath*' - -test_expect_success \ - 'commit all this.' \ - 'git commit -m "first set of changes"' - -mv c/* a/ -test_expect_success \ - 'move all of the new dir contents into a preexisting dir.' \ - 'git update-index --add --remove a/* c/apath0 c/apath1 c/apath2' - -test_expect_success \ - 'git diff-index --detect-dir-renames without full-dir rename.' \ - 'git diff-index --detect-dir-renames HEAD >current' -grep -v "^\[DBG\] " <current >current.filtered -cat >expected <<\EOF -:040000 040000 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 R100 c/* a/ -:100644 100644 fdbec444a77953b1bcc899d9fabfa202e5e68f08 fdbec444a77953b1bcc899d9fabfa202e5e68f08 R100 c/apath0 a/apath0 -:100644 100644 2f1f8d70c0fdad990819dfe37a31deb010805161 2f1f8d70c0fdad990819dfe37a31deb010805161 R100 c/apath1 a/apath1 -:100644 100644 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 R100 c/apath2 a/apath2 -EOF - -test_expect_failure \ - 'validate the output for bulk rename without full-dir rename.' \ - 'compare_diff_patch expected current.filtered' - -git reset -q --hard - -# now test moves to toplevel - -mv c/* . -test_expect_success \ - 'move all of the new dir contents into toplevel.' \ - 'git update-index --add --remove apath* c/apath0 c/apath1 c/apath2' - -test_expect_success \ - 'git diff-index --detect-dir-renames files bulk-moved to toplevel.' \ - 'git diff-index --detect-dir-renames HEAD >current' -grep -v "^\[DBG\] " <current >current.filtered -cat >expected <<\EOF -:040000 040000 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 R100 c/* ./ -:100644 100644 fdbec444a77953b1bcc899d9fabfa202e5e68f08 fdbec444a77953b1bcc899d9fabfa202e5e68f08 R100 c/apath0 apath0 -:100644 100644 2f1f8d70c0fdad990819dfe37a31deb010805161 2f1f8d70c0fdad990819dfe37a31deb010805161 R100 c/apath1 apath1 -:100644 100644 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 R100 c/apath2 apath2 -EOF - -test_expect_failure \ - 'validate the output for files bulk-moved to toplevel.' \ - 'compare_diff_patch expected current.filtered' -git reset -q --hard - -# now test renaming with subdirs (does not take subdirs into account) - -mv c a/ -test_expect_success \ - 'move the new dir as subdir of another.' \ - 'git update-index --add --remove a/c/* c/apath0 c/apath1 c/apath2' - -test_expect_success \ - 'commit all this.' \ - 'git commit -m "move as subdir"' - -mv a b -echo foo >> b/c/apath0 -test_expect_success \ - 'rename the directory with some changes.' \ - 'git update-index --add --remove a/path0 a/path1 a/path2 a/path3 a/c/apath0 a/c/apath1 a/c/apath2 b/path* b/c/apath*' - -test_expect_success \ - 'git diff-index --detect-dir-renames on a move including a subdir.' \ - 'git diff-index --detect-dir-renames HEAD >current' -grep -v "^\[DBG\] " <current >current.filtered -cat >expected <<\EOF -:040000 040000 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 R100 a/ b/ -:100644 100644 fdbec444a77953b1bcc899d9fabfa202e5e68f08 00084e5ea68b5ae339b7c4b429e4a70fe25d069b R096 a/c/apath0 b/c/apath0 -:100644 100644 2f1f8d70c0fdad990819dfe37a31deb010805161 2f1f8d70c0fdad990819dfe37a31deb010805161 R100 a/c/apath1 b/c/apath1 -:100644 100644 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 R100 a/c/apath2 b/c/apath2 -:100644 100644 fdbec444a77953b1bcc899d9fabfa202e5e68f08 fdbec444a77953b1bcc899d9fabfa202e5e68f08 R100 a/path0 b/path0 -:100644 100644 2f1f8d70c0fdad990819dfe37a31deb010805161 2f1f8d70c0fdad990819dfe37a31deb010805161 R100 a/path1 b/path1 -:100644 100644 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 R100 a/path2 b/path2 -:100644 100644 c6971ab9f08a6cd9c89a0f87d94ae347aad6144a c6971ab9f08a6cd9c89a0f87d94ae347aad6144a R100 a/path3 b/path3 -EOF - -test_expect_failure \ - 'validate the output for a move including a subdir.' \ - 'compare_diff_patch expected current.filtered' +test_expect_success 'diff-index --detect-dir-renames after directory move.' ' + cat >expected <<-EOF && + :040000 040000 $zeroes $zeroes R100 a/ b/ + :100644 100644 $path0_id $path0_id R100 a/path0 b/path0 + :100644 100644 $path1_id $path1_id R100 a/path1 b/path1 + :100644 100644 $path2_id $path2_id R100 a/path2 b/path2 + :100644 100644 $path3_id $path3_id R100 a/path3 b/path3 + EOF + git diff-index --detect-dir-renames HEAD >current && + grep -v "^\[DBG\] " <current >current.filtered && + compare_diff_patch expected current.filtered +' -git reset -q --hard +test_expect_success 'setup non-100% rename' ' + echo "Line 16" >>b/path0 && + mv b/path2 b/2path && + rm b/path3 && + echo anything >b/path100 && + git update-index --add --remove b/* b/path2 b/path3 +' -# now check that moving all files but not subdirs is not mistaken for dir move +test_expect_success 'diff-index --detect-dir-renames after content changes.' ' + path0_id2=$(git rev-parse :b/path0) && + path100_id=$(git rev-parse :b/path100) && + cat >expected <<-EOF && + :040000 040000 $zeroes $zeroes R100 a/ b/ + :100644 000000 $path3_id $zeroes D a/path3 + :100644 100644 $path2_id $path2_id R100 a/path2 b/2path + :100644 100644 $path0_id $path0_id2 R093 a/path0 b/path0 + :100644 100644 $path1_id $path1_id R100 a/path1 b/path1 + :000000 100644 $zeroes $path100_id A b/path100 + EOF + git diff-index --detect-dir-renames HEAD >current && + grep -v "^\[DBG\] " <current >current.filtered && + compare_diff_patch expected current.filtered +' -mkdir b -mv a/path* b/ -test_expect_success \ - 'rename files in the directory but not subdir.' \ - 'git update-index --add --remove a/path0 a/path1 a/path2 a/path3 b/path*' +test_expect_success 'setup bulk move that is not directory move' ' + git reset -q --hard && + + mkdir c && + ( + for i in 0 1 2 + do + cp a/path$i c/apath$i || + exit + done + ) && + git update-index --add c/apath* && + git commit -m "first set of changes" && + + mv c/* a/ && + git update-index --add --remove a/* c/apath0 c/apath1 c/apath2 +' -test_expect_success \ - 'git diff-index --detect-dir-renames on a move without a subdir.' \ - 'git diff-index --detect-dir-renames HEAD >current' -grep -v "^\[DBG\] " <current >current.filtered -cat >expected <<\EOF -:100644 100644 fdbec444a77953b1bcc899d9fabfa202e5e68f08 fdbec444a77953b1bcc899d9fabfa202e5e68f08 R100 a/path0 b/path0 -:100644 100644 2f1f8d70c0fdad990819dfe37a31deb010805161 2f1f8d70c0fdad990819dfe37a31deb010805161 R100 a/path1 b/path1 -:100644 100644 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 R100 a/path2 b/path2 -:100644 100644 c6971ab9f08a6cd9c89a0f87d94ae347aad6144a c6971ab9f08a6cd9c89a0f87d94ae347aad6144a R100 a/path3 b/path3 -EOF +test_expect_failure 'diff-index --detect-dir-renames without full-dir rename.' ' + cat >expected <<-EOF && + :040000 040000 $zeroes $zeroes R100 c/* a/ + :100644 100644 $path0_id $path0_id R100 c/apath0 a/apath0 + :100644 100644 $path1_id $path1_id R100 c/apath1 a/apath1 + :100644 100644 $path2_id $path2_id R100 c/apath2 a/apath2 + EOF + git diff-index --detect-dir-renames HEAD >current + grep -v "^\[DBG\] " <current >current.filtered && + compare_diff_patch expected current.filtered +' -test_expect_success \ - 'validate the output for a move without a subdir.' \ - 'compare_diff_patch expected current.filtered' +test_expect_success 'setup bulk move to toplevel' ' + git reset -q --hard && + mv c/* . && + git update-index --add --remove apath* c/apath0 c/apath1 c/apath2 +' -git reset -q --hard +test_expect_failure 'diff-index --detect-dir-renames bulk move to toplevel.' ' + cat >expected <<-EOF && + :040000 040000 $zeroes $zeroes R100 c/* ./ + :100644 100644 $path0_id $path0_id R100 c/apath0 apath0 + :100644 100644 $path1_id $path1_id R100 c/apath1 apath1 + :100644 100644 $path2_id $path2_id R100 c/apath2 apath2 + EOF + git diff-index --detect-dir-renames HEAD >current && + grep -v "^\[DBG\] " <current >current.filtered && + compare_diff_patch expected current.filtered +' -# now check that moving subdirs into one dir and files into another is not mistaken for dir move -# (well, clearly it is ...) +test_expect_success 'setup move including a subdir, with some content changes' ' + git reset -q --hard && + mv c a/ && + git update-index --add --remove a/c/* c/apath0 c/apath1 c/apath2 && + git commit -m "move as subdir" && -mv a/c b -mv a d -test_expect_success \ - 'rename subdir and files into different places.' \ - 'git update-index --add --remove a/path0 a/path1 a/path2 a/path3 a/c/apath0 a/c/apath1 a/c/apath2 d/path* b/apath*' + mv a b && + echo foo >>b/c/apath0 && + git update-index --add --remove a/path0 a/path1 a/path2 a/path3 \ + a/c/apath0 a/c/apath1 a/c/apath2 b/path* b/c/apath* +' -test_expect_success \ - 'git diff-index --detect-dir-renames on a split of subdir and files into different places.' \ - 'git diff-index --detect-dir-renames HEAD >current' -grep -v "^\[DBG\] " <current >current.filtered -cat >expected <<\EOF -:040000 040000 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 R100 a/c/ b/ -:100644 100644 fdbec444a77953b1bcc899d9fabfa202e5e68f08 fdbec444a77953b1bcc899d9fabfa202e5e68f08 R100 a/c/apath0 b/apath0 -:100644 100644 2f1f8d70c0fdad990819dfe37a31deb010805161 2f1f8d70c0fdad990819dfe37a31deb010805161 R100 a/c/apath1 b/apath1 -:100644 100644 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 R100 a/c/apath2 b/apath2 -:100644 100644 fdbec444a77953b1bcc899d9fabfa202e5e68f08 fdbec444a77953b1bcc899d9fabfa202e5e68f08 R100 a/path0 d/path0 -:100644 100644 2f1f8d70c0fdad990819dfe37a31deb010805161 2f1f8d70c0fdad990819dfe37a31deb010805161 R100 a/path1 d/path1 -:100644 100644 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 R100 a/path2 d/path2 -:100644 100644 c6971ab9f08a6cd9c89a0f87d94ae347aad6144a c6971ab9f08a6cd9c89a0f87d94ae347aad6144a R100 a/path3 d/path3 -EOF +test_expect_failure 'diff-index --detect-dir-renames on a move including a subdir.' ' + path0_id2=$(git rev-parse :b/c/apath0) && + cat >expected <<-EOF && + :040000 040000 $zeroes $zeroes R100 a/ b/ + :100644 100644 $path0_id $path0_id2 R096 a/c/apath0 b/c/apath0 + :100644 100644 $path1_id $path1_id R100 a/c/apath1 b/c/apath1 + :100644 100644 $path2_id $path2_id R100 a/c/apath2 b/c/apath2 + :100644 100644 $path0_id $path0_id R100 a/path0 b/path0 + :100644 100644 $path1_id $path1_id R100 a/path1 b/path1 + :100644 100644 $path2_id $path2_id R100 a/path2 b/path2 + :100644 100644 $path3_id $path3_id R100 a/path3 b/path3 + EOF + git diff-index --detect-dir-renames HEAD >current && + grep -v "^\[DBG\] " <current >current.filtered && + compare_diff_patch expected current.filtered +' -test_expect_failure \ - 'validate the output for a split of subdir and files into different places.' \ - 'compare_diff_patch expected current.filtered' +test_expect_success 'setup move without a subdir' ' + git reset -q --hard && + mkdir b && + mv a/path* b/ && + : rename files in the directory but not subdir. && + git update-index --add --remove a/path0 a/path1 a/path2 a/path3 b/path* +' -# now test moving a dir with no files but only subdirs -# (only factorizes lowest-level directories - not a big deal, just not perfect) +test_expect_success 'moving files but not subdirs is not mistaken for dir move' ' + cat >expected <<-EOF && + :100644 100644 $path0_id $path0_id R100 a/path0 b/path0 + :100644 100644 $path1_id $path1_id R100 a/path1 b/path1 + :100644 100644 $path2_id $path2_id R100 a/path2 b/path2 + :100644 100644 $path3_id $path3_id R100 a/path3 b/path3 + EOF + git diff-index --detect-dir-renames HEAD >current && + grep -v "^\[DBG\] " <current >current.filtered && + compare_diff_patch expected current.filtered +' -git reset -q --hard -mkdir a/b -mv a/path* a/b/ -test_expect_success \ - 'setup the directory with only subdirs, no direct child files.' \ - 'git update-index --add --remove a/path0 a/path1 a/path2 a/path3 a/b/path*' -test_expect_success \ - 'commit all this.' \ - 'git commit -m "move all toplevel files down one level"' +test_expect_success 'setup move of files and subdirs to different places' ' + git reset -q --hard && + mv a/c b && + mv a d && + git update-index --add --remove a/path0 a/path1 a/path2 a/path3 \ + a/c/apath0 a/c/apath1 a/c/apath2 d/path* b/apath* +' -test_expect_success \ - 'move this dir.' \ - 'git mv a z' +test_expect_failure 'moving subdirs into one dir and files into another is not mistaken for dir move' ' + cat >expected <<-EOF && + :040000 040000 $zeroes $zeroes R100 a/c/ b/ + :100644 100644 $path0_id $path0_id R100 a/c/apath0 b/apath0 + :100644 100644 $path1_id $path1_id R100 a/c/apath1 b/apath1 + :100644 100644 $path2_id $path2_id R100 a/c/apath2 b/apath2 + :100644 100644 $path0_id $path0_id R100 a/path0 d/path0 + :100644 100644 $path1_id $path1_id R100 a/path1 d/path1 + :100644 100644 $path2_id $path2_id R100 a/path2 d/path2 + :100644 100644 $path3_id $path3_id R100 a/path3 d/path3 + EOF + git diff-index --detect-dir-renames HEAD >current && + grep -v "^\[DBG\] " <current >current.filtered && + compare_diff_patch expected current.filtered +' -test_expect_success \ - 'git diff-index --detect-dir-renames with only subdirs' \ - 'git diff-index --detect-dir-renames HEAD >current' -grep -v "^\[DBG\] " <current >current.filtered -cat >expected <<\EOF -:040000 040000 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 R100 a/ z/ -:040000 040000 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 R100 a/b/ z/b/ -:040000 040000 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 R100 a/c/ z/c/ -:100644 100644 fdbec444a77953b1bcc899d9fabfa202e5e68f08 fdbec444a77953b1bcc899d9fabfa202e5e68f08 R100 a/b/path0 z/b/path0 -:100644 100644 2f1f8d70c0fdad990819dfe37a31deb010805161 2f1f8d70c0fdad990819dfe37a31deb010805161 R100 a/b/path1 z/b/path1 -:100644 100644 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 R100 a/b/path2 z/b/path2 -:100644 100644 c6971ab9f08a6cd9c89a0f87d94ae347aad6144a c6971ab9f08a6cd9c89a0f87d94ae347aad6144a R100 a/b/path3 z/b/path3 -:100644 100644 fdbec444a77953b1bcc899d9fabfa202e5e68f08 fdbec444a77953b1bcc899d9fabfa202e5e68f08 R100 a/c/apath0 z/c/apath0 -:100644 100644 2f1f8d70c0fdad990819dfe37a31deb010805161 2f1f8d70c0fdad990819dfe37a31deb010805161 R100 a/c/apath1 z/c/apath1 -:100644 100644 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 R100 a/c/apath2 z/c/apath2 -EOF +test_expect_success 'setup move of dir with only subdirs' ' + git reset -q --hard && + mkdir a/b && + mv a/path* a/b/ && + git update-index --add --remove a/path0 a/path1 a/path2 a/path3 a/b/path* && + git commit -m "move all toplevel files down one level" && -test_expect_failure \ - 'validate the output for a move with only subdirs.' \ - 'compare_diff_patch expected current.filtered' + git mv a z +' +test_expect_failure 'moving a dir with no files' ' + cat >expected <<-EOF && + :040000 040000 $zeroes $zeroes R100 a/ z/ + :040000 040000 $zeroes $zeroes R100 a/b/ z/b/ + :040000 040000 $zeroes $zeroes R100 a/c/ z/c/ + :100644 100644 $path0_id $path0_id R100 a/b/path0 z/b/path0 + :100644 100644 $path1_id $path1_id R100 a/b/path1 z/b/path1 + :100644 100644 $path2_id $path2_id R100 a/b/path2 z/b/path2 + :100644 100644 $path3_id $path3_id R100 a/b/path3 z/b/path3 + :100644 100644 $path0_id $path0_id R100 a/c/apath0 z/c/apath0 + :100644 100644 $path1_id $path1_id R100 a/c/apath1 z/c/apath1 + :100644 100644 $path2_id $path2_id R100 a/c/apath2 z/c/apath2 + EOF + git diff-index --detect-dir-renames HEAD >current && + grep -v "^\[DBG\] " <current >current.filtered && + compare_diff_patch expected current.filtered +' # now test moving all files from toplevel into subdir (does not hides file moves) (needs consensus on syntax) # Note: this is a special case of move of a dir into one of its own subdirs, which in # turn is a variant of new files/dirs being added into a dir after all its contents # are moved away -git reset -q --hard HEAD~3 +test_expect_success 'setup move from toplevel to subdir' ' + git reset -q --hard HEAD~3 && + mv a/* . && + git update-index --add --remove a/path0 a/path1 a/path2 a/path3 path* && + git commit -m "move all files to toplevel" && -mv a/* . -test_expect_success \ - 'rename the directory with some changes.' \ - 'git update-index --add --remove a/path0 a/path1 a/path2 a/path3 path*' + mkdir z && + mv path* z/ && + git update-index --add --remove path0 path1 path2 path3 z/path* +' -test_expect_success \ - 'commit all this.' \ - 'git commit -m "move all files to toplevel"' - -mkdir z -mv path* z/ -test_expect_success \ - 'rename the directory with some changes.' \ - 'git update-index --add --remove path0 path1 path2 path3 z/path*' - -test_expect_success \ - 'git diff-index --detect-dir-renames everything from toplevel.' \ - 'git diff-index --detect-dir-renames HEAD >current' -grep -v "^\[DBG\] " <current >current.filtered -cat >expected <<\EOF -:040000 040000 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 R100 ./* z/ -:100644 100644 fdbec444a77953b1bcc899d9fabfa202e5e68f08 fdbec444a77953b1bcc899d9fabfa202e5e68f08 R100 path0 z/path0 -:100644 100644 2f1f8d70c0fdad990819dfe37a31deb010805161 2f1f8d70c0fdad990819dfe37a31deb010805161 R100 path1 z/path1 -:100644 100644 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 R100 path2 z/path2 -:100644 100644 c6971ab9f08a6cd9c89a0f87d94ae347aad6144a c6971ab9f08a6cd9c89a0f87d94ae347aad6144a R100 path3 z/path3 -EOF - -test_expect_failure \ - 'validate the output for a move of everything from toplevel.' \ - 'compare_diff_patch expected current.filtered' +test_expect_failure '--detect-dir-renames everything from toplevel.' ' + cat >expected <<-EOF && + :040000 040000 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 R100 ./* z/ + :100644 100644 $path0_id $path0_id R100 path0 z/path0 + :100644 100644 $path1_id $path1_id R100 path1 z/path1 + :100644 100644 $path2_id $path2_id R100 path2 z/path2 + :100644 100644 $path3_id $path3_id R100 path3 z/path3 + EOF + git diff-index --detect-dir-renames HEAD >current && + grep -v "^\[DBG\] " <current >current.filtered && + compare_diff_patch expected current.filtered +' test_done
Re: [PATCH v4 2/4] Add testcases for the --detect-dir-renames diffcore flag.
Quoted 7 lines · Jonathan Nieder
Quoted 2 lines · Yann Dirson
From: Yann Dirson <[email protected]>
Subject: Add testcases for the --detect-dir-renames diffcore flagProbably better for the test to be squashed with the patch that adds that option.
I tend to agree, but we see quite a lot of patches split this way it seems. Junio, any preference here ?
Quoted 24 lines
Quoted 3 lines
t/t4046-diff-rename-factorize.sh new file +0
-0@@ -0,0 +1,326 @@[...]
t/t4046-diff-rename-factorize.sh
Quoted 3 lines
t/t4046-diff-rename-factorize.sh new file +3
-0+test_expect_success \ + 'commit the index.' \ + 'git update-ref HEAD $(echo "original empty commit" | git commit-tree $(git write-tree))'Nit:
Now test assertions tend to be written as
test_expect_success '...' ' command && command && ... 'The tabs to indent and opening ' at the end of the first line means less fuss in lining things up. Making each test assertion include its setup means tests don't pass if something gets messed up in the setup, and using multiline test assertions with && means there is no confusion about what the tests were written to test.
Quoted 3 lines
As you mentioned, this has the unfortunate downside of messing with syntax highlighting. Maybe the common text editors need a new mode for git-style test scripts?
There used to be a mmm-mode thingie for emacs which made such things easier (multiple major modes with custom delimiters) - but that did not handle quoting issues automagically either :)
One solution seen in some test scripts already is to define functions outside of the test_expect* clauses.
Quoted 7 lines
Or maybe it would make sense to implement
test_begin_expecting_success '...' ... test_end--- it would certainly make quoting easier.
Not sure it would be possible without a macro processor. And not sure a macro processor would be welcome by everyone here :)
Quoted 2 lines
It might make sense to compute the tree, commit, etc one at a time instead of this long one-liner.
If moved into a function which would make it readable, yes.
Quoted 7 lines
To avoid quoting troubles and since these are innocuous commands, it could be nice to put this before the first test assertion. Or even better:
mkdir a && printf "Line %s\n" 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 >a/path0 && ...
Well, when seeing for the first time such a construct, i tend to wonder how portable it is amont printf implementations. But hey, I'm always thrilled to discover parts of the standard tools I had missed until now :)
Quoted 16 lines
[...]
t/t4046-diff-rename-factorize.sh
Quoted 11 lines
t/t4046-diff-rename-factorize.sh new file +11
-0+test_expect_success \ + 'git diff-index --detect-dir-renames after directory move.' \ + 'git diff-index --detect-dir-renames HEAD >current' +grep -v "^\[DBG\] " <current >current.filtered +cat >expected <<\EOF +:040000 040000 0000000000000000000000000000000000000000 0000000000000000000000000000000000000000 R100 a/ b/ +:100644 100644 fdbec444a77953b1bcc899d9fabfa202e5e68f08 fdbec444a77953b1bcc899d9fabfa202e5e68f08 R100 a/path0 b/path0 +:100644 100644 2f1f8d70c0fdad990819dfe37a31deb010805161 2f1f8d70c0fdad990819dfe37a31deb010805161 R100 a/path1 b/path1 +:100644 100644 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 dbde7141d737c8aa0003672c1bc21ded48c6c3b9 R100 a/path2 b/path2 +:100644 100644 c6971ab9f08a6cd9c89a0f87d94ae347aad6144a c6971ab9f08a6cd9c89a0f87d94ae347aad6144a R100 a/path3 b/path3 +EOFNit: although
compare_diff_patchensures the result is not dependent on the hash function, these hard-coded hashes are still hard for a human to read. Could they be computed instead?
Well, that would just make the test harder to read imho. Using regexps would help for readability, but that would require another compare_diff variant. BTW, I should be using compare_diff_raw not compare_diff_patch here.
Re: [PATCH v4 2/4] Add testcases for the --detect-dir-renames diffcore flag.
Quoted 6 lines · Yann Dirson
Quoted 2 lines · Jonathan Nieder
It might make sense to compute the tree, commit, etc one at a time instead of this long one-liner.
If moved into a function which would make it readable, yes.
Something like
commit_index () {
test_tick &&
tree=$(git write-tree) &&
commit=$(
printf "%s\n" "$*" |
git commit-tree $tree
) &&
git update-ref HEAD $commit
}? Maybe the following (similar to what you use in later tests) would be even better, for more verbose output when running with -v.
commit_index () {
test_tick &&
git commit -m "$*"
}Quoted 5 lines
Quoted 2 lines
printf "Line %s\n" 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 >a/path0 && ...Well, when seeing for the first time such a construct, i tend to wonder how portable it is amont printf implementations.
Yes, it's portable.
Quoted 6 lines
Quoted 3 lines
Nit: although
compare_diff_patchensures the result is not dependent on the hash function, these hard-coded hashes are still hard for a human to read. Could they be computed instead?Well, that would just make the test harder to read imho. Using regexps would help for readability
If you had said "writability" I would agree with you here. And that's an important concern, too.
What I was suggesting looks like this:
path0_id2=$(git rev-parse :b/path0) && path100_id=$(git rev-parse :b/path100) && cat >expected <<-EOF && :040000 040000 $zeroes $zeroes R100 a/ b/ :100644 000000 $path3_id $zeroes D a/path3 :100644 100644 $path2_id $path2_id R100 a/path2 b/2path :100644 100644 $path0_id $path0_id2 R093 a/path0 b/path0 :100644 100644 $path1_id $path1_id R100 a/path1 b/path1 :000000 100644 $zeroes $path100_id A b/path100 EOF ...
Maybe it would be better to do
cat >expected <<-\EOF && :040000 040000 X X R100 a/ b/ :100644 000000 X X D a/path3 :100644 100644 X X R100 a/path2 b/2path :100644 100644 X X R093 a/path0 b/path0 :100644 100644 X X R100 a/path1 b/path1 :000000 100644 X X A b/path100 EOF
since the hashes are not being checked, anyway.
Re: [PATCH v4 2/4] Add testcases for the --detect-dir-renames diffcore flag.
In reply to Yann DirsonQuoted 12 lines · Yann Dirson
Quoted 7 lines · Jonathan Nieder
Quoted 2 lines · Yann Dirson
From: Yann Dirson<[email protected]>
Subject: Add testcases for the --detect-dir-renames diffcore flagProbably better for the test to be squashed with the patch that adds that option.
I tend to agree, but we see quite a lot of patches split this way it seems. Junio, any preference here ?
Personally, I prefer adding the test first with "test_expect_failure" and then changing it to test_expect_success in the patch that has the real code changes. That way it's really easy to accept a testcase and still keep altering how the test itself is fixed, which is usually a far more complex and nitpickley process than showing that the bug is there in the first place.
I know Junio has voiced the same preference for patchseries before and especially when the solution isn't obvious. Note that I obviously can't speak for him in any particular case.
Patches adding features should ofcourse have the code first and the tests after, since the tests in that case are only about preventing future breakage rather than showing that there is one now.
Signature · 7 lines
Andreas Ericsson [email protected] OP5 AB www.op5.se Tel: +46 8-230225 Fax: +46 8-230231 Considering the successes of the wars on alcohol, poverty, drugs and terror, I think we should give some serious thought to declaring war on peace.
Re: [PATCH v4 1/4] Introduce wholesame directory move detection in diffcore.
In reply to Yann Dirson(+cc: Duy and Baz, who commented before)
Quoted 9 lines · Yann Dirson
This feature tries to group together files moving from and to identical directories - the most common case being directory renames.
It is activated by the new --detect-dir-renames diffcore flag.
This is only the first step, adding the basic functionnality and adding support to raw diff output (and it breaks unified-diff output which does not know how to handle that stuff yet).
[...]
Quoted 5 lines
For now both the result
of "mv a b" and "mv a/* b/" are displayed as "Rnnn a/ b/", which is probably not what we want. "Rnnn a/* b/" could be a good choice for the latter if we want them to be distinguished, and even if we want them to look the same.
Example?
Quoted 2 lines
Other future developements to be made on top of this include:
* extension of unified-diff format to express this
[...]
Quoted 1 line
---
Oh, so this is for diff --raw only.
For the confused: the discussion from v3 perhaps explains why. But:
1. Just like --word-diff, this could be used as a user-facing feature for diffs that cannot be applied
2. Even if no one agrees on the proper diff format, it would be good plumbing to have.
v3: http://thread.gmane.org/gmane.comp.version-control.git/99780/focus=99782
v2: http://thread.gmane.org/gmane.comp.version-control.git/99529/focus=99528
v1: http://thread.gmane.org/gmane.comp.version-control.git/94612/focus=96807
I kind of like the cover letter to v1:
The idea is to add a new pass to diffcore-rename, to group file renames looking like a full directory move, and then to hide those file renames which do not carry any additionnal information.
Here is a sample run:
$ ./git-diff-tree --abbrev=6 ee491 --factorize-renames -r [DBG] possible rename from arm/ to bar/ [DBG] possible rename from ppc/ to moved/ [DBG] discarding dir rename of arm/, mixing moved/ and bar/ [DBG] ppc/* -> moved/* makes ppc/sha1ppc.S -> moved/sha1ppc.S uninteresting [DBG] ppc/* -> moved/* makes ppc/sha1.c -> moved/sha1.c uninteresting ee491a42190ec6e716f46a55fa0a7f4e307f1629 :040000 040000 000000... 000000... R100 ppc/ moved/ :100644 100644 9e3ae0... 9e3ae0... R100 arm/sha1.c bar/sha1.c :100644 100644 395264... 395264... R100 arm/sha1.h bar/sha1.h :100644 100644 c3c51a... c065ee... R099 ppc/sha1.h moved/sha1.h :100644 100644 8c1cb9... 8c1cb9... R100 arm/sha1_arm.S moved/sha1_arm.S
Presumably this patch takes care of the first step (grouping potential full-directory moves) and not the second (hiding the redundant file renames).
Naïve review:
diff-lib.c int run_diff_files(struct rev_info *revs, unsigned int option) · line 213
Quoted 10 lines
diff-lib.c +2
-1@@ -208,7 +208,8 @@ int run_diff_files(struct rev_info *revs, unsigned int option) ce_option, &dirty_submodule);if (!changed && !dirty_submodule) {ce_mark_uptodate (ce); -if (!DIFF_OPT_TST (&revs->diffopt,FIND_COPIES_HARDER )) +if (!DIFF_OPT_TST (&revs->diffopt,FIND_COPIES_HARDER ) && + !DIFF_OPT_TST (&revs->diffopt,DETECT_DIR_RENAMES ))continue ;
Hm, why?
diff-lib.c static int show_modified(struct rev_info *revs,
Quoted 1 line
diff-lib.c +0
-0@@ -338,7 +339,8 @@ static int show_modified(struct rev_info *revs,
[...]
Likewise.
[...]
diffcore-rename.c line 15
Quoted 8 lines
diffcore-rename.c +1
-0@@ -11,6 +11,7 @@static struct diff_rename_dst {struct diff_filespec *two;struct diff_filepair *pair; +int i_am_not_single:1 ;// does not look for a match, only here to be looked at } *rename_dst;
What does single mean?
Quoted 1 line
diffcore-rename.c +0
-0@@ -49,9 +50,36 @@ static struct diff_rename_dst *locate_rename_dst(struct diff_filespec *two,
[...]
Quoted 16 lines
diffcore-rename.c +15
-0+static struct diff_rename_dst *locate_rename_dst_dir (struct diff_filespec *dir) +{ +/* code mostly duplicated from locate_rename_dst - not sure we +* could merge them efficiently,though +*/ +int first, last; +int prefixlength =strlen (dir->path); + + first =0 ; + last = rename_dst_nr; +while (last > first) { +int next = (last + first) >>1 ; +struct diff_rename_dst *dst = &(rename_dst[next]); +int cmp =strncmp (dir->path, dst->two->path, prefixlength); +if (!cmp)
diffcore-rename.c static struct diff_rename_dst *locate_rename_dst(struct diff_filespec *two, · line 81
Quoted 10 lines
diffcore-rename.c +10
-0+return dst; +if (cmp <0 ) { + last = next; +continue ; + } + first = next+1 ; + } +/* not found */ +return NULL ; +}
Binary search --- this is just a way to index into the sorted list of rename_dsts, right?
At first I thought it was searching for a good rename destination. A comment (or overview in the log message) could help clarify.
diffcore-rename.c static int find_exact_renames(void) · line 421
Quoted 10 lines
diffcore-rename.c +4
-1@@ -386,8 +414,11 @@ static int find_exact_renames(void)for (i =0 ; i < rename_src_nr; i++)insert_file_table (&file_table,-1 , i, rename_src[i].one); -for (i =0 ; i < rename_dst_nr; i++) +for (i =0 ; i < rename_dst_nr; i++) { +if (rename_dst[i].i_am_not_single) +continue ;insert_file_table (&file_table,1 , i, rename_dst[i].two); + }
What is this code path for? (Sorry for the tedious questions. My thinking is, if I cannot answer them without doing some legwork, how could the person about to break your code who is only glancing for a moment before plunging forward on an exciting new feature?)
diffcore-rename.c static void record_if_better(struct diff_score m[], struct diff_score *o) · line 453
Quoted 10 lines
diffcore-rename.c +6
-0@@ -414,6 +445,180 @@ static void record_if_better(struct diff_score m[], struct diff_score *o) m[worst] = *o; } +struct diff_dir_rename { +struct diff_filespec *one; +struct diff_filespec *two; +int discarded; +struct diff_dir_rename * next; +};
Okay.
Quoted 6 lines
diffcore-rename.c +6
-0+// FIXME: prevent possible overflow +/* +* Copy dirname of src into dst, suitable to append a filename without +* an additional "/". +* Only handles relative paths since there is no absolute path in a git repo. +* Writes "" when there is no "/" in src.
diffcore-rename.c static void record_if_better(struct diff_score m[], struct diff_score *o) · line 470
Quoted 10 lines
diffcore-rename.c +10
-0+* May overwrite more chars than really needed, if src ends with a "/". +*/ +static const char *copy_dirname (char * dst,const char * src) +{ +char * lastslash =strrchr (src,'/' ); +if (!lastslash) { + *dst ='\0' ; +return dst; + } +strncpy (dst, src, lastslash - src +1 );
memcpy?
diffcore-rename.c static void record_if_better(struct diff_score m[], struct diff_score *o) · line 479
Quoted 9 lines
diffcore-rename.c +9
-0+ dst[lastslash - src +1 ] ='\0' ; + +// if src ends with a "/" strip the last component +if (lastslash[1 ] =='\0' ) { + lastslash =strrchr (dst,'/' ); +if (!lastslash) +return strcpy (dst,"." ); + lastslash[1 ] ='\0' ; + }
It might be easier to read like this:
/* Write name of the parent directory of src to dest. */ static char *remove_last_component(char *dst, const char *src)
{
size_t len = strlen(src);
const char *slash;
if (len > 0 && src[len - 1] == '/')
/* Trailing slash. Ignore it. */
len--;
slash = memrchr(src, '/', len);
if (!slash) {
*dst = '\0';
return dst;
}
*mempcpy(dst, src, slash - src) = '\0';
return dst;
}Requires the glibc-specific function memrchr(), but that is easy to implement. Compare strchrnul() [in git-compat-util.h].
[...]
diffcore-rename.c static void record_if_better(struct diff_score m[], struct diff_score *o) · line 500
Quoted 2 lines
diffcore-rename.c +2
-0+static struct diff_dir_rename * factorization_candidates =NULL ; +static void diffcore_factorize_renames (void )
Maybe this could be refactored into multiple functions?
Quoted 5 lines
diffcore-rename.c +1
-0@@ -451,13 +656,22 @@ void diffcore_rename(struct diff_options *options) p->one->rename_used++;register_rename_src (p->one, p->score); } +else {
[...]
diffcore-rename.c void diffcore_rename(struct diff_options *options) · line 673
Quoted 6 lines
diffcore-rename.c +6
-0+if (DIFF_OPT_TST (options,DETECT_DIR_RENAMES )) { +/* similarly, rename factorization needs to +* see all files from second tree, but we don't +* want them to be matched against single sources. +*/ +locate_rename_dst (p->two,1 )->i_am_not_single =1 ;
Huh?
diffcore-rename.c void diffcore_rename(struct diff_options *options) · line 727
Quoted 6 lines
diffcore-rename.c +2
-0@@ -509,6 +723,8 @@ void diffcore_rename(struct diff_options *options)if (rename_dst[i].pair)continue ;/* dealt with exact match already. */ +if (rename_dst[i].i_am_not_single) +continue ;/* not looking for a match. */
Oh, not single means "not seeking a new relationship".
diffcore-rename.c void diffcore_rename(struct diff_options *options) · line 791
Quoted 8 lines
diffcore-rename.c +4
-0@@ -569,7 +785,28 @@ void diffcore_rename(struct diff_options *options)/* At this point, we have found some renames and copies and they * are recorded in rename_dst. The original list is still in *q. */ + +/* Now possibly factorize those renames and copies. */ +if (DIFF_OPT_TST (options,DETECT_DIR_RENAMES )) +diffcore_factorize_renames ();
Huh? Factorize?
Quoted 6 lines
diffcore-rename.c +5
-0DIFF_QUEUE_CLEAR (&outq); + +// Now turn non-discarded factorization_candidates into real renames +struct diff_dir_rename * candidate; +for (candidate=factorization_candidates; candidate; candidate = candidate->next) { +struct diff_filepair * pair;
diffcore-rename.c void diffcore_rename(struct diff_options *options) · line 808
Quoted 10 lines
diffcore-rename.c +10
-0+if (candidate->discarded)continue ; +// visualize toplevel dir if needed - FIXME: wrong place for this ? +if (!*candidate->one->path) + candidate->one->path ="./" ; +if (!*candidate->two->path) + candidate->two->path ="./" ; + pair =diff_queue (&outq, candidate->one, candidate->two); + pair->score =MAX_SCORE ; + pair->renamed_pair =1 ; + }
Looks like I'll need to read diffcore_factorize_renames() after all. :(
Sorry, not much to say yet. Hopefully some of that can be useful nonetheless.
Jonathan
Re: [PATCH v4 1/4] Introduce wholesame directory move detection in diffcore.
Quoted 6 lines · Jonathan Nieder
Quoted 4 lines
of "mv a b" and "mv a/* b/" are displayed as "Rnnn a/ b/", which is probably not what we want. "Rnnn a/* b/" could be a good choice for the latter if we want them to be distinguished, and even if we want them to look the same.
Example?
It's just that every time we detect a bulk rename (of all files of a directory) it is currently shown as a directory rename, even if they all get moved into an existing dir. Technically, a "directory rename" (which I'd label "mv a b") is a special case of "bulk rename" (for which "mv a/* b/" is more correct), where the target directory is empty. Currently the distinction between the two is not done, but everything is incorrectly advertized as a rename. I shall fix that.
Quoted 6 lines
Quoted 2 lines
Other future developements to be made on top of this include:
* extension of unified-diff format to express this[...]
Quoted 1 line
---
Oh, so this is for diff --raw only.
Right, but that TODO list has not be revised in this iteration. --detect-dir-renames does not need much to gain unified-diff, it is mostly --hide-dir-rename-details which needs thougth and work - but as it is now targetted at humans only, we can start with whatever seems adequate and rewrite/refine as see fit.
Quoted 1 line
For the confused: the discussion from v3 perhaps explains why.
Thanks for this work of putting everything into context - I should have done that myself.
Quoted 13 lines
$ ./git-diff-tree --abbrev=6 ee491 --factorize-renames -r [DBG] possible rename from arm/ to bar/ [DBG] possible rename from ppc/ to moved/ [DBG] discarding dir rename of arm/, mixing moved/ and bar/ [DBG] ppc/* -> moved/* makes ppc/sha1ppc.S -> moved/sha1ppc.S uninteresting [DBG] ppc/* -> moved/* makes ppc/sha1.c -> moved/sha1.c uninteresting ee491a42190ec6e716f46a55fa0a7f4e307f1629 :040000 040000 000000... 000000... R100 ppc/ moved/ :100644 100644 9e3ae0... 9e3ae0... R100 arm/sha1.c bar/sha1.c :100644 100644 395264... 395264... R100 arm/sha1.h bar/sha1.h :100644 100644 c3c51a... c065ee... R099 ppc/sha1.h moved/sha1.h :100644 100644 8c1cb9... 8c1cb9... R100 arm/sha1_arm.S moved/sha1_arm.S
Quoted 2 lines
Presumably this patch takes care of the first step (grouping potential full-directory moves) and not the second (hiding the redundant file renames).
No, it did hide the two moves labeled as "uninteresting" in the debug traces above. The ones left are one which is part of the dif rename but was also modified, and the other ones (from arm/) are part of a directory split (see 3rd debug line).
Quoted 14 lines
Naïve review:
diff-lib.c int run_diff_files(struct rev_info *revs, unsigned int option) · line 213
Quoted 10 lines
diff-lib.c +2
-1@@ -208,7 +208,8 @@ int run_diff_files(struct rev_info *revs, unsigned int option) ce_option, &dirty_submodule);if (!changed && !dirty_submodule) {ce_mark_uptodate (ce); -if (!DIFF_OPT_TST (&revs->diffopt,FIND_COPIES_HARDER )) +if (!DIFF_OPT_TST (&revs->diffopt,FIND_COPIES_HARDER ) && + !DIFF_OPT_TST (&revs->diffopt,DETECT_DIR_RENAMES ))continue ;Hm, why?
Quoted 4 lines
diff-lib.c static int show_modified(struct rev_info *revs,
Quoted 1 line
diff-lib.c +0
-0@@ -338,7 +339,8 @@ static int show_modified(struct rev_info *revs,[...]
Likewise.
The code below that (calls to diff_change) was previously only used for FIND_COPIES_HARDER, but we also need to go there for DETECT_DIR_RENAMES. Removing those eg. make test "validate the output for a move without a subdir." fail. I admit I have not re-analyzed this part recently, those tests were already parts of the first iterations.
Quoted 11 lines
[...]
diffcore-rename.c line 15
Quoted 8 lines
diffcore-rename.c +1
-0@@ -11,6 +11,7 @@static struct diff_rename_dst {struct diff_filespec *two;struct diff_filepair *pair; +int i_am_not_single:1 ;// does not look for a match, only here to be looked at } *rename_dst;What does single mean?
"looking for a match" :)
Quoted 19 lines
Quoted 5 lines
diffcore-rename.c +1
-0@@ -451,13 +656,22 @@ void diffcore_rename(struct diff_options *options) p->one->rename_used++;register_rename_src (p->one, p->score); } +else {[...]
diffcore-rename.c void diffcore_rename(struct diff_options *options) · line 673
Quoted 6 lines
diffcore-rename.c +6
-0+if (DIFF_OPT_TST (options,DETECT_DIR_RENAMES )) { +/* similarly, rename factorization needs to +* see all files from second tree, but we don't +* want them to be matched against single sources. +*/ +locate_rename_dst (p->two,1 )->i_am_not_single =1 ;Huh?
Quoted 4 lines
diffcore-rename.c +0
-0@@ -386,8 +414,11 @@ static int find_exact_renames(void)for (i =0 ; i < rename_src_nr; i++)insert_file_table (&file_table,-1 , i, rename_src[i].one);
Quoted 11 lines
diffcore-rename.c static int find_exact_renames(void) · line 421
Quoted 6 lines
diffcore-rename.c +4
-1-for (i =0 ; i < rename_dst_nr; i++) +for (i =0 ; i < rename_dst_nr; i++) { +if (rename_dst[i].i_am_not_single) +continue ;insert_file_table (&file_table,1 , i, rename_dst[i].two); + }What is this code path for? (Sorry for the tedious questions. My thinking is, if I cannot answer them without doing some legwork, how could the person about to break your code who is only glancing for a moment before plunging forward on an exciting new feature?)
Those are definitely related to the above. This mimics the FIND_COPIES_HARDER behaviour (which is adding untouched files from *src* tree to the rename_dst list so the -C algorithm works): here we need to see all files from *dst* tree so we can tell if there was any file left in a dir after a move. This has a cost: we must ensure in other places that codepaths not intended for those extra files are ok.
Hm, thinking quickly about it, maybe we could use a separate list to store them, and only run through them when explicitely needed, that would cost less and avoid the polutions you pointed at. I shall have a look at that.
Quoted 23 lines
Quoted 1 line
diffcore-rename.c +0
-0@@ -49,9 +50,36 @@ static struct diff_rename_dst *locate_rename_dst(struct diff_filespec *two,[...]
Quoted 21 lines
diffcore-rename.c +20
-0+static struct diff_rename_dst *locate_rename_dst_dir (struct diff_filespec *dir) +{ +/* code mostly duplicated from locate_rename_dst - not sure we +* could merge them efficiently,though +*/ +int first, last; +int prefixlength =strlen (dir->path); + + first =0 ; + last = rename_dst_nr; +while (last > first) { +int next = (last + first) >>1 ; +struct diff_rename_dst *dst = &(rename_dst[next]); +int cmp =strncmp (dir->path, dst->two->path, prefixlength); +if (!cmp) +return dst; +if (cmp <0 ) { + last = next; +continue ; + }
Quoted 8 lines
diffcore-rename.c static struct diff_rename_dst *locate_rename_dst(struct diff_filespec *two, · line 81
Quoted 5 lines
diffcore-rename.c +5
-0+ first = next+1 ; + } +/* not found */ +return NULL ; +}Binary search --- this is just a way to index into the sorted list of
rename_dsts, right?
right.
Quoted 2 lines
At first I thought it was searching for a good rename destination. A comment (or overview in the log message) could help clarify.
Ok. More comments are surely good in this hairy piece of code :)
Quoted 10 lines
Quoted 10 lines
diffcore-rename.c +10
-0+// FIXME: prevent possible overflow +/* +* Copy dirname of src into dst, suitable to append a filename without +* an additional "/". +* Only handles relative paths since there is no absolute path in a git repo. +* Writes "" when there is no "/" in src. +* May overwrite more chars than really needed, if src ends with a "/". +*/ +static const char *copy_dirname (char * dst,const char * src) +{
Quoted 8 lines
diffcore-rename.c static void record_if_better(struct diff_score m[], struct diff_score *o) · line 470
Quoted 6 lines
diffcore-rename.c +6
-0+char * lastslash =strrchr (src,'/' ); +if (!lastslash) { + *dst ='\0' ; +return dst; + } +strncpy (dst, src, lastslash - src +1 );memcpy?
right
Quoted 32 lines
diffcore-rename.c static void record_if_better(struct diff_score m[], struct diff_score *o) · line 479
Quoted 9 lines
diffcore-rename.c +9
-0+ dst[lastslash - src +1 ] ='\0' ; + +// if src ends with a "/" strip the last component +if (lastslash[1 ] =='\0' ) { + lastslash =strrchr (dst,'/' ); +if (!lastslash) +return strcpy (dst,"." ); + lastslash[1 ] ='\0' ; + }It might be easier to read like this:
/* Write name of the parent directory of src to dest. */ static char *remove_last_component(char *dst, const char *src)
{ size_t len = strlen(src); const char *slash; if (len > 0 && src[len - 1] == '/') /* Trailing slash. Ignore it. */ len--; slash = memrchr(src, '/', len); if (!slash) { *dst = '\0'; return dst; } *mempcpy(dst, src, slash - src) = '\0'; return dst; }
Quoted 2 lines
Requires the glibc-specific function
memrchr(), but that is easy to implement. Comparestrchrnul()[ingit-compat-util.h].
Right, that's much more elegant - and should also be more efficient in the "ends with /" case.
Quoted 15 lines
[...]
diffcore-rename.c static void record_if_better(struct diff_score m[], struct diff_score *o) · line 500
Quoted 2 lines
diffcore-rename.c +2
-0+static struct diff_dir_rename * factorization_candidates =NULL ; +static void diffcore_factorize_renames (void )Maybe this could be refactored into multiple functions?
diffcore-rename.c void diffcore_rename(struct diff_options *options) · line 727
Quoted 6 lines
diffcore-rename.c +2
-0@@ -509,6 +723,8 @@ void diffcore_rename(struct diff_options *options)if (rename_dst[i].pair)continue ;/* dealt with exact match already. */ +if (rename_dst[i].i_am_not_single) +continue ;/* not looking for a match. */ Oh, not single means "not seeking a new relationship".
Quoted 10 lines
diffcore-rename.c void diffcore_rename(struct diff_options *options) · line 791
Quoted 8 lines
diffcore-rename.c +4
-0@@ -569,7 +785,28 @@ void diffcore_rename(struct diff_options *options)/* At this point, we have found some renames and copies and they * are recorded in rename_dst. The original list is still in *q. */ + +/* Now possibly factorize those renames and copies. */ +if (DIFF_OPT_TST (options,DETECT_DIR_RENAMES )) +diffcore_factorize_renames ();Huh? Factorize?
"--factorize-renames" was how the single-option was called in the previous series. It has still not turned completely false, but sure using "bulk rename" would be easier to read.
Quoted 1 line
Looks like I'll need to read
diffcore_factorize_renames()after all. :(
Hopefully that part is better commented :)
Quoted 2 lines
Sorry, not much to say yet. Hopefully some of that can be useful nonetheless.
Thanks for this review, that work is always useful.
Re: [PATCH v4 1/4] Introduce wholesame directory move detection in diffcore.
In reply to Yann DirsonQuoted 2 lines · Yann Dirson
This feature tries to group together files moving from and to identical directories - the most common case being directory renames.
You should change the C++/C99 comments to C89 comments since we target the latter.
Re: [PATCH v4 1/4] Introduce wholesame directory move detection in diffcore.
Quoted 6 lines · ?var Arnfj?r? Bjarmason
Quoted 2 lines · Yann Dirson
This feature tries to group together files moving from and to identical directories - the most common case being directory renames.
You should change the C++/C99 comments to C89 comments since we target the latter.
Right. Most of them are here as reminders to either fix something or be sure fixing it was considered before submission for inclusion - hopefully using unacceptable syntax helps here :)
Re: [PATCH v4 1/4] Introduce wholesame directory move detection in diffcore.
In reply to Yann DirsonQuoted 2 lines · Yann Dirson
This feature tries to group together files moving from and to identical directories - the most common case being directory renames.
Intro for the uninitiated (e.g., me):
1. Entry point for the existing rename detection.
The entry point to diffcore's rename translator is diffcore_rename(), which is used, naturally enough, by diffcore_std() if and only if the detect_rename option is enabled.
2. Semantics of the existing rename detection.
When diffcore_std() is called, the caller has already queued up a nice list of files to diff (see Documentation/technical/api-diff.txt). The rename detection pass replaces the queue with a new list of filepairs that reflects the detected renames, guided by the detect_rename, rename_score, and rename_limit options.
3. Implementation of the existing rename detection.
I. find rename candidates (all files, if detecting copies; added and
deleted files, if not)
II. find exact renames and copies
III. for each target file not accounted for yet, find the best 4
candidate source files
IV. record rename winners, in order from most to least similar pair V. record copy winners, in order from most to least similar pair VI. queue up results
4. Data maintained by the existing rename detection.
rename_dst, a list of rename candidates from step I, sorted by name For each rename dest candidate: a filespec and a filepair.
filepair::one source filespec
filepair::two destination filespec
filepair::score similarity score
rename_src, a list of rename candidates from step I, sorted by name For each rename source candidate: a filespec and a score
filespec::rename_used number of filespecs it's paired against
(to distinguish renames from copies)
diff-lib.c int run_diff_files(struct rev_info *revs, unsigned int option) · line 215
Quoted 12 lines
diff-lib.c +2
-1@@ -208,7 +208,8 @@ int run_diff_files(struct rev_info *revs, unsigned int option) ce_option, &dirty_submodule);if (!changed && !dirty_submodule) {ce_mark_uptodate (ce); -if (!DIFF_OPT_TST (&revs->diffopt,FIND_COPIES_HARDER )) +if (!DIFF_OPT_TST (&revs->diffopt,FIND_COPIES_HARDER ) && + !DIFF_OPT_TST (&revs->diffopt,DETECT_DIR_RENAMES ))continue ; } oldmode = ce->ce_mode;
Sane. diff-files and friends need to pass unchanged files to the diffcore lib when directory renames are being considered, just like they would with -C -C.
Otherwise, we cannot distinguish between
a/1 stays a/2 stays a/3 stays a/4 stays ... a/1000 stays a/1001 -> b/1001 a/1002 -> b/1002
which is clearly not a directory rename, and
a/1001 -> b/1001 a/1002 -> b/1002
which sure looks like one.
Reference: v0.99~515 (Diff overhaul, adding the other half of copy detection, 2005-05-21).
diff-lib.c static int show_modified(struct rev_info *revs, · line 344
Quoted 8 lines
diff-lib.c +2
-1@@ -338,7 +339,8 @@ static int show_modified(struct rev_info *revs, oldmode = old->ce_mode;if (mode == oldmode && !hashcmp (sha1, old->sha1) && !dirty_submodule && - !DIFF_OPT_TST (&revs->diffopt,FIND_COPIES_HARDER )) + !DIFF_OPT_TST (&revs->diffopt,FIND_COPIES_HARDER ) && + !DIFF_OPT_TST (&revs->diffopt,DETECT_DIR_RENAMES ))return 0 ;
The diff-index part.
Would we also need something like v1.6.4-rc0~45^2 (Avoid "diff-index --cached" optimization under --find-copies-harder, 2009-05-22) in order to grok unchanged subtrees, or is that taken care of some other way?
[...]
diffcore-rename.c line 15
Quoted 8 lines
diffcore-rename.c +1
-0@@ -11,6 +11,7 @@static struct diff_rename_dst {struct diff_filespec *two;struct diff_filepair *pair; +int i_am_not_single:1 ;// does not look for a match, only here to be looked at } *rename_dst;
If we're looking for directory renames but not finding-copies-harder, then unmodified files, while interesting and deserving of dst entries, should not be considered candidates for tracing individual files.
Yes? If so, maybe the less colorful
unsigned rename_target_candidate:1;
would make that clearer (and more importantly, the new kind of dst entry should be mentioned in the commit log message).
diffcore-rename.c static struct diff_rename_dst *locate_rename_dst(struct diff_filespec *two, · line 55
Quoted 7 lines
diffcore-rename.c +1
-0@@ -49,9 +50,36 @@ static struct diff_rename_dst *locate_rename_dst(struct diff_filespec *two, rename_dst[first].two =alloc_filespec (two->path);fill_filespec (rename_dst[first].two, two->sha1, two->mode); rename_dst[first].pair =NULL ; + rename_dst[first].i_am_not_single =0 ;return &(rename_dst[first]); }
All new files are rename target candidates.
Quoted 5 lines
diffcore-rename.c +4
-0+static struct diff_rename_dst *locate_rename_dst_dir (struct diff_filespec *dir) +{ +/* code mostly duplicated from locate_rename_dst - not sure we +* could merge them efficiently,though
diffcore-rename.c static struct diff_rename_dst *locate_rename_dst(struct diff_filespec *two, · line 70
Quoted 10 lines
diffcore-rename.c +10
-0+*/ +int first, last; +int prefixlength =strlen (dir->path); + + first =0 ; + last = rename_dst_nr; +while (last > first) { +int next = (last + first) >>1 ; +struct diff_rename_dst *dst = &(rename_dst[next]); +int cmp =strncmp (dir->path, dst->two->path, prefixlength);
prefixcmp?
diffcore-rename.c static struct diff_rename_dst *locate_rename_dst(struct diff_filespec *two, · line 81
Quoted 11 lines
diffcore-rename.c +11
-0+if (!cmp) +return dst; +if (cmp <0 ) { + last = next; +continue ; + } + first = next+1 ; + } +/* not found */ +return NULL ; +}
Hmm, so this is like locate_rename_dst(..., 0), except it matches prefixes. Result is a rename_dst entry within that directory (why not spend the time to find the first?).
Quoted 5 lines
diffcore-rename.c +1
-0+/* Table of rename/copy src files */ static struct diff_rename_src { struct diff_filespec *one;
diffcore-rename.c static int find_exact_renames(void) · line 421
Quoted 9 lines
diffcore-rename.c +4
-1@@ -386,8 +414,11 @@ static int find_exact_renames(void)for (i =0 ; i < rename_src_nr; i++)insert_file_table (&file_table,-1 , i, rename_src[i].one); -for (i =0 ; i < rename_dst_nr; i++) +for (i =0 ; i < rename_dst_nr; i++) { +if (rename_dst[i].i_am_not_single) +continue ;insert_file_table (&file_table,1 , i, rename_dst[i].two); + }
Prevents the new feature from turning on --find-copies-harder in step II. Good.
diffcore-rename.c static void record_if_better(struct diff_score m[], struct diff_score *o) · line 453
Quoted 13 lines
diffcore-rename.c +6
-0/* Find the renames */ i =for_each_hash (&file_table, find_same_files);@@ -414,6 +445,180 @@ static void record_if_better(struct diff_score m[], struct diff_score *o) m[worst] = *o; } +struct diff_dir_rename { +struct diff_filespec *one; +struct diff_filespec *two; +int discarded; +struct diff_dir_rename * next; +};
A linked list of renamed directories. What functions maintain it?
[...]
diffcore-rename.c static void record_if_better(struct diff_score m[], struct diff_score *o) · line 463
Quoted 1 line
diffcore-rename.c +1
-0+static const char *copy_dirname (char * dst,const char * src)
Discussed already.
[...]
diffcore-rename.c static void record_if_better(struct diff_score m[], struct diff_score *o) · line 505
Quoted 7 lines
diffcore-rename.c +7
-0+static struct diff_dir_rename * factorization_candidates =NULL ; +static void diffcore_factorize_renames (void ) +{ +char one_parent_path[PATH_MAX ], two_parent_path[PATH_MAX ]; +int i; + +for (i =0 ; i < rename_dst_nr; i++) {
For each rename target candidate: ...
Would it make sense to give the body of this loop its own function?
diffcore-rename.c static void record_if_better(struct diff_score m[], struct diff_score *o) · line 510
Quoted 5 lines
diffcore-rename.c +5
-0+struct diff_dir_rename * seen; + +// FIXME: what are those ? +if (!rename_dst[i].pair) +continue ;
Genuine new files are not interesting to us (since they do not provide evidence about directory renames, postive or negative).
diffcore-rename.c static void record_if_better(struct diff_score m[], struct diff_score *o) · line 513
Quoted 3 lines
diffcore-rename.c +3
-0+// dummy renames used by copy detection +if (!strcmp (rename_dst[i].pair->one->path, rename_dst[i].pair->two->path)) +continue ;
Whether a file was copied is probably interesting to us, but the corresponding filepair isn't.
diffcore-rename.c static void record_if_better(struct diff_score m[], struct diff_score *o) · line 520
Quoted 7 lines
diffcore-rename.c +7
-0+ +copy_dirname (one_parent_path, rename_dst[i].pair->one->path); +copy_dirname (two_parent_path, rename_dst[i].pair->two->path); + +// simple rename with no directory change +if (!strcmp (one_parent_path, two_parent_path)) +continue ;
File renames within a flat directory are not interesting to us.
diffcore-rename.c static void record_if_better(struct diff_score m[], struct diff_score *o) · line 523
Quoted 3 lines
diffcore-rename.c +3
-0+ +struct diff_filespec * one_parent =alloc_filespec (one_parent_path); +fill_filespec (one_parent, null_sha1/*FIXME*/ ,S_IFDIR );
Initialize a filespec, for use in filepairs in case the containing directories are a source/target pair. (?)
Quoted 4 lines
diffcore-rename.c +4
-0+ +// After this commit, are there any files still under one_parent_path ? +struct diff_rename_dst * one_leftover =locate_rename_dst_dir (one_parent); +if (one_leftover) {// FIXME: should only be run if !seen
[...]
diffcore-rename.c
Quoted 1 line
diffcore-rename.c +1
-0+ }
A candidate for a separate function, no? The idea, if I understand
it: too many unchanged files can disqualify a rename source.
diffcore-rename.c static void record_if_better(struct diff_score m[], struct diff_score *o) · line 591
Quoted 4 lines
diffcore-rename.c +4
-0+ +// already considered ? +for (seen=factorization_candidates; seen; seen = seen->next) +if (!strcmp (seen->one->path, one_parent_path))break ;
Lookup. A candidate for a separate function, I think.
diffcore-rename.c static void record_if_better(struct diff_score m[], struct diff_score *o) · line 604
Quoted 13 lines
diffcore-rename.c +13
-0+if (!seen) { +// record potential dir rename + seen =xmalloc (sizeof (*seen)); + seen->one = one_parent; + seen->two =alloc_filespec (two_parent_path); +fill_filespec (seen->two, null_sha1/*FIXME*/ ,S_IFDIR ); + seen->discarded =0 ; + seen->next = factorization_candidates; + factorization_candidates = seen; +fprintf (stderr,"[DBG] %s -> %s suggests possible rename from %s to %s\n" , + rename_dst[i].pair->one->path, + rename_dst[i].pair->two->path, + one_parent_path, two_parent_path);
Insertion. Likewise.
diffcore-rename.c static void record_if_better(struct diff_score m[], struct diff_score *o) · line 608
Quoted 4 lines
diffcore-rename.c +4
-0+continue ; + } +if (seen->discarded) +continue ;
One disqualification disqualifies a directory completely.
diffcore-rename.c static void record_if_better(struct diff_score m[], struct diff_score *o) · line 614
Quoted 6 lines
diffcore-rename.c +6
-0+// check that seen entry matches this rename +if (strcmp (two_parent_path, seen->two->path)) { +fprintf (stderr,"[DBG] discarding dir %s from renames (split into %s and %s)\n" , + one_parent_path, two_parent_path, seen->two->path); + seen->discarded =1 ; + }
If a directory split in two, it gets disqualified. Ok.
Quoted 11 lines
diffcore-rename.c +7
-0+ +/* all checks ok, we keep that entry */ + } + +return ; +} +void diffcore_rename (struct diff_options *options) {int detect_rename = options->detect_rename;@@ -451,13 +656,22 @@ void diffcore_rename(struct diff_options *options)
[...]
Quoted 4 lines
diffcore-rename.c +0
-0if (detect_rename ==DIFF_DETECT_COPY ) {/* * Increment the "rename_used" score by * one, to indicate ourselves as a user.
diffcore-rename.c void diffcore_rename(struct diff_options *options) · line 673
Quoted 10 lines
diffcore-rename.c +6
-0*/ p->one->rename_used++;register_rename_src (p->one, p->score); } +if (DIFF_OPT_TST (options,DETECT_DIR_RENAMES )) { +/* similarly, rename factorization needs to +* see all files from second tree, but we don't +* want them to be matched against single sources. +*/ +locate_rename_dst (p->two,1 )->i_am_not_single =1 ;
... except when --find-copies-harder is being used, right?
So I suspect there needs to be a
DIFF_OPT_TST(options, FIND_COPIES_HARDER)
somewhere around here.
diffcore-rename.c void diffcore_rename(struct diff_options *options) · line 730
Quoted 9 lines
diffcore-rename.c +2
-0@@ -509,6 +723,8 @@ void diffcore_rename(struct diff_options *options)if (rename_dst[i].pair)continue ;/* dealt with exact match already. */ +if (rename_dst[i].i_am_not_single) +continue ;/* not looking for a match. */ m = &mx[dst_cnt *NUM_CANDIDATE_PER_DST ];for (j =0 ; j <NUM_CANDIDATE_PER_DST ; j++)
Prevents the new feature from turning on --find-copies-harder in step III. Good.
Quoted 14 lines
diffcore-rename.c +9
-0@@ -569,7 +785,28 @@ void diffcore_rename(struct diff_options *options)/* At this point, we have found some renames and copies and they * are recorded in rename_dst. The original list is still in *q. */ + +/* Now possibly factorize those renames and copies. */ +if (DIFF_OPT_TST (options,DETECT_DIR_RENAMES )) +diffcore_factorize_renames (); +DIFF_QUEUE_CLEAR (&outq); + +// Now turn non-discarded factorization_candidates into real renames +struct diff_dir_rename * candidate; +for (candidate=factorization_candidates; candidate; candidate = candidate->next) {
diffcore-rename.c void diffcore_rename(struct diff_options *options) · line 807
Quoted 10 lines
diffcore-rename.c +10
-0+struct diff_filepair * pair; +if (candidate->discarded)continue ; +// visualize toplevel dir if needed - FIXME: wrong place for this ? +if (!*candidate->one->path) + candidate->one->path ="./" ; +if (!*candidate->two->path) + candidate->two->path ="./" ; + pair =diff_queue (&outq, candidate->one, candidate->two); + pair->score =MAX_SCORE ; + pair->renamed_pair =1 ;
Outputting the discovered directory renames.
Quoted 10 lines
diffcore-rename.c +2
-0+ } +for (i =0 ; i < q->nr; i++) {struct diff_filepair *p = q->queue[i];struct diff_filepair *pair_to_free =NULL ;tree-diff.c +0
-0
tree-diff.c static int compare_tree_entry(struct tree_desc *t1, struct tree_desc *t2, const · line 55
Quoted 8 lines
tree-diff.c +3
-1@@ -49,7 +49,9 @@ static int compare_tree_entry(struct tree_desc *t1, struct tree_desc *t2, constshow_entry (opt,"+" , t2, base, baselen);return 1 ; } -if (!DIFF_OPT_TST (opt,FIND_COPIES_HARDER ) && !hashcmp (sha1, sha2) && mode1 == mode2) +if (!DIFF_OPT_TST (opt,FIND_COPIES_HARDER ) && + !DIFF_OPT_TST (opt,DETECT_DIR_RENAMES ) && + !hashcmp (sha1, sha2) && mode1 == mode2)return 0 ;
Right, makes sense.
Conclusions:
- the basic idea looks sane
- the main function would benefit a lot from being split up a bit
- would be nice to have an overview of the design (especially, a quick description of the heuristics used) for the commit message
Looks promising, and a lot simpler than I feared. :)
Thanks.
Re: [PATCH v4 1/4] Introduce wholesame directory move detection in diffcore.
Quoted 11 lines · Jonathan Nieder
diff-lib.c static int show_modified(struct rev_info *revs, · line 344
Quoted 8 lines
diff-lib.c +2
-1@@ -338,7 +339,8 @@ static int show_modified(struct rev_info *revs, oldmode = old->ce_mode;if (mode == oldmode && !hashcmp (sha1, old->sha1) && !dirty_submodule && - !DIFF_OPT_TST (&revs->diffopt,FIND_COPIES_HARDER )) + !DIFF_OPT_TST (&revs->diffopt,FIND_COPIES_HARDER ) && + !DIFF_OPT_TST (&revs->diffopt,DETECT_DIR_RENAMES ))return 0 ;The diff-index part.
Quoted 4 lines
Would we also need something like v1.6.4-rc0~45^2 (Avoid "diff-index --cached" optimization under --find-copies-harder, 2009-05-22) in order to grok unchanged subtrees, or is that taken care of some other way?
Note to self: look at that
Quoted 14 lines
[...]
diffcore-rename.c line 15
Quoted 8 lines
diffcore-rename.c +1
-0@@ -11,6 +11,7 @@static struct diff_rename_dst {struct diff_filespec *two;struct diff_filepair *pair; +int i_am_not_single:1 ;// does not look for a match, only here to be looked at } *rename_dst;If we're looking for directory renames but not finding-copies-harder, then unmodified files, while interesting and deserving of dst entries, should not be considered candidates for tracing individual files.
Quoted 5 lines
Yes? If so, maybe the less colorful
unsigned rename_target_candidate:1;would make that clearer
Right - but maybe we can use a separate list for those non-candidates as I suggested elsewhere, to make things even more clear.
Quoted 2 lines
(and more importantly, the new kind of dst entry should be mentioned in the commit log message).
Hm, the commit message is a bit long already, isn't that too much detail ?
Quoted 8 lines
Quoted 8 lines
diffcore-rename.c +8
-0+static struct diff_rename_dst *locate_rename_dst_dir (struct diff_filespec *dir) +{ +/* code mostly duplicated from locate_rename_dst - not sure we +* could merge them efficiently,though +*/ +int first, last; +int prefixlength =strlen (dir->path); +
Quoted 8 lines
diffcore-rename.c static struct diff_rename_dst *locate_rename_dst(struct diff_filespec *two, · line 70
Quoted 6 lines
diffcore-rename.c +6
-0+ first =0 ; + last = rename_dst_nr; +while (last > first) { +int next = (last + first) >>1 ; +struct diff_rename_dst *dst = &(rename_dst[next]); +int cmp =strncmp (dir->path, dst->two->path, prefixlength);prefixcmp?
Since prefixlength is constant accross the loop, strncmp spares comparing every char with \0 - we only do that once. I guess it's worth keeping.
Quoted 7 lines
Quoted 7 lines
diffcore-rename.c +7
-0+if (!cmp) +return dst; +if (cmp <0 ) { + last = next; +continue ; + } + first = next+1 ;
Quoted 8 lines
diffcore-rename.c static struct diff_rename_dst *locate_rename_dst(struct diff_filespec *two, · line 81
Quoted 4 lines
diffcore-rename.c +4
-0+ } +/* not found */ +return NULL ; +}Hmm, so this is like
locate_rename_dst(..., 0), except it matches prefixes. Result is arename_dstentry within that directory (why not spend the time to find the first?).
We just don't need the first. In most cases, finding one entry just invalidates the possiblity of a bulk move. Then when we want all of them, we have to scan the list in both directions - and yes, a clean way to avoid code duplication would be nice for those.
Quoted 7 lines
Quoted 7 lines
diffcore-rename.c +0
-0/* Find the renames */ i =for_each_hash (&file_table, find_same_files);@@ -414,6 +445,180 @@ static void record_if_better(struct diff_score m[], struct diff_score *o) m[worst] = *o; }
Quoted 8 lines
diffcore-rename.c static void record_if_better(struct diff_score m[], struct diff_score *o) · line 453
Quoted 6 lines
diffcore-rename.c +6
-0+struct diff_dir_rename { +struct diff_filespec *one; +struct diff_filespec *two; +int discarded; +struct diff_dir_rename * next; +};A linked list of renamed directories. What functions maintain it?
That is just the type decl. For this patch it is only used from diffcore_factorize_renames() and diffcore_rename(), but the --hide-dir-rename-details part needs it for maybe_mark_uninteresting, which explains its position in the source file.
Quoted 11 lines
[...]
diffcore-rename.c static void record_if_better(struct diff_score m[], struct diff_score *o) · line 505
Quoted 7 lines
diffcore-rename.c +7
-0+static struct diff_dir_rename * factorization_candidates =NULL ; +static void diffcore_factorize_renames (void ) +{ +char one_parent_path[PATH_MAX ], two_parent_path[PATH_MAX ]; +int i; + +for (i =0 ; i < rename_dst_nr; i++) {For each rename target candidate: ...
Quoted 1 line
Would it make sense to give the body of this loop its own function?
Yes, that makes things more readable.
Quoted 8 lines
diffcore-rename.c static void record_if_better(struct diff_score m[], struct diff_score *o) · line 510
Quoted 5 lines
diffcore-rename.c +5
-0+struct diff_dir_rename * seen; + +// FIXME: what are those ? +if (!rename_dst[i].pair) +continue ;Genuine new files are not interesting to us (since they do not provide evidence about directory renames, postive or negative).
Well, maybe except for Junio's idea of using bulk-rename detection to bump rename scores (we may want to look at that afterwards, but I fear it will require heavy surgery).
Quoted 6 lines
diffcore-rename.c static void record_if_better(struct diff_score m[], struct diff_score *o) · line 513
Quoted 3 lines
diffcore-rename.c +3
-0+// dummy renames used by copy detection +if (!strcmp (rename_dst[i].pair->one->path, rename_dst[i].pair->two->path)) +continue ;Whether a file was copied is probably interesting to us, but the corresponding filepair isn't.
Good point: bulk-rename can already be made to direct move+copy detection to select the correct dst for the move.
Quoted 6 lines
diffcore-rename.c static void record_if_better(struct diff_score m[], struct diff_score *o) · line 523
Quoted 3 lines
diffcore-rename.c +3
-0+ +struct diff_filespec * one_parent =alloc_filespec (one_parent_path); +fill_filespec (one_parent, null_sha1/*FIXME*/ ,S_IFDIR );Initialize a filespec, for use in filepairs in case the containing directories are a source/target pair. (?)
We need a filespec for locate_rename_dst_dir, but it is silly, we can just pass it a path string, and initialize the filespec when really needed. Good catch.
Quoted 8 lines
Quoted 4 lines
diffcore-rename.c +4
-0+ +// After this commit, are there any files still under one_parent_path ? +struct diff_rename_dst * one_leftover =locate_rename_dst_dir (one_parent); +if (one_leftover) {// FIXME: should only be run if !seen [...]
diffcore-rename.c
Quoted 1 line
diffcore-rename.c +1
-0+ }A candidate for a separate function, no?
Right.
Quoted 2 lines
The idea, if I understand it: too many unchanged files can disqualify a rename source.
Not exactly: any file left in a dir disqualifies this dir for bulk-rename.
Quoted 7 lines
diffcore-rename.c static void record_if_better(struct diff_score m[], struct diff_score *o) · line 591
Quoted 4 lines
diffcore-rename.c +4
-0+ +// already considered ? +for (seen=factorization_candidates; seen; seen = seen->next) +if (!strcmp (seen->one->path, one_parent_path))break ;Lookup. A candidate for a separate function, I think.
Quoted 15 lines
diffcore-rename.c static void record_if_better(struct diff_score m[], struct diff_score *o) · line 604
Quoted 13 lines
diffcore-rename.c +13
-0+if (!seen) { +// record potential dir rename + seen =xmalloc (sizeof (*seen)); + seen->one = one_parent; + seen->two =alloc_filespec (two_parent_path); +fill_filespec (seen->two, null_sha1/*FIXME*/ ,S_IFDIR ); + seen->discarded =0 ; + seen->next = factorization_candidates; + factorization_candidates = seen; +fprintf (stderr,"[DBG] %s -> %s suggests possible rename from %s to %s\n" , + rename_dst[i].pair->one->path, + rename_dst[i].pair->two->path, + one_parent_path, two_parent_path);Insertion. Likewise.
Hm, it's only used once, and once the other splits are done the func fits on a screen (at least on my display ;)
OTOH, if we decide for a sorted list, to allow for binary search to be more efficient, then yes.
Quoted 12 lines
Quoted 11 lines
diffcore-rename.c +7
-0+ +/* all checks ok, we keep that entry */ + } + +return ; +} +void diffcore_rename (struct diff_options *options) {int detect_rename = options->detect_rename;@@ -451,13 +656,22 @@ void diffcore_rename(struct diff_options *options)[...]
Quoted 16 lines
Quoted 4 lines
diffcore-rename.c +0
-0if (detect_rename ==DIFF_DETECT_COPY ) {/* * Increment the "rename_used" score by * one, to indicate ourselves as a user. diffcore-rename.c void diffcore_rename(struct diff_options *options) · line 673
Quoted 10 lines
diffcore-rename.c +6
-0*/ p->one->rename_used++;register_rename_src (p->one, p->score); } +if (DIFF_OPT_TST (options,DETECT_DIR_RENAMES )) { +/* similarly, rename factorization needs to +* see all files from second tree, but we don't +* want them to be matched against single sources. +*/ +locate_rename_dst (p->two,1 )->i_am_not_single =1 ;... except when --find-copies-harder is being used, right?
I have not investigated interactions with copy detection yet.
Quoted 18 lines
Quoted 18 lines
diffcore-rename.c +13
-0@@ -569,7 +785,28 @@ void diffcore_rename(struct diff_options *options)/* At this point, we have found some renames and copies and they * are recorded in rename_dst. The original list is still in *q. */ + +/* Now possibly factorize those renames and copies. */ +if (DIFF_OPT_TST (options,DETECT_DIR_RENAMES )) +diffcore_factorize_renames (); +DIFF_QUEUE_CLEAR (&outq); + +// Now turn non-discarded factorization_candidates into real renames +struct diff_dir_rename * candidate; +for (candidate=factorization_candidates; candidate; candidate = candidate->next) { +struct diff_filepair * pair; +if (candidate->discarded)continue ; +// visualize toplevel dir if needed - FIXME: wrong place for this ? +if (!*candidate->one->path)
Quoted 8 lines
diffcore-rename.c void diffcore_rename(struct diff_options *options) · line 807
Quoted 6 lines
diffcore-rename.c +6
-0+ candidate->one->path ="./" ; +if (!*candidate->two->path) + candidate->two->path ="./" ; + pair =diff_queue (&outq, candidate->one, candidate->two); + pair->score =MAX_SCORE ; + pair->renamed_pair =1 ;Outputting the discovered directory renames.
Not really outputting, just queueing for ouput here.
Quoted 6 lines
Conclusions:
- the basic idea looks sane
- the main function would benefit a lot from being split up a bit
- would be nice to have an overview of the design (especially, a quick description of the heuristics used) for the commit message
Thanks for this detailed review. Pushing to
http://repo.or.cz/w/git/ydirson.git a set of patches addressing those remarks and others from this thread.
Quoted 1 line · Yann Dirson
At last a reroll of that old series.
v3: http://thread.gmane.org/gmane.comp.version-control.git/99780
v2: http://thread.gmane.org/gmane.comp.version-control.git/99529
v1: http://thread.gmane.org/gmane.comp.version-control.git/94612
In particular, from the cover letter to v1:
| I often found myself lost when looking at a diff where a couple of | large dirs where renamed, and a handful of files were modified to take | the rename into account - not a rare situation, I'd say. In such a | case, the diffs themselves are mostly hidden among numerous rename | entries. | | To that, I felt that git ought to know better, and could easily | present a directory rename as such.
Yes!
Quoted 4 lines
I hope we can turn the simple detection feature to something acceptable for inclusion - that is, once unified diff output shows the detected renames as annotations, and once the known major holes are plugged (both from FIXME and testcases).
Okay, I look forward to learning the output format.
Quoted 4 lines · Yann Dirson
The most prominent change is a
split of the old --factorize-renames flag into --detect-dir-renames and --hide-dir-rename-details - hopefully following Junio's comments dated nearly 2 years ago.
I suspect that makes the patches more readable, too.
Quoted 6 lines
I hope we can turn the simple detection feature to something acceptable for inclusion - that is, once unified diff output shows the detected renames as annotations, and once the known major holes are plugged (both from FIXME and testcases).
From my point of view, I think it would be best to start with
the smallest usable piece, which is the raw format. It probably makes the most sense to error out when -u and --detect-dir-renames are used together. Then unified diff could be reenabled in a separate patch series on top of this one.
Another nice feature might be to let the directory-move detection feed back into file-move detection to make it more accurate. Have you thought about this? Would it be feasible, and if so, would it be useful?
Quoted 5 lines · Jonathan Nieder
Quoted 1 line
From my point of view, I think it would be best to start with
the smallest usable piece, which is the raw format. It probably makes the most sense to error out when -u and --detect-dir-renames are used together. Then unified diff could be reenabled in a separate patch series on top of this one.
Well, there's probably not much to do - in the same order of work than adding the check to error out :)
Quoted 4 lines
Another nice feature might be to let the directory-move detection feed back into file-move detection to make it more accurate. Have you thought about this? Would it be feasible, and if so, would it be useful?
That was suggested by Junio at that time. It would surely be useful in some cases, but it's not clear to me how frequently it would. And I'd suspect that quite some design changes would be necessary: that would be a 2-way influence, in that a nearly-complete bulkmove could bump the score of the latest rename that would make it a full bulkmove. Probably not trivial to do right.