What's cooking in git.git (May 2011, #06; Fri, 13)

Here are the topics that have been cooking. Commits prefixed with '-' are only in 'pu' while commits prefixed with '+' are in 'next'.

I am somewhat frustrated that I see almost no reviews on gitweb patches. Does this mean nobody is interested in viewing the projects on gitweb? How can we improve this lack of review bandwidth situation?

The main part of this cycle is expected to run thru May, aiming for a feature freeze in early June.

We are about to finish the week #3 of this cycle.

--------------------------------------------------
[Graduated to "master"]

* bf/commit-template-no-cleanup (2011-05-11) 1 commit
(merged to 'next' on 2011-05-11 at 01fde0d)
+ Do not strip empty lines / trailing spaces from a commit message template

* cn/log-parse-opt (2011-04-14) 1 commit
(merged to 'next' on 2011-04-28 at 02f2eac)
+ log: convert to parse-options

* jc/fix-add-u-unmerged (2011-04-23) 1 commit (merged to 'next' on 2011-04-28 at f7ed821)
+ Fix "add -u" that sometimes fails to resolve unmerged paths (this branch is used by jc/add-delete-default.)

* jc/t1506-shell-param-expansion-gotcha (2011-05-09) 1 commit (merged to 'next' on 2011-05-11 at 6c8619e)
+ t1507: avoid "${parameter<op>'word'}" inside double-quotes

* jh/dirstat-lines (2011-04-29) 8 commits
(merged to 'next' on 2011-04-29 at a302674)
+ Mark dirstat error messages for translation
+ Improve error handling when parsing dirstat parameters
+ New --dirstat=lines mode, doing dirstat analysis based on diffstat
+ Allow specifying --dirstat cut-off percentage as a floating point number
+ Add config variable for specifying default --dirstat behavior
+ Refactor --dirstat parsing; deprecate --cumulative and --dirstat-by-file
+ Make --dirstat=0 output directories that contribute < 0.1% of changes
+ Add several testcases for --dirstat and friends

* jn/setup-revisions-glob-and-friends-passthru (2011-04-21) 2 commits (merged to 'next' on 2011-04-28 at 6006cc4)
+ revisions: allow --glob and friends in parse_options-enabled commands
+ revisions: split out handle_revision_pseudo_opt function

* js/maint-1.6.6-send-pack-stateless-rpc-deadlock-fix (2011-05-05) 2 commits
+ send-pack: unbreak push over stateless rpc
+ send-pack: avoid deadlock when pack-object dies early
(this branch is used by js/maint-send-pack-stateless-rpc-deadlock-fix.)

* js/maint-send-pack-stateless-rpc-deadlock-fix (2011-05-05) 2 commits
(merged to 'next' on 2011-05-05 at 3f4ffb9)
+ Merge branch 'js/maint-1.6.6-send-pack-stateless-rpc-deadlock-fix' into js/maint-send-pack-stateless-rpc-deadlock-fix (merged to 'next' on 2011-04-28 at db7e04a)
+ Merge branch 'js/maint-1.6.6-send-pack-stateless-rpc-deadlock-fix' into js/maint-send-pack-stateless-rpc-deadlock-fix (this branch uses js/maint-1.6.6-send-pack-stateless-rpc-deadlock-fix.)

* rr/rerere-libify-clear-gc (2011-05-08) 1 commit
(merged to 'next' on 2011-05-11 at b1d8c88)
+ rerere: libify rerere_clear() and rerere_gc()

This is pure code movement. Changing them to suit the need of other callers can be built on it as separate patches.

--------------------------------------------------
[New Topics]

* jc/maint-1.6.6-pathspec-stdin-and-cmdline (2011-05-11) 1 commit
+ setup_revisions(): take pathspec from command line and --stdin correctly
(this branch is used by jc/magic-pathspec, jc/maint-1.7.4-pathspec-stdin-and-cmdline and jc/maint-pathspec-stdin-and-cmdline.)

* jc/maint-1.7.4-pathspec-stdin-and-cmdline (2011-05-11) 1 commit
- Merge branch 'jc/maint-1.6.6-pathspec-stdin-and-cmdline'
(this branch uses jc/maint-1.6.6-pathspec-stdin-and-cmdline; is tangled with jc/magic-pathspec and jc/maint-pathspec-stdin-and-cmdline.)

* jc/maint-pathspec-stdin-and-cmdline (2011-05-11) 2 commits
- Merge branch 'jc/maint-1.7.4-pathspec-stdin-and-cmdline'
- Merge branch 'jc/maint-1.6.6-pathspec-stdin-and-cmdline'
(this branch uses jc/maint-1.6.6-pathspec-stdin-and-cmdline; is tangled with jc/magic-pathspec and jc/maint-1.7.4-pathspec-stdin-and-cmdline.)

These are to fix an ancient bug where feeding "git log --stdin path" with an input that also has additional pathspec corrupted memory.

* jl/read-tree-m-dry-run (2011-05-11) 1 commit
- Teach read-tree the -n|--dry-run option

Looked good. We _might_ want to make -n simply ignore -u, though.

* jc/streaming (2011-05-12) 5 commits
- write_entry(): use streaming API for writing working tree files
- stream: a streaming interface to read from the object store
- sha1_object_info_extended(): expose a bit more information from object-info
- write_entry(): separate two helper functions out
- packed_object_info_detail(): do not return a string
(this branch uses jc/convert.)

The counterpart to jc/bigfile to cover the write-out codepath. This does not yet have "read directly from pack or loose object without slurping the whole thing in memory" yet, which is still work in progress.

--------------------------------------------------
[Stalled]

* mg/diff-stat-count (2011-05-03) 2 commits
- diff-options.txt: describe --stat-{width,name-width,count}
- diff: introduce --stat-count to limit the stat lines

There was a miscounting spotted. Need another round.

* jn/gitweb-js (2011-04-28) 13 commits
- gitweb: Make JavaScript ability to adjust timezones configurable
- gitweb.js: Add UI for selecting common timezone to display dates
- gitweb: JavaScript ability to adjust time based on timezone
- gitweb: Unify the way long timestamp is displayed
- gitweb: Refactor generating of long dates into format_timestamp_html
- gitweb.js: Provide getElementsByClassName method (if it not exists)
- gitweb.js: Introduce code to handle cookies from JavaScript
- gitweb.js: Extract and improve datetime handling
- gitweb.js: Provide default values for padding in padLeftStr and padLeft
- gitweb.js: Update and improve comments in JavaScript files
- gitweb: Split JavaScript for maintability, combining on build
- Remove gitweb/gitweb.cgi and other legacy targets from main Makefile
- git-instaweb: Simplify build dependency on gitweb

Rerolled. Waiting for comments.

* jn/ctags-more (2011-04-29) 3 commits
- gitweb: Optional grouping of projects by category
- gitweb: Modularized git_get_project_description to be more generic
- gitweb: Split git_project_list_body in two functions

Waiting for comments.

* jc/require-work-tree-exists (2011-05-08) 1 commit
- require-work-tree wants more than what its name says

Make "git pull" run from a random place work as long as GIT_DIR and GIT_WORK_TREE are set up correctly. I am not absolutely sure if that is a sane use case, though. May drop.

* jc/add-delete-default (2011-04-27) 1 commit
- git add: notice removal of tracked paths by default

* jk/maint-merge-rename-create (2011-03-25) 3 commits
- merge: turn on rewrite detection
- merge: handle renames with replacement content
- t3030: fix accidental success in symlink rename

Peff wanted to reroll this.

* jc/index-pack (2011-02-27) 5 commits
- index-pack --verify: read anomalous offsets from v2 idx file
- write_idx_file: need_large_offset() helper function
- index-pack: --verify
- write_idx_file: introduce a struct to hold idx customization options
- index-pack: group the delta-base array entries also by type

Still a WIP. Need to put histogram output into index-pack --verify to really kill verify-pack.

* jc/dotdot-is-parent-directory (2011-05-04) 1 commit
- specifying ranges: we did not mean to make ".." an empty set

Updated documentation, but I do not feel a strong enough annoyance to push this forward. May drop.

* jk/tag-contains (2010-07-05) 4 commits
- Why is "git tag --contains" so slow?
- default core.clockskew variable to one day
- limit "contains" traversals based on commit timestamp
- tag: speed up --contains calculation

The idea of the bottom one is probably Ok, except that the use of object flags needs to be rethought, or at least the helper needs to be moved to builtin/tag.c to make it clear that it should not be used outside the current usage context.

* jc/advice-about-to-lose-commit (2011-05-06) 1 commit
- checkout: honor advice.detachedHead when reattaching to a branch

Needs to tighten the advice to only govern the generic How-to part.

--------------------------------------------------
[Cooking]

* jc/bigfile (2011-05-12) 3 commits
- Bigfile: teach "git add" to send a large file straight to a pack
- index_fd(): split into two helper functions
- index_fd(): turn write_object and format_check arguments into one flag

This covers the entry point for a big file to the system. Other parts that need to know about them are the exit point (i.e. write_entry()), packing and repacking (as long as bigfilethreshold is sane this should work), diffs and status (avoid slurping large binary in core only to do nothing, which we already should), and transport (receive-pack/fetch-pack call either unpack-objects or index-pack, which still want to hold the full object in-core and need to be fixed).

* jc/convert (2011-05-09) 4 commits
- convert: make it harder to screw up adding a conversion attribute
- convert: make it safer to add conversion attributes
- convert: give saner names to crlf/eol variables, types and functions
- convert: rename the "eol" global variable to "core_eol" (this branch is used by jc/streaming.)

Just a clean-up.
Will merge to "next" by the end of week #3.

* jk/blame-line-porcelain (2011-05-09) 3 commits
- blame: add --line-porcelain output format
- blame: refactor porcelain output
- add tests for various blame formats

* ms/tagname-does-not-begin-with-dash (2011-05-10) 1 commit
- tag: disallow '-' as tag name

Will merge to "next" by the end of week #3.

* fc/completion-zsh (2011-05-10) 1 commit
+ git-completion: fix regression in zsh support
(this branch is used by sg/completion-updates.)

* nd/sparse-co-fix (2011-05-10) 2 commits
- sparse checkout: do not eagerly decide the fate for whole directory
- t1011: fix sparse-checkout initialization and add new file

* kk/maint-prefix-in-config-mak (2011-05-09) 3 commits
(merged to 'next' on 2011-05-09 at 267a3aa)
+ Honor $(prefix) set in config.mak* when defining ETC_GIT*
+ Revert "Honor $(prefix) set in config.mak* when defining ETC_GIT* and sysconfdir" (merged to 'next' on 2011-05-02 at c747ba3)
+ Honor $(prefix) set in config.mak* when defining ETC_GIT* and sysconfdir

One revert and replace the whole thing from J6t. The end result looked reasonable.

Will merge to "master" by the end of week #4.

* ci/commit--interactive-atomic (2011-05-10) 6 commits (merged to 'next' on 2011-05-11 at 7f36fd8)
+ Test atomic git-commit --interactive
+ Add commit to list of config.singlekey commands
+ Add support for -p/--patch to git-commit
+ Allow git commit --interactive with paths
+ t7501.8: feed a meaningful command
+ Use a temporary index for git commit --interactive

Will merge to "master" by the end of week #4.

* ab/i18n-scripts (2011-05-08) 48 commits
- i18n: git-bisect bisect_next_check "You need to" message
- i18n: git-bisect [Y/n] messages
- i18n: git-bisect bisect_replay + $1 messages
- i18n: git-bisect bisect_reset + $1 messages
- i18n: git-bisect bisect_run + $@ messages
- i18n: git-bisect die + eval_gettext messages
- i18n: git-bisect die + gettext messages
- i18n: git-bisect echo + eval_gettext message
- i18n: git-bisect echo + gettext messages
- i18n: git-bisect gettext + echo message
- i18n: git-bisect add git-sh-i18n
- i18n: git-stash drop_stash say/die messages
- i18n: git-stash "unknown option" message
- i18n: git-stash die + eval_gettext $1 messages
- i18n: git-stash die + eval_gettext $* messages
- i18n: git-stash die + eval_gettext messages
- i18n: git-stash die + gettext messages
- i18n: git-stash say + gettext messages
- i18n: git-stash echo + gettext message
- i18n: git-stash add git-sh-i18n
- i18n: git-submodule "blob" and "submodule" messages
- i18n: git-submodule "path not initialized" message
- i18n: git-submodule "[...] path is ignored" message
- i18n: git-submodule "Entering [...]" message
- i18n: git-submodule $errmsg messages
- i18n: git-submodule "Submodule change[...]" messages
- i18n: git-submodule "cached cannot be used" message
- i18n: git-submodule $update_module say + die messages
- i18n: git-submodule die + eval_gettext messages
- i18n: git-submodule say + eval_gettext messages
- i18n: git-submodule echo + eval_gettext messages
- i18n: git-submodule add git-sh-i18n
- i18n: git-pull eval_gettext + warning message
- i18n: git-pull eval_gettext + die message
- i18n: git-pull die messages
- i18n: git-pull add git-sh-i18n
- i18n: git-am printf(1) message to eval_gettext
- i18n: git-am core say messages
- i18n: git-am "Falling back" say message
- i18n: git-am "Apply?" message
- i18n: git-am clean_abort messages
- i18n: git-am cannot_fallback messages
- i18n: git-am die messages
- i18n: git-am echo + gettext message
- i18n: git-am eval_gettext messages
- i18n: git-am multi-line getttext $msg; echo
- i18n: git-am one-line gettext $msg; echo
- i18n: git-am add git-sh-i18n
(this branch uses ab/i18n-scripts-basic.)

* ab/i18n-scripts-basic (2011-05-08) 4 commits
- Makefile: add xgettext target for *.sh files
- git-sh-i18n.sh: add GIT_GETTEXT_POISON support
- git-sh-i18n.sh: add no-op gettext() and eval_gettext() wrappers
- git-sh-i18n--envsubst: our own envsubst(1) for eval_gettext() (this branch is used by ab/i18n-scripts.)

Will be re-rolled to work around systems whose environment variable names are case insensitive.

* mg/merge-ff-config (2011-05-06) 3 commits
(merged to 'next' on 2011-05-08 at 977b432)
+ tests: check git does not barf on merge.ff values for future versions of git
+ merge: introduce merge.ff configuration variable
+ Merge branch 'jc/maint-branch-mergeoptions' into mg/merge-ff-config

Will merge to "master" by the end of week #4.

* mk/grep-pcre (2011-05-09) 10 commits
- git-grep: do not die upon -F/-P when grep.extendedRegexp is set.
- git-grep: Bail out when -P is used with -F or -E
- grep: Add basic tests
- configure: Check for libpcre
- git-grep: Learn PCRE
- grep: Extract compile_regexp_failed() from compile_regexp()
- grep: Fix a typo in a comment
(merged to 'next' on 2011-05-08 at 5d3bede)
+ grep: Put calls to fixmatch() and regmatch() into patmatch()
+ contrib/completion: --line-number to git grep
+ Documentation: Add --line-number to git-grep synopsis

Will merge to "next" by the end of week #3.

* sg/completion-updates (2011-05-10) 4 commits
(merged to 'next' on 2011-05-10 at eaf5398)
+ Merge branch 'fc/completion-zsh' into sg/completion-updates
+ Revert "completion: don't declare 'local words' to make zsh happy" (merged to 'next' on 2011-05-08 at 761178c)
+ completion: move private shopt shim for zsh to __git_ namespace (merged to 'next' on 2011-05-02 at 0fd443a)
+ completion: don't declare 'local words' to make zsh happy (this branch uses fc/completion-zsh.)

This reverts the "local words only on bash" thingy and uses the workaround blessed by zsh folks instead.

Will merge to "master" by the end of week #4.

* ld/p4-preserve-user-names (2011-05-10) 3 commits
- git-p4: warn if git authorship won't be retained (merged to 'next' on 2011-05-11 at bc14314)
+ git-p4: small improvements to user-preservation (merged to 'next' on 2011-04-29 at 25116c8)
+ git-p4: add option to preserve user names

A re-roll of the tip one looked good but did not come in time for morning cycle today.

Will merge to "next".

* jc/magic-pathspec (2011-05-12) 13 commits
(merged to 'next' on 2011-05-12 at 656c8b5)
+ t3703: Skip tests using directory name ":" on Windows
(merged to 'next' on 2011-05-11 at 86d1d6d)
+ revision.c: leave a note for "a lone :" enhancement
+ Merge branch 'jc/maint-1.6.6-pathspec-stdin-and-cmdline'
(merged to 'next' on 2011-05-11 at aaad95e)
+ t3703, t4208: add test cases for magic pathspec
+ rev/path disambiguation: further restrict "misspelled index entry" diag
+ fix overslow :/no-such-string-ever-existed diagnostics
+ fix overstrict :<path> diagnosis
+ grep: use get_pathspec() correctly
+ pathspec: drop "lone : means no pathspec" from get_pathspec()
+ Revert "magic pathspec: add ":(icase)path" to match case insensitively"
(merged to 'next' on 2011-04-25 at 788cd46)
+ magic pathspec: add ":(icase)path" to match case insensitively
+ magic pathspec: futureproof shorthand form
+ magic pathspec: add tentative ":/path/from/top/level" pathspec support
(this branch uses jc/maint-1.6.6-pathspec-stdin-and-cmdline; is tangled with jc/maint-1.7.4-pathspec-stdin-and-cmdline and jc/maint-pathspec-stdin-and-cmdline.)

This seems to need more time to mature than I expected.

Am 13.05.2011 22:03, schrieb Junio C Hamano:

Fine by me. Do you want me to send an updated version? If so, should a warning be issued in that case or should that just happen silently?

Later we _may_ have a condition where "read-tree -m" with some options (except "-n") may succeed but "read-tree -m -u" with the same other options may fail, and the reason to have "-n" is so that the caller can tell if the operation may or may not fail in advance without changing the state. Singling out "do not write INDEX" was a mistake to begin with.

How about "simply ignore" aka "silently"? We do not even have to make it sound like an user error, if we explain the option like this.

I think the change to the implementation would be trivial, but you would probably want to update the tests in 1002 (remove the one at the end that checks -u/-n incompatibility, and perhaps make sure "-n -u" combination does not touch anything as advertised).

Thanks.

index a35849f..46a96f2 100644
@@ -55,8 +55,8 @@ OPTIONS
 
 -n::
 --dry-run::
-	Don't write the index file. This option isn't allowed together
-	with -u.
+	Check if the command would error out, without updating the index
+	nor the files in the working tree for real.
 
 -v::
 	Show the progress of checking files out.

Using this option tells read-tree to not update the index. That makes it possible to check if updating the index would be successful without changing anything. Using this option will silently ignore -u.

The relevant tests in the t10?? range were extended to do a read-tree -n before the real read-tree to make sure neither the index nor any local files were changed with -n and the same exit code as without -n is returned.

---

Am 15.05.2011 22:23, schrieb Junio C Hamano:

I added two convenience functions read_tree_u_must_[succeed|fail]() which work pretty much the way read_tree_must_[succeed|fail]() do but with the added functionality that they check git diff-files before and after the read-tree -n too.

I went through all the test cases in the t10?? range and replaced every relevant call to "git read-tree" with a call to one of those functions (the only exception being plain resets of the work tree in tests that did not test read-tree themselves).

The only exceptions where I could not make that replacement without breaking a test were:

* t1001:381 (#25) new file conflicts with untracked directory This should have been detected by verify_absent_1(), but that function is left early when opts->update is 0. Will look into that.

* t1004:209 (#13) unlinking an un-unlink-able symlink
This is an issue of wrong directory permissions, it should be ok that -n doesn't detect this as this is not a version control issue. I will document that in the commit message.

* 1011:95 (#5) read-tree with empty .git/info/sparse-checkout
* t1011:163 (#11) read-tree updates worktree, dirty case
Not sure about those two as I haven't got much experience with sparse checkout. Will dig further, pointers welcome.

Apart from these four all other tests look good.

 Documentation/git-read-tree.txt      |    5 ++
 builtin/read-tree.c                  |    7 ++-
 t/t1000-read-tree-m-3way.sh          |   96 ++++++++++++++++++++--------------
 t/t1001-read-tree-m-2way.sh          |   61 ++++++++++++++-------
 t/t1002-read-tree-m-u-2way.sh        |   91 ++++++++++++++++++--------------
 t/t1004-read-tree-m-u-wf.sh          |   44 ++++++++++++----
 t/t1005-read-tree-reset.sh           |   23 ++++++--
 t/t1008-read-tree-overlay.sh         |   10 +++-
 t/t1011-read-tree-sparse-checkout.sh |   33 ++++++++----
 t/t1012-read-tree-df.sh              |   19 +++++--
 t/t1020-subdirectory.sh              |   15 +++++-
 11 files changed, 266 insertions(+), 138 deletions(-)

index 26fdadc..46a96f2 100644
@@ -53,6 +53,11 @@ OPTIONS
 	trees that are not directly related to the current
 	working tree status into a temporary index file.
 
+-n::
+--dry-run::
+	Check if the command would error out, without updating the index
+	nor the files in the working tree for real.
+
 -v::
 	Show the progress of checking files out.
 

index 93c9281..c0da227 100644
@@ -98,7 +98,7 @@ static struct lock_file lock_file;
 
 int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)
 {
-	int i, newfd, stage = 0;
+	int i, newfd, stage = 0, dry_run = 0;
 	unsigned char sha1[20];
 	struct tree_desc t[MAX_UNPACK_TREES];
 	struct unpack_trees_options opts;

@@ -130,6 +130,7 @@ int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)
 		  PARSE_OPT_NONEG, exclude_per_directory_cb },
 		OPT_SET_INT('i', NULL, &opts.index_only,
 			    "don't check the working tree after merging", 1),
+		OPT__DRY_RUN(&dry_run, "don't update the index or the work tree"),
 		OPT_SET_INT(0, "no-sparse-checkout", &opts.skip_sparse_checkout,
 			    "skip applying sparse checkout filter", 1),
 		OPT_SET_INT(0, "debug-unpack", &opts.debug_unpack,

@@ -183,6 +184,8 @@ int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)
 		die("--exclude-per-directory is meaningless unless -u");
 	if (opts.merge && !opts.index_only)
 		setup_work_tree();
+	if (opts.update && dry_run)
+		opts.update = 0;
 
 	if (opts.merge) {
 		if (stage < 2)

@@ -219,7 +222,7 @@ int cmd_read_tree(int argc, const char **argv, const char *unused_prefix)
 	if (unpack_trees(nr_trees, t, &opts))
 		return 128;
 
-	if (opts.debug_unpack)
+	if (opts.debug_unpack || dry_run)
 		return 0; /* do not write the index out */
 
 	/*

index 8499116..33a6a18 100755
@@ -6,6 +6,17 @@ test_description='read-tree -u --reset'
 
 # two-tree test
 
+read_tree_u_must_succeed () {
+    git ls-files -s >pre-dry-run &&
+    git diff-files >pre-dry-run-wt &&
+    git read-tree -n "$@" &&
+    git ls-files -s >post-dry-run &&
+    git diff-files >post-dry-run-wt &&
+    test_cmp pre-dry-run post-dry-run &&
+    test_cmp pre-dry-run-wt post-dry-run-wt &&
+    git read-tree "$@"
+}
+
 test_expect_success 'setup' '
   git init &&
   mkdir df &&

@@ -22,13 +33,13 @@ test_expect_success 'setup' '
 '
 
 test_expect_success 'reset should work' '
-  git read-tree -u --reset HEAD^ &&
+  read_tree_u_must_succeed -u --reset HEAD^ &&
   git ls-files >actual &&
   test_cmp expect actual
 '
 
 test_expect_success 'reset should remove remnants from a failed merge' '
-  git read-tree --reset -u HEAD &&
+  read_tree_u_must_succeed --reset -u HEAD &&
   git ls-files -s >expect &&
   sha1=$(git rev-parse :new) &&
   (

@@ -37,13 +48,13 @@ test_expect_success 'reset should remove remnants from a failed merge' '
   ) | git update-index --index-info &&
   >old &&
   git ls-files -s &&
-  git read-tree --reset -u HEAD &&
+  read_tree_u_must_succeed --reset -u HEAD &&
   git ls-files -s >actual &&
   ! test -f old
 '
 
 test_expect_success 'Porcelain reset should remove remnants too' '
-  git read-tree --reset -u HEAD &&
+  read_tree_u_must_succeed --reset -u HEAD &&
   git ls-files -s >expect &&
   sha1=$(git rev-parse :new) &&
   (

@@ -58,7 +69,7 @@ test_expect_success 'Porcelain reset should remove remnants too' '
 '
 
 test_expect_success 'Porcelain checkout -f should remove remnants too' '
-  git read-tree --reset -u HEAD &&
+  read_tree_u_must_succeed --reset -u HEAD &&
   git ls-files -s >expect &&
   sha1=$(git rev-parse :new) &&
   (

@@ -73,7 +84,7 @@ test_expect_success 'Porcelain checkout -f should remove remnants too' '
 '
 
 test_expect_success 'Porcelain checkout -f HEAD should remove remnants too' '
-  git read-tree --reset -u HEAD &&
+  read_tree_u_must_succeed --reset -u HEAD &&
   git ls-files -s >expect &&
   sha1=$(git rev-parse :new) &&
   (

index f9e0028..b88a35f 100755
@@ -4,6 +4,14 @@ test_description='test multi-tree read-tree without merging'
 
 . ./test-lib.sh
 
+read_tree_must_succeed () {
+    git ls-files -s >pre-dry-run &&
+    git read-tree -n "$@" &&
+    git ls-files -s >post-dry-run &&
+    test_cmp pre-dry-run post-dry-run &&
+    git read-tree "$@"
+}
+
 test_expect_success setup '
 	echo one >a &&
 	git add a &&

@@ -21,7 +29,7 @@ test_expect_success setup '
 '
 
 test_expect_success 'multi-read' '
-	git read-tree initial master side &&
+	read_tree_must_succeed initial master side &&
 	(echo a; echo b/c) >expect &&
 	git ls-files >actual &&
 	test_cmp expect actual

index de84e35..6d47c37 100755
@@ -13,6 +13,17 @@ test_description='sparse checkout tests
 
 . ./test-lib.sh
 
+read_tree_u_must_succeed () {
+    git ls-files -s >pre-dry-run &&
+    git diff-files >pre-dry-run-wt &&
+    git read-tree -n "$@" &&
+    git ls-files -s >post-dry-run &&
+    git diff-files >post-dry-run-wt &&
+    test_cmp pre-dry-run post-dry-run &&
+    test_cmp pre-dry-run-wt post-dry-run-wt &&
+    git read-tree "$@"
+}
+
 test_expect_success 'setup' '
 	cat >expected <<-\EOF &&
 	100644 77f0ba1734ed79d12881f81b36ee134de6a3327b 0	init.t

@@ -41,7 +52,7 @@ test_expect_success 'setup' '
 '
 
 test_expect_success 'read-tree without .git/info/sparse-checkout' '
-	git read-tree -m -u HEAD &&
+	read_tree_u_must_succeed -m -u HEAD &&
 	git ls-files --stage >result &&
 	test_cmp expected result &&
 	git ls-files -t >result &&

@@ -50,7 +61,7 @@ test_expect_success 'read-tree without .git/info/sparse-checkout' '
 
 test_expect_success 'read-tree with .git/info/sparse-checkout but disabled' '
 	echo >.git/info/sparse-checkout &&
-	git read-tree -m -u HEAD &&
+	read_tree_u_must_succeed -m -u HEAD &&
 	git ls-files -t >result &&
 	test_cmp expected.swt result &&
 	test -f init.t &&

@@ -60,7 +71,7 @@ test_expect_success 'read-tree with .git/info/sparse-checkout but disabled' '
 test_expect_success 'read-tree --no-sparse-checkout with empty .git/info/sparse-checkout and enabled' '
 	git config core.sparsecheckout true &&
 	echo >.git/info/sparse-checkout &&
-	git read-tree --no-sparse-checkout -m -u HEAD &&
+	read_tree_u_must_succeed --no-sparse-checkout -m -u HEAD &&
 	git ls-files -t >result &&
 	test_cmp expected.swt result &&
 	test -f init.t &&

@@ -87,7 +98,7 @@ test_expect_success 'match directories with trailing slash' '
 	EOF
 
 	echo sub/ > .git/info/sparse-checkout &&
-	git read-tree -m -u HEAD &&
+	read_tree_u_must_succeed -m -u HEAD &&
 	git ls-files -t > result &&
 	test_cmp expected.swt-noinit result &&
 	test ! -f init.t &&

@@ -96,7 +107,7 @@ test_expect_success 'match directories with trailing slash' '
 
 test_expect_success 'match directories without trailing slash' '
 	echo sub >>.git/info/sparse-checkout &&
-	git read-tree -m -u HEAD &&
+	read_tree_u_must_succeed -m -u HEAD &&
 	git ls-files -t >result &&
 	test_cmp expected.swt-noinit result &&
 	test ! -f init.t &&

@@ -105,7 +116,7 @@ test_expect_success 'match directories without trailing slash' '
 
 test_expect_success 'match directory pattern' '
 	echo "s?b" >>.git/info/sparse-checkout &&
-	git read-tree -m -u HEAD &&
+	read_tree_u_must_succeed -m -u HEAD &&
 	git ls-files -t >result &&
 	test_cmp expected.swt-noinit result &&
 	test ! -f init.t &&

@@ -120,7 +131,7 @@ test_expect_success 'checkout area changes' '
 	EOF
 
 	echo init.t >.git/info/sparse-checkout &&
-	git read-tree -m -u HEAD &&
+	read_tree_u_must_succeed -m -u HEAD &&
 	git ls-files -t >result &&
 	test_cmp expected.swt-nosub result &&
 	test -f init.t &&

@@ -130,7 +141,7 @@ test_expect_success 'checkout area changes' '
 test_expect_success 'read-tree updates worktree, absent case' '
 	echo sub/added >.git/info/sparse-checkout &&
 	git checkout -f top &&
-	git read-tree -m -u HEAD^ &&
+	read_tree_u_must_succeed -m -u HEAD^ &&
 	test ! -f init.t
 '
 

@@ -147,14 +158,14 @@ test_expect_success 'read-tree removes worktree, dirty case' '
 	echo init.t >.git/info/sparse-checkout &&
 	git checkout -f top &&
 	echo dirty >added &&
-	git read-tree -m -u HEAD^ &&
+	read_tree_u_must_succeed -m -u HEAD^ &&
 	grep -q dirty added
 '
 
 test_expect_success 'read-tree adds to worktree, absent case' '
 	echo init.t >.git/info/sparse-checkout &&
 	git checkout -f removed &&
-	git read-tree -u -m HEAD^ &&
+	read_tree_u_must_succeed -u -m HEAD^ &&
 	test ! -f sub/added
 '
 

@@ -163,7 +174,7 @@ test_expect_success 'read-tree adds to worktree, dirty case' '
 	git checkout -f removed &&
 	mkdir sub &&
 	echo dirty >sub/added &&
-	git read-tree -u -m HEAD^ &&
+	read_tree_u_must_succeed -u -m HEAD^ &&
 	grep -q dirty sub/added
 '
 

index 9811d46..d3e814d 100755
@@ -36,6 +36,17 @@ checkindex () {
 	test_cmp expect current
 }
 
+read_tree_u_must_succeed () {
+    git ls-files -s >pre-dry-run &&
+    git diff-files >pre-dry-run-wt &&
+    git read-tree -n "$@" &&
+    git ls-files -s >post-dry-run &&
+    git diff-files >post-dry-run-wt &&
+    test_cmp pre-dry-run post-dry-run &&
+    test_cmp pre-dry-run-wt post-dry-run-wt &&
+    git read-tree "$@"
+}
+
 test_expect_success setup '
 	maketree O-000 a/b-2/c/d a/b/c/d a/x &&
 	maketree A-000 a/b-2/c/d a/b/c/d a/x &&

@@ -53,7 +64,7 @@ test_expect_success setup '
 
 test_expect_success '3-way (1)' '
 	settree A-000 &&
-	git read-tree -m -u O-000 A-000 B-000 &&
+	read_tree_u_must_succeed -m -u O-000 A-000 B-000 &&
 	checkindex <<-EOF
 	3 a/b
 	0 a/b-2/c/d

@@ -65,7 +76,7 @@ test_expect_success '3-way (1)' '
 
 test_expect_success '3-way (2)' '
 	settree A-001 &&
-	git read-tree -m -u O-000 A-001 B-000 &&
+	read_tree_u_must_succeed -m -u O-000 A-001 B-000 &&
 	checkindex <<-EOF
 	3 a/b
 	0 a/b-2/c/d

@@ -78,7 +89,7 @@ test_expect_success '3-way (2)' '
 
 test_expect_success '3-way (3)' '
 	settree A-010 &&
-	git read-tree -m -u O-010 A-010 B-010 &&
+	read_tree_u_must_succeed -m -u O-010 A-010 B-010 &&
 	checkindex <<-EOF
 	2 t
 	1 t-0

@@ -92,7 +103,7 @@ test_expect_success '3-way (3)' '
 
 test_expect_success '2-way (1)' '
 	settree O-020 &&
-	git read-tree -m -u O-020 A-020 &&
+	read_tree_u_must_succeed -m -u O-020 A-020 &&
 	checkindex <<-EOF
 	0 ds/dma/ioat/Makefile
 	0 ds/dma/ioat/registers.h

index ddc3921..a9d2a71 100755
@@ -8,6 +8,17 @@ test_description='Try various core-level commands in subdirectory.
 
 . ./test-lib.sh
 
+read_tree_u_must_succeed () {
+    git ls-files -s >pre-dry-run &&
+    git diff-files >pre-dry-run-wt &&
+    git read-tree -n "$@" &&
+    git ls-files -s >post-dry-run &&
+    git diff-files >post-dry-run-wt &&
+    test_cmp pre-dry-run post-dry-run &&
+    test_cmp pre-dry-run-wt post-dry-run-wt &&
+    git read-tree "$@"
+}
+
 test_expect_success setup '
 	long="a b c d e f g h i j k l m n o p q r s t u v w x y z" &&
 	for c in $long; do echo $c; done >one &&

@@ -98,13 +109,13 @@ test_expect_success 'checkout-index' '
 test_expect_success 'read-tree' '
 	rm -f one dir/two &&
 	tree=`git write-tree` &&
-	git read-tree --reset -u "$tree" &&
+	read_tree_u_must_succeed --reset -u "$tree" &&
 	cmp one original.one &&
 	cmp dir/two original.two &&
 	(
 		cd dir &&
 		rm -f two &&
-		git read-tree --reset -u "$tree" &&
+		read_tree_u_must_succeed --reset -u "$tree" &&
 		cmp two ../original.two &&
 		cmp ../one ../original.one
 	)

1.7.5.1.340.g084d4.dirty

Am 17.05.2011 21:43, schrieb Jens Lehmann:

I found the reason for all three failures, patch coming.

Sounds sensible.

Thanks. This is the kind of corner case I was worried about when I said "we might have a case where read-tree -m succeeds but read-tree -m -u doesn't". Fixing it would mean that ...

To enable the users of unpack_trees() to check if a work tree update would succeed without actually touching the work tree the new check_worktree flag is added to struct unpack_trees_options. This is necessary because when update is set to 0 testing for absent new files and sparse handling were disabled. This is fine when e.g. only the index should be updated but is not sufficient when the user wants to check if a work tree update would succeed.

Now users can set update to 0 and check_worktree to 1 to see if an unpack_tree() with update=1 run would succeed. The only class of problems that are not detected at the moment are file system conditions like ENOSPC or missing permissions.

---
Am 18.05.2011 00:47, schrieb Junio C Hamano:

Maybe something like this patch?

But this is the "don't update the work tree when -n is used together with -u" part, so it is needed, no? With this patch applied first and opts.check_worktree set to 1 inside that if() added there all tests succeed.

Sure, will do in the next round.

 unpack-trees.c |    4 ++--
 unpack-trees.h |    3 ++-
 2 files changed, 4 insertions(+), 3 deletions(-)

index 500ebcf..b6dca8f 100644
@@ -1001,7 +1001,7 @@ int unpack_trees(unsigned len, struct tree_desc *t, struct unpack_trees_options
 	state.refresh_cache = 1;
 
 	memset(&el, 0, sizeof(el));
-	if (!core_apply_sparse_checkout || !o->update)
+	if (!core_apply_sparse_checkout || (!o->update && !o->check_worktree))
 		o->skip_sparse_checkout = 1;
 	if (!o->skip_sparse_checkout) {
 		if (add_excludes_from_file_to_list(git_path("info/sparse-checkout"), "", 0, NULL, &el, 0) < 0)

@@ -1368,7 +1368,7 @@ static int verify_absent_1(struct cache_entry *ce,
 	int len;
 	struct stat st;
 
-	if (o->index_only || o->reset || !o->update)
+	if (o->index_only || o->reset || (!o->update && !o->check_worktree))
 		return 0;
 
 	len = check_leading_path(ce->name, ce_namelen(ce));

index cd11a08..402bbdb 100644
@@ -46,7 +46,8 @@ struct unpack_trees_options {
 		     debug_unpack,
 		     skip_sparse_checkout,
 		     gently,
-		     show_all_errors;
+		     show_all_errors,
+		     check_worktree; /* check work tree contents even if !update */
 	const char *prefix;
 	int cache_bottom;
 	struct dir_struct *dir;

1.7.5.1.340.g5ecaf

I would say the natural way to do your "dry-run" would be to change the inner guts of unpack_trees() codepath that currently does

	if (opts.update) {
		if (do something to the work tree and get non-zero on failure)
			die("... cannot update '%s'", path);
	}

with your "-n" work to

	if (opts.update) {
        	if (opts.dry_run) {
			if (would the work tree operation fail?)
				say("... update would fail because ... '%s'", path);
		} else {
			if (do something to the work tree and get non-zero on failure)
				die("... cannot update '%s'", path);
		}
	}

and that was why I thought you would want to keep the original value of opts.update. I wouldn't think of a good way to make the code that kicks in when both update and dry_run are set if you clear update that early in the codepath.

Of course you could implement a roundabout logic that says something like

	if (!update && !pretending_to_update)
        	return 0; /* we are not updating */

	if (update)
        	do work tree operations;
	else if (pretending_to_update)
        	check if work tree operations would fail;
	else
		do nothing;

But I think that is going backwards. The point of "-n" == dry-run is to cover all cases, and your initial round was only covering "no -u" case and in this round we are trying to also cover "-u" case. With the approach to add "check-worktree", if we have the third mode of operation (the first two being the existing "without -u" and "with -u"), would you add yet another "check-distim"?

Wouldn't the interaction with these true "operation modes" and "dry run" be a lot simpler to read and implement that way? In other words, which one do you find easier between these two to follow?

	if (opts.update) {
		if (opts.dry-run)
                	check if we can update but do not update in real;
		else
                	do update and die if we can't;
	} else if (opts.distim) {
		if (opts.dry-run)
                	check if we can distim but do not actually distim it;
		else
                	do distim and die if we cannot;
	} ...

vs

	if (!opts.update && !opts.distim && !opts.check_worktree && !opts.check_distim)
		return;

	if (opts.update || opts.check_worktree) {
		if (opts.check_worktree)
                	check if we can update but do not update in real;
		else
                	do update and die if we can't;
	} else if (opts.distim || opts.check_distim) {
		if (opts.check_distim)
                	check if we can distim but do not distim in real;
		else
                	do distim and die if we can't;
        }

Am 19.05.2011 20:14, schrieb Junio C Hamano:

Ah, now I get it ... Thanks, will add a dry_run flag that keeps update from changing anything.

You haven't sent this to git@vger, or it failed to deliver to me.

Anyway, do you plan to extend sha1_object_info_extended() further, to support external storage too (git-annex way for example)?

Duy

That is because it is still in flux and not ready for public viewing yet.

By definition a topic that is still in flux does not have concrete plans for everything. I haven't gone far enough to be interested in adding a support for something the other parts of the system do not support.

It is a lot more important to support the data types and representations that are supported by the other parts of the system already, and fill the missing parts. "git add" will learn to handle huge blobs natively in the jc/bigfile topic. This topic is about handling huge blobs in the checkout codepath. Without it, the system is useless if you cannot check them out even if you can add them.

The sending side of "git push" (aka "send-pack") and "git fetch" (aka "upload-pack") should also be Ok because pack-objects has known to handle large blob in a pack by copying the bits straight without re-deltifying.

I however think the receiving end still wants to hold everything in core. It would be the natural next thing to fix around this area before doing anything else. If somebody is interested, look at the receiving end ("index-pack" and "unpack-objects") while I am still busy in this topic.

Multiple people working on separate independent areas who share a reasonably narrow common agenda tend to speed up the development. At the very end of Linus era of git, after he wrote pack-objects, he worked on the object transfer protocols and unpack-objects, and I took the local use of the data in packed objects in read_sha1_file() codepath. There weren't much overlap or conflicting design decision in our works, and it turned out great. The common agenda in this case was to make the new object representation (i.e. packs) useful throughout the system.

On the other hand, multiple people working in a similar area with different agenda tend to step on each other's toes.

Sending a lot increases chance of unexpected disconnecting. But that's not the focus now.

I looked at index-pack before. Will see how to fix it (that is if I'm not busy with pathspec stuff)

Duy