Improve git-status --ignored

This patch series addresses several bugs and performance issues in .gitignore processing.

Patches #1 - #6 fix bugs and add appropriate test cases.

Patch #7 changes handling of "ignored tracked" directories, as I discovered that with the current bahavior git-clean can delete tracked content.

Patches #8 - #14 are performance optimizations.

Also available here:
https://github.com/kblees/git/tree/kb/improve-git-status-ignored-v2
git pull git://github.com/kblees/git.git kb/improve-git-status-ignored-v2

Changes since v1 (old#->new#: description):

1->1: dir.c: git-status --ignored: don't drop ignored directories 2->2: dir.c: git-status --ignored: don't list files in ignored directories 3->3: dir.c: git-status --ignored: don't list empty ignored directories

*->4: dir.c: git-ls-files --directories: don't hide empty directories
- new bugfix

4->5: dir.c: git-status --ignored: don't list empty directories as ignored
- fixed typo in commit message thanks to Eric Sunshine
- patch is reduced to a one-liner (the DIR_HIDE_EMPTY_DIRECTORIES flag has already been fixed in patch 4, renaming the variable and tweaking DIR_SHOW_OTHER_DIRECTORIES is not strictly necessary)

*->6: dir.c: make 'git-status --ignored' work within leading directories
- new bugfix

*->7: dir.c: git-clean -d -X: don't delete tracked directories
- changes handling of "ignored tracked" directories

5->8: dir.c: factor out parts of last_exclude_matching for later reuse 5->9: dir.c: move prep_exclude
- split in two patches for cleaner diff
- removed unnecessary ';' after '}'

6->10: dir.c: unify is_excluded and is_path_excluded APIs
- fixed another typo in commit message

7->11: dir.c: replace is_path_excluded with now equivalent is_excluded API 8->12: dir.c: git-status: avoid is_excluded checks for tracked files

*->13: dir.c: git-status --ignored: don't scan the work tree three times
- new optimization

*->14: dir.c: git-status --ignored: don't scan the work tree twice
- new optimization

Karsten Blees (14):
dir.c: git-status --ignored: don't drop ignored directories dir.c: git-status --ignored: don't list files in ignored directories dir.c: git-status --ignored: don't list empty ignored directories dir.c: git-ls-files --directories: don't hide empty directories dir.c: git-status --ignored: don't list empty directories as ignored dir.c: make 'git-status --ignored' work within leading directories dir.c: git-clean -d -X: don't delete tracked directories dir.c: factor out parts of last_exclude_matching for later reuse dir.c: move prep_exclude
dir.c: unify is_excluded and is_path_excluded APIs
dir.c: replace is_path_excluded with now equivalent is_excluded API dir.c: git-status: avoid is_excluded checks for tracked files dir.c: git-status --ignored: don't scan the work tree three times dir.c: git-status --ignored: don't scan the work tree twice

 Documentation/technical/api-directory-listing.txt |  25 +-
 builtin/add.c                                     |   5 +-
 builtin/check-ignore.c                            |   7 +-
 builtin/ls-files.c                                |  15 +-
 dir.c                                             | 499 +++++++++-------------
 dir.h                                             |  25 +-
 t/t3001-ls-files-others-exclude.sh                |  49 +++
 t/t7061-wtstatus-ignore.sh                        | 125 +++++-
 t/t7300-clean.sh                                  |  34 ++
 unpack-trees.c                                    |  10 +-
 unpack-trees.h                                    |   1 -
 wt-status.c                                       |  24 +-
 12 files changed, 455 insertions(+), 364 deletions(-)