rev-cache

SUGGESTED FOR 'PU':

Traversing objects is currently very costly, as every commit and tree must be loaded and parsed. Much time and energy could be saved by caching metadata and topological info in an efficient, easily accessible manner. Furthermore, this could improve git's interfacing potential, by providing a condensed summary of a repository's commit tree.

This is a series to implement such a revision caching mechanism, aptly named rev-cache. The series will provide:
- a core API to manipulate and traverse caches
- an integration into the internal revision walker
- a porcelain front-end providing access to users and (shell) applications
- a series of tests to verify/demonstrate correctness
- documentation of the API, porcelain and core concepts

In cold starts rev-cache has sped up packing and walking by a factor of 4, and over twice that on warm starts. The mechanism is minimally intrusive: most of the changes take place in seperate files, and only a handful of git's existing functions are modified.

Slides from the presentation I gave at GitTogether'09 give a good overview of the mechanism, usage and performance of rev-cache. (well, they complement the documentation well. they're a bit bare without the dialogue...) You can find them at http://www.flickr.com/photos/sirnot/sets/72157623652754819/

The patchset is structured so that each patchfile represents a fully functional, incremental development of rev-cache. Unfortunately the addition of several files makes for some obscenely large patchfiles. I apologize and regret that this is somewhat unavoidable; my only consolation is that patchfiles are less 'patches' and more additions of files.

Hope you find this useful.

- Nick

---
Hi everyone. I'm sorry about the (very) belated patchset -- University has a tendency of being very absorbing. It's spring break now though, so I've finally got around to finalizing rev-cache! The only significant change from last time (besides the bugfixes) is how structures are read/written -- excessive care is now taken to ensure a standardized file format.

This time round I've also got some swanky slides I made for GitTogether. Hehe.

Documentation/git-rev-cache.txt       |  194 +++
Documentation/technical/rev-cache.txt |  634 +++++++++
Makefile                              |    2 +
builtin.h                             |    1 +
builtin/gc.c                          |    9 +
builtin/rev-cache.c                   |  339 +++++
command-list.txt                      |    1 +
commit.c                              |   36 +-
git.c                                 |    1 +
list-objects.c                        |   46 +-
object.h                              |    3 +-
rev-cache.c                           | 2468 +++++++++++++++++++++++++++++++++
rev-cache.h                           |  123 ++
revision.c                            |   90 +-
revision.h                            |   44 +-
t/t6019-rev-cache-list.sh             |  263 ++++
16 files changed, 4227 insertions(+), 27 deletions(-)

create mode 100644 Documentation/git-rev-cache.txt create mode 100644 Documentation/technical/rev-cache.txt create mode 100644 builtin/rev-cache.c
create mode 100644 rev-cache.c
create mode 100644 rev-cache.h
create mode 100644 t/t6019-rev-cache-list.sh