address packed-refs speed regressions
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v2 Latest9 patches 8 repliesIn review
- v16 patches 19 repliesSuperseded
Patches in v2 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/9 | strbuf_getwholeline: use getc macrostrbuf.c | +1 | 0 replies | |
| 2/9 | git-compat-util: add fallbacks for unlocked stdiogit-compat-util.h | +6 | 0 replies | |
| 3/9 | strbuf_getwholeline: use getc_unlockedstrbuf.c | +3 | 0 replies | |
| 4/9 | config: use getc_unlocked when reading from fileconfig.c | +3 | 0 replies | |
| 5/9 | strbuf_addch: avoid calling strbuf_growstrbuf.h | +2 | 0 replies | |
| 6/9 | strbuf_getwholeline: avoid calling strbuf_growstrbuf.c | +2 | 0 replies | |
| 7/9 | strbuf_getwholeline: use getdelim if it is availableMakefile config.mak.uname strbuf.c | +49 | 7 replies | |
| 8/9 | read_packed_refs: avoid double-checking sane refsrefs.c | +4 | 0 replies | |
| 9/9 | t1430: add another refs-escape testt/ | +8 | 0 replies |
Reviews trailers per patch
None of the 9 patches has a Reviewed-by or Acked-by.
| Patch | Eric Sunshine |
|---|---|
| 1/9 | |
| 2/9 | |
| 3/9 | |
| 4/9 | |
| 5/9 | |
| 6/9 | Helped-by |
| 7/9 | |
| 8/9 | |
| 9/9 |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.
Cover letter [PATCH v2 0/9] address packed-refs speed regressions
Quoted 9 lines · Jeff King
As I've mentioned before, I have some repositories with rather large numbers of refs. The worst one has ~13 million refs, for a 1.6GB packed-refs file. So I was saddened by this:
$ time git.v2.0.0 rev-parse refs/heads/foo >/dev/null 2>&1 real 0m6.840s user 0m6.404s sys 0m0.440s
Quoted 4 lines
$ time git.v2.4.0-rc1 rev-parse refs/heads/foo >/dev/null 2>&1 real 0m19.432s user 0m18.996s sys 0m0.456s
Here's a re-roll incorporating feedback from the list. Thanks everybody for your comments. Last time the final number was ~8.5s, which was disappointingly slower than v2.0.0. In this iteration, my final numbers are:
real 0m5.703s user 0m5.276s sys 0m0.432s
which is quite pleasing.
The big changes that resulted in this additional speedup are:
1. Use getdelim() when it is available. This is much faster than even a getc_unlocked() loop.
2. The slowdown from d0f810f was from adding in refname_is_safe calls. But what I didn't notice before is that we run them in _addition_ to check_refname_format, rather than instead of it. So in the common case of a sanely-formatted refname, we can skip the call, rather than writing a lot of code to micro-optimize it.
It was also mentioned in a nearby thread that the config code could benefit from some of the same micro-optimizations. It can't make use of getdelim(), as it really does want to do character-by-character parsing. But it can still use getc_unlocked() and the strbuf_avail() trick, which speeds up config reading by 47%. Those patches are included here.
[1/9]: strbuf_getwholeline: use getc macro
[2/9]: git-compat-util: add fallbacks for unlocked stdio [3/9]: strbuf_getwholeline: use getc_unlocked [4/9]: config: use getc_unlocked when reading from file [5/9]: strbuf_addch: avoid calling strbuf_grow [6/9]: strbuf_getwholeline: avoid calling strbuf_grow [7/9]: strbuf_getwholeline: use getdelim if it is available [8/9]: read_packed_refs: avoid double-checking sane refs [9/9]: t1430: add another refs-escape test
-Peff