Threaded loose object and pack access
Get series with b4
b4 am [email protected]Versions newest first
- v1 Latest5 patches 5 repliesIn review
Patches in v1 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/5 | Turn grep's use_threads into a global flagbuiltin/ thread-utils.c thread-utils.h | +40 | 0 replies | |
| 2/5 | grep: push locking into read_sha1_*builtin/ sha1_file.c thread-utils.c +1 more | +31 | 0 replies | |
| 3/5 | sha1_file_name_buf(): sha1_file_name in caller's buffersha1_file.c | +19 | 0 replies | |
| 4/5 | sha1_file: stuff various pack reading variables into a structsha1_file.c | +67 | 0 replies | |
| 5/5 | sha1_file: make the pack machinery thread-safebuiltin/ cache.h replace_object.c +3 more | +93 | 3 replies |
Reviews trailers per patch
No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.
Cover letter [POC PATCH 0/5] Threaded loose object and pack access
Well, just to make sure we're all left in a confused mess of partly conflicting patches, here's another angle on the same thing:
Quoted 3 lines · Jeff King
Wow, that's horrible. Leaving aside the parallelism, it's just terrible that reading from the cache is 20 times slower than the worktree. I get similar results on my quad-core machine.
By poking around in sha1_file.c I got that down to about 10. It's not great yet, but it seems a start.
The goal would be to improve it to the point where a patch lookup that already has all relevant packs open and windows mapped can proceed without locking. I'm not sure that's doable short of duplicating the whole pack state (including fds and windows) across threads, but I'll give it some more thought before going that route.
Thomas Rast (5):
Turn grep's use_threads into a global flag
grep: push locking into read_sha1_* sha1_file_name_buf(): sha1_file_name in caller's buffer
sha1_file: stuff various pack reading variables into a struct
sha1_file: make the pack machinery thread-safe
builtin/grep.c | 60 +++++----------- cache.h | 1 + replace_object.c | 5 +- sha1_file.c | 213 +++++++++++++++++++++++++++++++++++++++++------------- thread-utils.c | 30 ++++++++ thread-utils.h | 23 ++++++ 6 files changed, 240 insertions(+), 92 deletions(-)
Signature · 1 line
1.7.8.431.g2abf2