dir: avoid allocation in fill_directory()
PATCH 1/1 dir: avoid allocation in fill_directory()
Pass the match member of the first pathspec item directly to read_directory() instead of using common_prefix() to duplicate it first, thus avoiding memory duplication, strlen(3) and free(3).
---
This updates 966de3028 (dir: convert fill_directory to use the pathspec struct interface). The result is closer to the original, yet still using the modern interface.
This patch eerily resembles 2b189435 (dir: simplify fill_directory()).
dir.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-)
dir.c +3 -4
@@ -174,20 +174,19 @@ char *common_prefix(const struct pathspec *pathspec)int fill_directory (struct dir_struct *dir,const struct pathspec *pathspec) { -char *prefix; +const char *prefix;size_t prefix_len;/* * Calculate common prefix for the pathspec, and * use that to optimize the directory walk */ - prefix =common_prefix (pathspec); - prefix_len = prefix ?strlen (prefix) :0 ; + prefix_len =common_prefix_len (pathspec); + prefix = prefix_len ? pathspec->items[0 ].match :"" ;/* Read the directory and prune it */ read_directory (dir, prefix, prefix_len, pathspec); -free (prefix);return prefix_len; }
Signature · 1 line
2.11.1
Quoted 3 lines · René Scharfe
Pass the match member of the first pathspec item directly to
read_directory()instead of usingcommon_prefix()to duplicate it first, thus avoiding memory duplication, strlen(3) and free(3).
How about killing common_prefix()? There are two other callers in ls-files.c and commit.c and it looks safe to do (but I didn't look very hard).
Quoted 10 lines
dir.c +1
-1@@ -174,20 +174,19 @@ char *common_prefix(const struct pathspec *pathspec)int fill_directory (struct dir_struct *dir,const struct pathspec *pathspec) { -char *prefix; +const char *prefix;
dir.c char *common_prefix(const struct pathspec *pathspec) · line 185
Quoted 10 lines
dir.c +2
-2size_t prefix_len;/* * Calculate common prefix for the pathspec, and * use that to optimize the directory walk */ - prefix =common_prefix (pathspec); - prefix_len = prefix ?strlen (prefix) :0 ; + prefix_len =common_prefix_len (pathspec); + prefix = prefix_len ? pathspec->items[0 ].match :"" ;
There's a subtle difference. Before the patch, prefix[prefix_len] is NUL. After the patch, it's not always true. If some code (incorrectly) depends on that, this patch exposes it. I had a look inside read_directory() though and it looks like no such code exists. So, all good.
Quoted 7 lines
dir.c +0
-1/* Read the directory and prune it */ read_directory (dir, prefix, prefix_len, pathspec); -free (prefix);return prefix_len; }
Signature · 1 line
Duy
Quoted 21 lines · Duy Nguyen
Quoted 3 lines · René Scharfe
Pass the match member of the first pathspec item directly to
read_directory()instead of usingcommon_prefix()to duplicate it first, thus avoiding memory duplication, strlen(3) and free(3).How about killing
common_prefix()? There are two other callers inls-files.candcommit.cand it looks safe to do (but I didn't look very hard).Quoted 12 lines
dir.c +1
-1@@ -174,20 +174,19 @@ char *common_prefix(const struct pathspec *pathspec)int fill_directory (struct dir_struct *dir,const struct pathspec *pathspec) { -char *prefix; +const char *prefix;size_t prefix_len;
Quoted 14 lines
dir.c char *common_prefix(const struct pathspec *pathspec) · line 185
Quoted 8 lines
dir.c +2
-2/* * Calculate common prefix for the pathspec, and * use that to optimize the directory walk */ - prefix =common_prefix (pathspec); - prefix_len = prefix ?strlen (prefix) :0 ; + prefix_len =common_prefix_len (pathspec); + prefix = prefix_len ? pathspec->items[0 ].match :"" ;There's a subtle difference. Before the patch, prefix[prefix_len] is NUL. After the patch, it's not always true. If some code (incorrectly) depends on that, this patch exposes it. I had a look inside
read_directory()though and it looks like no such code exists. So, all good.
Yeah I had the exact same thought when looking at this, but I agree everything looks fine. And if something does indeed depend on prefix having a \0 at prefix_len then this will allow us to more easily find the bug and fix it.
Quoted 10 lines
Quoted 8 lines
dir.c +0
-1/* Read the directory and prune it */ read_directory (dir, prefix, prefix_len, pathspec); -free (prefix);return prefix_len; }Signature · 1 line
Duy
Signature · 1 line
Brandon Williams
Am 08.02.2017 um 07:22 schrieb Duy Nguyen:
Quoted 8 lines · René Scharfe
Quoted 3 lines
Pass the match member of the first pathspec item directly to
read_directory()instead of usingcommon_prefix()to duplicate it first, thus avoiding memory duplication, strlen(3) and free(3).How about killing
common_prefix()? There are two other callers inls-files.candcommit.cand it looks safe to do (but I didn't look very hard).
I would like that, but it doesn't look like it's worth it. I have a patch series for making overlay_tree_on_cache() take pointer+length, but it's surprisingly long and bloats the code. Duplicating a small piece of memory once per command doesn't look so bad in comparison.
(The payoff for avoiding an allocation is higher for library functions like fill_directory().)
But while working on that I found two opportunities for improvement in prune_cache(). I'll send patches shortly.
Quoted 5 lines
There's a subtle difference. Before the patch, prefix[prefix_len] is NUL. After the patch, it's not always true. If some code (incorrectly) depends on that, this patch exposes it. I had a look inside
read_directory()though and it looks like no such code exists. So, all good.
Thanks for checking.
NB: The code before 966de302 (dir: convert fill_directory to use the pathspec struct interface, committed 2017-01-04) made the same assumption, i.e. that NUL is not needed.
René