recursive support for ls-files
cover recursive support for ls-files
After looking at the feedback I rerolled a few things, in particular the --submodule_prefix option that existed to give a submodule context about where it had been invoked from. People didn't seem to like the idea of exposing this to the users (yet anyways) so I removed it as an option and instead have it being passed to a child process via an environment variable GIT_INTERNAL_SUBMODULE_PREFIX. This way we don't have to support anything to external users at the moment.
Also fixed a bug (and added a test) for the -z options as pointed out by Jeff King.
Brandon Williams (3):
submodules: make submodule-prefix option an envvar
ls-files: optionally recurse into submodules
ls-files: add pathspec matching for submodules
Documentation/git-ls-files.txt | 7 +- builtin/ls-files.c | 173 ++++++++++++++++++++------- cache.h | 1 + dir.c | 46 +++++++- dir.h | 4 + environment.c | 1 + t/t3007-ls-files-recurse-submodules.sh | 209 +++++++++++++++++++++++++++++++++ 7 files changed, 398 insertions(+), 43 deletions(-) create mode 100755 t/t3007-ls-files-recurse-submodules.sh
Signature · 1 line
2.8.0.rc3.226.g39d4020
PATCH v3 1/3 submodules: make submodule-prefix option an envvar
Add a submodule-prefix enviorment variable 'GIT_INTERNAL_SUBMODULE_PREFIX' which can be used by commands which have --recurse-submodule options.
--- cache.h | 1 + environment.c | 1 + 2 files changed, 2 insertions(+)
cache.h +1 -0
@@ -408,6 +408,7 @@ static inline enum object_type object_type(unsigned int mode)#define GIT_NAMESPACE_ENVIRONMENT "GIT_NAMESPACE"#define GIT_WORK_TREE_ENVIRONMENT "GIT_WORK_TREE"#define GIT_PREFIX_ENVIRONMENT "GIT_PREFIX" +#define GIT_SUBMODULE_PREFIX_ENVIRONMENT "GIT_INTERNAL_SUBMODULE_PREFIX"#define DEFAULT_GIT_DIR_ENVIRONMENT ".git"#define DB_ENVIRONMENT "GIT_OBJECT_DIRECTORY"#define INDEX_ENVIRONMENT "GIT_INDEX_FILE"
environment.c +1 -0
@@ -120,6 +120,7 @@ const char * const local_repo_env[] = {NO_REPLACE_OBJECTS_ENVIRONMENT ,GIT_REPLACE_REF_BASE_ENVIRONMENT ,GIT_PREFIX_ENVIRONMENT , +GIT_SUBMODULE_PREFIX_ENVIRONMENT ,GIT_SHALLOW_FILE_ENVIRONMENT ,GIT_COMMON_DIR_ENVIRONMENT ,NULL
Signature · 1 line
2.8.0.rc3.226.g39d4020
Re: [PATCH 1/3 v3] submodules: make submodule-prefix option an envvar
Quoted 1 line · Brandon Williams
Add a submodule-prefix enviorment variable
environment?
Quoted 33 lines
'
GIT_INTERNAL_SUBMODULE_PREFIX' which can be used by commands which have --recurse-submodule options.Signed-off-by Brandon Williams <[email protected]>--- cache.h | 1 + environment.c | 1 + 2 files changed, 2 insertions(+)cache.h +1
-0@@ -408,6 +408,7 @@ static inline enum object_type object_type(unsigned int mode)#define GIT_NAMESPACE_ENVIRONMENT "GIT_NAMESPACE"#define GIT_WORK_TREE_ENVIRONMENT "GIT_WORK_TREE"#define GIT_PREFIX_ENVIRONMENT "GIT_PREFIX" +#define GIT_SUBMODULE_PREFIX_ENVIRONMENT "GIT_INTERNAL_SUBMODULE_PREFIX"#define DEFAULT_GIT_DIR_ENVIRONMENT ".git"#define DB_ENVIRONMENT "GIT_OBJECT_DIRECTORY"#define INDEX_ENVIRONMENT "GIT_INDEX_FILE"environment.c +1
-0@@ -120,6 +120,7 @@ const char * const local_repo_env[] = {NO_REPLACE_OBJECTS_ENVIRONMENT ,GIT_REPLACE_REF_BASE_ENVIRONMENT ,GIT_PREFIX_ENVIRONMENT , +GIT_SUBMODULE_PREFIX_ENVIRONMENT ,GIT_SHALLOW_FILE_ENVIRONMENT ,GIT_COMMON_DIR_ENVIRONMENT ,NULL
PATCH v3 2/3 ls-files: optionally recurse into submodules
In reply to Brandon WilliamsAllow ls-files to recognize submodules in order to retrieve a list of files from a repository's submodules. This is done by forking off a process to recursively call ls-files on all submodules. Use environment variable GIT_INTERNAL_SUBMODULE_PREFIX to pass a path to the submodule which it can use to prepend to output or pathspec matching logic.
--- Documentation/git-ls-files.txt | 7 ++- builtin/ls-files.c | 63 ++++++++++++++++++++++ t/t3007-ls-files-recurse-submodules.sh | 99 ++++++++++++++++++++++++++++++++++ 3 files changed, 168 insertions(+), 1 deletion(-) create mode 100755 t/t3007-ls-files-recurse-submodules.sh
Documentation/git-ls-files.txt +6 -1
@@ -18,7 +18,8 @@ SYNOPSIS [--exclude-per-directory=<file>] [--exclude-standard] [--error-unmatch] [--with-tree=<tree-ish>] - [--full-name] [--abbrev] [--] [<file>...] + [--full-name] [--recurse-submodules] + [--abbrev] [--] [<file>...] DESCRIPTION -----------
@@ -137,6 +138,10 @@ a space) at the start of each line: option forces paths to be output relative to the project top directory. +--recurse-submodules:: + Recursively calls ls-files on each submodule in the repository. + Currently there is only support for the --cached mode. + --abbrev[=<n>]:: Instead of showing the full 40-byte hexadecimal object lines, show only a partial prefix.
builtin/ls-files.c +63 -0
105 lines not on this page · raw message
t/t3007-ls-files-recurse-submodules.sh new file +99 -0
99 lines not on this page · raw message
Signature · 1 line
2.8.0.rc3.226.g39d4020
PATCH v3 3/3 ls-files: add pathspec matching for submodules
In reply to Brandon WilliamsPathspecs can be a bit tricky when trying to apply them to submodules. The main challenge is that the pathspecs will be with respect to the superproject and not with respect to paths in the submodule. The approach this patch takes is to pass in the identical pathspec from the superproject to the submodule in addition to the submodule-prefix, which is the path from the root of the superproject to the submodule, and then we can compare an entry in the submodule prepended with the submodule-prefix to the pathspec in order to determine if there is a match.
This patch also permits the pathspec logic to perform a prefix match against submodules since a pathspec could refer to a file inside of a submodule. Due to limitations in the wildmatch logic, a prefix match is only done literally. If any wildcard character is encountered we'll simply punt and produce a false positive match. More accurate matching will be done once inside the submodule. This is due to the superproject not knowing what files could exist in the submodule.
--- builtin/ls-files.c | 134 ++++++++++++++++++++------------- dir.c | 46 ++++++++++- dir.h | 4 + t/t3007-ls-files-recurse-submodules.sh | 126 +++++++++++++++++++++++++++++-- 4 files changed, 248 insertions(+), 62 deletions(-)
builtin/ls-files.c +82 -52
@@ -177,6 +177,7 @@ static void show_gitlink(const struct cache_entry *ce) {struct child_process cp =CHILD_PROCESS_INIT ;int status; +int i;argv_array_push (&cp.args,"ls-files" );argv_array_push (&cp.args,"--recurse-submodules" );
@@ -184,6 +185,29 @@ static void show_gitlink(const struct cache_entry *ce)GIT_SUBMODULE_PREFIX_ENVIRONMENT , submodule_prefix ? submodule_prefix :"" , ce->name); +/* add options */ +if (show_eol) +argv_array_push (&cp.args,"--eol" ); +if (show_valid_bit) +argv_array_push (&cp.args,"-v" ); +if (show_stage) +argv_array_push (&cp.args,"--stage" ); +if (show_cached) +argv_array_push (&cp.args,"--cached" ); +if (debug_mode) +argv_array_push (&cp.args,"--debug" ); +if (line_terminator =='\0' ) +argv_array_push (&cp.args,"-z" ); + +/* +* Pass in the original pathspec args. The submodule will be +* responsible for prepending the 'submodule_prefix' prior to comparing +* against the pathspec for matches. +*/ +argv_array_push (&cp.args,"--" ); +for (i =0 ; i < pathspec.nr; i++) +argv_array_push (&cp.args, pathspec.items[i].original); + cp.git_cmd =1 ; cp.dir = ce->name; status =run_command (&cp);
@@ -193,57 +217,62 @@ static void show_gitlink(const struct cache_entry *ce)static void show_ce_entry (const char *tag,const struct cache_entry *ce) { +struct strbuf name =STRBUF_INIT ;int len = max_prefix_len; +if (submodule_prefix) +strbuf_addstr (&name, submodule_prefix); +strbuf_addstr (&name, ce->name);if (len >=ce_namelen (ce))die ("git ls-files: internal error - cache entry not superset of prefix" ); -if (!match_pathspec (&pathspec, ce->name,ce_namelen (ce), - len, ps_matched, -S_ISDIR (ce->ce_mode) ||S_ISGITLINK (ce->ce_mode))) -return ; -if (recurse_submodules &&S_ISGITLINK (ce->ce_mode)) { +if (recurse_submodules &&S_ISGITLINK (ce->ce_mode) && +submodule_path_match (&pathspec, name.buf, ps_matched)) {show_gitlink (ce); -return ; - } + }else if (match_pathspec (&pathspec, name.buf, name.len, + len, ps_matched, +S_ISDIR (ce->ce_mode) || +S_ISGITLINK (ce->ce_mode))) { +if (tag && *tag && show_valid_bit && + (ce->ce_flags &CE_VALID )) { +static char alttag[4 ]; +memcpy (alttag, tag,3 ); +if (isalpha (tag[0 ])) + alttag[0 ] =tolower (tag[0 ]); +else if (tag[0 ] =='?' ) + alttag[0 ] ='!' ; +else { + alttag[0 ] ='v' ; + alttag[1 ] = tag[0 ]; + alttag[2 ] =' ' ; + alttag[3 ] =0 ; + } + tag = alttag; + } -if (tag && *tag && show_valid_bit && - (ce->ce_flags &CE_VALID )) { -static char alttag[4 ]; -memcpy (alttag, tag,3 ); -if (isalpha (tag[0 ])) - alttag[0 ] =tolower (tag[0 ]); -else if (tag[0 ] =='?' ) - alttag[0 ] ='!' ; -else { - alttag[0 ] ='v' ; - alttag[1 ] = tag[0 ]; - alttag[2 ] =' ' ; - alttag[3 ] =0 ; +if (!show_stage) { +fputs (tag, stdout); + }else { +printf ("%s%06o %s %d\t" , + tag, + ce->ce_mode, +find_unique_abbrev (ce->sha1,abbrev), +ce_stage (ce)); + } +write_eolinfo (ce, ce->name); +write_name (ce->name); +if (debug_mode) { +const struct stat_data *sd = &ce->ce_stat_data; + +printf (" ctime: %d:%d\n" , sd->sd_ctime.sec, sd->sd_ctime.nsec); +printf (" mtime: %d:%d\n" , sd->sd_mtime.sec, sd->sd_mtime.nsec); +printf (" dev: %d\tino: %d\n" , sd->sd_dev, sd->sd_ino); +printf (" uid: %d\tgid: %d\n" , sd->sd_uid, sd->sd_gid); +printf (" size: %d\tflags: %x\n" , sd->sd_size, ce->ce_flags); } - tag = alttag; } -if (!show_stage) { -fputs (tag, stdout); - }else { -printf ("%s%06o %s %d\t" , - tag, - ce->ce_mode, -find_unique_abbrev (ce->sha1,abbrev), -ce_stage (ce)); - } -write_eolinfo (ce, ce->name); -write_name (ce->name); -if (debug_mode) { -const struct stat_data *sd = &ce->ce_stat_data; - -printf (" ctime: %d:%d\n" , sd->sd_ctime.sec, sd->sd_ctime.nsec); -printf (" mtime: %d:%d\n" , sd->sd_mtime.sec, sd->sd_mtime.nsec); -printf (" dev: %d\tino: %d\n" , sd->sd_dev, sd->sd_ino); -printf (" uid: %d\tgid: %d\n" , sd->sd_uid, sd->sd_gid); -printf (" size: %d\tflags: %x\n" , sd->sd_size, ce->ce_flags); - } +strbuf_release (&name); }static void show_ru_info (void )
@@ -568,27 +597,28 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix) submodule_prefix =getenv (GIT_SUBMODULE_PREFIX_ENVIRONMENT );if (recurse_submodules && - (show_stage || show_deleted || show_others || show_unmerged || - show_killed || show_modified || show_resolve_undo || - show_valid_bit || show_tag || show_eol)) -die ("ls-files --recurse-submodules can only be used in " -"--cached mode" ); + (show_deleted || show_others || show_unmerged || + show_killed || show_modified || show_resolve_undo)) +die ("ls-files --recurse-submodules unsupported mode" );if (recurse_submodules && error_unmatch)die ("ls-files --recurse-submodules does not support " "--error-unmatch" ); -if (recurse_submodules && argc) -die ("ls-files --recurse-submodules does not support path " -"arguments" ); -parse_pathspec (&pathspec,0 ,PATHSPEC_PREFER_CWD |PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP , prefix, argv); -/* Find common prefix for all pathspec's */ - max_prefix =common_prefix (&pathspec); +/* +* Find common prefix for all pathspec's +* This is used as a performance optimization which unfortunately cannot +* be done when recursing into submodules +*/ +if (recurse_submodules) + max_prefix =NULL ; +else + max_prefix =common_prefix (&pathspec); max_prefix_len = max_prefix ?strlen (max_prefix) :0 ;/* Treat unmatching pathspec elements as errors */
dir.c +44 -2
64 lines not on this page · raw message
dir.h +4 -0
@@ -304,6 +304,10 @@ extern int git_fnmatch(const struct pathspec_item *item, const char *pattern, const char *string, int prefix); +extern int submodule_path_match (const struct pathspec *ps, +const char *submodule_name, +char *seen); +static inline int ce_path_match (const struct cache_entry *ce,const struct pathspec *pathspec,char *seen)
t/t3007-ls-files-recurse-submodules.sh +118 -8
151 lines not on this page · raw message
Signature · 1 line
2.8.0.rc3.226.g39d4020
Quoted 7 lines · Brandon Williams
After looking at the feedback I rerolled a few things, in particular the --submodule_prefix option that existed to give a submodule context about where it had been invoked from. People didn't seem to like the idea of exposing this to the users (yet anyways) so I removed it as an option and instead have it being passed to a child process via an environment variable
GIT_INTERNAL_SUBMODULE_PREFIX. This way we don't have to support anything to external users at the moment.
I think we can still have it as a command-line argument and declare it internal. It's not like environment variables cannot also be set by our callers. :)
I don't mind it as an environment variable, though. In some ways it makes things easier. I just think "internal versus external" and the exact implementation are orthogonal.
Quoted 2 lines
Also fixed a bug (and added a test) for the -z options as pointed out by Jeff King.
Hmm. It is broken after patch 2, and then fixed in patch 3. Usually we'd try not to have a broken state in the history. It's less important in this case, because the breakage is not a regression (--recurse-submodules is a new feature, so you could consider it "not working" until the 3rd patch). But I think it's still a good rule to follow, because it makes the commits easier to review, look at later, etc.
For that matter, I do not understand why options like "-s" get enabled in patch 3. I do not mind them starting as disabled in patch 2, but it seems like "pass along some known-safe options" should be its own patch somewhere between patches 2 and 3.
There are some other options that are ignored (neither disabled nor passed along to children). Most of them are related to exclusions, which I _think_ are safe to ignore (they do not do anything interesting unless you specify "-o", which is explicitly disabled). I'm not sure about --with-tree, though (or what it would even mean in the context of recursing).
-Peff
Quoted 10 lines · Jeff King
Quoted 7 lines · Brandon Williams
After looking at the feedback I rerolled a few things, in particular the --submodule_prefix option that existed to give a submodule context about where it had been invoked from. People didn't seem to like the idea of exposing this to the users (yet anyways) so I removed it as an option and instead have it being passed to a child process via an environment variable
GIT_INTERNAL_SUBMODULE_PREFIX. This way we don't have to support anything to external users at the moment.
Quoted 8 lines
I think we can still have it as a command-line argument and declare it internal. It's not like environment variables cannot also be set by our callers. :)
I don't mind it as an environment variable, though. In some ways it makes things easier. I just think "internal versus external" and the exact implementation are orthogonal.
We may still want it to be an option at some point in the future. This way we can revisit making it an option once we know more about the other uses it could have (aside from just being for submodules as someone suggested).
Quoted 11 lines
Quoted 2 lines
Also fixed a bug (and added a test) for the -z options as pointed out by Jeff King.
Hmm. It is broken after patch 2, and then fixed in patch 3. Usually we'd try not to have a broken state in the history. It's less important in this case, because the breakage is not a regression (--recurse-submodules is a new feature, so you could consider it "not working" until the 3rd patch). But I think it's still a good rule to follow, because it makes the commits easier to review, look at later, etc.
Quoted 4 lines
For that matter, I do not understand why options like "-s" get enabled in patch 3. I do not mind them starting as disabled in patch 2, but it seems like "pass along some known-safe options" should be its own patch somewhere between patches 2 and 3.
I'll keep that in mind for future patches. I figured that since it was fixed in the end that would be fine but if things shouldn't be broken at any state in the patch series I'll make sure to not do that in the future.
Quoted 6 lines
There are some other options that are ignored (neither disabled nor passed along to children). Most of them are related to exclusions, which I _think_ are safe to ignore (they do not do anything interesting unless you specify "-o", which is explicitly disabled). I'm not sure about --with-tree, though (or what it would even mean in the context of recursing).
These other features that are disabled now could be enabled in a future patch. You're right though I'd have to think about the --with-tree option a bit more and what it would mean with submodules.
-Brandon
Quoted 15 lines · Brandon Williams
Quoted 14 lines · Jeff King
Quoted 7 lines · Brandon Williams
After looking at the feedback I rerolled a few things, in particular the --submodule_prefix option that existed to give a submodule context about where it had been invoked from. People didn't seem to like the idea of exposing this to the users (yet anyways) so I removed it as an option and instead have it being passed to a child process via an environment variable
GIT_INTERNAL_SUBMODULE_PREFIX. This way we don't have to support anything to external users at the moment.I think we can still have it as a command-line argument and declare it internal. It's not like environment variables cannot also be set by our callers. :)
Quoted 8 lines
Quoted 3 lines
I don't mind it as an environment variable, though. In some ways it makes things easier. I just think "internal versus external" and the exact implementation are orthogonal.
We may still want it to be an option at some point in the future. This way we can revisit making it an option once we know more about the other uses it could have (aside from just being for submodules as someone suggested).
I do not think it makes too much of a difference between environment and command line option. We need an update to the "git" potty to say "you told me to use the submodule-prefix feature, but this subcommand is not prepared to accept it (yet)" and cause it to error out either way, which would mean that a series that introduces the feature needs to touch "git.c" anyway, so I would have expected us to add command line option first, simply because "git.c" is where it happens, optionally with the support for the environment variable, not the other way around.
Quoted 11 lines
Quoted 11 lines
Quoted 2 lines
Also fixed a bug (and added a test) for the -z options as pointed out by Jeff King.
Hmm. It is broken after patch 2, and then fixed in patch 3. Usually we'd try not to have a broken state in the history. It's less important in this case, because the breakage is not a regression (--recurse-submodules is a new feature, so you could consider it "not working" until the 3rd patch). But I think it's still a good rule to follow, because it makes the commits easier to review, look at later, etc.
Quoted 4 lines
Quoted 4 lines
For that matter, I do not understand why options like "-s" get enabled in patch 3. I do not mind them starting as disabled in patch 2, but it seems like "pass along some known-safe options" should be its own patch somewhere between patches 2 and 3.
Yes, exactly.
An obvious lazy way out to avoid breakage-in-the-middle and make incremental progress would be to squash everything into one patch, but we should and we should be able to do better.
I'd imagine this three-patch series would be more pleasant for future readers if it were structured like:
[1/3] introduces the submodule-prefix as a global feature; at the
least it needs a way to invoke (either an environment, or an option to "git" potty, or both) and prevent mistakes by erroring out when it is attempted to call a subcommand that does not support the feature (yet).
[2/3] adds the --recurse-submodule feature in a limited form to
"ls-files". I'd suggest for this step to pass through all options and arguments that are safe and reasonably useful to pass through without needing anything more than "ah, this option was given, so let's stuff it to the argv-array". An attempt to give things that are not yet passed through until 3/3 to lead to an error that says it is not allowed (yet).
[3-N] each of the remaining steps after 3/N adds support for one
more thing to be passed that 2/3 refrained from doing, by doing more than just "pass it in argv-array", and then remove the "not yet supported" error that added by 2/3 for that one thing. The first of these "more things" would be to support pathspecs as the receiving side would need code changes for the matching logic. There may be more, or there may be nothing else that requires 4/N, 5/N, etc.
Quoted 4 lines
Quoted 4 lines
There are some other options that are ignored (neither disabled nor passed along to children). Most of them are related to exclusions, which I _think_ are safe to ignore (they do not do anything interesting unless you specify "-o", which is explicitly disabled).
Sure. I agree that some options fall outside of "safe and reasonably useful to pass through" criteria and it is OK not to support passing them through. I however think we should detect mistakes in the caller, though.
I wonder if this will be common enough in the future that we are better off adding a bit or two to the parse-options infrastructure. Thinking out loud, would an enhancement like this be sufficient for this particular series, and still useful in more general cases?
- allow the caller of parse_options() to mark each entry with a handful of bits with no meaning to parse-options API. "ls-files" may use this mechanism to mark options that to be passed through and options that must be rejected when --recurse-submodules is asked.
- parse_options() will mark each entry as "this option was given" while doing its work. Note that "--no-foo" counts as "option 'foo' was given".
With these, after your call to parse_options() returns, you could notice that recurse-submodules was asked for by inspecting your own "static int recurse_submodules" global variable, and then iterate over builtin_ls_files_options[] array yourself to see if an option that you marked with "must be rejected while recursing" bit using mechanism #1 was seen by parse_options() by relying on mechanism #2.
You could also add a new helper macro to parse-options API that iterates over the options[] array, i.e.
for_each_parse_options_array(opt) {
if (!opt_was_used(opt))
continue;
if (opt_custom_bit(opt) & NOT_IN_RECURSIVE)
die("'%s' cannot be used with --recurse-submodules",
parse_options_name(opt))
}or something like that, but we'd need to gain experience with the mechanism #1 and #2 first before deciding if such a helper is useful (iow, I do not think we need it from day one, if we go this route).
Quoted 26 lines · Junio C Hamano
Quoted 15 lines · Brandon Williams
Quoted 14 lines · Jeff King
Quoted 7 lines · Brandon Williams
After looking at the feedback I rerolled a few things, in particular the --submodule_prefix option that existed to give a submodule context about where it had been invoked from. People didn't seem to like the idea of exposing this to the users (yet anyways) so I removed it as an option and instead have it being passed to a child process via an environment variable
GIT_INTERNAL_SUBMODULE_PREFIX. This way we don't have to support anything to external users at the moment.I think we can still have it as a command-line argument and declare it internal. It's not like environment variables cannot also be set by our callers. :)
Quoted 8 lines
Quoted 3 lines
I don't mind it as an environment variable, though. In some ways it makes things easier. I just think "internal versus external" and the exact implementation are orthogonal.
We may still want it to be an option at some point in the future. This way we can revisit making it an option once we know more about the other uses it could have (aside from just being for submodules as someone suggested).
Quoted 9 lines
I do not think it makes too much of a difference between environment and command line option. We need an update to the "git" potty to say "you told me to use the submodule-prefix feature, but this subcommand is not prepared to accept it (yet)" and cause it to error out either way, which would mean that a series that introduces the feature needs to touch "
git.c" anyway, so I would have expected us to add command line option first, simply because "git.c" is where it happens, optionally with the support for the environment variable, not the other way around.
In a previous email you mentioned that this feature should be completely hidden from users, which is why I removed the command line option for this latest series. If that isn't what you intended that I can definitely add the option to git.c. And you would rather we perform the checking in git.c to see if a subcommand supports the prefix versus silently ignoring it if it hasn't? I'm assuming this checking would also be done in git.c?
Quoted 40 lines
Quoted 12 lines
Quoted 12 lines
Quoted 3 lines
Also fixed a bug (and added a test) for the -z options as pointed out by Jeff King.
Hmm. It is broken after patch 2, and then fixed in patch 3. Usually we'd try not to have a broken state in the history. It's less important in this case, because the breakage is not a regression (--recurse-submodules is a new feature, so you could consider it "not working" until the 3rd patch). But I think it's still a good rule to follow, because it makes the commits easier to review, look at later, etc.
Quoted 4 lines
Quoted 4 lines
For that matter, I do not understand why options like "-s" get enabled in patch 3. I do not mind them starting as disabled in patch 2, but it seems like "pass along some known-safe options" should be its own patch somewhere between patches 2 and 3.
Yes, exactly.
An obvious lazy way out to avoid breakage-in-the-middle and make incremental progress would be to squash everything into one patch, but we should and we should be able to do better.
I'd imagine this three-patch series would be more pleasant for future readers if it were structured like:
[1/3] introduces the submodule-prefix as a global feature; at the
least it needs a way to invoke (either an environment, or an option to "git" potty, or both) and prevent mistakes by erroring out when it is attempted to call a subcommand that does not support the feature (yet).[2/3] adds the --recurse-submodule feature in a limited form to
"ls-files". I'd suggest for this step to pass through all options and arguments that are safe and reasonably useful to pass through without needing anything more than "ah, this option was given, so let's stuff it to the argv-array". An attempt to give things that are not yet passed through until 3/3 to lead to an error that says it is not allowed (yet).
Quoted 8 lines
[3-N] each of the remaining steps after 3/N adds support for one
more thing to be passed that 2/3 refrained from doing, by doing more than just "pass it in argv-array", and then remove the "not yet supported" error that added by 2/3 for that one thing. The first of these "more things" would be to support pathspecs as the receiving side would need code changes for the matching logic. There may be more, or there may be nothing else that requires 4/N, 5/N, etc.
I can do another rework and structure the patch series more inline with what you are suggesting here.
Signature · 1 line
Brandon Williams
Quoted 3 lines · Brandon Williams
In a previous email you mentioned that this feature should be completely hidden from users, which is why I removed the command line option for this latest series.
I may have said something like that; I do not recall, though, so a more accurate description might be "I may have said something that can be (mis)interpreted like so". Sorry for the confusion.
In any case, an environment is not "completely hidden from users", so it is not fundamentallly different from a command line option anyway ;-)
Quoted 5 lines
If that isn't what you intended that I can
definitely add the option togit.c. And you would rather we perform the checking ingit.cto see if a subcommand supports the prefix versus silently ignoring it if it hasn't? I'm assuming this checking would also be done ingit.c?
I actually do not care strongly _where_ the check happens. It was just that in the subcommand dispatcher would be the single place that is easiest-to-implement to perform that check, that made me suggest that. We already have various bits like NEEDS_WORK_TREE, RUN_SETUP, etc. so REJECT_EXTERNAL_PREFIX (or whatever its name be; I do not offhand recall current proposal) bit would fit there naturally, I would think. Of course, non-built-in commands need to protect themselves separately, if they want to.
Quoted 6 lines · Junio C Hamano
Quoted 3 lines · Brandon Williams
In a previous email you mentioned that this feature should be completely hidden from users, which is why I removed the command line option for this latest series.
Quoted 7 lines
I may have said something like that; I do not recall, though, so a more accurate description might be "I may have said something that can be (mis)interpreted like so". Sorry for the confusion.
In any case, an environment is not "completely hidden from users", so it is not fundamentallly different from a command line option anyway ;-)
No worries, I'm still trying to get a hang of all this :) True, that wouldn't be completely hidden either.
Quoted 6 lines
Quoted 5 lines
If that isn't what you intended that I can
definitely add the option togit.c. And you would rather we perform the checking ingit.cto see if a subcommand supports the prefix versus silently ignoring it if it hasn't? I'm assuming this checking would also be done ingit.c?
Quoted 8 lines
I actually do not care strongly _where_ the check happens. It was just that in the subcommand dispatcher would be the single place that is easiest-to-implement to perform that check, that made me suggest that. We already have various bits like
NEEDS_WORK_TREE,RUN_SETUP, etc. soREJECT_EXTERNAL_PREFIX(or whatever its name be; I do not offhand recall current proposal) bit would fit there naturally, I would think. Of course, non-built-in commands need to protect themselves separately, if they want to.
That makes sense. I have an idea of where the check could be made. And with those flags it may make sense to have the flag be an indicator that the builtin is ready for submodule type commands ie SUPPORTS_SUBMODULES or something along those lines. That way we only need to add the flag to each command as we go (instead of all commands which don't support it) and just make sure that the flag is set if the submodule prefix option is being used. How does that sound?
Signature · 1 line
Brandon Williams
Quoted 2 lines · Brandon Williams
or something along those lines. That way we only need to add the flag to each command as we go ...
Sounds good. Thanks.
v4 cover recursive support for ls-files
In reply to Brandon WilliamsA couple things have changed in v4:
- Restructured the patch series to prevent a breakage mid-way.
- Added an additional patch in the middle to pass through safe options. This way the series is structured in a more coherent manor.
- Added --submodule-prefix to top-level git.c
Hopefully this series addresses some of issues brought up in v3
Brandon Williams (4):
submodules: make submodule-prefix option
ls-files: optionally recurse into submodules
ls-files: pass through safe options for --recurse-submodules
ls-files: add pathspec matching for submodules
Documentation/git-ls-files.txt | 7 +- Documentation/git.txt | 5 + builtin/ls-files.c | 187 ++++++++++++++++++++++------- cache.h | 1 + dir.c | 46 +++++++- dir.h | 4 + environment.c | 1 + git.c | 21 +++- t/t3007-ls-files-recurse-submodules.sh | 209 +++++++++++++++++++++++++++++++++ 9 files changed, 437 insertions(+), 44 deletions(-) create mode 100755 t/t3007-ls-files-recurse-submodules.sh
Signature · 1 line
2.8.0.rc3.226.g39d4020
PATCH v4 1/4 submodules: make submodule-prefix option
Add a submodule-prefix environment variable 'GIT_INTERNAL_SUBMODULE_PREFIX' which can be used by commands which have --recurse-submodule options to give context to submodules about how they were invoked. This option is only allowed for builtins which have submodule support.
--- Documentation/git.txt | 5 +++++ cache.h | 1 + environment.c | 1 + git.c | 19 +++++++++++++++++++ 4 files changed, 26 insertions(+)
Documentation/git.txt +5 -0
@@ -13,6 +13,7 @@ SYNOPSIS [--exec-path[=<path>]] [--html-path] [--man-path] [--info-path] [-p|--paginate|--no-pager] [--no-replace-objects] [--bare] [--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>] + [--submodule-prefix=<path>] <command> [<args>] DESCRIPTION
@@ -601,6 +602,10 @@ foo.bar= ...`) sets `foo.bar` to the empty string. details. Equivalent to setting the `GIT_NAMESPACE` environment variable. +--submodule-prefix=<path>:: + Set a prefix which gives submodules context about the superproject that + invoked it. Only allowed for commands which support submodules. + --bare:: Treat the repository as a bare repository. If GIT_DIR environment is not set, it is set to the current working
cache.h +1 -0
@@ -408,6 +408,7 @@ static inline enum object_type object_type(unsigned int mode)#define GIT_NAMESPACE_ENVIRONMENT "GIT_NAMESPACE"#define GIT_WORK_TREE_ENVIRONMENT "GIT_WORK_TREE"#define GIT_PREFIX_ENVIRONMENT "GIT_PREFIX" +#define GIT_SUBMODULE_PREFIX_ENVIRONMENT "GIT_INTERNAL_SUBMODULE_PREFIX"#define DEFAULT_GIT_DIR_ENVIRONMENT ".git"#define DB_ENVIRONMENT "GIT_OBJECT_DIRECTORY"#define INDEX_ENVIRONMENT "GIT_INDEX_FILE"
environment.c +1 -0
@@ -120,6 +120,7 @@ const char * const local_repo_env[] = {NO_REPLACE_OBJECTS_ENVIRONMENT ,GIT_REPLACE_REF_BASE_ENVIRONMENT ,GIT_PREFIX_ENVIRONMENT , +GIT_SUBMODULE_PREFIX_ENVIRONMENT ,GIT_SHALLOW_FILE_ENVIRONMENT ,GIT_COMMON_DIR_ENVIRONMENT ,NULL
git.c +19 -0
@@ -164,6 +164,20 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)setenv (GIT_WORK_TREE_ENVIRONMENT , cmd,1 );if (envchanged) *envchanged =1 ; + }else if (!strcmp (cmd,"--submodule-prefix" )) { +if (*argc <2 ) { +fprintf (stderr,"No prefix given for --submodule-prefix.\n" ); +usage (git_usage_string); + } +setenv (GIT_SUBMODULE_PREFIX_ENVIRONMENT , (*argv)[1 ],1 ); +if (envchanged) + *envchanged =1 ; + (*argv)++; + (*argc)--; + }else if (skip_prefix (cmd,"--submodule-prefix=" , &cmd)) { +setenv (GIT_SUBMODULE_PREFIX_ENVIRONMENT , cmd,1 ); +if (envchanged) + *envchanged =1 ; }else if (!strcmp (cmd,"--bare" )) {char *cwd =xgetcwd (); is_bare_repository_cfg =1 ;
@@ -310,6 +324,7 @@ static int handle_alias(int *argcp, const char ***argv)* RUN_SETUP for reading from the configuration file. */ #define NEED_WORK_TREE (1<<3) +#define SUPPORT_SUBMODULES (1<<4)struct cmd_struct {const char *cmd;
@@ -344,6 +359,10 @@ static int run_builtin(struct cmd_struct *p, int argc, const char **argv) }commit_pager_choice (); +if (!help && (getenv (GIT_SUBMODULE_PREFIX_ENVIRONMENT ) && + !(p->option &SUPPORT_SUBMODULES ))) +die ("%s doesn't support submodules" , p->cmd); +if (!help && p->option &NEED_WORK_TREE )setup_work_tree ();
Signature · 1 line
2.8.0.rc3.226.g39d4020
Re: [PATCH 1/4 v4] submodules: make submodule-prefix option
Documentation/git.txt foo.bar= ...`) sets `foo.bar` to the empty string. · line 607
Quoted 3 lines · Brandon Williams
+--submodule-prefix=<path>::
Documentation/git.txt +2
-0+ Set a prefix which gives submodules context about the superproject that + invoked it. Only allowed for commands which support submodules.
This, and also the message in die(), uses a phrase "support submodules", but it is unclear what it exactly means to the end users and readers.
A "ls-files" that is recursively run as an implementation detail of the "grep --recurse-submodules" would be taught to support this option with this series. Who is supporting submodules in that context?
I'd imagine (close to) 100% of the people would say it is "grep" that is supporting submodules, not "ls-files", but what this paragraph and die() message want to express by the phrase "support submodules" is the fact that "ls-files" knows how to react to "--submodule-prefix" option.
I'd suggest not to worry too much about this phrasing at this point, until we figure out exactly how we want to present these to end users. For now, perhaps drop the second sentence and replace it with "The end-users are not expected to use this option" or something like that?
Quoted 34 lines
git.c +15
-0@@ -164,6 +164,20 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)setenv (GIT_WORK_TREE_ENVIRONMENT , cmd,1 );if (envchanged) *envchanged =1 ; + }else if (!strcmp (cmd,"--submodule-prefix" )) { +if (*argc <2 ) { +fprintf (stderr,"No prefix given for --submodule-prefix.\n" ); +usage (git_usage_string); + } +setenv (GIT_SUBMODULE_PREFIX_ENVIRONMENT , (*argv)[1 ],1 ); +if (envchanged) + *envchanged =1 ; + (*argv)++; + (*argc)--; + }else if (skip_prefix (cmd,"--submodule-prefix=" , &cmd)) { +setenv (GIT_SUBMODULE_PREFIX_ENVIRONMENT , cmd,1 ); +if (envchanged) + *envchanged =1 ; }else if (!strcmp (cmd,"--bare" )) {char *cwd =xgetcwd (); is_bare_repository_cfg =1 ;@@ -310,6 +324,7 @@ static int handle_alias(int *argcp, const char ***argv)* RUN_SETUP for reading from the configuration file. */ #define NEED_WORK_TREE (1<<3) +#define SUPPORT_SUBMODULES (1<<4)struct cmd_struct {const char *cmd;
git.c static int run_builtin(struct cmd_struct *p, int argc, const char **argv) · line 364
Quoted 6 lines
git.c +3
-0@@ -344,6 +359,10 @@ static int run_builtin(struct cmd_struct *p, int argc, const char **argv) }commit_pager_choice (); +if (!help && (getenv (GIT_SUBMODULE_PREFIX_ENVIRONMENT ) && + !(p->option &SUPPORT_SUBMODULES ))) +die ("%s doesn't support submodules" , p->cmd);
s/submodules/submodule-prefix/ at least.
Quoted 2 lines
git.c +0
-0if (!help && p->option &NEED_WORK_TREE )setup_work_tree ();
Re: [PATCH 1/4 v4] submodules: make submodule-prefix option
Quoted 21 lines · Junio C Hamano
Documentation/git.txt foo.bar= ...`) sets `foo.bar` to the empty string. · line 607
Quoted 3 lines · Brandon Williams
+--submodule-prefix=<path>::
Documentation/git.txt +2
-0+ Set a prefix which gives submodules context about the superproject that + invoked it. Only allowed for commands which support submodules.This, and also the message in
die(), uses a phrase "support submodules", but it is unclear what it exactly means to the end users and readers.A "ls-files" that is recursively run as an implementation detail of the "grep --recurse-submodules" would be taught to support this option with this series. Who is supporting submodules in that context?
I'd imagine (close to) 100% of the people would say it is "grep" that is supporting submodules, not "ls-files", but what this paragraph and
die()message want to express by the phrase "support submodules" is the fact that "ls-files" knows how to react to "--submodule-prefix" option.
Quoted 5 lines
I'd suggest not to worry too much about this phrasing at this point, until we figure out exactly how we want to present these to end users. For now, perhaps drop the second sentence and replace it with "The end-users are not expected to use this option" or something like that?
K can do. The intention is that each command has to do whatever internal rework needed so that it understands how to interact with subomodules (be that calling another command or internally supporting it).
Quoted 3 lines
git.c static int run_builtin(struct cmd_struct *p, int argc, const char **argv) · line 364
Quoted 1 line
git.c +1
-0+die ("%s doesn't support submodules" , p->cmd);s/submodules/submodule-prefix/ at least.
So should the #define be SUPPORT_SUBMODULE_PREFIX instead? That may be too narrow minded and not looking toward future submodule options support but I'm not sure.
Signature · 1 line
Brandon Williams
Re: [PATCH 1/4 v4] submodules: make submodule-prefix option
Quoted 5 lines · Brandon Williams
Quoted 1 line
s/submodules/submodule-prefix/ at least.
So should the #define be
SUPPORT_SUBMODULE_PREFIXinstead? That may be too narrow minded and not looking toward future submodule options support but I'm not sure.
I am not convinced that this prefix needs to be tied/limited to submodule, at least not yet, though. I view it as a prefix that points from above the repository's top, of which submodule support may be a good sample user, but the caller may not necessarily be doing or interested in "submodule support".
That's also part of figuring out how we want define the semantics of this thing and how we want to present it to the end-users, I guess, so we may have to rename it when we know more, but that's OK.
Re: [PATCH 1/4 v4] submodules: make submodule-prefix option
Quoted 14 lines · Junio C Hamano
Quoted 5 lines · Brandon Williams
Quoted 1 line
s/submodules/submodule-prefix/ at least.
So should the #define be
SUPPORT_SUBMODULE_PREFIXinstead? That may be too narrow minded and not looking toward future submodule options support but I'm not sure.I am not convinced that this prefix needs to be tied/limited to submodule, at least not yet, though. I view it as a prefix that points from above the repository's top, of which submodule support may be a good sample user, but the caller may not necessarily be doing or interested in "submodule support".
Quoted 3 lines
That's also part of figuring out how we want define the semantics of this thing and how we want to present it to the end-users, I guess, so we may have to rename it when we know more, but that's OK.
K we can leave the name as is for now then. Thankfully it should be a simple name change since it only lives in 1 file.
Signature · 1 line
Brandon Williams
PATCH v4 2/4 ls-files: optionally recurse into submodules
In reply to Brandon WilliamsAllow ls-files to recognize submodules in order to retrieve a list of files from a repository's submodules. This is done by forking off a process to recursively call ls-files on all submodules. Use top-level --submodule_prefix option to pass a path to the submodule which it can use to prepend to output or pathspec matching logic.
--- Documentation/git-ls-files.txt | 7 +- builtin/ls-files.c | 143 ++++++++++++++++++++++++--------- git.c | 2 +- t/t3007-ls-files-recurse-submodules.sh | 100 +++++++++++++++++++++++ 4 files changed, 212 insertions(+), 40 deletions(-) create mode 100755 t/t3007-ls-files-recurse-submodules.sh
Documentation/git-ls-files.txt +6 -1
@@ -18,7 +18,8 @@ SYNOPSIS [--exclude-per-directory=<file>] [--exclude-standard] [--error-unmatch] [--with-tree=<tree-ish>] - [--full-name] [--abbrev] [--] [<file>...] + [--full-name] [--recurse-submodules] + [--abbrev] [--] [<file>...] DESCRIPTION -----------
@@ -137,6 +138,10 @@ a space) at the start of each line: option forces paths to be output relative to the project top directory. +--recurse-submodules:: + Recursively calls ls-files on each submodule in the repository. + Currently there is only support for the --cached mode. + --abbrev[=<n>]:: Instead of showing the full 40-byte hexadecimal object lines, show only a partial prefix.
builtin/ls-files.c +105 -38
190 lines not on this page · raw message
git.c +1 -1
@@ -440,7 +440,7 @@ static struct cmd_struct commands[] = { { "init-db", cmd_init_db }, { "interpret-trailers", cmd_interpret_trailers, RUN_SETUP_GENTLY }, { "log", cmd_log, RUN_SETUP }, - { "ls-files", cmd_ls_files, RUN_SETUP }, + { "ls-files", cmd_ls_files, RUN_SETUP | SUPPORT_SUBMODULES }, { "ls-remote", cmd_ls_remote, RUN_SETUP_GENTLY }, { "ls-tree", cmd_ls_tree, RUN_SETUP }, { "mailinfo", cmd_mailinfo },
t/t3007-ls-files-recurse-submodules.sh new file +100 -0
100 lines not on this page · raw message
Signature · 1 line
2.8.0.rc3.226.g39d4020
Re: [PATCH 2/4 v4] ls-files: optionally recurse into submodules
Quoted 53 lines · Brandon Williams
Allow ls-files to recognize submodules in order to retrieve a list of files from a repository's submodules. This is done by forking off a process to recursively call ls-files on all submodules. Use top-level --submodule_prefix option to pass a path to the submodule which it can use to prepend to output or pathspec matching logic.
Signed-off-by Brandon Williams <[email protected]>--- Documentation/git-ls-files.txt | 7 +- builtin/ls-files.c | 143 ++++++++++++++++++++++++--------- git.c | 2 +- t/t3007-ls-files-recurse-submodules.sh | 100 +++++++++++++++++++++++ 4 files changed, 212 insertions(+), 40 deletions(-) create mode 100755 t/t3007-ls-files-recurse-submodules.shDocumentation/git-ls-files.txt +6
-1@@ -18,7 +18,8 @@ SYNOPSIS [--exclude-per-directory=<file>] [--exclude-standard] [--error-unmatch] [--with-tree=<tree-ish>] - [--full-name] [--abbrev] [--] [<file>...] + [--full-name] [--recurse-submodules] + [--abbrev] [--] [<file>...] DESCRIPTION -----------@@ -137,6 +138,10 @@ a space) at the start of each line: option forces paths to be output relative to the project top directory. +--recurse-submodules:: + Recursively calls ls-files on each submodule in the repository. + Currently there is only support for the --cached mode. + --abbrev[=<n>]:: Instead of showing the full 40-byte hexadecimal object lines, show only a partial prefix.builtin/ls-files.c +1
-0@@ -14,6 +14,7 @@#include "resolve-undo.h" #include "string-list.h" #include "pathspec.h" +#include "run-command.h" static int abbrev;static int show_deleted;
builtin/ls-files.c static int show_valid_bit; · line 33
Quoted 5 lines
builtin/ls-files.c +2
-0@@ -28,6 +29,8 @@ static int show_valid_bit;static int line_terminator ='\n' ;static int debug_mode;static int show_eol; +static int recurse_submodules; +static const char *submodule_prefix;
I would have expected this to added to environment.c in the previous step, but it is OK--I'd imagine you'd grab this from the environment and carrying a piece of information from git.c to here by setenv() followed by getenv() feels somewhat roundabout, though.
builtin/ls-files.c static void write_eolinfo(const struct cache_entry *ce, const char *path) · line 75
Quoted 8 lines
builtin/ls-files.c +2
-0static const char *prefix;static int max_prefix_len;@@ -68,6 +71,21 @@ static void write_eolinfo(const struct cache_entry *ce, const char *path)static void write_name (const char *name) {/* +* NEEDSWORK: To make this thread-safe, full_name would have to be owned +* by the caller.
As Peff mentioned in his review in another thread, a large number of functions in git are not reentrant, and I do not think we would want to give the impression that those missing a warning are safe to use.
Other than that, this step looks OK. 3/4 and later would be a lot more fun to review ;-)
Re: [PATCH 2/4 v4] ls-files: optionally recurse into submodules
Quoted 7 lines · Junio C Hamano
Quoted 1 line · Brandon Williams
+static const char
*submodule_prefix;I would have expected this to added to
environment.cin the previous step, but it is OK--I'd imagine you'd grab this from the environment and carrying a piece of information fromgit.cto here bysetenv()followed bygetenv()feels somewhat roundabout, though.
If it would make sense to do the caching of prefix string in environment.c I can move it there and add a get_submodule_prefix() function which either reads the envvar or the cached value if its already been read. Would that be a better route?
Quoted 14 lines
builtin/ls-files.c static void write_eolinfo(const struct cache_entry *ce, const char *path) · line 75
Quoted 9 lines
static const char *prefix;
static intmax_prefix_len;builtin/ls-files.c +2
-0@@ -68,6 +71,21 @@ static void write_eolinfo(const struct cache_entry *ce, const char *path)static void write_name (const char *name) {/* +* NEEDSWORK: To make this thread-safe, full_name would have to be owned +* by the caller. As Peff mentioned in his review in another thread, a large number of functions in git are not reentrant, and I do not think we would want to give the impression that those missing a warning are safe to use.
Quoted 2 lines
Other than that, this step looks OK. 3/4 and later would be a lot more fun to review ;-)
Oh yes, I can remove the comment. Seemed to miss that bit while rerolling the series.
Signature · 1 line
Brandon Williams
PATCH v4 3/4 ls-files: pass through safe options for --recurse-submodules
In reply to Brandon WilliamsPass through some known-safe options when recursing into submodules. (--cached, --stage, -v, -t, -z, --debug, --eol)
--- builtin/ls-files.c | 34 ++++++++++++++++++++++++++++++---- t/t3007-ls-files-recurse-submodules.sh | 17 ++++++++++++----- 2 files changed, 42 insertions(+), 9 deletions(-)
builtin/ls-files.c +30 -4
@@ -31,6 +31,7 @@ static int debug_mode;static int show_eol;static int recurse_submodules;static const char *submodule_prefix; +static struct argv_array recurse_submodules_opts =ARGV_ARRAY_INIT ;static const char *prefix;static int max_prefix_len;
@@ -170,6 +171,27 @@ static void show_killed_files(struct dir_struct *dir) } } +/* +* Compile an argv_array with all of the options supported by --recurse_submodules +*/ +static void compile_submodule_options (int show_tag) +{ +if (show_cached) +argv_array_push (&recurse_submodules_opts,"--cached" ); +if (show_stage) +argv_array_push (&recurse_submodules_opts,"--stage" ); +if (show_valid_bit) +argv_array_push (&recurse_submodules_opts,"-v" ); +if (show_tag) +argv_array_push (&recurse_submodules_opts,"-t" ); +if (line_terminator =='\0' ) +argv_array_push (&recurse_submodules_opts,"-z" ); +if (debug_mode) +argv_array_push (&recurse_submodules_opts,"--debug" ); +if (show_eol) +argv_array_push (&recurse_submodules_opts,"--eol" ); +} +/** * Recursively call ls-files on a submodule */
@@ -184,6 +206,9 @@ static void show_gitlink(const struct cache_entry *ce)argv_array_push (&cp.args,"ls-files" );argv_array_push (&cp.args,"--recurse-submodules" ); +/* add supported options */ +argv_array_pushv (&cp.args, recurse_submodules_opts.argv); + cp.git_cmd =1 ; cp.dir = ce->name; status =run_command (&cp);
@@ -568,14 +593,15 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)if (require_work_tree && !is_inside_work_tree ())setup_work_tree (); -if (recurse_submodules) +if (recurse_submodules) { submodule_prefix =getenv (GIT_SUBMODULE_PREFIX_ENVIRONMENT ); +compile_submodule_options (show_tag); + }if (recurse_submodules && - (show_stage || show_deleted || show_others || show_unmerged || + (show_deleted || show_others || show_unmerged || show_killed || show_modified || show_resolve_undo || - show_valid_bit || show_tag || show_eol || with_tree || - (line_terminator =='\0' ))) + with_tree))die ("ls-files --recurse-submodules unsupported mode" );if (recurse_submodules && error_unmatch)
t/t3007-ls-files-recurse-submodules.sh +12 -5
@@ -34,6 +34,18 @@ test_expect_success 'ls-files correctly outputs files in submodule' ' test_cmp expect actual ' +test_expect_success 'ls-files correctly outputs files in submodule with -z' ' + cat | tr "\n" "\0" >expect <<-\EOF && + .gitmodules + a + b/b + submodule/c + EOF + + git ls-files --recurse-submodules -z >actual && + test_cmp expect actual +' + test_expect_success 'ls-files does not output files not added to a repo' ' cat >expect <<-\EOF && .gitmodules
@@ -86,15 +98,10 @@ test_incompatible_with_recurse_submodules () { " } -test_incompatible_with_recurse_submodules -z -test_incompatible_with_recurse_submodules -v -test_incompatible_with_recurse_submodules -t test_incompatible_with_recurse_submodules --deleted test_incompatible_with_recurse_submodules --modified test_incompatible_with_recurse_submodules --others -test_incompatible_with_recurse_submodules --stage test_incompatible_with_recurse_submodules --killed test_incompatible_with_recurse_submodules --unmerged -test_incompatible_with_recurse_submodules --eol test_done
Signature · 1 line
2.8.0.rc3.226.g39d4020
Re: [PATCH 3/4 v4] ls-files: pass through safe options for --recurse-submodules
Quoted 14 lines · Brandon Williams
Pass through some known-safe options when recursing into submodules. (--cached, --stage, -v, -t, -z, --debug, --eol)
Signed-off-by Brandon Williams <[email protected]>--- builtin/ls-files.c | 34 ++++++++++++++++++++++++++++++---- t/t3007-ls-files-recurse-submodules.sh | 17 ++++++++++++----- 2 files changed, 42 insertions(+), 9 deletions(-)builtin/ls-files.c +0
-0
builtin/ls-files.c static int debug_mode; · line 34
Quoted 4 lines
builtin/ls-files.c +1
-0@@ -31,6 +31,7 @@ static int debug_mode;static int show_eol;static int recurse_submodules;static const char *submodule_prefix; +static struct argv_array recurse_submodules_opts =ARGV_ARRAY_INIT ;
I'd imagine that this is also thread-unsafe, but we do not have to comment it ;-)
Quoted 14 lines
builtin/ls-files.c +10
-0@@ -170,6 +171,27 @@ static void show_killed_files(struct dir_struct *dir) } } +/* +* Compile an argv_array with all of the options supported by --recurse_submodules +*/ +static void compile_submodule_options (int show_tag) +{ +if (show_cached) +argv_array_push (&recurse_submodules_opts,"--cached" ); +if (show_stage) +argv_array_push (&recurse_submodules_opts,"--stage" ); +if (show_valid_bit)
builtin/ls-files.c static void show_killed_files(struct dir_struct *dir) · line 193
Quoted 10 lines
builtin/ls-files.c +10
-0+argv_array_push (&recurse_submodules_opts,"-v" ); +if (show_tag) +argv_array_push (&recurse_submodules_opts,"-t" ); +if (line_terminator =='\0' ) +argv_array_push (&recurse_submodules_opts,"-z" ); +if (debug_mode) +argv_array_push (&recurse_submodules_opts,"--debug" ); +if (show_eol) +argv_array_push (&recurse_submodules_opts,"--eol" ); +}
OK. These are only the safe ones to pass through? "compile" or "assemble" is much less important thing to say than how these are chosen. "pass_supported_options()" or something? I dunno.
builtin/ls-files.c int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix) · line 605
Quoted 8 lines
builtin/ls-files.c +2
-3if (recurse_submodules && - (show_stage || show_deleted || show_others || show_unmerged || + (show_deleted || show_others || show_unmerged || show_killed || show_modified || show_resolve_undo || - show_valid_bit || show_tag || show_eol || with_tree || - (line_terminator =='\0' ))) + with_tree))die ("ls-files --recurse-submodules unsupported mode" );
Makes sense.
builtin/ls-files.c
Quoted 7 lines
builtin/ls-files.c +7
-0+test_expect_success 'ls-files correctly outputs files in submodule with -z' ' + cat | tr "\n" "\0" >expect <<-\EOF && + .gitmodules + a + b/b + submodule/c +EOF
Hmm, what do you need "cat" for here?
In nul_to_q and q_to_nul implementations (t/test-lib-functions.sh) we seem to avoid using "tr", even though q_to_cr and others do use it. I wonder if we had some portability issues with passing NUL through tr or something?
... digs and finds e85fe4d8 ("more tr portability test script fixes", 2008-03-12)
So use something like
perl -pe 'y/\012/\000/' <<\-EOF
...
EOFinstead, perhaps?
Quoted 3 lines
builtin/ls-files.c +3
-0+ git ls-files --recurse-submodules -z >actual && + test_cmp expect actual +'
Re: [PATCH 3/4 v4] ls-files: pass through safe options for --recurse-submodules
Quoted 8 lines · Junio C Hamano
In
nul_to_qandq_to_nulimplementations (t/test-lib-functions.sh) we seem to avoid using "tr", even thoughq_to_crand others do use it. I wonder if we had some portability issues with passing NUL through tr or something?... digs and finds
e85fe4d8("more tr portability test script fixes", 2008-03-12)
Quoted 7 lines
So use something like
perl -pe 'y/\012/\000/' <<\-EOF ... EOFinstead, perhaps?
I actually think it would make more sense to add
lf_to_nul () {
perl -pe 'y/\012/\000/'
}to t/test-lib-functions.sh somewhere near q_to_nul if we were to go this route.
Re: [PATCH 3/4 v4] ls-files: pass through safe options for --recurse-submodules
Quoted 18 lines · Junio C Hamano
Quoted 8 lines · Junio C Hamano
In
nul_to_qandq_to_nulimplementations (t/test-lib-functions.sh) we seem to avoid using "tr", even thoughq_to_crand others do use it. I wonder if we had some portability issues with passing NUL through tr or something?... digs and finds
e85fe4d8("more tr portability test script fixes", 2008-03-12)Quoted 7 lines
So use something like
perl -pe 'y/\012/\000/' <<\-EOF ... EOFinstead, perhaps?
Quoted 8 lines
I actually think it would make more sense to add
lf_to_nul () { perl -pe 'y/\012/\000/' }to
t/test-lib-functions.shsomewhere nearq_to_nulif we were to go this route.
my mind is drawing a blank, what does the 'lf' in 'lf_to_nul' stand for? line feed?
Signature · 1 line
Brandon Williams
Re: [PATCH 3/4 v4] ls-files: pass through safe options for --recurse-submodules
Quoted 2 lines · Brandon Williams
my mind is drawing a blank, what does the 'lf' in '
lf_to_nul' stand for? line feed?
Yup. "man 7 ascii" ;-)
Re: [PATCH 3/4 v4] ls-files: pass through safe options for --recurse-submodules
In reply to Brandon WilliamsQuoted 25 lines · Brandon Williams
Quoted 24 lines · Junio C Hamano
Quoted 8 lines · Junio C Hamano
In
nul_to_qandq_to_nulimplementations (t/test-lib-functions.sh) we seem to avoid using "tr", even thoughq_to_crand others do use it. I wonder if we had some portability issues with passing NUL through tr or something?... digs and finds
e85fe4d8("more tr portability test script fixes", 2008-03-12)Quoted 7 lines
So use something like
perl -pe 'y/\012/\000/' <<\-EOF ... EOFinstead, perhaps?
I actually think it would make more sense to add
lf_to_nul () { perl -pe 'y/\012/\000/' }
Quoted 6 lines
Quoted 2 lines
to
t/test-lib-functions.shsomewhere nearq_to_nulif we were to go this route.my mind is drawing a blank, what does the 'lf' in '
lf_to_nul' stand for? line feed?
Yes, line feed. (Note that Git has to deal with this cross platform new lines e.g. CRLF is common on Windows, CR was common on MAC, and LF is Windows, so naming the new line as they are, makes sense here.)
Re: [PATCH 3/4 v4] ls-files: pass through safe options for --recurse-submodules
In reply to Junio C HamanoQuoted 18 lines · Junio C Hamano
Quoted 8 lines · Junio C Hamano
In
nul_to_qandq_to_nulimplementations (t/test-lib-functions.sh) we seem to avoid using "tr", even thoughq_to_crand others do use it. I wonder if we had some portability issues with passing NUL through tr or something?... digs and finds
e85fe4d8("more tr portability test script fixes", 2008-03-12)Quoted 7 lines
So use something like
perl -pe 'y/\012/\000/' <<\-EOF ... EOFinstead, perhaps?
Quoted 8 lines
I actually think it would make more sense to add
lf_to_nul () { perl -pe 'y/\012/\000/' }to
t/test-lib-functions.shsomewhere nearq_to_nulif we were to go this route.
Turns out this function already exists in test-lib-functions.sh
Signature · 1 line
Brandon Williams
Re: [PATCH 3/4 v4] ls-files: pass through safe options for --recurse-submodules
Quoted 10 lines · Brandon Williams
Quoted 8 lines
I actually think it would make more sense to add
lf_to_nul () { perl -pe 'y/\012/\000/' }to
t/test-lib-functions.shsomewhere nearq_to_nulif we were to go this route.Turns out this function already exists in
test-lib-functions.sh
;-)
Re: [PATCH 3/4 v4] ls-files: pass through safe options for --recurse-submodules
In reply to Junio C HamanoQuoted 21 lines · Junio C Hamano
Quoted 20 lines · Brandon Williams
builtin/ls-files.c +16
-0@@ -170,6 +171,27 @@ static void show_killed_files(struct dir_struct *dir) } } +/* +* Compile an argv_array with all of the options supported by --recurse_submodules +*/ +static void compile_submodule_options (int show_tag) +{ +if (show_cached) +argv_array_push (&recurse_submodules_opts,"--cached" ); +if (show_stage) +argv_array_push (&recurse_submodules_opts,"--stage" ); +if (show_valid_bit) +argv_array_push (&recurse_submodules_opts,"-v" ); +if (show_tag) +argv_array_push (&recurse_submodules_opts,"-t" ); +if (line_terminator =='\0' ) +argv_array_push (&recurse_submodules_opts,"-z" ); +if (debug_mode)
Quoted 8 lines
builtin/ls-files.c static void show_killed_files(struct dir_struct *dir) · line 193
Quoted 4 lines
builtin/ls-files.c +4
-0+argv_array_push (&recurse_submodules_opts,"--debug" ); +if (show_eol) +argv_array_push (&recurse_submodules_opts,"--eol" ); +}OK. These are only the safe ones to pass through? "compile" or "assemble" is much less important thing to say than how these are chosen. "
pass_supported_options()" or something? I dunno.
Alternatively we can change this to compile all potential options (not just the ones that are supported now) and then just error the caller out, as you've suggested, if an unsupported option used. 'pass' may not be the most descriptive word for this function as it isn't actually doing the passing but rather generating an argv of options that will be passed in the event a submodule is found and we need to kick off a child process for it.
Signature · 1 line
Brandon Williams
Re: [PATCH 3/4 v4] ls-files: pass through safe options for --recurse-submodules
In reply to Brandon Williamsbuiltin/ls-files.c int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix) · line 605
Quoted 8 lines · Brandon Williams
builtin/ls-files.c +2
-3if (recurse_submodules && - (show_stage || show_deleted || show_others || show_unmerged || + (show_deleted || show_others || show_unmerged || show_killed || show_modified || show_resolve_undo || - show_valid_bit || show_tag || show_eol || with_tree || - (line_terminator =='\0' ))) + with_tree))die ("ls-files --recurse-submodules unsupported mode" );
Ahh, one more thing, and this comment probably applies to 2/4 not this one, but if the intention is to shrink this "not supported yet" check as the series progresses, in the earlier step the check would need to make sure no pathspec is given, which is first supported in 4/4, I think. It is not a big deal to require rerolling by itself, bit if you are rerolling 2/4 for other reasons, then...
Re: [PATCH 3/4 v4] ls-files: pass through safe options for --recurse-submodules
Quoted 17 lines · Junio C Hamano
builtin/ls-files.c int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix) · line 605
Quoted 8 lines · Brandon Williams
builtin/ls-files.c +2
-3if (recurse_submodules && - (show_stage || show_deleted || show_others || show_unmerged || + (show_deleted || show_others || show_unmerged || show_killed || show_modified || show_resolve_undo || - show_valid_bit || show_tag || show_eol || with_tree || - (line_terminator =='\0' ))) + with_tree))die ("ls-files --recurse-submodules unsupported mode" );Ahh, one more thing, and this comment probably applies to 2/4 not this one, but if the intention is to shrink this "not supported yet" check as the series progresses, in the earlier step the check would need to make sure no pathspec is given, which is first supported in 4/4, I think. It is not a big deal to require rerolling by itself, bit if you are rerolling 2/4 for other reasons, then...
Oh there is a separate if gaurd for pathspecs which is introduced in 2/4 and then removed once pathspec support has been added in 4/4.
Signature · 1 line
Brandon Williams
Re: [PATCH 3/4 v4] ls-files: pass through safe options for --recurse-submodules
Quoted 2 lines · Brandon Williams
Oh there is a separate if gaurd for pathspecs which is introduced in 2/4 and then removed once pathspec support has been added in 4/4.
Thanks; I missed to spot that when I wrote the message you are responding to, but it indeed is there ;-)
PATCH v4 4/4 ls-files: add pathspec matching for submodules
In reply to Brandon WilliamsPathspecs can be a bit tricky when trying to apply them to submodules. The main challenge is that the pathspecs will be with respect to the superproject and not with respect to paths in the submodule. The approach this patch takes is to pass in the identical pathspec from the superproject to the submodule in addition to the submodule-prefix, which is the path from the root of the superproject to the submodule, and then we can compare an entry in the submodule prepended with the submodule-prefix to the pathspec in order to determine if there is a match.
This patch also permits the pathspec logic to perform a prefix match against submodules since a pathspec could refer to a file inside of a submodule. Due to limitations in the wildmatch logic, a prefix match is only done literally. If any wildcard character is encountered we'll simply punt and produce a false positive match. More accurate matching will be done once inside the submodule. This is due to the superproject not knowing what files could exist in the submodule.
--- builtin/ls-files.c | 28 ++++++--- dir.c | 46 +++++++++++++- dir.h | 4 ++ t/t3007-ls-files-recurse-submodules.sh | 108 ++++++++++++++++++++++++++++++++- 4 files changed, 174 insertions(+), 12 deletions(-)
builtin/ls-files.c +21 -7
@@ -199,6 +199,7 @@ static void show_gitlink(const struct cache_entry *ce) {struct child_process cp =CHILD_PROCESS_INIT ;int status; +int i; argv_array_pushf(&cp.args, "--submodule-prefix=%s%s/", submodule_prefix ? submodule_prefix : "",
@@ -209,6 +210,15 @@ static void show_gitlink(const struct cache_entry *ce)/* add supported options */ argv_array_pushv (&cp.args, recurse_submodules_opts.argv); +/* +* Pass in the original pathspec args. The submodule will be +* responsible for prepending the 'submodule_prefix' prior to comparing +* against the pathspec for matches. +*/ +argv_array_push (&cp.args,"--" ); +for (i =0 ; i < pathspec.nr; i++) +argv_array_push (&cp.args, pathspec.items[i].original); + cp.git_cmd =1 ; cp.dir = ce->name; status =run_command (&cp);
@@ -227,7 +237,8 @@ static void show_ce_entry(const char *tag, const struct cache_entry *ce)if (len >=ce_namelen (ce))die ("git ls-files: internal error - cache entry not superset of prefix" ); -if (recurse_submodules &&S_ISGITLINK (ce->ce_mode)) { +if (recurse_submodules &&S_ISGITLINK (ce->ce_mode) && +submodule_path_match (&pathspec, name.buf, ps_matched)) {show_gitlink (ce); } else if (match_pathspec(&pathspec, name.buf, name.len, len, ps_matched,
@@ -608,17 +619,20 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)die ("ls-files --recurse-submodules does not support " "--error-unmatch" ); -if (recurse_submodules && argc) -die ("ls-files --recurse-submodules does not support path " -"arguments" ); -parse_pathspec (&pathspec,0 ,PATHSPEC_PREFER_CWD |PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP , prefix, argv); -/* Find common prefix for all pathspec's */ - max_prefix =common_prefix (&pathspec); +/* +* Find common prefix for all pathspec's +* This is used as a performance optimization which unfortunately cannot +* be done when recursing into submodules +*/ +if (recurse_submodules) + max_prefix =NULL ; +else + max_prefix =common_prefix (&pathspec); max_prefix_len = max_prefix ?strlen (max_prefix) :0 ;/* Treat unmatching pathspec elements as errors */
dir.c +44 -2
64 lines not on this page · raw message
dir.h +4 -0
@@ -304,6 +304,10 @@ extern int git_fnmatch(const struct pathspec_item *item, const char *pattern, const char *string, int prefix); +extern int submodule_path_match (const struct pathspec *ps, +const char *submodule_name, +char *seen); +static inline int ce_path_match (const struct cache_entry *ce,const struct pathspec *pathspec,char *seen)
t/t3007-ls-files-recurse-submodules.sh +105 -3
114 lines not on this page · raw message
Signature · 1 line
2.8.0.rc3.226.g39d4020
Re: [PATCH 4/4 v4] ls-files: add pathspec matching for submodules
builtin/ls-files.c int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix) · line 636
Quoted 12 lines · Brandon Williams
builtin/ls-files.c +9
-2-/* Find common prefix for all pathspec's */ - max_prefix =common_prefix (&pathspec); +/* +* Find common prefix for all pathspec's +* This is used as a performance optimization which unfortunately cannot +* be done when recursing into submodules +*/ +if (recurse_submodules) + max_prefix =NULL ; +else + max_prefix =common_prefix (&pathspec); max_prefix_len = max_prefix ?strlen (max_prefix) :0 ;
I still wonder if we can do better than this, as this would be a big cycle-saver especially in recurse-submodules case.
When you get max_prefix that is "a/b/c", there are three cases:
* a/b/c is a path prefix for an entry in the index, e.g. a/b/c/d; you then can safely use it and you do not have to do any recursive invocation of ls-files outside "a/b/c". You may match a/b/c/d in the toplevel, or you may recurse a/b/c/e that is a submodule, but you won't have to pay attention to submodules outside.
* a leading path of a/b/c, e.g. a/b, is a gitlink or a blob in the index; you can use a/b and you only have to recurse into a/b if that is a submodule; if a/b is a blob, you'd show nothing.
* a/b/c itself and no leading path of it appears in the index; you know that nothing will match once you know that you are in this situation.
Because a gitlink "a/b" sorts at the same location in the index as a regular blob "a/b" would, by feeding the max_prefix common_prefix() gives you (i.e. "a/b/c") to index_name_pos() to see which one of the three situations you are in can be done fairly cheaply, I would think. The index_name_pos() call may find "a/b/c" exactly (case 1), or return a location where "a/b/c" would be inserted in the list of existing entries. If there were "a/b" (or "a") in the index, there wouldn't be any "a/b/x" (or "a/x") at the same time, so a query for "a/b/c" would land you next to (just after) an existing entry that is a leading path of it, if such an entry exists, no? That would allow you to tell case 2 above fairly cheaply, I would expect.
It is a separate issue if adding that support to 4/4 is a good idea; I personally think doing it as a separate follow-up patch would make more sense, so all of the above is tangent.
Re: [PATCH 4/4 v4] ls-files: add pathspec matching for submodules
Quoted 47 lines · Junio C Hamano
builtin/ls-files.c int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix) · line 636
Quoted 12 lines · Brandon Williams
builtin/ls-files.c +9
-2-/* Find common prefix for all pathspec's */ - max_prefix =common_prefix (&pathspec); +/* +* Find common prefix for all pathspec's +* This is used as a performance optimization which unfortunately cannot +* be done when recursing into submodules +*/ +if (recurse_submodules) + max_prefix =NULL ; +else + max_prefix =common_prefix (&pathspec); max_prefix_len = max_prefix ?strlen (max_prefix) :0 ;I still wonder if we can do better than this, as this would be a big cycle-saver especially in recurse-submodules case.
When you get
max_prefixthat is "a/b/c", there are three cases:* a/b/c is a path prefix for an entry in the index, e.g. a/b/c/d; you then can safely use it and you do not have to do any recursive invocation of ls-files outside "a/b/c". You may match a/b/c/d in the toplevel, or you may recurse a/b/c/e that is a submodule, but you won't have to pay attention to submodules outside.
* a leading path of a/b/c, e.g. a/b, is a gitlink or a blob in the index; you can use a/b and you only have to recurse into a/b if that is a submodule; if a/b is a blob, you'd show nothing.
* a/b/c itself and no leading path of it appears in the index; you know that nothing will match once you know that you are in this situation.
Because a gitlink "a/b" sorts at the same location in the index as a regular blob "a/b" would, by feeding the
max_prefixcommon_prefix()gives you (i.e. "a/b/c") toindex_name_pos()to see which one of the three situations you are in can be done fairly cheaply, I would think. Theindex_name_pos()call may find "a/b/c" exactly (case 1), or return a location where "a/b/c" would be inserted in the list of existing entries. If there were "a/b" (or "a") in the index, there wouldn't be any "a/b/x" (or "a/x") at the same time, so a query for "a/b/c" would land you next to (just after) an existing entry that is a leading path of it, if such an entry exists, no? That would allow you to tell case 2 above fairly cheaply, I would expect.
Quoted 3 lines
It is a separate issue if adding that support to 4/4 is a good idea; I personally think doing it as a separate follow-up patch would make more sense, so all of the above is tangent.
I agree that this could be a big cycle saver. At the present I was working towards getting a working implementation but this should definitely be addressed in a follow-up patch to introduce the optimization to the recurse-submodule mode. It hopefully wouldn't be too hard to implement seeing as its using string literals.
Signature · 1 line
Brandon Williams
v5 cover recursive support for ls-files
In reply to Brandon WilliamsThe big change in this version is the introduction of a --super-prefix option to the top level git. After much discussion this seemed to be a better naming scheme than 'submodule-prefix' as it could be an option other cmds could use independent of submodules.
In 3/4 I changed the compile_submodule_options function to compile all options that can be realistically passed through and when an option that isn't supported (or rather safe) yet is provided the caller will be errored out.
Brandon Williams (4):
git: make super-prefix option
ls-files: optionally recurse into submodules
ls-files: pass through safe options for --recurse-submodules
ls-files: add pathspec matching for submodules
Documentation/git-ls-files.txt | 7 +- Documentation/git.txt | 6 + builtin/ls-files.c | 202 ++++++++++++++++++++++++------- cache.h | 2 + dir.c | 46 +++++++- dir.h | 4 + environment.c | 10 ++ git.c | 24 +++- t/t3007-ls-files-recurse-submodules.sh | 209 +++++++++++++++++++++++++++++++++ 9 files changed, 466 insertions(+), 44 deletions(-) create mode 100755 t/t3007-ls-files-recurse-submodules.sh
Signature · 1 line
2.8.0.rc3.226.g39d4020
PATCH v5 1/4 git: make super-prefix option
Add a super-prefix environment variable 'GIT_INTERNAL_SUPER_PREFIX' which can be used to specify a path from above a repository down to its root. The immediate use of this option is by commands which have a --recurse-submodule option in order to give context to submodules about how they were invoked. This option is currently only allowed for builtins which support a super-prefix.
--- Documentation/git.txt | 6 ++++++ cache.h | 2 ++ environment.c | 10 ++++++++++ git.c | 22 ++++++++++++++++++++++ 4 files changed, 40 insertions(+)
Documentation/git.txt +6 -0
@@ -13,6 +13,7 @@ SYNOPSIS [--exec-path[=<path>]] [--html-path] [--man-path] [--info-path] [-p|--paginate|--no-pager] [--no-replace-objects] [--bare] [--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>] + [--super-prefix=<path>] <command> [<args>] DESCRIPTION
@@ -601,6 +602,11 @@ foo.bar= ...`) sets `foo.bar` to the empty string. details. Equivalent to setting the `GIT_NAMESPACE` environment variable. +--super-prefix=<path>:: + Set a prefix which gives a path from above a repository down to its + root. One use is to give submodules context about the superproject that + invoked it. Currently for internal use only. + --bare:: Treat the repository as a bare repository. If GIT_DIR environment is not set, it is set to the current working
cache.h +2 -0
@@ -408,6 +408,7 @@ static inline enum object_type object_type(unsigned int mode)#define GIT_NAMESPACE_ENVIRONMENT "GIT_NAMESPACE"#define GIT_WORK_TREE_ENVIRONMENT "GIT_WORK_TREE"#define GIT_PREFIX_ENVIRONMENT "GIT_PREFIX" +#define GIT_SUPER_PREFIX_ENVIRONMENT "GIT_INTERNAL_SUPER_PREFIX"#define DEFAULT_GIT_DIR_ENVIRONMENT ".git"#define DB_ENVIRONMENT "GIT_OBJECT_DIRECTORY"#define INDEX_ENVIRONMENT "GIT_INDEX_FILE"
@@ -468,6 +469,7 @@ extern int get_common_dir_noenv(struct strbuf *sb, const char *gitdir);extern int get_common_dir (struct strbuf *sb,const char *gitdir);extern const char *get_git_namespace (void );extern const char *strip_namespace (const char *namespaced_ref); +extern const char *get_super_prefix ();extern const char *get_git_work_tree (void );/*
environment.c +10 -0
@@ -100,6 +100,8 @@ static char *work_tree;static const char *namespace;static size_t namespace_len; +static const char *super_prefix; +static const char *git_dir, *git_common_dir;static char *git_object_dir, *git_index_file, *git_graft_file;int git_db_env, git_index_env, git_graft_env, git_common_dir_env;
@@ -120,6 +122,7 @@ const char * const local_repo_env[] = {NO_REPLACE_OBJECTS_ENVIRONMENT ,GIT_REPLACE_REF_BASE_ENVIRONMENT ,GIT_PREFIX_ENVIRONMENT , +GIT_SUPER_PREFIX_ENVIRONMENT ,GIT_SHALLOW_FILE_ENVIRONMENT ,GIT_COMMON_DIR_ENVIRONMENT ,NULL
@@ -222,6 +225,13 @@ const char *strip_namespace(const char *namespaced_ref)return namespaced_ref + namespace_len; } +const char *get_super_prefix () +{ +if (!super_prefix) + super_prefix =getenv (GIT_SUPER_PREFIX_ENVIRONMENT ); +return super_prefix; +} +static int git_work_tree_initialized;/*
git.c +22 -0
@@ -164,6 +164,20 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)setenv (GIT_WORK_TREE_ENVIRONMENT , cmd,1 );if (envchanged) *envchanged =1 ; + }else if (!strcmp (cmd,"--super-prefix" )) { +if (*argc <2 ) { +fprintf (stderr,"No prefix given for --super-prefix.\n" ); +usage (git_usage_string); + } +setenv (GIT_SUPER_PREFIX_ENVIRONMENT , (*argv)[1 ],1 ); +if (envchanged) + *envchanged =1 ; + (*argv)++; + (*argc)--; + }else if (skip_prefix (cmd,"--super-prefix=" , &cmd)) { +setenv (GIT_SUPER_PREFIX_ENVIRONMENT , cmd,1 ); +if (envchanged) + *envchanged =1 ; }else if (!strcmp (cmd,"--bare" )) {char *cwd =xgetcwd (); is_bare_repository_cfg =1 ;
@@ -310,6 +324,7 @@ static int handle_alias(int *argcp, const char ***argv)* RUN_SETUP for reading from the configuration file. */ #define NEED_WORK_TREE (1<<3) +#define SUPPORT_SUPER_PREFIX (1<<4)struct cmd_struct {const char *cmd;
@@ -344,6 +359,13 @@ static int run_builtin(struct cmd_struct *p, int argc, const char **argv) }commit_pager_choice (); +if (!help &&get_super_prefix ()) { +if (!(p->option &SUPPORT_SUPER_PREFIX )) +die ("%s doesn't support --super-prefix" , p->cmd); +if (prefix) +die ("can't have both a prefix and super-prefix" ); + } +if (!help && p->option &NEED_WORK_TREE )setup_work_tree ();
Signature · 1 line
2.8.0.rc3.226.g39d4020
Re: [PATCH v5 1/4] git: make super-prefix option
Documentation/git.txt foo.bar= ...`) sets `foo.bar` to the empty string. · line 608
Quoted 10 lines · Brandon Williams
DESCRIPTION
Documentation/git.txt +4
-0@@ -601,6 +602,11 @@ foo.bar= ...`) sets `foo.bar` to the empty string. details. Equivalent to setting the `GIT_NAMESPACE` environment variable. +--super-prefix=<path>:: + Set a prefix which gives a path from above a repository down to its + root. One use is to give submodules context about the superproject that + invoked it. Currently for internal use only.
I would put the "Currently for internal use only." at the beginning, so end users don't have to bother reading though the description, when they want to only use kosher flags. (Well, checking man git fetch and searching for 'internal',
there doesn't seem to be a consistent way how to document internal flags :(. It doesn't however advertise the flag in the SYNOPSIS. Ok it doesn't advertise a lot of flags in its SYNOPSIS)
environment.c const char *strip_namespace(const char *namespaced_ref) · line 231
Quoted 4 lines
environment.c +4
-0+const char *get_super_prefix () +{ +if (!super_prefix) + super_prefix =getenv (GIT_SUPER_PREFIX_ENVIRONMENT );
The getenv() function returns a pointer to the value in the environment, or NULL if there is no match.
So in case this is not set (when e.g. the user did not specify the super prefix),
we would probe it a couple of times.
The caching effect only occurs when the string is set. So this looks like we save repetitive
calls, but we do not always do that.
Re: [PATCH v5 1/4] git: make super-prefix option
In reply to Brandon WilliamsQuoted 6 lines · Brandon Williams
Add a super-prefix environment variable '
GIT_INTERNAL_SUPER_PREFIX' which can be used to specify a path from above a repository down to its root. The immediate use of this option is by commands which have a --recurse-submodule option in order to give context to submodules about how they were invoked. This option is currently only allowed for builtins which support a super-prefix.
Yes, it can be used to specify that, but it is unclear (1) how such a path thusly specified is used, (2) how it affects the outcome of the operations, and (3) what it means (the same applies to the documentation update).
It would help future readers of "git log", "tig blame", etc. to have a few sentences after the first sentence above. Perhaps
... specify a path from above ... down to its root. When such a super-prefix is specified, the paths reported by Git are prefixed with it to make them relative to that directory "above". The paths given by the users on the command line (e.g. "git subcmd --output-file=path/to/a/file" and pathspecs) are taken relative to that directory "above" to match.
or something like that?
cache.h extern int get_common_dir_noenv(struct strbuf *sb, const char *gitdir); · line 472
Quoted 5 lines
cache.h +1
-0@@ -468,6 +469,7 @@ extern int get_common_dir_noenv(struct strbuf *sb, const char *gitdir);extern int get_common_dir (struct strbuf *sb,const char *gitdir);extern const char *get_git_namespace (void );extern const char *strip_namespace (const char *namespaced_ref); +extern const char *get_super_prefix ();
Nice, but
extern const char *get_super_prefix(void);
environment.c const char *strip_namespace(const char *namespaced_ref) · line 228
Quoted 1 line
environment.c +1
-0+const char *get_super_prefix ()
Likewise.
environment.c const char *strip_namespace(const char *namespaced_ref) · line 233
Quoted 5 lines
environment.c +5
-0+{ +if (!super_prefix) + super_prefix =getenv (GIT_SUPER_PREFIX_ENVIRONMENT ); +return super_prefix; +}
Good.
git.c static int run_builtin(struct cmd_struct *p, int argc, const char **argv) · line 367
Quoted 8 lines
git.c +6
-0commit_pager_choice (); +if (!help &&get_super_prefix ()) { +if (!(p->option &SUPPORT_SUPER_PREFIX )) +die ("%s doesn't support --super-prefix" , p->cmd); +if (prefix) +die ("can't have both a prefix and super-prefix" ); + }
Nice. I'd phrase the latter as "can't use--super-prefix from a subdirectory", though.
Re: [PATCH v5 1/4] git: make super-prefix option
In reply to Brandon WilliamsQuoted 6 lines · Brandon Williams
Add a super-prefix environment variable '
GIT_INTERNAL_SUPER_PREFIX' which can be used to specify a path from above a repository down to its root. The immediate use of this option is by commands which have a --recurse-submodule option in order to give context to submodules about how they were invoked. This option is currently only allowed for builtins which support a super-prefix.
What about non-builtins?
E.g., what should
git --super-prefix=foo bar
do? Should the externals and scripts check the presence of GIT_INTERNAL_SUPER_PREFIX and barf if it is set? Most scripts would probably notice eventually when calling some other builtin that doesn't support SUPER_PREFIX, but it seems hacky to count on that.
There's also the question of 3rd-party programs. If we want to be conservative, I think you'd want to just always bail in execv_dashed_external() if --super-prefix is in use. That doesn't give an option for scripts to say "hey, I support this", but we can perhaps worry about loosening later.
-Peff
Re: [PATCH v5 1/4] git: make super-prefix option
Quoted 20 lines · Jeff King
Quoted 6 lines · Brandon Williams
Add a super-prefix environment variable '
GIT_INTERNAL_SUPER_PREFIX' which can be used to specify a path from above a repository down to its root. The immediate use of this option is by commands which have a --recurse-submodule option in order to give context to submodules about how they were invoked. This option is currently only allowed for builtins which support a super-prefix.What about non-builtins?
E.g., what should
git --super-prefix=foo bardo? Should the externals and scripts check the presence of
GIT_INTERNAL_SUPER_PREFIXand barf if it is set? Most scripts would probably notice eventually when calling some other builtin that doesn't supportSUPER_PREFIX, but it seems hacky to count on that.
Quoted 7 lines
There's also the question of 3rd-party programs. If we want to be conservative, I think you'd want to just always bail in
execv_dashed_external()if --super-prefix is in use. That doesn't give an option for scripts to say "hey, I support this", but we can perhaps worry about loosening later.-Peff
That makes sense.
Signature · 1 line
Brandon Williams
PATCH v5 2/4 ls-files: optionally recurse into submodules
In reply to Brandon WilliamsAllow ls-files to recognize submodules in order to retrieve a list of files from a repository's submodules. This is done by forking off a process to recursively call ls-files on all submodules. Use top-level --super-prefix option to pass a path to the submodule which it can use to prepend to output or pathspec matching logic.
--- Documentation/git-ls-files.txt | 7 +- builtin/ls-files.c | 139 ++++++++++++++++++++++++--------- git.c | 2 +- t/t3007-ls-files-recurse-submodules.sh | 100 ++++++++++++++++++++++++ 4 files changed, 208 insertions(+), 40 deletions(-) create mode 100755 t/t3007-ls-files-recurse-submodules.sh
Documentation/git-ls-files.txt +6 -1
@@ -18,7 +18,8 @@ SYNOPSIS [--exclude-per-directory=<file>] [--exclude-standard] [--error-unmatch] [--with-tree=<tree-ish>] - [--full-name] [--abbrev] [--] [<file>...] + [--full-name] [--recurse-submodules] + [--abbrev] [--] [<file>...] DESCRIPTION -----------
@@ -137,6 +138,10 @@ a space) at the start of each line: option forces paths to be output relative to the project top directory. +--recurse-submodules:: + Recursively calls ls-files on each submodule in the repository. + Currently there is only support for the --cached mode. + --abbrev[=<n>]:: Instead of showing the full 40-byte hexadecimal object lines, show only a partial prefix.
builtin/ls-files.c +101 -38
194 lines not on this page · raw message
git.c +1 -1
@@ -443,7 +443,7 @@ static struct cmd_struct commands[] = { { "init-db", cmd_init_db }, { "interpret-trailers", cmd_interpret_trailers, RUN_SETUP_GENTLY }, { "log", cmd_log, RUN_SETUP }, - { "ls-files", cmd_ls_files, RUN_SETUP }, + { "ls-files", cmd_ls_files, RUN_SETUP | SUPPORT_SUPER_PREFIX }, { "ls-remote", cmd_ls_remote, RUN_SETUP_GENTLY }, { "ls-tree", cmd_ls_tree, RUN_SETUP }, { "mailinfo", cmd_mailinfo },
t/t3007-ls-files-recurse-submodules.sh new file +100 -0
100 lines not on this page · raw message
Signature · 1 line
2.8.0.rc3.226.g39d4020
Re: [PATCH v5 2/4] ls-files: optionally recurse into submodules
Quoted 64 lines · Brandon Williams
Allow ls-files to recognize submodules in order to retrieve a list of files from a repository's submodules. This is done by forking off a process to recursively call ls-files on all submodules. Use top-level --super-prefix option to pass a path to the submodule which it can use to prepend to output or pathspec matching logic.
Signed-off-by Brandon Williams <[email protected]>--- Documentation/git-ls-files.txt | 7 +- builtin/ls-files.c | 139 ++++++++++++++++++++++++--------- git.c | 2 +- t/t3007-ls-files-recurse-submodules.sh | 100 ++++++++++++++++++++++++ 4 files changed, 208 insertions(+), 40 deletions(-) create mode 100755 t/t3007-ls-files-recurse-submodules.shDocumentation/git-ls-files.txt +6
-1@@ -18,7 +18,8 @@ SYNOPSIS [--exclude-per-directory=<file>] [--exclude-standard] [--error-unmatch] [--with-tree=<tree-ish>] - [--full-name] [--abbrev] [--] [<file>...] + [--full-name] [--recurse-submodules] + [--abbrev] [--] [<file>...] DESCRIPTION -----------@@ -137,6 +138,10 @@ a space) at the start of each line: option forces paths to be output relative to the project top directory. +--recurse-submodules:: + Recursively calls ls-files on each submodule in the repository. + Currently there is only support for the --cached mode. + --abbrev[=<n>]:: Instead of showing the full 40-byte hexadecimal object lines, show only a partial prefix.builtin/ls-files.c +3
-0@@ -14,6 +14,7 @@#include "resolve-undo.h" #include "string-list.h" #include "pathspec.h" +#include "run-command.h" static int abbrev;static int show_deleted;@@ -28,8 +29,10 @@ static int show_valid_bit;static int line_terminator ='\n' ;static int debug_mode;static int show_eol; +static int recurse_submodules;static const char *prefix; +static const char *super_prefix;static int max_prefix_len;static int prefix_len;static struct pathspec pathspec;
builtin/ls-files.c static void write_eolinfo(const struct cache_entry *ce, const char *path) · line 77
Quoted 7 lines
builtin/ls-files.c +4
-0@@ -68,6 +71,19 @@ static void write_eolinfo(const struct cache_entry *ce, const char *path)static void write_name (const char *name) {/* +* Prepend the super_prefix to name to construct the full_name to be +* written. 'full_name' gets reused across output lines to minimize the +* allocation churn. +*/
When doing these tricks with the allocation churn (i.e. we make it hard to read and understand
for a reader, then we should do it completely, i.e. keep full_name in the strbuf and
only do a strbuf_setlen to reset the buffer just a bit. With this implementation we burden the
reader/user to understand how the memory is kept over multiple calls to this function,
but we still do more work than expected). So either I'd not worry about performance
and provide an 'obvious correct' implementation, with e.g. no static here and we free the memory
correctly. Or you'd go the performance route, but then we'd usually ask for numbers.
(How much faster is it; Does the trickyness trade off well to the performance gain?)
builtin/ls-files.c static void write_eolinfo(const struct cache_entry *ce, const char *path) · line 79
Quoted 2 lines
builtin/ls-files.c +2
-0+static struct strbuf full_name =STRBUF_INIT ; +if (super_prefix && *super_prefix) {
Why do we have to check twice here? Wouldn't just
if (super_prefix) {
...be enough?
Re: [PATCH v5 2/4] ls-files: optionally recurse into submodules
In reply to Brandon WilliamsDocumentation/git-ls-files.txt a space) at the start of each line: · line 143
Quoted 3 lines · Brandon Williams
+--recurse-submodules::
Documentation/git-ls-files.txt +2
-0+ Recursively calls ls-files on each submodule in the repository. + Currently there is only support for the --cached mode.
Good to describe it, but at this step, there is only support for the "--cached" mode and it does not take a pathspec.
Also as the series progresses, I would expect this to be updated in patches $n/4 (2 < $n).
builtin/ls-files.c int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix) · line 583
Quoted 3 lines
builtin/ls-files.c +3
-0+if (recurse_submodules && argc) +die ("ls-files --recurse-submodules does not support path " +"arguments" );
Hmm, s/path arguments/pathspec/, perhaps, as the latter is specified in the glossary?
PATCH v5 3/4 ls-files: pass through safe options for --recurse-submodules
In reply to Brandon WilliamsPass through some known-safe options when recursing into submodules. (--cached, --stage, -v, -t, -z, --debug, --eol)
Other options are compiled into an argv_array but if an unsafe option is given the caller will be errored out.
--- builtin/ls-files.c | 51 ++++++++++++++++++++++++++++++++-- t/t3007-ls-files-recurse-submodules.sh | 17 ++++++++---- 2 files changed, 60 insertions(+), 8 deletions(-)
builtin/ls-files.c +48 -3
@@ -30,6 +30,7 @@ static int line_terminator = '\n';static int debug_mode;static int show_eol;static int recurse_submodules; +static struct argv_array submodules_options =ARGV_ARRAY_INIT ;static const char *prefix;static const char *super_prefix;
@@ -168,6 +169,45 @@ static void show_killed_files(struct dir_struct *dir) } } +/* +* Compile an argv_array with all of the options supported by --recurse_submodules +*/ +static void compile_submodule_options (const struct dir_struct *dir,int show_tag) +{ +if (line_terminator =='\0' ) +argv_array_push (&submodules_options,"-z" ); +if (show_tag) +argv_array_push (&submodules_options,"-t" ); +if (show_valid_bit) +argv_array_push (&submodules_options,"-v" ); +if (show_cached) +argv_array_push (&submodules_options,"--cached" ); +if (show_deleted) +argv_array_push (&submodules_options,"--deleted" ); +if (show_modified) +argv_array_push (&submodules_options,"--modified" ); +if (show_others) +argv_array_push (&submodules_options,"--others" ); +if (dir->flags &DIR_SHOW_IGNORED ) +argv_array_push (&submodules_options,"--ignored" ); +if (show_stage) +argv_array_push (&submodules_options,"--stage" ); +if (show_killed) +argv_array_push (&submodules_options,"--killed" ); +if (dir->flags &DIR_SHOW_OTHER_DIRECTORIES ) +argv_array_push (&submodules_options,"--directory" ); +if (!(dir->flags &DIR_SHOW_OTHER_DIRECTORIES )) +argv_array_push (&submodules_options,"--empty-directory" ); +if (show_unmerged) +argv_array_push (&submodules_options,"--unmerged" ); +if (show_resolve_undo) +argv_array_push (&submodules_options,"--resolve-undo" ); +if (show_eol) +argv_array_push (&submodules_options,"--eol" ); +if (debug_mode) +argv_array_push (&submodules_options,"--debug" ); +} +/** * Recursively call ls-files on a submodule */
@@ -182,6 +222,9 @@ static void show_gitlink(const struct cache_entry *ce)argv_array_push (&cp.args,"ls-files" );argv_array_push (&cp.args,"--recurse-submodules" ); +/* add supported options */ +argv_array_pushv (&cp.args, submodules_options.argv); + cp.git_cmd =1 ; cp.dir = ce->name; status =run_command (&cp);
@@ -567,11 +610,13 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)if (require_work_tree && !is_inside_work_tree ())setup_work_tree (); +if (recurse_submodules) +compile_submodule_options (&dir, show_tag); +if (recurse_submodules && - (show_stage || show_deleted || show_others || show_unmerged || + (show_deleted || show_others || show_unmerged || show_killed || show_modified || show_resolve_undo || - show_valid_bit || show_tag || show_eol || with_tree || - (line_terminator =='\0' ))) + with_tree))die ("ls-files --recurse-submodules unsupported mode" );if (recurse_submodules && error_unmatch)
t/t3007-ls-files-recurse-submodules.sh +12 -5
@@ -34,6 +34,18 @@ test_expect_success 'ls-files correctly outputs files in submodule' ' test_cmp expect actual ' +test_expect_success 'ls-files correctly outputs files in submodule with -z' ' + lf_to_nul >expect <<-\EOF && + .gitmodules + a + b/b + submodule/c + EOF + + git ls-files --recurse-submodules -z >actual && + test_cmp expect actual +' + test_expect_success 'ls-files does not output files not added to a repo' ' cat >expect <<-\EOF && .gitmodules
@@ -86,15 +98,10 @@ test_incompatible_with_recurse_submodules () { " } -test_incompatible_with_recurse_submodules -z -test_incompatible_with_recurse_submodules -v -test_incompatible_with_recurse_submodules -t test_incompatible_with_recurse_submodules --deleted test_incompatible_with_recurse_submodules --modified test_incompatible_with_recurse_submodules --others -test_incompatible_with_recurse_submodules --stage test_incompatible_with_recurse_submodules --killed test_incompatible_with_recurse_submodules --unmerged -test_incompatible_with_recurse_submodules --eol test_done
Signature · 1 line
2.8.0.rc3.226.g39d4020
PATCH v5 4/4 ls-files: add pathspec matching for submodules
In reply to Brandon WilliamsPathspecs can be a bit tricky when trying to apply them to submodules. The main challenge is that the pathspecs will be with respect to the superproject and not with respect to paths in the submodule. The approach this patch takes is to pass in the identical pathspec from the superproject to the submodule in addition to the submodule-prefix, which is the path from the root of the superproject to the submodule, and then we can compare an entry in the submodule prepended with the submodule-prefix to the pathspec in order to determine if there is a match.
This patch also permits the pathspec logic to perform a prefix match against submodules since a pathspec could refer to a file inside of a submodule. Due to limitations in the wildmatch logic, a prefix match is only done literally. If any wildcard character is encountered we'll simply punt and produce a false positive match. More accurate matching will be done once inside the submodule. This is due to the superproject not knowing what files could exist in the submodule.
--- builtin/ls-files.c | 28 ++++++--- dir.c | 46 +++++++++++++- dir.h | 4 ++ t/t3007-ls-files-recurse-submodules.sh | 108 ++++++++++++++++++++++++++++++++- 4 files changed, 174 insertions(+), 12 deletions(-)
builtin/ls-files.c +21 -7
@@ -215,6 +215,7 @@ static void show_gitlink(const struct cache_entry *ce) {struct child_process cp =CHILD_PROCESS_INIT ;int status; +int i; argv_array_pushf(&cp.args, "--super-prefix=%s%s/", super_prefix ? super_prefix : "",
@@ -225,6 +226,15 @@ static void show_gitlink(const struct cache_entry *ce)/* add supported options */ argv_array_pushv (&cp.args, submodules_options.argv); +/* +* Pass in the original pathspec args. The submodule will be +* responsible for prepending the 'submodule_prefix' prior to comparing +* against the pathspec for matches. +*/ +argv_array_push (&cp.args,"--" ); +for (i =0 ; i < pathspec.nr; i++) +argv_array_push (&cp.args, pathspec.items[i].original); + cp.git_cmd =1 ; cp.dir = ce->name; status =run_command (&cp);
@@ -243,7 +253,8 @@ static void show_ce_entry(const char *tag, const struct cache_entry *ce)if (len >=ce_namelen (ce))die ("git ls-files: internal error - cache entry not superset of prefix" ); -if (recurse_submodules &&S_ISGITLINK (ce->ce_mode)) { +if (recurse_submodules &&S_ISGITLINK (ce->ce_mode) && +submodule_path_match (&pathspec, name.buf, ps_matched)) {show_gitlink (ce); } else if (match_pathspec(&pathspec, name.buf, name.len, len, ps_matched,
@@ -623,17 +634,20 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)die ("ls-files --recurse-submodules does not support " "--error-unmatch" ); -if (recurse_submodules && argc) -die ("ls-files --recurse-submodules does not support path " -"arguments" ); -parse_pathspec (&pathspec,0 ,PATHSPEC_PREFER_CWD |PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP , prefix, argv); -/* Find common prefix for all pathspec's */ - max_prefix =common_prefix (&pathspec); +/* +* Find common prefix for all pathspec's +* This is used as a performance optimization which unfortunately cannot +* be done when recursing into submodules +*/ +if (recurse_submodules) + max_prefix =NULL ; +else + max_prefix =common_prefix (&pathspec); max_prefix_len = max_prefix ?strlen (max_prefix) :0 ;/* Treat unmatching pathspec elements as errors */
dir.c +44 -2
64 lines not on this page · raw message
dir.h +4 -0
@@ -304,6 +304,10 @@ extern int git_fnmatch(const struct pathspec_item *item, const char *pattern, const char *string, int prefix); +extern int submodule_path_match (const struct pathspec *ps, +const char *submodule_name, +char *seen); +static inline int ce_path_match (const struct cache_entry *ce,const struct pathspec *pathspec,char *seen)
t/t3007-ls-files-recurse-submodules.sh +105 -3
114 lines not on this page · raw message
Signature · 1 line
2.8.0.rc3.226.g39d4020
v6 cover recursive support for ls-files
In reply to Brandon WilliamsMinor fixes per the comments on version 5.
Brandon Williams (4):
git: make super-prefix option
ls-files: optionally recurse into submodules
ls-files: pass through safe options for --recurse-submodules
ls-files: add pathspec matching for submodules
Documentation/git-ls-files.txt | 7 +- Documentation/git.txt | 6 + builtin/ls-files.c | 202 ++++++++++++++++++++++++------- cache.h | 2 + dir.c | 46 +++++++- dir.h | 4 + environment.c | 10 ++ git.c | 28 ++++- t/t3007-ls-files-recurse-submodules.sh | 209 +++++++++++++++++++++++++++++++++ 9 files changed, 470 insertions(+), 44 deletions(-) create mode 100755 t/t3007-ls-files-recurse-submodules.sh
Signature · 1 line
2.10.0
PATCH v6 1/4 git: make super-prefix option
Add a super-prefix environment variable 'GIT_INTERNAL_SUPER_PREFIX' which can be used to specify a path from above a repository down to its root. When such a super-prefix is specified, the paths reported by Git are prefixed with it to make them relative to that directory "above". The paths given by the user on the command line
(e.g. "git subcmd --output-file=path/to/a/file" and pathspecs) are taken relative to the directory "above" to match.
The immediate use of this option is by commands which have a --recurse-submodule option in order to give context to submodules about how they were invoked. This option is currently only allowed for builtins which support a super-prefix.
--- Documentation/git.txt | 6 ++++++ cache.h | 2 ++ environment.c | 10 ++++++++++ git.c | 26 ++++++++++++++++++++++++++ 4 files changed, 44 insertions(+)
Documentation/git.txt +6 -0
@@ -13,6 +13,7 @@ SYNOPSIS [--exec-path[=<path>]] [--html-path] [--man-path] [--info-path] [-p|--paginate|--no-pager] [--no-replace-objects] [--bare] [--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>] + [--super-prefix=<path>] <command> [<args>] DESCRIPTION
@@ -601,6 +602,11 @@ foo.bar= ...`) sets `foo.bar` to the empty string. details. Equivalent to setting the `GIT_NAMESPACE` environment variable. +--super-prefix=<path>:: + Currently for internal use only. Set a prefix which gives a path from + above a repository down to its root. One use is to give submodules + context about the superproject that invoked it. + --bare:: Treat the repository as a bare repository. If GIT_DIR environment is not set, it is set to the current working
cache.h +2 -0
@@ -408,6 +408,7 @@ static inline enum object_type object_type(unsigned int mode)#define GIT_NAMESPACE_ENVIRONMENT "GIT_NAMESPACE"#define GIT_WORK_TREE_ENVIRONMENT "GIT_WORK_TREE"#define GIT_PREFIX_ENVIRONMENT "GIT_PREFIX" +#define GIT_SUPER_PREFIX_ENVIRONMENT "GIT_INTERNAL_SUPER_PREFIX"#define DEFAULT_GIT_DIR_ENVIRONMENT ".git"#define DB_ENVIRONMENT "GIT_OBJECT_DIRECTORY"#define INDEX_ENVIRONMENT "GIT_INDEX_FILE"
@@ -468,6 +469,7 @@ extern int get_common_dir_noenv(struct strbuf *sb, const char *gitdir);extern int get_common_dir (struct strbuf *sb,const char *gitdir);extern const char *get_git_namespace (void );extern const char *strip_namespace (const char *namespaced_ref); +extern const char *get_super_prefix (void );extern const char *get_git_work_tree (void );/*
environment.c +10 -0
@@ -100,6 +100,8 @@ static char *work_tree;static const char *namespace;static size_t namespace_len; +static const char *super_prefix; +static const char *git_dir, *git_common_dir;static char *git_object_dir, *git_index_file, *git_graft_file;int git_db_env, git_index_env, git_graft_env, git_common_dir_env;
@@ -120,6 +122,7 @@ const char * const local_repo_env[] = {NO_REPLACE_OBJECTS_ENVIRONMENT ,GIT_REPLACE_REF_BASE_ENVIRONMENT ,GIT_PREFIX_ENVIRONMENT , +GIT_SUPER_PREFIX_ENVIRONMENT ,GIT_SHALLOW_FILE_ENVIRONMENT ,GIT_COMMON_DIR_ENVIRONMENT ,NULL
@@ -222,6 +225,13 @@ const char *strip_namespace(const char *namespaced_ref)return namespaced_ref + namespace_len; } +const char *get_super_prefix (void ) +{ +if (!super_prefix) + super_prefix =getenv (GIT_SUPER_PREFIX_ENVIRONMENT ); +return super_prefix; +} +static int git_work_tree_initialized;/*
git.c +26 -0
@@ -164,6 +164,20 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)setenv (GIT_WORK_TREE_ENVIRONMENT , cmd,1 );if (envchanged) *envchanged =1 ; + }else if (!strcmp (cmd,"--super-prefix" )) { +if (*argc <2 ) { +fprintf (stderr,"No prefix given for --super-prefix.\n" ); +usage (git_usage_string); + } +setenv (GIT_SUPER_PREFIX_ENVIRONMENT , (*argv)[1 ],1 ); +if (envchanged) + *envchanged =1 ; + (*argv)++; + (*argc)--; + }else if (skip_prefix (cmd,"--super-prefix=" , &cmd)) { +setenv (GIT_SUPER_PREFIX_ENVIRONMENT , cmd,1 ); +if (envchanged) + *envchanged =1 ; }else if (!strcmp (cmd,"--bare" )) {char *cwd =xgetcwd (); is_bare_repository_cfg =1 ;
@@ -310,6 +324,7 @@ static int handle_alias(int *argcp, const char ***argv)* RUN_SETUP for reading from the configuration file. */ #define NEED_WORK_TREE (1<<3) +#define SUPPORT_SUPER_PREFIX (1<<4)struct cmd_struct {const char *cmd;
@@ -344,6 +359,13 @@ static int run_builtin(struct cmd_struct *p, int argc, const char **argv) }commit_pager_choice (); +if (!help &&get_super_prefix ()) { +if (!(p->option &SUPPORT_SUPER_PREFIX )) +die ("%s doesn't support --super-prefix" , p->cmd); +if (prefix) +die ("can't use --super-prefix from a subdirectory" ); + } +if (!help && p->option &NEED_WORK_TREE )setup_work_tree ();
@@ -558,6 +580,10 @@ static void execv_dashed_external(const char **argv)const char *tmp;int status; +if (get_super_prefix ()) { +die ("%s doesn't support --super-prefix" , argv[0 ]); + } +if (use_pager ==-1 ) use_pager =check_pager_config (argv[0 ]);commit_pager_choice ();
Signature · 1 line
2.10.0
Re: [PATCH v6 1/4] git: make super-prefix option
environment.c const char *strip_namespace(const char *namespaced_ref) · line 234
Quoted 8 lines · Brandon Williams
environment.c +7
-0+const char *get_super_prefix (void ) +{ +if (!super_prefix) + super_prefix =getenv (GIT_SUPER_PREFIX_ENVIRONMENT ); +return super_prefix; +} +
As said earlier, is the following a valid thought:
git.c static void execv_dashed_external(const char **argv) · line 586
Quoted 12 lines
The
getenv()function returns a pointer to the value in the environment, or NULL if there is no match.
So in case this is not set (when e.g. the user did not specify the super prefix), we would probe it a couple of times.
The caching effect only occurs when the string is set. So this looks like we save repetitive calls, but we do not always do that.git.c +4
-0+if (get_super_prefix ()) { +die ("%s doesn't support --super-prefix" , argv[0 ]); + } +
Nit: no braces, please.
Re: [PATCH v6 1/4] git: make super-prefix option
Quoted 19 lines · Stefan Beller
environment.c const char *strip_namespace(const char *namespaced_ref) · line 234
Quoted 8 lines · Brandon Williams
environment.c +7
-0+const char *get_super_prefix (void ) +{ +if (!super_prefix) + super_prefix =getenv (GIT_SUPER_PREFIX_ENVIRONMENT ); +return super_prefix; +} +As said earlier, is the following a valid thought:
Quoted 6 lines
The
getenv()function returns a pointer to the value in the environment, or NULL if there is no match.
So in case this is not set (when e.g. the user did not specify the super prefix), we would probe it a couple of times.
The caching effect only occurs when the string is set. So this looks like we save repetitive calls, but we do not always do that.
That reading is correct. If the code wants to do the caching, it should do so correctly. Unless we expect that some callers might want to be able to invalidate the cache,
get_super_prefix(void)
{
static int initialized;
if (!initialized) {
super_prefix = getenv(...);
initialized = 1;
}
return super_prefix;
} would suffice.
Thanks for careful reading.
Re: [PATCH v6 1/4] git: make super-prefix option
In reply to Stefan BellerQuoted 19 lines · Stefan Beller
environment.c const char *strip_namespace(const char *namespaced_ref) · line 234
Quoted 8 lines · Brandon Williams
environment.c +7
-0+const char *get_super_prefix (void ) +{ +if (!super_prefix) + super_prefix =getenv (GIT_SUPER_PREFIX_ENVIRONMENT ); +return super_prefix; +} +As said earlier, is the following a valid thought:
Quoted 6 lines
The
getenv()function returns a pointer to the value in the environment, or NULL if there is no match.
So in case this is not set (when e.g. the user did not specify the super prefix), we would probe it a couple of times.
The caching effect only occurs when the string is set. So this looks like we save repetitive calls, but we do not always do that.
I think your concern is valid. If it is not set, we will do an O(n) search through the whole environment on each call.
I also think the result of getenv() needs to be copied. In some implementations it persists for the life of the program, but that's not guaranteed; it may be overwritten by unrelated calls to getenv() or setenv().
-Peff
PATCH v6 2/4 ls-files: optionally recurse into submodules
In reply to Brandon WilliamsAllow ls-files to recognize submodules in order to retrieve a list of files from a repository's submodules. This is done by forking off a process to recursively call ls-files on all submodules. Use top-level --super-prefix option to pass a path to the submodule which it can use to prepend to output or pathspec matching logic.
--- Documentation/git-ls-files.txt | 8 +- builtin/ls-files.c | 138 ++++++++++++++++++++++++--------- git.c | 2 +- t/t3007-ls-files-recurse-submodules.sh | 100 ++++++++++++++++++++++++ 4 files changed, 208 insertions(+), 40 deletions(-) create mode 100755 t/t3007-ls-files-recurse-submodules.sh
Documentation/git-ls-files.txt +7 -1
@@ -18,7 +18,8 @@ SYNOPSIS [--exclude-per-directory=<file>] [--exclude-standard] [--error-unmatch] [--with-tree=<tree-ish>] - [--full-name] [--abbrev] [--] [<file>...] + [--full-name] [--recurse-submodules] + [--abbrev] [--] [<file>...] DESCRIPTION -----------
@@ -137,6 +138,11 @@ a space) at the start of each line: option forces paths to be output relative to the project top directory. +--recurse-submodules:: + Recursively calls ls-files on each submodule in the repository. + Currently there is only support for the --cached mode without a + pathspec. + --abbrev[=<n>]:: Instead of showing the full 40-byte hexadecimal object lines, show only a partial prefix.
builtin/ls-files.c +100 -38
198 lines not on this page · raw message
git.c +1 -1
@@ -443,7 +443,7 @@ static struct cmd_struct commands[] = { { "init-db", cmd_init_db }, { "interpret-trailers", cmd_interpret_trailers, RUN_SETUP_GENTLY }, { "log", cmd_log, RUN_SETUP }, - { "ls-files", cmd_ls_files, RUN_SETUP }, + { "ls-files", cmd_ls_files, RUN_SETUP | SUPPORT_SUPER_PREFIX }, { "ls-remote", cmd_ls_remote, RUN_SETUP_GENTLY }, { "ls-tree", cmd_ls_tree, RUN_SETUP }, { "mailinfo", cmd_mailinfo },
t/t3007-ls-files-recurse-submodules.sh new file +100 -0
100 lines not on this page · raw message
Signature · 1 line
2.10.0
PATCH v6 3/4 ls-files: pass through safe options for --recurse-submodules
In reply to Brandon WilliamsPass through some known-safe options when recursing into submodules. (--cached, --stage, -v, -t, -z, --debug, --eol)
Other options are compiled into an argv_array but if an unsafe option is given the caller will be errored out.
--- builtin/ls-files.c | 51 ++++++++++++++++++++++++++++++++-- t/t3007-ls-files-recurse-submodules.sh | 17 ++++++++---- 2 files changed, 60 insertions(+), 8 deletions(-)
builtin/ls-files.c +48 -3
@@ -30,6 +30,7 @@ static int line_terminator = '\n';static int debug_mode;static int show_eol;static int recurse_submodules; +static struct argv_array submodules_options =ARGV_ARRAY_INIT ;static const char *prefix;static const char *super_prefix;
@@ -168,6 +169,45 @@ static void show_killed_files(struct dir_struct *dir) } } +/* +* Compile an argv_array with all of the options supported by --recurse_submodules +*/ +static void compile_submodule_options (const struct dir_struct *dir,int show_tag) +{ +if (line_terminator =='\0' ) +argv_array_push (&submodules_options,"-z" ); +if (show_tag) +argv_array_push (&submodules_options,"-t" ); +if (show_valid_bit) +argv_array_push (&submodules_options,"-v" ); +if (show_cached) +argv_array_push (&submodules_options,"--cached" ); +if (show_deleted) +argv_array_push (&submodules_options,"--deleted" ); +if (show_modified) +argv_array_push (&submodules_options,"--modified" ); +if (show_others) +argv_array_push (&submodules_options,"--others" ); +if (dir->flags &DIR_SHOW_IGNORED ) +argv_array_push (&submodules_options,"--ignored" ); +if (show_stage) +argv_array_push (&submodules_options,"--stage" ); +if (show_killed) +argv_array_push (&submodules_options,"--killed" ); +if (dir->flags &DIR_SHOW_OTHER_DIRECTORIES ) +argv_array_push (&submodules_options,"--directory" ); +if (!(dir->flags &DIR_SHOW_OTHER_DIRECTORIES )) +argv_array_push (&submodules_options,"--empty-directory" ); +if (show_unmerged) +argv_array_push (&submodules_options,"--unmerged" ); +if (show_resolve_undo) +argv_array_push (&submodules_options,"--resolve-undo" ); +if (show_eol) +argv_array_push (&submodules_options,"--eol" ); +if (debug_mode) +argv_array_push (&submodules_options,"--debug" ); +} +/** * Recursively call ls-files on a submodule */
@@ -182,6 +222,9 @@ static void show_gitlink(const struct cache_entry *ce)argv_array_push (&cp.args,"ls-files" );argv_array_push (&cp.args,"--recurse-submodules" ); +/* add supported options */ +argv_array_pushv (&cp.args, submodules_options.argv); + cp.git_cmd =1 ; cp.dir = ce->name; status =run_command (&cp);
@@ -567,11 +610,13 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)if (require_work_tree && !is_inside_work_tree ())setup_work_tree (); +if (recurse_submodules) +compile_submodule_options (&dir, show_tag); +if (recurse_submodules && - (show_stage || show_deleted || show_others || show_unmerged || + (show_deleted || show_others || show_unmerged || show_killed || show_modified || show_resolve_undo || - show_valid_bit || show_tag || show_eol || with_tree || - (line_terminator =='\0' ))) + with_tree))die ("ls-files --recurse-submodules unsupported mode" );if (recurse_submodules && error_unmatch)
t/t3007-ls-files-recurse-submodules.sh +12 -5
@@ -34,6 +34,18 @@ test_expect_success 'ls-files correctly outputs files in submodule' ' test_cmp expect actual ' +test_expect_success 'ls-files correctly outputs files in submodule with -z' ' + lf_to_nul >expect <<-\EOF && + .gitmodules + a + b/b + submodule/c + EOF + + git ls-files --recurse-submodules -z >actual && + test_cmp expect actual +' + test_expect_success 'ls-files does not output files not added to a repo' ' cat >expect <<-\EOF && .gitmodules
@@ -86,15 +98,10 @@ test_incompatible_with_recurse_submodules () { " } -test_incompatible_with_recurse_submodules -z -test_incompatible_with_recurse_submodules -v -test_incompatible_with_recurse_submodules -t test_incompatible_with_recurse_submodules --deleted test_incompatible_with_recurse_submodules --modified test_incompatible_with_recurse_submodules --others -test_incompatible_with_recurse_submodules --stage test_incompatible_with_recurse_submodules --killed test_incompatible_with_recurse_submodules --unmerged -test_incompatible_with_recurse_submodules --eol test_done
Signature · 1 line
2.10.0
Re: [PATCH v6 3/4] ls-files: pass through safe options for --recurse-submodules
Quoted 25 lines · Brandon Williams
builtin/ls-files.c +25
-0+static void compile_submodule_options (const struct dir_struct *dir,int show_tag) +{ +if (line_terminator =='\0' ) +argv_array_push (&submodules_options,"-z" ); +if (show_tag) +argv_array_push (&submodules_options,"-t" ); +if (show_valid_bit) +argv_array_push (&submodules_options,"-v" ); +if (show_cached) +argv_array_push (&submodules_options,"--cached" ); +if (show_deleted) +argv_array_push (&submodules_options,"--deleted" ); +if (show_modified) +argv_array_push (&submodules_options,"--modified" ); +if (show_others) +argv_array_push (&submodules_options,"--others" ); +if (dir->flags &DIR_SHOW_IGNORED ) +argv_array_push (&submodules_options,"--ignored" ); +if (show_stage) +argv_array_push (&submodules_options,"--stage" ); +if (show_killed) +argv_array_push (&submodules_options,"--killed" ); +if (dir->flags &DIR_SHOW_OTHER_DIRECTORIES ) +argv_array_push (&submodules_options,"--directory" ); +if (!(dir->flags &DIR_SHOW_OTHER_DIRECTORIES ))
builtin/ls-files.c static void show_killed_files(struct dir_struct *dir) · line 209
Quoted 10 lines
builtin/ls-files.c +10
-0+argv_array_push (&submodules_options,"--empty-directory" ); +if (show_unmerged) +argv_array_push (&submodules_options,"--unmerged" ); +if (show_resolve_undo) +argv_array_push (&submodules_options,"--resolve-undo" ); +if (show_eol) +argv_array_push (&submodules_options,"--eol" ); +if (debug_mode) +argv_array_push (&submodules_options,"--debug" ); +}
With this and 4/4 applied, the documentation still says "--cached" is the only supported option.
Does it really make sense to pass all of these? I understand "-z" and I suspect things like "-t" and "-v" that affect "how" things are shown may also happen to work, but I am not sure how much it makes sense for options that affect "what" things are shown.
What does it even mean to ask for say "--unmerged" to be shown, for example, from the superproject? Recurse into submodules whose cache entries in the index of the superproject are unmerged, or something else?
I am inclined to say that it is probably better to keep the "--cached only" as documented, at least on the "what are shown" side.
Thanks.
Re: [PATCH v6 3/4] ls-files: pass through safe options for --recurse-submodules
Quoted 51 lines · Junio C Hamano
Quoted 25 lines · Brandon Williams
builtin/ls-files.c +25
-0+static void compile_submodule_options (const struct dir_struct *dir,int show_tag) +{ +if (line_terminator =='\0' ) +argv_array_push (&submodules_options,"-z" ); +if (show_tag) +argv_array_push (&submodules_options,"-t" ); +if (show_valid_bit) +argv_array_push (&submodules_options,"-v" ); +if (show_cached) +argv_array_push (&submodules_options,"--cached" ); +if (show_deleted) +argv_array_push (&submodules_options,"--deleted" ); +if (show_modified) +argv_array_push (&submodules_options,"--modified" ); +if (show_others) +argv_array_push (&submodules_options,"--others" ); +if (dir->flags &DIR_SHOW_IGNORED ) +argv_array_push (&submodules_options,"--ignored" ); +if (show_stage) +argv_array_push (&submodules_options,"--stage" ); +if (show_killed) +argv_array_push (&submodules_options,"--killed" ); +if (dir->flags &DIR_SHOW_OTHER_DIRECTORIES ) +argv_array_push (&submodules_options,"--directory" ); +if (!(dir->flags &DIR_SHOW_OTHER_DIRECTORIES ))builtin/ls-files.c static void show_killed_files(struct dir_struct *dir) · line 209
Quoted 10 lines
builtin/ls-files.c +10
-0+argv_array_push (&submodules_options,"--empty-directory" ); +if (show_unmerged) +argv_array_push (&submodules_options,"--unmerged" ); +if (show_resolve_undo) +argv_array_push (&submodules_options,"--resolve-undo" ); +if (show_eol) +argv_array_push (&submodules_options,"--eol" ); +if (debug_mode) +argv_array_push (&submodules_options,"--debug" ); +}With this and 4/4 applied, the documentation still says "--cached" is the only supported option.
Does it really make sense to pass all of these? I understand "-z" and I suspect things like "-t" and "-v" that affect "how" things are shown may also happen to work, but I am not sure how much it makes sense for options that affect "what" things are shown.
What does it even mean to ask for say "--unmerged" to be shown, for example, from the superproject? Recurse into submodules whose cache entries in the index of the superproject are unmerged, or something else?
Quoted 5 lines
I am inclined to say that it is probably better to keep the "--cached only" as documented, at least on the "what are shown" side.
Thanks.
You're right that probably makes the most sense for now.
Signature · 1 line
Brandon Williams
Re: [PATCH v6 3/4] ls-files: pass through safe options for --recurse-submodules
Pass through some known-safe options when recursing into submodules. (--cached, -v, -t, -z, --debug, --eol)
If other unsafe options are given the caller will be errored out.
---
Something more like this correct? I ditched the extra parameters and reworded the commit msg to reflect this.
builtin/ls-files.c | 30 +++++++++++++++++++++++++++--- t/t3007-ls-files-recurse-submodules.sh | 16 ++++++++++++---- 2 files changed, 39 insertions(+), 7 deletions(-)
builtin/ls-files.c +27 -3
@@ -30,6 +30,7 @@ static int line_terminator = '\n';static int debug_mode;static int show_eol;static int recurse_submodules; +static struct argv_array submodules_options =ARGV_ARRAY_INIT ;static const char *prefix;static const char *super_prefix;
@@ -168,6 +169,25 @@ static void show_killed_files(struct dir_struct *dir) } } +/* +* Compile an argv_array with all of the options supported by --recurse_submodules +*/ +static void compile_submodule_options (const struct dir_struct *dir,int show_tag) +{ +if (line_terminator =='\0' ) +argv_array_push (&submodules_options,"-z" ); +if (show_tag) +argv_array_push (&submodules_options,"-t" ); +if (show_valid_bit) +argv_array_push (&submodules_options,"-v" ); +if (show_cached) +argv_array_push (&submodules_options,"--cached" ); +if (show_eol) +argv_array_push (&submodules_options,"--eol" ); +if (debug_mode) +argv_array_push (&submodules_options,"--debug" ); +} +/** * Recursively call ls-files on a submodule */
@@ -182,6 +202,9 @@ static void show_gitlink(const struct cache_entry *ce)argv_array_push (&cp.args,"ls-files" );argv_array_push (&cp.args,"--recurse-submodules" ); +/* add supported options */ +argv_array_pushv (&cp.args, submodules_options.argv); + cp.git_cmd =1 ; cp.dir = ce->name; status =run_command (&cp);
@@ -567,11 +590,12 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)if (require_work_tree && !is_inside_work_tree ())setup_work_tree (); +if (recurse_submodules) +compile_submodule_options (&dir, show_tag); +if (recurse_submodules && (show_stage || show_deleted || show_others || show_unmerged || - show_killed || show_modified || show_resolve_undo || - show_valid_bit || show_tag || show_eol || with_tree || - (line_terminator =='\0' ))) + show_killed || show_modified || show_resolve_undo || with_tree))die ("ls-files --recurse-submodules unsupported mode" );if (recurse_submodules && error_unmatch)
t/t3007-ls-files-recurse-submodules.sh +12 -4
@@ -34,6 +34,18 @@ test_expect_success 'ls-files correctly outputs files in submodule' ' test_cmp expect actual ' +test_expect_success 'ls-files correctly outputs files in submodule with -z' ' + lf_to_nul >expect <<-\EOF && + .gitmodules + a + b/b + submodule/c + EOF + + git ls-files --recurse-submodules -z >actual && + test_cmp expect actual +' + test_expect_success 'ls-files does not output files not added to a repo' ' cat >expect <<-\EOF && .gitmodules
@@ -86,15 +98,11 @@ test_incompatible_with_recurse_submodules () { " } -test_incompatible_with_recurse_submodules -z -test_incompatible_with_recurse_submodules -v -test_incompatible_with_recurse_submodules -t test_incompatible_with_recurse_submodules --deleted test_incompatible_with_recurse_submodules --modified test_incompatible_with_recurse_submodules --others test_incompatible_with_recurse_submodules --stage test_incompatible_with_recurse_submodules --killed test_incompatible_with_recurse_submodules --unmerged -test_incompatible_with_recurse_submodules --eol test_done
Signature · 1 line
2.10.0
PATCH v6 4/4 ls-files: add pathspec matching for submodules
In reply to Brandon WilliamsPathspecs can be a bit tricky when trying to apply them to submodules. The main challenge is that the pathspecs will be with respect to the superproject and not with respect to paths in the submodule. The approach this patch takes is to pass in the identical pathspec from the superproject to the submodule in addition to the submodule-prefix, which is the path from the root of the superproject to the submodule, and then we can compare an entry in the submodule prepended with the submodule-prefix to the pathspec in order to determine if there is a match.
This patch also permits the pathspec logic to perform a prefix match against submodules since a pathspec could refer to a file inside of a submodule. Due to limitations in the wildmatch logic, a prefix match is only done literally. If any wildcard character is encountered we'll simply punt and produce a false positive match. More accurate matching will be done once inside the submodule. This is due to the superproject not knowing what files could exist in the submodule.
--- Documentation/git-ls-files.txt | 3 +- builtin/ls-files.c | 27 +++++++-- dir.c | 46 +++++++++++++- dir.h | 4 ++ t/t3007-ls-files-recurse-submodules.sh | 108 ++++++++++++++++++++++++++++++++- 5 files changed, 175 insertions(+), 13 deletions(-)
Documentation/git-ls-files.txt +1 -2
@@ -140,8 +140,7 @@ a space) at the start of each line: --recurse-submodules:: Recursively calls ls-files on each submodule in the repository. - Currently there is only support for the --cached mode without a - pathspec. + Currently there is only support for the --cached. --abbrev[=<n>]:: Instead of showing the full 40-byte hexadecimal object
builtin/ls-files.c +21 -6
@@ -215,6 +215,7 @@ static void show_gitlink(const struct cache_entry *ce) {struct child_process cp =CHILD_PROCESS_INIT ;int status; +int i; argv_array_pushf(&cp.args, "--super-prefix=%s%s/", super_prefix ? super_prefix : "",
@@ -225,6 +226,15 @@ static void show_gitlink(const struct cache_entry *ce)/* add supported options */ argv_array_pushv (&cp.args, submodules_options.argv); +/* +* Pass in the original pathspec args. The submodule will be +* responsible for prepending the 'submodule_prefix' prior to comparing +* against the pathspec for matches. +*/ +argv_array_push (&cp.args,"--" ); +for (i =0 ; i < pathspec.nr; i++) +argv_array_push (&cp.args, pathspec.items[i].original); + cp.git_cmd =1 ; cp.dir = ce->name; status =run_command (&cp);
@@ -243,7 +253,8 @@ static void show_ce_entry(const char *tag, const struct cache_entry *ce)if (len >=ce_namelen (ce))die ("git ls-files: internal error - cache entry not superset of prefix" ); -if (recurse_submodules &&S_ISGITLINK (ce->ce_mode)) { +if (recurse_submodules &&S_ISGITLINK (ce->ce_mode) && +submodule_path_match (&pathspec, name.buf, ps_matched)) {show_gitlink (ce); } else if (match_pathspec(&pathspec, name.buf, name.len, len, ps_matched,
@@ -623,16 +634,20 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)die ("ls-files --recurse-submodules does not support " "--error-unmatch" ); -if (recurse_submodules && argc) -die ("ls-files --recurse-submodules does not support pathspec" ); -parse_pathspec (&pathspec,0 ,PATHSPEC_PREFER_CWD |PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP , prefix, argv); -/* Find common prefix for all pathspec's */ - max_prefix =common_prefix (&pathspec); +/* +* Find common prefix for all pathspec's +* This is used as a performance optimization which unfortunately cannot +* be done when recursing into submodules +*/ +if (recurse_submodules) + max_prefix =NULL ; +else + max_prefix =common_prefix (&pathspec); max_prefix_len = max_prefix ?strlen (max_prefix) :0 ;/* Treat unmatching pathspec elements as errors */
dir.c +44 -2
64 lines not on this page · raw message
dir.h +4 -0
@@ -304,6 +304,10 @@ extern int git_fnmatch(const struct pathspec_item *item, const char *pattern, const char *string, int prefix); +extern int submodule_path_match (const struct pathspec *ps, +const char *submodule_name, +char *seen); +static inline int ce_path_match (const struct cache_entry *ce,const struct pathspec *pathspec,char *seen)
t/t3007-ls-files-recurse-submodules.sh +105 -3
114 lines not on this page · raw message
Signature · 1 line
2.10.0
Re: [PATCH v6 4/4] ls-files: add pathspec matching for submodules
Quoted 32 lines · Brandon Williams
Pathspecs can be a bit tricky when trying to apply them to submodules. The main challenge is that the pathspecs will be with respect to the superproject and not with respect to paths in the submodule. The approach this patch takes is to pass in the identical pathspec from the superproject to the submodule in addition to the submodule-prefix, which is the path from the root of the superproject to the submodule, and then we can compare an entry in the submodule prepended with the submodule-prefix to the pathspec in order to determine if there is a match.
This patch also permits the pathspec logic to perform a prefix match against submodules since a pathspec could refer to a file inside of a submodule. Due to limitations in the wildmatch logic, a prefix match is only done literally. If any wildcard character is encountered we'll simply punt and produce a false positive match. More accurate matching will be done once inside the submodule. This is due to the superproject not knowing what files could exist in the submodule.
Signed-off-by Brandon Williams <[email protected]>--- Documentation/git-ls-files.txt | 3 +- builtin/ls-files.c | 27 +++++++-- dir.c | 46 +++++++++++++- dir.h | 4 ++ t/t3007-ls-files-recurse-submodules.sh | 108 ++++++++++++++++++++++++++++++++- 5 files changed, 175 insertions(+), 13 deletions(-)Documentation/git-ls-files.txt +0
-0
Documentation/git-ls-files.txt a space) at the start of each line: · line 143
Quoted 6 lines
Documentation/git-ls-files.txt +1
-2@@ -140,8 +140,7 @@ a space) at the start of each line: --recurse-submodules:: Recursively calls ls-files on each submodule in the repository. - Currently there is only support for the --cached mode without a - pathspec. + Currently there is only support for the --cached.
s/--cached/--cached mode/ ?
The "the" in front of --cached sounds a bit strange for a non native speaker here.
Documentation/git-ls-files.txt a space) at the start of each line:
Quoted 9 lines
Documentation/git-ls-files.txt +9
-0+ /* + * Find common prefix for all pathspec's + * This is used as a performance optimization which unfortunately cannot + * be done when recursing into submodules + */ + if (recurse_submodules) + max_prefix = NULL; + else + max_prefix = common_prefix(&pathspec);
Nit of the day:
While this is readable, you may want to explore how this reads shorter as
max_prefix = recurse_submodules ? NULL : common_prefix(&pathspec);
?
Quoted 10 lines
Documentation/git-ls-files.txt +10
-0+ git ls-files --recurse-submodules "sub" >actual && + test_cmp expect actual && + git ls-files --recurse-submodules "sub/" >actual && + test_cmp expect actual && + git ls-files --recurse-submodules "sub/file" >actual && + test_cmp expect actual && + git ls-files --recurse-submodules "su*/file" >actual && + test_cmp expect actual && + git ls-files --recurse-submodules "su?/file" >actual && + test_cmp expect actual
Documentation/git-ls-files.txt
Quoted 10 lines
Documentation/git-ls-files.txt +8
-0+' + + git ls-files --recurse-submodules "s??/file" >actual && + test_cmp expect actual && + git ls-files --recurse-submodules "s???file" >actual && + test_cmp expect actual && + git ls-files --recurse-submodules "s*file" >actual && + test_cmp expect actual '
Thanks for the tests!
v7 cover recursive support for ls-files
In reply to Brandon WilliamsFew minor fixes pointed out Stefan
Brandon Williams (4):
git: make super-prefix option
ls-files: optionally recurse into submodules
ls-files: pass through safe options for --recurse-submodules
ls-files: add pathspec matching for submodules
Documentation/git-ls-files.txt | 7 +- Documentation/git.txt | 6 + builtin/ls-files.c | 181 +++++++++++++++++++++------- cache.h | 2 + dir.c | 46 +++++++- dir.h | 4 + environment.c | 13 ++ git.c | 27 ++++- t/t3007-ls-files-recurse-submodules.sh | 210 +++++++++++++++++++++++++++++++++ 9 files changed, 452 insertions(+), 44 deletions(-) create mode 100755 t/t3007-ls-files-recurse-submodules.sh
Signature · 1 line
2.10.0
PATCH v7 1/4 git: make super-prefix option
Add a super-prefix environment variable 'GIT_INTERNAL_SUPER_PREFIX' which can be used to specify a path from above a repository down to its root. When such a super-prefix is specified, the paths reported by Git are prefixed with it to make them relative to that directory "above". The paths given by the user on the command line
(e.g. "git subcmd --output-file=path/to/a/file" and pathspecs) are taken relative to the directory "above" to match.
The immediate use of this option is by commands which have a --recurse-submodule option in order to give context to submodules about how they were invoked. This option is currently only allowed for builtins which support a super-prefix.
--- Documentation/git.txt | 6 ++++++ cache.h | 2 ++ environment.c | 13 +++++++++++++ git.c | 25 +++++++++++++++++++++++++ 4 files changed, 46 insertions(+)
Documentation/git.txt +6 -0
@@ -13,6 +13,7 @@ SYNOPSIS [--exec-path[=<path>]] [--html-path] [--man-path] [--info-path] [-p|--paginate|--no-pager] [--no-replace-objects] [--bare] [--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>] + [--super-prefix=<path>] <command> [<args>] DESCRIPTION
@@ -601,6 +602,11 @@ foo.bar= ...`) sets `foo.bar` to the empty string. details. Equivalent to setting the `GIT_NAMESPACE` environment variable. +--super-prefix=<path>:: + Currently for internal use only. Set a prefix which gives a path from + above a repository down to its root. One use is to give submodules + context about the superproject that invoked it. + --bare:: Treat the repository as a bare repository. If GIT_DIR environment is not set, it is set to the current working
cache.h +2 -0
@@ -408,6 +408,7 @@ static inline enum object_type object_type(unsigned int mode)#define GIT_NAMESPACE_ENVIRONMENT "GIT_NAMESPACE"#define GIT_WORK_TREE_ENVIRONMENT "GIT_WORK_TREE"#define GIT_PREFIX_ENVIRONMENT "GIT_PREFIX" +#define GIT_SUPER_PREFIX_ENVIRONMENT "GIT_INTERNAL_SUPER_PREFIX"#define DEFAULT_GIT_DIR_ENVIRONMENT ".git"#define DB_ENVIRONMENT "GIT_OBJECT_DIRECTORY"#define INDEX_ENVIRONMENT "GIT_INDEX_FILE"
@@ -468,6 +469,7 @@ extern int get_common_dir_noenv(struct strbuf *sb, const char *gitdir);extern int get_common_dir (struct strbuf *sb,const char *gitdir);extern const char *get_git_namespace (void );extern const char *strip_namespace (const char *namespaced_ref); +extern const char *get_super_prefix (void );extern const char *get_git_work_tree (void );/*
environment.c +13 -0
@@ -100,6 +100,8 @@ static char *work_tree;static const char *namespace;static size_t namespace_len; +static const char *super_prefix; +static const char *git_dir, *git_common_dir;static char *git_object_dir, *git_index_file, *git_graft_file;int git_db_env, git_index_env, git_graft_env, git_common_dir_env;
@@ -120,6 +122,7 @@ const char * const local_repo_env[] = {NO_REPLACE_OBJECTS_ENVIRONMENT ,GIT_REPLACE_REF_BASE_ENVIRONMENT ,GIT_PREFIX_ENVIRONMENT , +GIT_SUPER_PREFIX_ENVIRONMENT ,GIT_SHALLOW_FILE_ENVIRONMENT ,GIT_COMMON_DIR_ENVIRONMENT ,NULL
@@ -222,6 +225,16 @@ const char *strip_namespace(const char *namespaced_ref)return namespaced_ref + namespace_len; } +const char *get_super_prefix (void ) +{ +static int initialized; +if (!initialized) { + super_prefix =getenv (GIT_SUPER_PREFIX_ENVIRONMENT ); + initialized =1 ; + } +return super_prefix; +} +static int git_work_tree_initialized;/*
git.c +25 -0
@@ -164,6 +164,20 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)setenv (GIT_WORK_TREE_ENVIRONMENT , cmd,1 );if (envchanged) *envchanged =1 ; + }else if (!strcmp (cmd,"--super-prefix" )) { +if (*argc <2 ) { +fprintf (stderr,"No prefix given for --super-prefix.\n" ); +usage (git_usage_string); + } +setenv (GIT_SUPER_PREFIX_ENVIRONMENT , (*argv)[1 ],1 ); +if (envchanged) + *envchanged =1 ; + (*argv)++; + (*argc)--; + }else if (skip_prefix (cmd,"--super-prefix=" , &cmd)) { +setenv (GIT_SUPER_PREFIX_ENVIRONMENT , cmd,1 ); +if (envchanged) + *envchanged =1 ; }else if (!strcmp (cmd,"--bare" )) {char *cwd =xgetcwd (); is_bare_repository_cfg =1 ;
@@ -310,6 +324,7 @@ static int handle_alias(int *argcp, const char ***argv)* RUN_SETUP for reading from the configuration file. */ #define NEED_WORK_TREE (1<<3) +#define SUPPORT_SUPER_PREFIX (1<<4)struct cmd_struct {const char *cmd;
@@ -344,6 +359,13 @@ static int run_builtin(struct cmd_struct *p, int argc, const char **argv) }commit_pager_choice (); +if (!help &&get_super_prefix ()) { +if (!(p->option &SUPPORT_SUPER_PREFIX )) +die ("%s doesn't support --super-prefix" , p->cmd); +if (prefix) +die ("can't use --super-prefix from a subdirectory" ); + } +if (!help && p->option &NEED_WORK_TREE )setup_work_tree ();
@@ -558,6 +580,9 @@ static void execv_dashed_external(const char **argv)const char *tmp;int status; +if (get_super_prefix ()) +die ("%s doesn't support --super-prefix" , argv[0 ]); +if (use_pager ==-1 ) use_pager =check_pager_config (argv[0 ]);commit_pager_choice ();
Signature · 1 line
2.10.0
PATCH v7 2/4 ls-files: optionally recurse into submodules
In reply to Brandon WilliamsAllow ls-files to recognize submodules in order to retrieve a list of files from a repository's submodules. This is done by forking off a process to recursively call ls-files on all submodules. Use top-level --super-prefix option to pass a path to the submodule which it can use to prepend to output or pathspec matching logic.
--- Documentation/git-ls-files.txt | 8 +- builtin/ls-files.c | 138 ++++++++++++++++++++++++--------- git.c | 2 +- t/t3007-ls-files-recurse-submodules.sh | 100 ++++++++++++++++++++++++ 4 files changed, 208 insertions(+), 40 deletions(-) create mode 100755 t/t3007-ls-files-recurse-submodules.sh
Documentation/git-ls-files.txt +7 -1
@@ -18,7 +18,8 @@ SYNOPSIS [--exclude-per-directory=<file>] [--exclude-standard] [--error-unmatch] [--with-tree=<tree-ish>] - [--full-name] [--abbrev] [--] [<file>...] + [--full-name] [--recurse-submodules] + [--abbrev] [--] [<file>...] DESCRIPTION -----------
@@ -137,6 +138,11 @@ a space) at the start of each line: option forces paths to be output relative to the project top directory. +--recurse-submodules:: + Recursively calls ls-files on each submodule in the repository. + Currently there is only support for the --cached mode without a + pathspec. + --abbrev[=<n>]:: Instead of showing the full 40-byte hexadecimal object lines, show only a partial prefix.
builtin/ls-files.c +100 -38
198 lines not on this page · raw message
git.c +1 -1
@@ -443,7 +443,7 @@ static struct cmd_struct commands[] = { { "init-db", cmd_init_db }, { "interpret-trailers", cmd_interpret_trailers, RUN_SETUP_GENTLY }, { "log", cmd_log, RUN_SETUP }, - { "ls-files", cmd_ls_files, RUN_SETUP }, + { "ls-files", cmd_ls_files, RUN_SETUP | SUPPORT_SUPER_PREFIX }, { "ls-remote", cmd_ls_remote, RUN_SETUP_GENTLY }, { "ls-tree", cmd_ls_tree, RUN_SETUP }, { "mailinfo", cmd_mailinfo },
t/t3007-ls-files-recurse-submodules.sh new file +100 -0
100 lines not on this page · raw message
Signature · 1 line
2.10.0
PATCH v7 3/4 ls-files: pass through safe options for --recurse-submodules
In reply to Brandon WilliamsPass through some known-safe options when recursing into submodules. (--cached, -v, -t, -z, --debug, --eol)
--- builtin/ls-files.c | 30 +++++++++++++++++++++++++++--- t/t3007-ls-files-recurse-submodules.sh | 16 ++++++++++++---- 2 files changed, 39 insertions(+), 7 deletions(-)
builtin/ls-files.c +27 -3
@@ -30,6 +30,7 @@ static int line_terminator = '\n';static int debug_mode;static int show_eol;static int recurse_submodules; +static struct argv_array submodules_options =ARGV_ARRAY_INIT ;static const char *prefix;static const char *super_prefix;
@@ -168,6 +169,25 @@ static void show_killed_files(struct dir_struct *dir) } } +/* +* Compile an argv_array with all of the options supported by --recurse_submodules +*/ +static void compile_submodule_options (const struct dir_struct *dir,int show_tag) +{ +if (line_terminator =='\0' ) +argv_array_push (&submodules_options,"-z" ); +if (show_tag) +argv_array_push (&submodules_options,"-t" ); +if (show_valid_bit) +argv_array_push (&submodules_options,"-v" ); +if (show_cached) +argv_array_push (&submodules_options,"--cached" ); +if (show_eol) +argv_array_push (&submodules_options,"--eol" ); +if (debug_mode) +argv_array_push (&submodules_options,"--debug" ); +} +/** * Recursively call ls-files on a submodule */
@@ -182,6 +202,9 @@ static void show_gitlink(const struct cache_entry *ce)argv_array_push (&cp.args,"ls-files" );argv_array_push (&cp.args,"--recurse-submodules" ); +/* add supported options */ +argv_array_pushv (&cp.args, submodules_options.argv); + cp.git_cmd =1 ; cp.dir = ce->name; status =run_command (&cp);
@@ -567,11 +590,12 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)if (require_work_tree && !is_inside_work_tree ())setup_work_tree (); +if (recurse_submodules) +compile_submodule_options (&dir, show_tag); +if (recurse_submodules && (show_stage || show_deleted || show_others || show_unmerged || - show_killed || show_modified || show_resolve_undo || - show_valid_bit || show_tag || show_eol || with_tree || - (line_terminator =='\0' ))) + show_killed || show_modified || show_resolve_undo || with_tree))die ("ls-files --recurse-submodules unsupported mode" );if (recurse_submodules && error_unmatch)
t/t3007-ls-files-recurse-submodules.sh +12 -4
@@ -34,6 +34,18 @@ test_expect_success 'ls-files correctly outputs files in submodule' ' test_cmp expect actual ' +test_expect_success 'ls-files correctly outputs files in submodule with -z' ' + lf_to_nul >expect <<-\EOF && + .gitmodules + a + b/b + submodule/c + EOF + + git ls-files --recurse-submodules -z >actual && + test_cmp expect actual +' + test_expect_success 'ls-files does not output files not added to a repo' ' cat >expect <<-\EOF && .gitmodules
@@ -86,15 +98,11 @@ test_incompatible_with_recurse_submodules () { " } -test_incompatible_with_recurse_submodules -z -test_incompatible_with_recurse_submodules -v -test_incompatible_with_recurse_submodules -t test_incompatible_with_recurse_submodules --deleted test_incompatible_with_recurse_submodules --modified test_incompatible_with_recurse_submodules --others test_incompatible_with_recurse_submodules --stage test_incompatible_with_recurse_submodules --killed test_incompatible_with_recurse_submodules --unmerged -test_incompatible_with_recurse_submodules --eol test_done
Signature · 1 line
2.10.0
PATCH v7 4/4 ls-files: add pathspec matching for submodules
In reply to Brandon WilliamsPathspecs can be a bit tricky when trying to apply them to submodules. The main challenge is that the pathspecs will be with respect to the superproject and not with respect to paths in the submodule. The approach this patch takes is to pass in the identical pathspec from the superproject to the submodule in addition to the submodule-prefix, which is the path from the root of the superproject to the submodule, and then we can compare an entry in the submodule prepended with the submodule-prefix to the pathspec in order to determine if there is a match.
This patch also permits the pathspec logic to perform a prefix match against submodules since a pathspec could refer to a file inside of a submodule. Due to limitations in the wildmatch logic, a prefix match is only done literally. If any wildcard character is encountered we'll simply punt and produce a false positive match. More accurate matching will be done once inside the submodule. This is due to the superproject not knowing what files could exist in the submodule.
--- Documentation/git-ls-files.txt | 3 +- builtin/ls-files.c | 27 +++++++-- dir.c | 46 +++++++++++++- dir.h | 4 ++ t/t3007-ls-files-recurse-submodules.sh | 108 ++++++++++++++++++++++++++++++++- 5 files changed, 175 insertions(+), 13 deletions(-)
Documentation/git-ls-files.txt +1 -2
@@ -140,8 +140,7 @@ a space) at the start of each line: --recurse-submodules:: Recursively calls ls-files on each submodule in the repository. - Currently there is only support for the --cached mode without a - pathspec. + Currently there is only support for the --cached mode. --abbrev[=<n>]:: Instead of showing the full 40-byte hexadecimal object
builtin/ls-files.c +21 -6
@@ -195,6 +195,7 @@ static void show_gitlink(const struct cache_entry *ce) {struct child_process cp =CHILD_PROCESS_INIT ;int status; +int i; argv_array_pushf(&cp.args, "--super-prefix=%s%s/", super_prefix ? super_prefix : "",
@@ -205,6 +206,15 @@ static void show_gitlink(const struct cache_entry *ce)/* add supported options */ argv_array_pushv (&cp.args, submodules_options.argv); +/* +* Pass in the original pathspec args. The submodule will be +* responsible for prepending the 'submodule_prefix' prior to comparing +* against the pathspec for matches. +*/ +argv_array_push (&cp.args,"--" ); +for (i =0 ; i < pathspec.nr; i++) +argv_array_push (&cp.args, pathspec.items[i].original); + cp.git_cmd =1 ; cp.dir = ce->name; status =run_command (&cp);
@@ -223,7 +233,8 @@ static void show_ce_entry(const char *tag, const struct cache_entry *ce)if (len >=ce_namelen (ce))die ("git ls-files: internal error - cache entry not superset of prefix" ); -if (recurse_submodules &&S_ISGITLINK (ce->ce_mode)) { +if (recurse_submodules &&S_ISGITLINK (ce->ce_mode) && +submodule_path_match (&pathspec, name.buf, ps_matched)) {show_gitlink (ce); } else if (match_pathspec(&pathspec, name.buf, name.len, len, ps_matched,
@@ -602,16 +613,20 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)die ("ls-files --recurse-submodules does not support " "--error-unmatch" ); -if (recurse_submodules && argc) -die ("ls-files --recurse-submodules does not support pathspec" ); -parse_pathspec (&pathspec,0 ,PATHSPEC_PREFER_CWD |PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP , prefix, argv); -/* Find common prefix for all pathspec's */ - max_prefix =common_prefix (&pathspec); +/* +* Find common prefix for all pathspec's +* This is used as a performance optimization which unfortunately cannot +* be done when recursing into submodules +*/ +if (recurse_submodules) + max_prefix =NULL ; +else + max_prefix =common_prefix (&pathspec); max_prefix_len = max_prefix ?strlen (max_prefix) :0 ;/* Treat unmatching pathspec elements as errors */
dir.c +44 -2
64 lines not on this page · raw message
dir.h +4 -0
@@ -304,6 +304,10 @@ extern int git_fnmatch(const struct pathspec_item *item, const char *pattern, const char *string, int prefix); +extern int submodule_path_match (const struct pathspec *ps, +const char *submodule_name, +char *seen); +static inline int ce_path_match (const struct cache_entry *ce,const struct pathspec *pathspec,char *seen)
t/t3007-ls-files-recurse-submodules.sh +105 -3
114 lines not on this page · raw message
Signature · 1 line
2.10.0
Quoted 1 line · Brandon Williams
Few minor fixes pointed out Stefan
Nit: This is not very informative, so you could provide an "interdiff" to the last patch, (see https://github.com/trast/tbdiff for a sophisticated tool,
I usually do a git diff origin/sb/<what-junio-queued>)
That helps reviewers as they do not need to review it all again, but they may remember what they annotated and see how you addressed it.
Thanks,
Stefan
Quoted 3 lines · Stefan Beller
Quoted 1 line · Brandon Williams
Few minor fixes pointed out Stefan
The series itself looks good though. :)
Thanks,
Stefan
Quoted 9 lines · Stefan Beller
Quoted 3 lines · Stefan Beller
Quoted 1 line · Brandon Williams
Few minor fixes pointed out Stefan
The series itself looks good though. :)
Thanks,
Stefan
Thanks! And I'll keep that in mind for the future. Still lots to learn about contributing to the project.
Signature · 1 line
Brandon Williams