add regex match flags to git describe

git describe currently only supports glob matching with the --matches flag. It would be useful to support regular expressions.

For consistency, this uses the same regex flags as those used by git-grep.

Some old discussion of this as a candidate feature is here, though nobody put together a patch as far as I can see:
http://comments.gmane.org/gmane.comp.version-control.git/173873

Quoted 8 lines · Mostyn Bramley-Moore

git describe currently only supports glob matching with the --matches flag. It would be useful to support regular expressions.

For consistency, this uses the same regex flags as those used by git-grep.

Some old discussion of this as a candidate feature is here, though nobody put together a patch as far as I can see:
http://comments.gmane.org/gmane.comp.version-control.git/173873

Thanks.

I do not think it is wrong per-se to add an option to use regular expressions instead of globs, but if we are to do so, the endgame we aim for MUST be that we do so consistently to all the other commands that iterate over refs and limit their output to the ones that match given pattern (or a set of patterns), not just 'describe'.

Even if we are not adding such an option to these other commands right now (yet), we at least need to know what these commands are (e.g. "git tag -l" and "git for-each-ref" immediately come to mind, but there may be others), and make sure that the option names you choose here can be used sensibly in their context. I think "tag" and "for-each-ref" do no pattern matching against anything other than the refnames, so it would be clear what a new --perl-regexp option does in their contexts.

Unlike "grep" whose sole point is to perform pattern matching, the filtering of refs these commands do is merely a very small tweak in the overall picture (e.g. "git tag --contains $commit -l $pattern" does filter by matching $pattern against the refname, but that is a small detail compared to the filtering done by the reachability with the $commit), so I am not sure if short -E/-G/-F/-P should be given to these commands like "grep" does, though. These commands may have better uses for these shorter option names.

Quoted 5 lines

I do not think it is wrong per-se to add an option to use regular expressions instead of globs, but if we are to do so, the endgame we aim for MUST be that we do so consistently to all the other commands that iterate over refs and limit their output to the ones that match given pattern (or a set of patterns), not just 'describe'.

There is one important distinction between 'git describe' and the other commands that iterate through refs- it applies an internal search strategy and outputs at most one match. This makes it difficult to search for the closest matching tag...

Besides 'git grep', the only regex type flag that is given a short option seems to be -E for 'git log' and 'git rev-list'. I have no objection to dropping the short options, or leaving only -E.

Quoted 6 lines · Mostyn Bramley-Moore

Quoted 5 lines

I do not think it is wrong per-se to add an option to use regular expressions instead of globs, but if we are to do so, the endgame we aim for MUST be that we do so consistently to all the other commands that iterate over refs and limit their output to the ones that match given pattern (or a set of patterns), not just 'describe'.

Quoted 4 lines

There is one important distinction between 'git describe' and the other commands that iterate through refs- it applies an internal search strategy and outputs at most one match. This makes it difficult to search for the closest matching tag...

If that was what you were trying to solve, then it sounds like a typical XY problem. You do not need custom matching flags; you need a "give me N (or all) names based on possible tags" option.

And I do not think it is a bad thing to add. I already said that an option to match with a regular expression is not a bad thing to add, either ;-)

Quoted 3 lines

Besides 'git grep', the only regex type flag that is given a short option seems to be -E for 'git log' and 'git rev-list'. I have no objection to dropping the short options, or leaving only -E.

They also take -F, but "log" and friends do not pattern match the refnames, so I do not think you have to worry about them at the moment.

It is more important to envision what we would do in the future when a command that takes a pattern (or a set of patterns) to match the refnames with _and_ another pattern (or a set of patterns) to match something else, and take that into account when designing this "allowing matching logic for refnames to be customized from glob to something else" feature, so that we do not paint outselves into a corner we cannot later get out of. Imagine a hypothetical command 'git mgrep' that can look for a pattern in tips of multiple branches that can be used this way:

     $ git mgrep -e 'froo*tz' --refs 'refs/heads/release/*'

which may look for strings that match "froo*tz" in the trees of all branches whose name match the pattern 'release/*'. In this example, the pattern to match strings is a BRE (same default as "git grep"), and the pattern to match refnames is a glob.

Consistency & similarity with "git grep" would most likely lead us to add -E/-F/-G/-P options to this command and to make it affect how the pattern to match strings works. For example:

     $ git mgrep -E -e 'fro+tz' --match-refs 'refs/heads/release/*'

may look for the same strings that would match the first example, but the pattern is expressed in ERE. "-P", "-G", and "-F" options would also work the same way.

Now, the question is what this "-E" (or -P/-G/-F) should do with the matching the command does with the refnames. The easiest (and laziest) way out from the implementors' point of view might be to declare that they affect both at the same time. But is that useful in practice? It probably isn't, as it forces the users to write

     $ git mgrep -E -e 'fro+tz' --match-refs 'refs/heads/release/.*'

because the ref matching suddenly starts to use ERE (not glob), which most likely is not something users would expect. So we may need a separate set of options to affect the way how refs are matched.

We cannot just say "but we do not have that 'mgrep' command yet, so we can do whatever we want to do with 'describe' today". When the need eventually arises that requires us to be able to specify how strings are matched and how refnames are matched independently, we would end up with an inconsistent UI where 'describe' takes '-P' (or '--perl-regexp') to affect the way how refnames are matched, while commands like 'mgrep' would need to use '--refmatch-perl-regexp' (or any other name that can be distinguished from '--perl-regexp') to do the same thing because they do not want '--perl-regexp' to affect the matching of refnames.

And at that point in the future, it is too late to fix 'describe', as people are so used to use '--perl-regexp' to match with refs. We will forever regret that we did not give the option a name that can be used independently from the existing '--perl-regexp' that is about matching for strings, not refnames.

That is exactly the kind of thing that would paint us in a corner that we cannot get out of, which we need to avoid, hence we need to think ahead now.

OK, brainstorming a bit, how about either of these:

1) --match-pattern-type=<glob|fixed-strings|basic-regexp|extended-regexp|perl-regexp>

It's a bit lengthy (maybe --match-type would be sufficient), but I like that the value names are shared with git grep etc option names. And it seems future-proof- if we ever need to support different pattern types for other arguments, a --foo-pattern-type flag could be added and make obvious sense.

Quoted 10 lines · Mostyn Bramley-Moore

OK, brainstorming a bit, how about either of these:

1) --match-pattern-type=<glob|fixed-strings|basic-regexp|extended-regexp|perl-regexp>

It's a bit lengthy (maybe --match-type would be sufficient), but I like that the value names are shared with git grep etc option names. And it seems future-proof- if we ever need to support different pattern types for other arguments, a --foo-pattern-type flag could be added and make obvious sense.

Swapping the option key and value may not be a bad idea, but one problem that the above does not solve, which I outlined in the message you are responding to, is that "match-pattern-type" does not give any hint that this is about affecting the match that is done to "refs", e.g. you cannot tell in

   $ git mgrep --match-pattern-type=perl-regexp -e foo --refs 'release_*'

if the perl-regexp is to be used for matching branch names or for matching the strings the command looks for in the trees of the matching branches.

Magic pattern annotation like we do for pathspecs Duy raised may not be a bad idea, either, and would probably be easier to teach people. Just like in Perl "(?i)$any_pattern" is a way to introduce the case insensitive match with $any_pattern, we may be able to pick an extensible magic syntax and decorate the pattern you would specify for matching refnames to tell Git what kind of pattern it is, e.g.

   $ git mgrep -P -e foo --refs '/?glob/release_*'

I am not suggesting that we must use /?<pattern type name>/ prefix as the "extensible magic syntax" here--I am just illustrating what I mean by "extensible magic syntax".

On 12/31/2015 01:23 AM, Junio C Hamano wrote:
...

Quoted 6 lines

Swapping the option key and value may not be a bad idea, but one problem that the above does not solve, which I outlined in the message you are responding to, is that "match-pattern-type" does not give any hint that this is about affecting the match that is done to "refs", e.g. you cannot tell in

Quoted 5 lines

   $ git mgrep --match-pattern-type=perl-regexp -e foo --refs 'release_*'

if the perl-regexp is to be used for matching branch names or for matching the strings the command looks for in the trees of the matching branches.

There is a hint: --foo-pattern-type applies only to --foo.

In your mgrep example --match-pattern-type would apply to --match patterns only, and if we choose to implement it then --refs-pattern-type would apply to --refs patterns only. eg:

$ git mgrep --match '^foo' --match-pattern-type=perl-regexp --refs
release_*' --refs-pattern-type=glob

Quoted 9 lines

Magic pattern annotation like we do for pathspecs Duy raised may not be a bad idea, either, and would probably be easier to teach people. Just like in Perl "(?i)$any_pattern" is a way to introduce the case insensitive match with $any_pattern, we may be able to pick an extensible magic syntax and decorate the pattern you would specify for matching refnames to tell Git what kind of pattern it is, e.g.

   $ git mgrep -P -e foo --refs '/?glob/release_*'

Quoted 3 lines

I am not suggesting that we must use /?<pattern type name>/ prefix as the "extensible magic syntax" here--I am just illustrating what I mean by "extensible magic syntax".

I hadn't seen the pathspec magic patterns before- interesting. We could possibly share syntax with pathspecs, ie :(?pattern-options...)pattern

Would this be confusing for commands that already have --perl-regexp etc? What should happen if you specify both --perl-regexp and and a different type of pattern like :(glob)foo (error out, I suppose)?

Quoted 8 lines · Mostyn Bramley-Moore

On 12/31/2015 01:23 AM, Junio C Hamano wrote:
...

Quoted 6 lines

Swapping the option key and value may not be a bad idea, but one problem that the above does not solve, which I outlined in the message you are responding to, is that "match-pattern-type" does not give any hint that this is about affecting the match that is done to "refs", e.g. you cannot tell in

Quoted 7 lines

Quoted 5 lines

    $ git mgrep --match-pattern-type=perl-regexp -e foo --refs 'release_*'

if the perl-regexp is to be used for matching branch names or for matching the strings the command looks for in the trees of the matching branches.

There is a hint: --foo-pattern-type applies only to --foo.

Hmph.

Quoted 6 lines

In your mgrep example --match-pattern-type would apply to --match patterns only, and if we choose to implement it then --refs-pattern-type would apply to --refs patterns only. eg:

$ git mgrep --match '^foo' --match-pattern-type=perl-regexp --refs
release_*' --refs-pattern-type=glob

Most likely the hypothetical "mgrep" would not use "--match" but use "-e" to retain similarity to "grep", and "--e-pattern-type" would be confusing. But I agree that "--refs-pattern-type" uniformly used where we take pattern parameter on the command line to match with refs may make it clear that you are only affecting the matches against refs.

Quoted 9 lines

Quoted 9 lines

Magic pattern annotation like we do for pathspecs Duy raised may not be a bad idea, either, and would probably be easier to teach people. Just like in Perl "(?i)$any_pattern" is a way to introduce the case insensitive match with $any_pattern, we may be able to pick an extensible magic syntax and decorate the pattern you would specify for matching refnames to tell Git what kind of pattern it is, e.g.

    $ git mgrep -P -e foo --refs '/?glob/release_*'

Quoted 7 lines

Quoted 3 lines

I am not suggesting that we must use /?<pattern type name>/ prefix as the "extensible magic syntax" here--I am just illustrating what I mean by "extensible magic syntax".

I hadn't seen the pathspec magic patterns before- interesting. We could possibly share syntax with pathspecs, ie :(?pattern-options...)pattern

Even though we have DWIM between revisions and paths on the command line when the user omits "--" for disambiguation, I do not think we look at the shape of the string to DWIM/decide that it is a pattern, so as long as the magic syntax cannot be a valid pattern to match against refs right now (and your ":(?...)" qualifies as such, as a refname would not contain a component that begins with a colon), it would be possible to introduce it as the magic syntax for matching refs.

Or did you mean to use this syntax also for patterns that match strings in contents, e.g.

     $ git grep -e ':(?perl-regexp)...'

I am not bold enough to say that it would be a good idea, but I offhand do not think of a reason why we shouldn't go that route, either.

Quoted 3 lines

Would this be confusing for commands that already have --perl-regexp etc? What should happen if you specify both --perl-regexp and and a different type of pattern like :(glob)foo (error out, I suppose)?

If we were to go that route, ideally, I would say that

     $ git grep --perl-regexp -e 'A' -e ':(?basic-regexp)B' -e ':(?fixed-string)C'

should match with A as pcre, B as BRE and C as a fixed string.

I do not offhand remember if we built the underlying grep machinery in such a way that it is easy to extend it to allow such mixture, though.

Quoted 28 lines · Junio C Hamano

Quoted 9 lines

Quoted 9 lines

Magic pattern annotation like we do for pathspecs Duy raised may not be a bad idea, either, and would probably be easier to teach people. Just like in Perl "(?i)$any_pattern" is a way to introduce the case insensitive match with $any_pattern, we may be able to pick an extensible magic syntax and decorate the pattern you would specify for matching refnames to tell Git what kind of pattern it is, e.g.

    $ git mgrep -P -e foo --refs '/?glob/release_*'

Quoted 8 lines

Quoted 3 lines

I am not suggesting that we must use /?<pattern type name>/ prefix as the "extensible magic syntax" here--I am just illustrating what I mean by "extensible magic syntax".

I hadn't seen the pathspec magic patterns before- interesting. We could possibly share syntax with pathspecs, ie :(?pattern-options...)pattern

Even though we have DWIM between revisions and paths on the command line when the user omits "--" for disambiguation, I do not think we look at the shape of the string to DWIM/decide that it is a pattern, so as long as the magic syntax cannot be a valid pattern to match against refs right now (and your ":(?...)" qualifies as such, as a refname would not contain a component that begins with a colon), it would be possible to introduce it as the magic syntax for matching refs.

Quoted 4 lines

Or did you mean to use this syntax also for patterns that match strings in contents, e.g.

     $ git grep -e ':(?perl-regexp)...'

I think it would be nice to support this syntax in every command that does pattern matching.

Corner case: what if we want to search for ":(?perl-regexp)", eg in git's own source? I suppose this would do:
git grep -e ":(?fixed-strings):(?perl-regexp)"

It is more important to envision what we would do in the future when a command that takes a pattern (or a set of patterns) to match the refnames with _and_ another pattern (or a set of patterns) to match something else, and take that into account when designing this "allowing matching logic for refnames to be customized from glob to something else" feature, so that we do not paint outselves into a corner we cannot later get out of. Imagine a hypothetical command 'git mgrep' that can look for a pattern in tips of multiple branches that can be used this way:

    $ git mgrep -e 'froo*tz' --refs 'refs/heads/release/*'

which may look for strings that match "froo*tz" in the trees of all branches whose name match the pattern 'release/*'. In this example, the pattern to match strings is a BRE (same default as "git grep"), and the pattern to match refnames is a glob.

Consistency & similarity with "git grep" would most likely lead us to add -E/-F/-G/-P options to this command and to make it affect how the pattern to match strings works. For example:

    $ git mgrep -E -e 'fro+tz' --match-refs 'refs/heads/release/*'

may look for the same strings that would match the first example, but the pattern is expressed in ERE. "-P", "-G", and "-F" options would also work the same way.

Now, the question is what this "-E" (or -P/-G/-F) should do with the matching the command does with the refnames. The easiest (and laziest) way out from the implementors' point of view might be to declare that they affect both at the same time.
...

What about not using command line options? We could go with something like pathspec magic. I think as long as we provide three options:
literal, some pattern matching and backquote, then the user has enough flexibility to specify any set of refs. For pattern matching I'd prefer wildmatch. regex can be used via backquote, e.g. for-each-ref | grep ... (or teach for-each-ref about regex to avoid grep).