feature request: excluding files/paths from "git grep"

What would be nice is a per-user/repo config setting that excludes certain files and paths from the 'git grep' search.

Does this sound reasonable/acceptable?

Quoted 4 lines · Noel Grandin

What would be nice is a per-user/repo config setting that excludes certain files and paths from the 'git grep' search.

Does this sound reasonable/acceptable?

There is no config setting to do that, but since v1.9.5 you can use ':!' or ':(exclude) to exclude paths, for example

git grep foo -- '*.c' ':!src/ ':!*foo*.c'

will exclude .c files in src directory or contains "foo". If you use some exclude patterns often, you can write a short script. Perhaps we could support pathspec macros (similar to git-attr macros), stored in config file. You still need to type, but it'll be a lot shorter.

If it's an attribute of the file, and not the request, maybe gitattributes would be a better fit. You can already do this with:

*.foo -diff

in your .gitattributes file, though that _also_ marks the files as "not for diffing", which may not be desired. There's not a separate "grep" attribute, but I do not think it would be unreasonable to add one.

Quoted 8 lines · Jeff King

If it's an attribute of the file, and not the request, maybe gitattributes would be a better fit. You can already do this with:

*.foo -diff

in your .gitattributes file, though that _also_ marks the files as "not for diffing", which may not be desired. There's not a separate "grep" attribute, but I do not think it would be unreasonable to add one.

I have a vague recollection of having a discussion that started with something like this:

"diff" is named as if it is only for "diff" for historical reasons, but it is about "do we want to treat its raw contents as text?"

I do not recall its conclusion, but it it were "Yes, that is what it means", then it might be reasonable to:

- have "git grep" ignore paths marked with -diff by default (perhaps "-a" option to disable, just like GNU)

- have "git grep" pay attention to diff.textconv and search in the result of textconv filter.

So I think _if_ using "diff" attributes is enough for this purpose, then there is no code to be written. But if somebody wants to draw a distinction between the uses (I want to diff "foo" files, but never see them in grep) then we could introduce a "grep" attribute (with the fallback being the value of the "diff" attribute for that path).

Quoted 5 lines · Jeff King

So I think _if_ using "diff" attributes is enough for this purpose, then there is no code to be written. But if somebody wants to draw a distinction between the uses (I want to diff "foo" files, but never see them in grep) then we could introduce a "grep" attribute (with the fallback being the value of the "diff" attribute for that path).

That is all true.

If we were to have a new 'grep' attribute that can be used to express 'It is OK to diff two versions of this path, but hits by grep in this path is useless' (and verse versa), the built-in macro attribute 'binary' should also be updated with it. A path being 'binary' currently means '-diff -merge -text' but it should also mean '-grep' in the new world, if we were to go in that direction.

Quoted 9 lines

Quoted 5 lines · Jeff King

So I think _if_ using "diff" attributes is enough for this purpose, then there is no code to be written. But if somebody wants to draw a distinction between the uses (I want to diff "foo" files, but never see them in grep) then we could introduce a "grep" attribute (with the fallback being the value of the "diff" attribute for that path).

Quoted 8 lines

That is all true.

If we were to have a new 'grep' attribute that can be used to express 'It is OK to diff two versions of this path, but hits by grep in this path is useless' (and verse versa), the built-in macro attribute 'binary' should also be updated with it. A path being 'binary' currently means '-diff -merge -text' but it should also mean '-grep' in the new world, if we were to go in that direction.

I think it would do so automatically. There is no "grep" attribute given, so we fall back to the "-diff" attribute. But I do not mind modifying the macro to be more explicit.

Note also that I am not volunteering to work on this, nor am I convinced it's actually worth pursuing. I've yet to see a useful case where you would want text diffs but not greps (or vice versa), and if we can avoid cluttering the attribute space, we should. I was mostly pointing it out that it is not logically inconsistent to want such a thing. :)

If somebody does look into it, I suspect the place to start is modifying userdiff_find_by_path to optionally prefer "grep" to "diff".

-Peff

So, as a summary of the discussion, it seems it's time to switch the default to --textconv for git grep?

So, as a summary of the discussion, it seems it's time to switch the default to --textconv for git grep?

Quoted 2 lines · Michael J Gruber

So, as a summary of the discussion, it seems it's time to switch the default to --textconv for git grep?

Hmmm, why?

Nobody seems to be asking for such a change in this thread. The original issue IIRC was that the grep output was unnecessary for some paths and the repository did not mark these paths as such. Once they are marked as "-diff", there is no reason why you want to trigger textconv to squelch the hits from grep.

So that does not sound to me a summary of the discussion at all.

I do not recall its conclusion, but it it were "Yes, that is what it means", then it might be reasonable to:

- have "git grep" ignore paths marked with -diff by default (perhaps "-a" option to disable, just like GNU)

- have "git grep" pay attention to diff.textconv and search in the result of textconv filter.

Junio C Hamano venit, vidit, dixit 26.02.2015 21:59:

Quoted 1 line

So that does not sound to me a summary of the discussion at all.

Well, your conditional

Quoted 6 lines

I do not recall its conclusion, but it it were "Yes, that is what it means", then it might be reasonable to:

- have "git grep" ignore paths marked with -diff by default (perhaps "-a" option to disable, just like GNU)

Quoted 2 lines

- have "git grep" pay attention to diff.textconv and search in the result of textconv filter.

and Jeff's "Yes" on that condition certainly read like that to me: Make "git grep" react to "diff", "-diff" attributes in the same way as "git diff".

Quoted 8 lines · Junio C Hamano

Quoted 5 lines · Jeff King

So I think _if_ using "diff" attributes is enough for this purpose, then there is no code to be written. But if somebody wants to draw a distinction between the uses (I want to diff "foo" files, but never see them in grep) then we could introduce a "grep" attribute (with the fallback being the value of the "diff" attribute for that path).

Quoted 8 lines

That is all true.

If we were to have a new 'grep' attribute that can be used to express 'It is OK to diff two versions of this path, but hits by grep in this path is useless' (and verse versa), the built-in macro attribute 'binary' should also be updated with it. A path being 'binary' currently means '-diff -merge -text' but it should also mean '-grep' in the new world, if we were to go in that direction.

I think it would do so automatically. There is no "grep" attribute given, so we fall back to the "-diff" attribute. But I do not mind modifying the macro to be more explicit.

Note also that I am not volunteering to work on this, nor am I convinced it's actually worth pursuing. I've yet to see a useful case where you would want text diffs but not greps (or vice versa), and if we can avoid cluttering the attribute space, we should. I was mostly pointing it out that it is not logically inconsistent to want such a thing. :)

If somebody does look into it, I suspect the place to start is modifying userdiff_find_by_path to optionally prefer "grep" to "diff".

-Peff
--
To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

There have been cases where I wanted grep to always ignore certain files, but to still get text diffs for those files. One case is people insist on using ChangeLog files, and another is people who commit generated files of one sort or another.

Quoted 4 lines · Trevor Saunders

There have been cases where I wanted grep to always ignore certain files, but to still get text diffs for those files. One case is people insist on using ChangeLog files, and another is people who commit generated files of one sort or another.

The attributes are to say "the contents to be stored in this file is of this nature". Something inherent to the type of the contents, and that is why there is no way to countermand them from the command line.

The "nature of the content" may be "result of comparing two versions of them textually will never make sense to humans", or "result of finding substrings in them will never make sense to humans", which are what "-diff" and hypothetical "-grep" mean, respectively.

"It is inconvenient that I see hits in ChangeLog files when I look for string BUG" does not make ChangeLog inherently "result of finding substrings in it never makes sense to humans"-kind of file type. Maybe somebody who is playing a role of a coder right now may not look at existing ChangeLog entries, but when that same person plays the role of a release manager next day, running grep on older ChangeLog files may become necessary to find changes related to recent changes. For these "per-invocation" differences, attributes to declare permenent/inherent nature of the contents is much less suited than per-invocation inclusion/exclusion mechanism based on pathspecs, I would think.

Quoted 4 lines

... For these "per-invocation" differences, attributes to declare permenent/inherent nature of the contents is much less suited than per-invocation inclusion/exclusion mechanism based on pathspecs, I would think.

I think that makes some amount of sense, however typing stuff like --exclude=ChangeLog all the time is not terribly easy on the hands. Would it make sense to instead add a config variable grep.exclude?

Quoted 8 lines · Trevor Saunders

Quoted 4 lines

... For these "per-invocation" differences, attributes to declare permenent/inherent nature of the contents is much less suited than per-invocation inclusion/exclusion mechanism based on pathspecs, I would think.

I think that makes some amount of sense, however typing stuff like --exclude=ChangeLog all the time is not terribly easy on the hands. Would it make sense to instead add a config variable grep.exclude?

I do not think it makes much more sense for at least three reasons. for one thing, It still goes against "per-invocation" nature of what is being done. Your "for this invocation I do not want ChangeLog" does not have to be limited to 'grep'. And also "I end up having to give these pathspecs all the time" is not limited to negative ones.

We have magic pathspecs, like "This pattern is used to match the string case-insensitively", "This pattern specifies that the path should *not* match it", etc. How about adding a new feature that lets you say "This is a short hand to giving these pathspecs" and call that "pathspec macro"?

If you get tired to keep having to type

    $ git log -- Documentation/ ':!Documentation/technical/'

every time you want to check the end-user facing documentation pages, you could for example say (I am using a made-up 'macro' pathspec magic that is introduced by ':*' followed by a <macro name>):

    $ git log -- ':*userdoc'

and the same macro specification could be used for all the other things that take pathspecs (grep, add, diff, etc.).

You could then have something like this to define your own "nolog" macro:

    [pathspecMacro]
        nolog = ':!ChangeLog' ':!ChangeLog.*'

to shorten your invocation of "grep" by appending it when you want to exclude some files, i.e.

    $ git grep -e pattern -- \*.c ':*nolog'

and the same pathspec macro can be used in other places, not just "grep". Wouldn't it make more sense?

I think they solve somewhat different problems, but maybe my problem is so specialized I should just have a wrapper around grep that changes defaults. Trev

Quoted 3 lines · Trevor Saunders

I think they solve somewhat different problems, but maybe my problem is so specialized I should just have a wrapper around grep that changes defaults. Trev

I'm with Trevor on this one. While I see the appeal of the generality of a macro solution, this is really just about convenience for me on a per-project basis.

As in, while working on a specific project, I sometimes just want to exclude, for the time being, a bunch of stuff from 'git grep'.

Quoted 2 lines

As in, while working on a specific project, I sometimes just want to exclude, for the time being, a bunch of stuff from 'git grep'.

The key word here is "for the time being", though. What would you do once you are done with the "for the time being" activity? "git config --unset"?

Quoted 3 lines · Junio C Hamano

Quoted 2 lines

As in, while working on a specific project, I sometimes just want to exclude, for the time being, a bunch of stuff from 'git grep'.

Quoted 3 lines

The key word here is "for the time being", though. What would you do once you are done with the "for the time being" activity? "git config --unset"?

IMHO this is being too paternalistic. You can already shoot yourself in the foot by configuring an alias to grep, running your alias, and wondering why it does not produce the results you wanted.