Skip to contentSkip to conversation

Ignore file filter

PATCH In review David Greaves 5 participants 18 messages Jun 15, 2016 1 message not in the archive

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Ignore file filter

Hi Petr

This is an inline filter that introduces the concept of .git/ignore It is intended to be used within the cogito scripts like the other cg-X* files.

#!/usr/bin/env bash
#
# Takes a list of files on stdin and only passes valid ones agording to .git/ignore # Copyright (c) David Greaves, 2005
#
# This filter implements cogito ignore rules and should typically be used in find pipelines #
# Synopsis
# cg-Xignore [-debug] [-f] [-h] [-d] < file-list >useful-file-list
#
# Options
# -debug::
# produce helpful debug output
#
# -q::
# don't say what paths are ignored
#
# -f::
# passes files
#
# -d::
# passes directories
#
# -h::
# passes symbolic links
#
# The default is to pass all file types that are not ignored.
#
# Note that the .git/ignore file contains multiple expressions, 1 per line
# Lines beginning with a '#' are ignored (allowing comments)
# These are 'bash regular expressions' not glob patterns
# This allows ignore rules to take the directory into account
# Suggested contents:

#   # bash regexps (not globs)
#   ^\.[^/]
#   /\.
#   /$
#   .*\.o$

# This doesn't allow the -h which is the [ arg for symlinks... #. ${COGITO_LIB}cg-Xlib
_git=${GIT_DIR:-.git}

IGNORE_FILE="$_git/ignore"

if [ "$1" = "-0" ]; then

    # doesn't work :(
    zerosep=$'-d "\0"'
    shift

fi

# Defaults
pass_files=0
pass_dirs=0
pass_links=0
pass_all=1
while [ $# -gt 0 ]; do

    case $1 in
	"-f")
	    pass_all=0
	    pass_files=1
	    ;;
	"-d")
	    pass_all=0
	    pass_dirs=1
	    ;;
	"-h")
	    pass_all=0
	    pass_links=1
	    ;;
	"-q")
	    quiet=1
	    ;;
	"-debug")
	    debug=1
	    ;;
	esac
    shift

done

# save stderr
exec 5>&2
if [ $quiet ]; then
    # turn off noise
    exec 2>&-
fi

if [ $debug ]; then

    exec 4>&5

else
exec 4>/dev/null
fi

# Strip out the common leading ./ allowing "find ." sed 's:^./::' | \
while read $zerosep file; do

    echo "consider file: $file" >&4
    ignore=0
    if [ -f $IGNORE_FILE ]; then
	exec 3<$IGNORE_FILE
	while read -r -u3 patt ; do
	    if [[ $patt =~ "^\w*#" ]]; then
		continue
	    fi
	    echo "consider pattern: $patt" >&4
	    if [[ $file =~ $patt ]]; then
		ignore=1
		echo "Ignoring $file because of $patt" >&2
		break
	    fi
	done
    fi
    echo "passing file: $file" >&4
    
    if [ $ignore != "1" \
	-a \( $pass_all -eq 1 \
	   -o \( $pass_files -eq 1 -a -f $file \) \
	   -o \( $pass_dirs  -eq 1 -a -d $file \) \
	   -o \( $pass_links -eq 1 -a -h $file \) \
	   \) \
	]; then
	echo $file
    fi

done

#1 lore raw

Dear diary, on Thu, May 12, 2005 at 11:30:32PM CEST, I got a letter where David Greaves <[email protected]> told me that...

Quoted 1 line

# This doesn't allow the -h which is the [ arg for symlinks...

But so is -L. And I'd just use -l...

Quoted 2 lines

#. ${COGITO_LIB}cg-Xlib
_git=${GIT_DIR:-.git}

...but it makes no sense anyway I think to reinclude this stuff from a cg-Xfile you are including from other scripts anyway.

Quoted 1 line
	    if [[ $file =~ $patt ]]; then

I'm sorry but this is really nothing my bash-2.05.0(1)-release supports. We're already bash-only, but further reducing that to bash3 really won't work. I *might* get convinced to add some bash2+-only feature, but only if you'll be really good at explaining that it makes sense.

Besides, I'd prefer just the shell globs in the ignore file, as it is done in the rest of the world, and in all the real-world scenarios I've seen, the globs were powerful enough.

Also, how does this interact with git-ls-files --exclude and .git/exclude? We would have two ignoring mechanisms...

Signature · 3 lines
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor
#2 lore raw
Quoted 11 lines · Petr Baudis

Dear diary, on Thu, May 12, 2005 at 11:30:32PM CEST, I got a letter where David Greaves <[email protected]> told me that...

Quoted 3 lines

# This doesn't allow the -h which is the [ arg for symlinks...

But so is -L. And I'd just use -l...

OK

Quoted 9 lines
Quoted 4 lines

#. ${COGITO_LIB}cg-Xlib
_git=${GIT_DIR:-.git}

...but it makes no sense anyway I think to reinclude this stuff from a cg-Xfile you are including from other scripts anyway.

cg-Xignore isn't included - only called.
it's also just a library program.
Also I don't think cg-Xlib should be doing arg handling. As an include it should provide an arg handling function that the scripts call.

Quoted 10 lines
Quoted 3 lines
	    if [[ $file =~ $patt ]]; then

I'm sorry but this is really nothing my bash-2.05.0(1)-release supports. We're already bash-only, but further reducing that to bash3 really won't work. I *might* get convinced to add some bash2+-only feature, but only if you'll be really good at explaining that it makes sense.

Ah
OK
I don't know how to do that.
I was actually aiming for glob matching when I came upon this in the manpage.
I just thought it was bash and didn't think to check what version it was introduced with.

Quoted 9 lines

Besides, I'd prefer just the shell globs in the ignore file, as it is done in the rest of the world, and in all the real-world scenarios I've seen, the globs were powerful enough.

Also, how does this interact with git-ls-files --exclude and .git/exclude? We would have two ignoring mechanisms...

because one was cogito's and one was git's.
Cogitos was supposed to have a more powerful, pattern based abroach.

David

#3 lore raw

Dear diary, on Sat, May 14, 2005 at 10:28:24AM CEST, I got a letter where David Greaves <[email protected]> told me that...

Quoted 10 lines
Quoted 9 lines
Quoted 4 lines

#. ${COGITO_LIB}cg-Xlib
_git=${GIT_DIR:-.git}

...but it makes no sense anyway I think to reinclude this stuff from a cg-Xfile you are including from other scripts anyway.

cg-Xignore isn't included - only called.

Oh yes, I'm stupid.

Quoted 4 lines

it's also just a library program.
Also I don't think cg-Xlib should be doing arg handling. As an include it should provide an arg handling function that the scripts call.

I'd prefer the few and scattered users which don't want arg handling to explicitly set some magic variable before calling cg-Xlib rather than adding the arg parser function call everywhere else.

Quoted 10 lines
Quoted 10 lines
Quoted 3 lines
	    if [[ $file =~ $patt ]]; then

I'm sorry but this is really nothing my bash-2.05.0(1)-release supports. We're already bash-only, but further reducing that to bash3 really won't work. I *might* get convinced to add some bash2+-only feature, but only if you'll be really good at explaining that it makes sense.

Quoted 5 lines

Ah
OK
I don't know how to do that.
I was actually aiming for glob matching when I came upon this in the manpage.

Ok, so what's the outcome? Are you going to stop at this point, or will you change the scripts so that they use the glob list?

Signature · 3 lines
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor
#4 lore raw
Quoted 10 lines · Petr Baudis

Dear diary, on Sat, May 14, 2005 at 10:28:24AM CEST, I got a letter where David Greaves <[email protected]> told me that...

Quoted 5 lines

Also I don't think cg-Xlib should be doing arg handling. As an include it should provide an arg handling function that the scripts call.

Quoted 5 lines

I'd prefer the few and scattered users which don't want arg handling to explicitly set some magic variable before calling cg-Xlib rather than adding the arg parser function call everywhere else.

OK

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 3 lines
	    if [[ $file =~ $patt ]]; then

I'm sorry but this is really nothing my bash-2.05.0(1)-release supports. We're already bash-only, but further reducing that to bash3 really won't work. I *might* get convinced to add some bash2+-only feature, but only if you'll be really good at explaining that it makes sense.

Quoted 10 lines
Quoted 6 lines

OK
I don't know how to do that.
I was actually aiming for glob matching when I came upon this in the manpage.

Ok, so what's the outcome? Are you going to stop at this point, or will you change the scripts so that they use the glob list?

Well, Junio solved that for me - I'll gather the comments and resubmit.

David

#5 lore raw
In reply to David Greaves
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line

"DG" == David Greaves <[email protected]> writes:

if [[ $file =~ $patt ]]; then

I'm sorry but this is really nothing my bash-2.05.0(1)-release supports.

DG> OK
DG> I don't know how to do that.

Is that regexp or shell glob? If regexp, expr is your friend, like this:

    if expr "$file" : "$patt" >/dev/null; then

For a glob:

    patt='?.sh'
    file=1.sh

    case "$file" in
    $patt)
            echo Yeah ;;
    *)
            echo No ;;
    esac
#6 lore raw

Dear diary, on Sat, May 14, 2005 at 11:01:49AM CEST, I got a letter where Junio C Hamano <[email protected]> told me that...

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
Quoted 1 line
Quoted 1 line

"DG" == David Greaves <[email protected]> writes:

if [[ $file =~ $patt ]]; then

Quoted 8 lines
Quoted 1 line
Quoted 1 line

I'm sorry but this is really nothing my bash-2.05.0(1)-release supports.

DG> OK
DG> I don't know how to do that.

Is that regexp or shell glob? If regexp, expr is your friend, like this:

    if expr "$file" : "$patt" >/dev/null; then

Oh, this looks nice. I didn't know expr can do that. :-)

Still, I'd prefer the old-fashioned globs as primary matching mechanism.

Signature · 3 lines
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor
#7 lore raw
Quoted 19 lines · Petr Baudis

Dear diary, on Sat, May 14, 2005 at 11:01:49AM CEST, I got a letter where Junio C Hamano <[email protected]> told me that...

Quoted 15 lines
Quoted 9 lines
Quoted 9 lines
Quoted 6 lines
Quoted 3 lines
Quoted 3 lines

"DG" == David Greaves <[email protected]> writes:

if [[ $file =~ $patt ]]; then

I'm sorry but this is really nothing my bash-2.05.0(1)-release supports.

DG> OK
DG> I don't know how to do that.

Is that regexp or shell glob? If regexp, expr is your friend, like this:

Quoted 9 lines
Quoted 3 lines
   if expr "$file" : "$patt" >/dev/null; then

Oh, this looks nice. I didn't know expr can do that. :-)

Still, I'd prefer the old-fashioned globs as primary matching mechanism.

OK
I was wondering about supporting _both_ globs and re's right now my ignore file has a # to precede comment lines maybe re: precedes regexp lines and unadorned lines are globs.

However the re's provided by regex(7) are too weedy to be worth bothering with.
If however, there is a serious plan to go to perl, it may be worth providing for this now in the ignore syntax.

Additionally this causes problems with sharing the same exclude file as used by git.
However...
I really think git's exclude file capability and cogito's are different. Cogito is aiming to provide full-blown SCM capabilities - git isn't

I am also concerned that a centralised ignore file is not flexible enough. Certainly limiting if we support globs only.
It may be that you want different rules in different trees - someone on lkml mentioned that excludes vary in different parts of the source. Eg .s files may be generally ignored - but not in the asm parts of the tree.

Also... you haven't mentioned perl for a while - can you give us an update? I personally think we're making life needlessly unpleasant by sticking with shell.

David

#8 lore raw

[RFD] Ignore rules

Here, I would like more people to speak up, plaese, especially the authors of other layers over git than Cogito, since I think it'd be great if we could agree on common ignore rules format and we could just call the files ".gitignore" instead of ".cgignore", ".jitignore" etc.

Dear diary, on Sat, May 14, 2005 at 05:13:08PM CEST, I got a letter where David Greaves <[email protected]> told me that...

Quoted 2 lines

I was wondering about supporting _both_ globs and re's right now my ignore file has a # to precede comment lines

I assume \# will override that?

Quoted 1 line

maybe re: precedes regexp lines and unadorned lines are globs.

Or maybe even /, which is the common regexp prefix anyway...

To mention it in this mail too, I think leading '!' should do the "ignore exclude" - that is, it would override any possible previous ignore decisions about the file. E.g. '!*' would throw away all previously applied ignore rules.

Quoted 8 lines

However the re's provided by regex(7) are too weedy to be worth bothering with.
If however, there is a serious plan to go to perl, it may be worth providing for this now in the ignore syntax.

Also... you haven't mentioned perl for a while - can you give us an update? I personally think we're making life needlessly unpleasant by sticking with shell.

If there is still a serious plan, it is much more long-term now, since shell turns out to keep doing fine and everything we need, and that all reasonably fast.

That said, I think it's fine to use Perl regexps. I think they rule. :-) But what do others think? Should we stick with POSIX regexps (I assume at least extended instead of basic), or go with Perl regexps?

Quoted 5 lines

Additionally this causes problems with sharing the same exclude file as used by git.
However...
I really think git's exclude file capability and cogito's are different. Cogito is aiming to provide full-blown SCM capabilities - git isn't

If we get to agree on some common format, I'm thinking whether it wouldn't be actually good to extend the --exclude option to support it. How much of an issue would that be? What do others think?

Quoted 5 lines

I am also concerned that a centralised ignore file is not flexible enough. Certainly limiting if we support globs only.
It may be that you want different rules in different trees - someone on lkml mentioned that excludes vary in different parts of the source. Eg .s files may be generally ignored - but not in the asm parts of the tree.

I imagine it as (ignore rules applied in this order):

<default ignore list>:

Some builtin ignore list catching files like *.o, *.a and such. Remember that you can throw it away with !* if you don't like it.

/.git/ignore:

Per-repository ignore list, not version tracked etc; really a local thing.

/.gitignore
/**/.gitignore:

(Applied in the order from the project root to the current directory.) Version tracked ignore list, which concerns the current directory, BUT may match pathnames instead of just filenames (but no ..). That is, you could do something like

	echo '*.o' >.gitignore

to ignore all the object files in the current directory, and

	echo '**.o' >.gitignore

to also ignore the object files in all the subdirectories.

Opinions?

Signature · 3 lines
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor
#9 lore raw

Re: [RFD] Ignore rules

Quoted 10 lines · Petr Baudis

Here, I would like more people to speak up, plaese, especially the authors of other layers over git than Cogito, since I think it'd be great if we could agree on common ignore rules format and we could just call the files ".gitignore" instead of ".cgignore", ".jitignore" etc.

Dear diary, on Sat, May 14, 2005 at 05:13:08PM CEST, I got a letter where David Greaves <[email protected]> told me that...

Quoted 8 lines
Quoted 4 lines

I was wondering about supporting _both_ globs and re's right now my ignore file has a # to precede comment lines

I assume \# will override that?

doesn't match /^\w#/ so yes

Quoted 6 lines
Quoted 3 lines

maybe re: precedes regexp lines and unadorned lines are globs.

Or maybe even /, which is the common regexp prefix anyway...

I thought about it but thought it would look strange since we won't want a closing /
OTOH, no globs will start with a / since they have to be relative so maybe a good choice after all.

Quoted 6 lines

To mention it in this mail too, I think leading '!' should do the "ignore exclude" - that is, it would override any possible previous ignore decisions about the file. E.g. '!*' would throw away all previously applied ignore rules.

Well, CVS has a lone ! meaning just that.
I wonder if we should simply say that most patterns are 'ignore' patterns. '!' patterns are 'accept' patterns.
The last seen pattern wins

so given:
*.o
!fre*.o
freddy.o

frog.o is ignored
fred.o is seen
freddy.o is ignored

This means !* (or is that !**) has the same effect as a lone ! in CVS.

Quoted 11 lines
Quoted 10 lines

However the re's provided by regex(7) are too weedy to be worth bothering with.
If however, there is a serious plan to go to perl, it may be worth providing for this now in the ignore syntax.

Also... you haven't mentioned perl for a while - can you give us an update? I personally think we're making life needlessly unpleasant by sticking with shell.

Quoted 9 lines

If there is still a serious plan, it is much more long-term now, since shell turns out to keep doing fine and everything we need, and that all reasonably fast.

That said, I think it's fine to use Perl regexps. I think they rule. :-) But what do others think? Should we stick with POSIX regexps (I assume at least extended instead of basic), or go with Perl regexps?

I vote perl re's
You have the power if needed but most of the time it'll be basic regexps. I think if we have a central 'path' matching .git/ignore then regexps are needed.
Then if we have them, we may as well allow them.
OTOH: A plethora of .gitignore files with the !negation mechanism may make globs adequate.

Quoted 8 lines
Quoted 7 lines

Additionally this causes problems with sharing the same exclude file as used by git.
However...
I really think git's exclude file capability and cogito's are different. Cogito is aiming to provide full-blown SCM capabilities - git isn't

Quoted 5 lines

If we get to agree on some common format, I'm thinking whether it wouldn't be actually good to extend the --exclude option to support it. How much of an issue would that be? What do others think?

I'm not convinced it needs to be extended.
It's trivial to take git-ls-files and filter it's results. The rest of git will need filtering.

Quoted 8 lines
Quoted 7 lines

I am also concerned that a centralised ignore file is not flexible enough. Certainly limiting if we support globs only.
It may be that you want different rules in different trees - someone on lkml mentioned that excludes vary in different parts of the source. Eg .s files may be generally ignored - but not in the asm parts of the tree.

Quoted 7 lines

I imagine it as (ignore rules applied in this order):

<default ignore list>:

Some builtin ignore list catching files like *.o, *.a and such.

Why builtin?
Why not have a default setup for .git/ignore The way I'd do it is in cg-init :
[ -f cogito.ignore ] && mv cogito.ignore .git/ignore [ ! -f .git/ignore ] && cat <<STD_IGNORE > .git/ignore

Quoted 3 lines

Remember that you can throw it away with !* if you don't like it.

but it means reading the man pages to find it out whilst you're editing .git/gitignore
I could never remember what CVS ignored...

Quoted 5 lines

/.git/ignore:

Per-repository ignore list, not version tracked etc; really a local thing.

~/.gitignore ?? CVS has it - personal prefs for your editor's strange numbered backups etc

Quoted 10 lines

/.gitignore
/**/.gitignore:

(Applied in the order from the project root to the current directory.)

Version tracked ignore list, which concerns the current directory, BUT may match pathnames instead of just filenames (but no ..). That is, you could do something like

Quoted 9 lines
	echo '*.o' >.gitignore

to ignore all the object files in the current directory, and

	echo '**.o' >.gitignore

to also ignore the object files in all the subdirectories.

well, is the matching against files or paths?
And do .gitignores affect their sub-tree or just their dir? And if they're vc'ed then git needs to stop ignoring .files

Quoted 4 lines

Opinions?

I think the global ignores contain both:
* regexps against the path relative to the project root
* regexps against the filename
* shell globs against the filename
I think the .gitignore's operate on a per-tree basis (so their directory and lower)
I think .gitignore matching should be against filenames (not paths)

# comments
/regexp
!/inverted regexp
shellglob
!inverted shellglob

Lesson from the CVS manual:
Specifying -I !' to cvs import' will import everything, which is generally what you want to do if you are importing files from a pristine distribution or any other source which is known to not contain any extraneous files. However, looking at the rules above you will see there is a fly in the ointment; if the distribution contains any .cvsignore' files, then the patterns from those files will be processed even if -I !' is specified. The only workaround is to remove the .cvsignore' files in order to do the import. Because this is awkward, in the future -I !' might be modified to override `.cvsignore' files in each directory.

#10 lore raw

Re: [RFD] Ignore rules

In reply to Petr Baudis
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"PB" == Petr Baudis <[email protected]> writes:

PB> Here, I would like more people to speak up, plaese, especially the PB> authors of other layers over git than Cogito, since I think it'd be PB> great if we could agree on common ignore rules format and we could just PB> call the files ".gitignore" instead of ".cgignore", ".jitignore" etc.

Purely off the top of my head, without even regurgitating what you wrote in the message I am responding to (sorry, I am about to leave for the day).

* Two lists, one propagated with merges and another repository private one.

* GIT_DIR/info/ignore is the one propagated with merge but it is not a list itself. It records the name of a file that is GIT tracked. GIT_DIR/ignore is the repository private one.

* Repository private one and then merge propagated one are read in order, one line at a time. The first hit determines whether it is taken or ignored.

* Each entry in the list is not a shell glob but a regexp. The ignore list rarely changes, so more expressiveness with a bit higher learning curve and a bit more typing would not hurt much. The entry is implicitly anchored at the left and matched against the path relative to what is recorded in GIT_INDEX_FILE (so 'Makefile' matches "Makefile" in linux-2.6.git/ but not "fs/Makefile"). The regexp can optionally prefixed with a '!' to mean negation. A line that starts with a '#' is a comment.

#11 lore raw

Re: [RFD] Ignore rules

Is there value in:

a. pushing the ignore logic into the core git tools such as git-ls-files

b. including the current ignore .* rule as a default ignore rule that can be overridden by a .gitignore file

jon.

Signature · 2 lines
homepage: http://www.zeta.org.au/~jon/
blog: http://orwelliantremors.blogspot.com/
#12 lore raw

Re: [RFD] Ignore rules

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"JS" == Jon Seymour <[email protected]> writes:

JS> Is there value in:

JS> a. pushing the ignore logic into the core git tools such as git-ls-files

When there is an agreed upon Porcelain layer ignore logic, git-ls-files --others --exclude-from=... should be changed so that it uses the same file format and same semantics, and probably use the default ignore file without being explicitly told. Otherwise things would get quite confusing, so yes, I see value in there.

I do not however think this should apply to things like "git-update-cache --add"; because even though user may say "git-update-cache --add *" and wish it to ignore things on the ignore list, that is not how shell parameter expansion works.

I would like to see a git-path-helper command that can act as filter between "find -print0" and "xargs -0" like this:

    find * -print0 |
    git-path-helper -z --ignore-file=.git/info/ignore |
    xargs -r -0 git-update-cache --add --

I envision that we should not even need --ignore-file parameter, once we have "an agreed upon Porcelain layer ignore logic". It should read the "agreed upon" location, somewhere under the $GIT_DIR and perform the "agreed upon" filtering logic.

I further envision that this would work from anywhere in the work tree, not only from the directory that corresponds to the top of the tree structure GIT_INDEX_FILE describes. For example, in linux-2.6 git tree, you _ought_ to be able to say something like this:

cd fs
find ext? ../include/linux -type f -print0 |
git-path-helper -z |
xargs -r -0 git-update-cache --add --

The git-path-helper command could internally run getpwd() and find out the top directory (in this case, the parent directory of our current working directory "fs"), then canonicalize the incoming filenames (either relative to getpwd() or the full filesystem path) to paths relative to the top directory, apply the ignore filter and send the surviving paths downstream. Anything downstream driven with xargs -0 would get relative paths suitable for core GIT consumption.

JS> b. including the current ignore .* rule as a default ignore rule that JS> can be overridden by a .gitignore file

Knowing the number of places that assume .* are irrelevant, I would not be looking forward to doing that myself, but that behaviour would be ideal.

#13 lore raw

Re: [RFD] Ignore rules

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"JCH" == Junio C Hamano <[email protected]> writes:

JCH> ... For example, in linux-2.6 git tree, you _ought_ to be JCH> able to say something like this:

JCH>     cd fs
JCH>     find ext? ../include/linux -type f -print0 |
JCH>     git-path-helper -z |
JCH>     xargs -r -0 git-update-cache --add --

The above example has an obvious thinko/typo. It should have been like this:

     cd fs
     find ext? ../include/linux -type f -print0 |
     git-path-helper -z | {
         cd .. && xargs -r -0 git-update-cache --add --
     }
#14 lore raw

[RFD] git-run-with-user-path

In reply to Junio C Hamano

Petr, I have not written the implementation yet, but do you think Cogito would find something like this useful? I've seen some patch to add running from subdirectory support to cg-add in the mailing list and I assume Cogito is in the process of being enhanced in this area.

JIT has been supporting running from the subdirectory from the day one, but has been using a bit different mechanism. If we can agree on a core-ish helper (I consider this just as non-core as git-diff-helper is), I would want to switch to use something like this one.

This _deliberately_ conflates the user-path canonicalization with the ignore-list processing discussed recently on the list. When you think about it, they are very closely related and handling them at the same place makes more sense than having them separately. It is a way to munge paths given by users and "find -print0 | xargs -0" into a form appropriate for core GIT consumption. Similar reasoning with which you accepted the use of cg-Xignore by David Greaves in cg-commit.

Linus, and everybody else, suggestions and comments?

------------

git-run-with-user-path(1)
=========================
v0.1, May 2005

NAME
----
git-run-with-user-path - Run command from the top after canonicalizing paths.

SYNOPSIS
--------
'git-run-with-user-path' [options] <command> <argument>... '--' <path>...

DESCRIPTION
-----------
This command takes a <command>, zero or more <argument> and zero or more <path> arguments. <path> arguments name objects on the filesystem, <command> is typically a core GIT command, and <argument> are the initial arguments to the <command>.

It first finds the project top directory (the directory that corresponds to the top of the tree structure GIT_INDEX_FILE describes), and canonicalizes the given <path> to be relative to the project top. It then chdir(2)'s to the project top directory and runs the given <command>, with <argument> and these canonicalized <path> arguments.

This is useful for the Porcelain layer to run core GIT commands from subdirectories. For example, if linux-2.6.git tree is checked out in /usr/src/linux, you can do:

    $ cd /usr/src/linux/fs
    $ ... work in fs directory making changes ...
    $ git-run-with-user-path git-diff-tree -r HEAD -- ext? ../include/linux
    $ find ext? ! -type d -print0 |
      xargs -0 git-run-with-user-path git-update-cache --add -- --

The above is roughly equivalent to:

    $ cd /usr/src/linux
    $ git-diff-tree -r HEAD fs/ext? include/linux
    $ find fs/ext? include/linux ! -type d -print0 |
      xargs git-update-cache --add --

OPTIONS
-------
--no-ignore::

By default, the path arguments are filtered with the same ignore rules Porcelain layers use. With --no-ignore flag, there is no such filtering done.

Author
------
Written by Junio C Hamano <[email protected]>

Documentation
--------------
Documentation by Junio C Hamano.

GIT
---
Part of the link:git.html[git] suite

#15 lore raw

Re: [RFD] Ignore rules

In reply to Jon Seymour
Quoted 4 lines · Jon Seymour

a. pushing the ignore logic into the core git tools such as git-ls-files

b. including the current ignore .* rule as a default ignore rule that can be overridden by a .gitignore file

I'd say YES to both.

My preferred ignore file logic would be:

- stop at first match (that's more efficient)
- !pattern prevents exclusion of matching files
- bash-style shell globs, except that ...
- a pattern that starts with / is a regexp
- * doesn't cross directory boundaries, but ** does
- I don't need a per-repository (i.e. non-checked-in/propagated) ignore file.

Signature · 1 line
Matthias Urlichs   |   {M:U} IT Design @ m-u-it.de   |  [email protected]
#16 lore raw

Re: [RFD] Ignore rules

Quoted 11 lines · Matthias Urlichs
Quoted 6 lines · Jon Seymour

a. pushing the ignore logic into the core git tools such as git-ls-files

b. including the current ignore .* rule as a default ignore rule that can be overridden by a .gitignore file

Quoted 7 lines

I'd say YES to both.

My preferred ignore file logic would be:

- stop at first match (that's more efficient)

more efficient true - but then surely 98% of the time you have to check _all_ patterns since files aren't generally ignored.
And the ability to override earlier matches makes life much easier. So I say no shortcuts, last pattern to match decides ignore/accept status

Quoted 9 lines

- !pattern prevents exclusion of matching files
- bash-style shell globs, except that ...
- a pattern that starts with / is a regexp
- * doesn't cross directory boundaries, but ** does

- I don't need a per-repository (i.e. non-checked-in/propagated) ignore file.

I agree.
But for the sake of checking a couple of files it makes sense to define a complete set of locations.

David

A message that is not in the archive (sent off-list or not yet received): [email protected]

#18 lore raw
Quoted 9 lines · Junio C Hamano

Just a half theoretical question. How well does this perform when your filenames have:

- ' ' (ASCII 0x20)
- '\t' (ASCII 0x09)
- '\n' (ASCII 0x0a)
- '`' (ASCII 0x60) backtick
- '$' (ASCII 0x24) dollar sign

Quoted 5 lines

in them? Or is it the case that the rest of the Cogito is not careful enough and it does not matter to be careful only in this script?

That's what the:
zerosep=$'-d "\0"'
is for.
It looks like the shell 'read' doesn't honour it though - I couldn't make it work.

However, thanks, after I fixed the quotes I missed in:
-o \( $pass_files -eq 1 -a -f "$file" \) \
-o \( $pass_dirs -eq 1 -a -d "$file" \) \
-o \( $pass_links -eq 1 -a -h "$file" \) \

it handles all cases above except \n (of course)

Frankly I think we're beyond shell programming and we should be using perl (IMHO as the 'best' and most portable text handler) It also has a *fantastic* test harness available.

David