Skip to contentSkip to conversation

cpio command not found

PATCH In review Bill Lear 13 participants 26 messages Jun 15, 2016

lore
Share & tools
lore

Bill Lear

author
#0 lore raw

I just cloned the latest git and it builds ok, but fails when I clone:

% uname -a
Linux gronk.zopyra.com 2.4.9-21 #1 Thu Jan 17 14:16:30 EST 2002 i686 unknown

% git --version
git version gitgui.0.8.4.g9c514

% git clone --bare ~/my_repo
Initialized empty Git repository in /repos/git/my_repo/ /opt/git/bin/git-clone: line 297: cpio: command not found

So, should git not check this when it is building? I don't remember this dependence from earlier versions of git. I have been running git 1.4.xx on this machine for a while...

Bill

#1 lore raw

PATCH 1/1 Don't use cpio in git-clone when not installed

---
 git-clone.sh |    4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)
git-clone.sh +3 -1
index 0ea3c24..57e96ae 100755
@@ -191,7 +191,9 @@ fi
 # it is local
 if base=$(get_repo_base "$repo"); then
 	repo="$base"
-	local=yes
+	if type cpio > /dev/null 2>&1; then
+		local=yes
+	fi
 fi
 
 dir="$2"
Signature · 1 line
1.5.3.4
#2 lore raw

Re: [PATCH] Don't use cpio in git-clone when not installed

Damn, and I forgot git-send-email doesn't add the <> to In-Reply-To.

Mike

#3 lore raw

Re: [PATCH] Don't use cpio in git-clone when not installed

In reply to Mike Hommey

"type"? That's probably better than using "which", but sounds quite wrong. Why not add Makefile target that the shell scripts depend on to see if necessary tools are available and let the builder know if they are not?

#4 lore raw

Re: [PATCH] Don't use cpio in git-clone when not installed

Hi,

Quoted 4 lines · Junio C Hamano

"type"? That's probably better than using "which", but sounds quite wrong. Why not add Makefile target that the shell scripts depend on to see if necessary tools are available and let the builder know if they are not?

I thought the accord was that "cpio" is a _runtime_ dependency. But yes, I agree "type" is not necessarily the best choice. Maybe testing the exit code for 127 (not found) is the better choice?

Ciao,
Dscho

#5 lore raw

Re: [PATCH] Don't use cpio in git-clone when not installed

In reply to Mike Hommey

git-clone.sh fi · line 196

Quoted 3 lines · Mike Hommey
git-clone.sh +3 -0
+	if type cpio > /dev/null 2>&1; then
+		local=yes
+	fi

Isn't "type" a bashism?

Maybe just do

	if echo . | cpio -o > /dev/null 2>&1; then

instead? Maybe even doing this at install time to avoid the overhead of executing another process..

Linus

#6 lore raw

Re: [PATCH] Don't use cpio in git-clone when not installed

Quoted 8 lines · Linus Torvalds

git-clone.sh fi · line 196

Quoted 3 lines · Mike Hommey
git-clone.sh +3 -0
+	if type cpio > /dev/null 2>&1; then
+		local=yes
+	fi

Isn't "type" a bashism?

I think it's POSIX, and since I found the same construct in git-mergetool.sh, I thought it would be okay for git.

Mike

#7 lore raw

Re: [PATCH] Don't use cpio in git-clone when not installed

In reply to Linus Torvalds
Quoted 6 lines · Linus Torvalds

git-clone.sh fi · line 196

Quoted 3 lines · Mike Hommey
git-clone.sh +3 -0
+	if type cpio > /dev/null 2>&1; then
+		local=yes
+	fi

Isn't "type" a bashism?

I seem to recall that it is in POSIX.

#8 lore raw

Re: [PATCH] Don't use cpio in git-clone when not installed

In reply to Linus Torvalds
Quoted 8 lines · Linus Torvalds

git-clone.sh fi · line 196

Quoted 3 lines · Mike Hommey
git-clone.sh +3 -0
+     if type cpio > /dev/null 2>&1; then
+             local=yes
+     fi

Isn't "type" a bashism?

busybox ash has "type". I'm happy.

Signature · 1 line
Duy
#9 lore raw

Re: [PATCH] Don't use cpio in git-clone when not installed

In reply to Linus Torvalds
Quoted 7 lines · Linus Torvalds

git-clone.sh fi · line 196

Quoted 3 lines · Mike Hommey
git-clone.sh +3 -0
+    if type cpio > /dev/null 2>&1; then
+            local=yes
+    fi
Quoted 8 lines

Isn't "type" a bashism?

Maybe just do

      if echo . | cpio -o > /dev/null 2>&1; then

instead? Maybe even doing this at install time to avoid the overhead of executing another process..

Or perhaps trap / check actual execution of cpio in git-clone, and fallback to ln -s / link if it fails...

Signature · 3 lines
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git
#10 lore raw

Re: [PATCH] Don't use cpio in git-clone when not installed

In reply to Linus Torvalds
Quoted 8 lines · Linus Torvalds <torvalds <at> linux-foundation.org> writes:

git-clone.sh fi · line 196

Quoted 3 lines · Mike Hommey
git-clone.sh +3 -0
+	if type cpio > /dev/null 2>&1; then
+		local=yes
+	fi

Isn't "type" a bashism?

ksh also has type. But SUS states, that "command -v" can be used.

Options

[...]

-v
(On systems supporting the User Portability Utilities option.) Write a string to standard output that indicates the pathname or command that will be used by the shell, in the current shell execution environment (see Shell Execution Environment ), to invoke command_name, but do not invoke command_name.

* Utilities, regular built-in utilities, command_names including a slash character, and any implementation-defined functions that are found using the PATH variable (as described in Command Search and Execution ), shall be written as absolute pathnames.

* Shell functions, special built-in utilities, regular built-in utilities not associated with a PATH search, and shell reserved words shall be written as just their names.

* An alias shall be written as a command line that represents its alias definition.

* Otherwise, no output shall be written and the exit status shall reflect that the name was not found.

Regards,
Alexander

#11 lore raw

Re: [PATCH] Don't use cpio in git-clone when not installed

In reply to Mike Hommey

BTW, you have workaround for git-merge also? It uses cpio to save/restore state.

Quoted 29 lines · Mike Hommey
---
 git-clone.sh |    4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)
git-clone.sh +3 -2
index 0ea3c24..57e96ae 100755
@@ -191,7 +191,9 @@ fi
 # it is local
 if base=$(get_repo_base "$repo"); then
        repo="$base"
-       local=yes
+       if type cpio > /dev/null 2>&1; then
+               local=yes
+       fi
 fi
 
 dir="$2"
--

1.5.3.4

-
To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Signature · 1 line
Duy
#12 lore raw

Re: [PATCH] Don't use cpio in git-clone when not installed

Quoted 1 line · Nguyen Thai Ngoc Duy

BTW, you have workaround for git-merge also? It uses cpio to save/restore state.

Why do people want "workaround"? Is installing cpio such a hassle?

#13 lore raw

Re: [PATCH] Don't use cpio in git-clone when not installed

Quoted 6 lines · Junio C Hamano
Quoted 1 line · Nguyen Thai Ngoc Duy

BTW, you have workaround for git-merge also? It uses cpio to save/restore state.

Why do people want "workaround"? Is installing cpio such a hassle?

Note that to do what git-merge does with cpio, i wonder if it wouldn't be sensible to use git stash, now.

Mike

#14 lore raw

Re: [PATCH] Don't use cpio in git-clone when not installed

Quoted 10 lines · Mike Hommey
Quoted 6 lines · Junio C Hamano
Quoted 1 line · Nguyen Thai Ngoc Duy

BTW, you have workaround for git-merge also? It uses cpio to save/restore state.

Why do people want "workaround"? Is installing cpio such a hassle?

Note that to do what git-merge does with cpio, i wonder if it wouldn't be sensible to use git stash, now.

Like this? That's an excellent suggestion.

The patch uses the 'git stash create' which is in 'next' (jc/stash-create topic).

Having said that, the savestate()/restorestate() codepaths are only relevant to the "try multiple strategies and pick the best one" feature of git-merge, and that is where cpio is used (and the patch rewrites it to use stash). I am not sure if anybody ever used it in practice. I admit I am guilty of inventing it, but I certainly do not.

It might make sense to remove that try-multiple-strategies feature from git-merge and be done with it. It would certainly make the eventual rewrite to C much easier.

---
 git-merge.sh |   11 +++++------
 1 files changed, 5 insertions(+), 6 deletions(-)
git-merge.sh +5 -6
index 3a01db0..e8916cc 100755
@@ -28,20 +28,19 @@ allow_trivial_merge=t
 
 dropsave() {
 	rm -f -- "$GIT_DIR/MERGE_HEAD" "$GIT_DIR/MERGE_MSG" \
-		 "$GIT_DIR/MERGE_SAVE" || exit 1
+		 "$GIT_DIR/MERGE_STASH" || exit 1
 }
 
 savestate() {
 	# Stash away any local modifications.
-	git diff-index -z --name-only $head |
-	cpio -0 -o >"$GIT_DIR/MERGE_SAVE"
+	git stash create >"$GIT_DIR/MERGE_STASH"
 }
 
 restorestate() {
-        if test -f "$GIT_DIR/MERGE_SAVE"
+        if test -f "$GIT_DIR/MERGE_STASH"
 	then
 		git reset --hard $head >/dev/null
-		cpio -iuv <"$GIT_DIR/MERGE_SAVE"
+		git stash apply --index $(cat "$GIT_DIR/MERGE_STASH")
 		git update-index --refresh >/dev/null
 	fi
 }
@@ -386,7 +385,7 @@ case "$use_strategies" in
     single_strategy=no
     ;;
 *)
-    rm -f "$GIT_DIR/MERGE_SAVE"
+    rm -f "$GIT_DIR/MERGE_STASH"
     single_strategy=yes
     ;;
 esac
#15 lore raw

Re: [PATCH] Don't use cpio in git-clone when not installed

In reply to Junio C Hamano
Quoted 6 lines · Junio C Hamano
Quoted 1 line · Nguyen Thai Ngoc Duy

BTW, you have workaround for git-merge also? It uses cpio to save/restore state.

Why do people want "workaround"? Is installing cpio such a hassle?

It is on Windows because busybox cpio is not really good and busybox tar is even worse (for cpio emulation). Maybe I should just improve busybox cpio :-)

Signature · 1 line
Duy
#16 lore raw

Re: [PATCH] Don't use cpio in git-clone when not installed

Quoted 3 lines · Nguyen Thai Ngoc Duy

It is on Windows because busybox cpio is not really good and busybox tar is even worse (for cpio emulation). Maybe I should just improve busybox cpio :-)

Sounds sensible, as it (at least its -p mode of operation) is one of the programs that is very useful yet not so well known to people new to UNIX.

#17 lore raw
In reply to Bill Lear
Quoted 2 lines · Bill Lear

I don't remember this dependence from earlier versions of git. I have been running git 1.4.xx on this machine for a while...

When you clone with -l, git uses cpio to hardlink to the original repository. What has changed is that -l is now used by default when cloning a repository that's accessed via the file system (as opposed to over some network protocol).

To work around this, specify the repository location with file://, and git won't try to hardlink (and hence won't try to use cpio).

Signature · 2 lines
Karl Hasselström, [email protected]
      www.treskal.com/kalle
author
#18 lore raw
Quoted 5 lines · On Wednesday, October 31, 2007 at 14:30:39 (+0100) Karl Hasselström writes:
Quoted 2 lines · Bill Lear

I don't remember this dependence from earlier versions of git. I have been running git 1.4.xx on this machine for a while...

Quoted 7 lines

When you clone with -l, git uses cpio to hardlink to the original repository. What has changed is that -l is now used by default when cloning a repository that's accessed via the file system (as opposed to over some network protocol).

To work around this, specify the repository location with file://, and git won't try to hardlink (and hence won't try to use cpio).

Hmm, thanks for the workaround, but I don't altogether like leaving things like this.

If the system does not have cpio, I think the build of git should complain and fail, or it should activate code that treats any repository accessed over the file system as it would file://.

No sense in leaving this surprise to the user so late in the cycle.

Bill

#19 lore raw
Quoted 3 lines · Bill Lear

If the system does not have cpio, I think the build of git should complain and fail, or it should activate code that treats any repository accessed over the file system as it would file://.

git may be build and run on two different hosts, so while the build-time check is good there should be a run-time check as well.

/Allan

#20 lore raw
In reply to Bill Lear
Quoted 14 lines · Bill Lear
Quoted 5 lines · On Wednesday, October 31, 2007 at 14:30:39 (+0100) Karl Hasselström writes:
Quoted 2 lines · Bill Lear

I don't remember this dependence from earlier versions of git. I have been running git 1.4.xx on this machine for a while...

Quoted 7 lines

When you clone with -l, git uses cpio to hardlink to the original repository. What has changed is that -l is now used by default when cloning a repository that's accessed via the file system (as opposed to over some network protocol).

To work around this, specify the repository location with file://, and git won't try to hardlink (and hence won't try to use cpio).

Quoted 6 lines

Hmm, thanks for the workaround, but I don't altogether like leaving things like this.

If the system does not have cpio, I think the build of git should complain and fail, or it should activate code that treats any repository accessed over the file system as it would file://.

Something like this could be done at run-time instead. You might install cpio, but shouldn't require a rebuild of git just to use it.

Dave.

#21 lore raw
In reply to Karl Hasselström
Quoted 9 lines · Karl Hasselstr?m
Quoted 2 lines · Bill Lear

I don't remember this dependence from earlier versions of git. I have been running git 1.4.xx on this machine for a while...

When you clone with -l, git uses cpio to hardlink to the original repository. What has changed is that -l is now used by default when cloning a repository that's accessed via the file system (as opposed to over some network protocol).

Why cpio? What is wrong with ln(1) (every Unix should have one) or link(2) ?

Erik

Signature · 2 lines
They're all fools. Don't worry. Darwin may be slow, but he'll
eventually get them. -- Matthew Lammers in alt.sysadmin.recovery
#22 lore raw
Quoted 6 lines · Erik Mouw
Quoted 5 lines · Karl Hasselstr?m
Quoted 2 lines · Bill Lear

I don't remember this dependence from earlier versions of git. I have been running git 1.4.xx on this machine for a while...

Quoted 7 lines
Quoted 4 lines

When you clone with -l, git uses cpio to hardlink to the original repository. What has changed is that -l is now used by default when cloning a repository that's accessed via the file system (as opposed to over some network protocol).

Why cpio? What is wrong with ln(1) (every Unix should have one) or link(2) ?

Since git-clone is not yet a builtin, and is actually a shell script, it makes more sense to use cpio than ln.

Mike

#23 lore raw
In reply to Erik Mouw

Hi,

Quoted 6 lines · Erik Mouw
Quoted 5 lines · Karl Hasselstr?m
Quoted 2 lines · Bill Lear

I don't remember this dependence from earlier versions of git. I have been running git 1.4.xx on this machine for a while...

Quoted 7 lines
Quoted 4 lines

When you clone with -l, git uses cpio to hardlink to the original repository. What has changed is that -l is now used by default when cloning a repository that's accessed via the file system (as opposed to over some network protocol).

Why cpio? What is wrong with ln(1) (every Unix should have one) or link(2) ?

Patch, please?

Ciao,
Dscho

#24 lore raw

PATCH 1/1 Get rid of cpio in git-clone (was: Re: cpio command not found)

Quoted 13 lines · Johannes Schindelin
Quoted 11 lines · Erik Mouw
Quoted 9 lines · Karl Hasselstr?m
Quoted 2 lines · Bill Lear

I don't remember this dependence from earlier versions of git. I have been running git 1.4.xx on this machine for a while...

When you clone with -l, git uses cpio to hardlink to the original repository. What has changed is that -l is now used by default when cloning a repository that's accessed via the file system (as opposed to over some network protocol).

Quoted 4 lines
Quoted 2 lines

Why cpio? What is wrong with ln(1) (every Unix should have one) or link(2) ?

Patch, please?

Here you go.

Remove dependency on cpio for git-clone. Apparently some POSIX systems out there don't have cpio, just assume cp is there.

INSTALL +1 -2
index f1eb404..9074563 100644
@@ -79,8 +79,7 @@ Issues of note:
 	- "perl" and POSIX-compliant shells are needed to use most of
 	  the barebone Porcelainish scripts.
 
-	- "cpio" is used by git-merge for saving and restoring the index,
-	  and by git-clone when doing a local (possibly hardlinked) clone.
+	- "cpio" is used by git-merge for saving and restoring the index.
 
  - Some platform specific issues are dealt with Makefile rules,
    but depending on your specific installation, you may not
git-clone.sh +1 -1
index 0ea3c24..061534c 100755
@@ -294,7 +294,7 @@ yes)
 			fi
 		fi &&
 		cd "$repo" &&
-		find objects -depth -print | cpio -pumd$l "$GIT_DIR/" || exit 1
+		cp -Rp$l objects/ "$GIT_DIR/" || exit 1
 	fi
 	git-ls-remote "$repo" >"$GIT_DIR/CLONE_HEAD" || exit 1
 	;;
Signature · 2 lines
They're all fools. Don't worry. Darwin may be slow, but he'll
eventually get them. -- Matthew Lammers in alt.sysadmin.recovery
#25 lore raw

Re: [PATCH] Get rid of cpio in git-clone (was: Re: cpio command not found)

Quoted 3 lines · Erik Mouw
Quoted 1 line

Patch, please?

Here you go.

Good, now we have something to critique. :)

git-clone.sh yes) · line 297

Quoted 2 lines
git-clone.sh +1 -1
-		find objects -depth -print | cpio -pumd$l "$GIT_DIR/" || exit 1
+		cp -Rp$l objects/ "$GIT_DIR/" || exit 1

cp -l isn't even close to portable. It's not in POSIX, and doesn't work on (at least) Solaris.

I think Mike's patch (cpio if available, copy otherwise) is a reasonable approach. If there are other methods (and I think cp -l is not unreasonable for systems where it is supported and cpio is unavailable), then perhaps it is worth trying them one by one and dropping back to full copy if all fail.

-Peff