Skip to contentSkip to conversation

Makefile targets: tar & rpm pkgs, while using O=<dir> as non-root

DISCUSSION Linda Walsh 8 participants 18 messages Dec 18, 2005 – Dec 30, 2005

lore
Share & tools
lore
#0 lore raw

Unpacked 2.6.13.3 and made it read-only.

Using the "O=" param, built output tree for another machine as a non-root user.

I wanted to create an installable kernel & module package to copy to the new machine & install.

I noted new targets:
binrpm-pkg [& rpm-pkg], and
tarbz2-pkg [& targz-pkg, & tar-pkg].

Both seem to fail either for reasons that appear to be related to not honoring the "O=" param, or attempting to actually install into the root of my build-machine.

Should these targets work or have they not yet been converted to work within the "O=" framework? In cases where the Makefile is attempting to install into "<Root>/boot" or "<Root>/lib/modules" ,should I expect the output to appear in "$O/boot" and "$O/lib/modules/"?

Please Cc responses to me, in addition to list, thanks.

-linda

#1 lore raw
Quoted 9 lines · Linda Walsh

Unpacked 2.6.13.3 and made it read-only.

I noted new targets:
binrpm-pkg [& rpm-pkg], and
tarbz2-pkg [& targz-pkg, & tar-pkg].

Both seem to fail either for reasons that appear to be related to not honoring the "O=" param, or attempting to actually install into the root of my build-machine.

I've not really used out-of-tree building for Linux, so the tar* targets are probably not really tested with that regard. Though I'll check that and see if it works (of if a patch is needed.)

The RPM targets have to be tested by somebody else :)

MfG, JBG

Signature · 4 lines
Jan-Benedict Glaw       [email protected]    . +49-172-7608481             _ O _
"Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg  _ _ O
 für einen Freien Staat voll Freier Bürger"  | im Internet! |   im Irak!   O O O
ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));
#2 lore raw
Quoted 7 lines · Jan-Benedict Glaw

I've not really used out-of-tree building for Linux, so the tar* targets are probably not really tested with that regard. Though I'll check that and see if it works (of if a patch is needed.)

The RPM targets have to be tested by somebody else :)

---
Thanks -- the error I got for the "tar"-based images started as:

ishtar:/usr/src/linux-2.6.14.3> make V=1 INSTALL_ROOT= ../build/ast-root O=../build/ast-26143/ tarbz2-pkg make -C /home/build/ast-26143 \
KBUILD_SRC=/usr/src/linux-2.6.14.3 \
KBUILD_EXTMOD="" -f /usr/src/linux-2.6.14.3/Makefile ../build/ast-root
make[1]: Nothing to be done for `/usr/src/linux-2.6.14.3/../build/ast-root'.
make -C /home/build/ast-26143 \
KBUILD_SRC=/usr/src/linux-2.6.14.3 \
KBUILD_EXTMOD="" -f /usr/src/linux-2.6.14.3/Makefile tarbz2-pkg
make -f /usr/src/linux-2.6.14.3/scripts/package/Makefile tarbz2-pkg
make
make -C /usr/src/linux-2.6.14.3 O=/home/build/ast-26143
Makefile:484: .config: No such file or directory
Using /usr/src/linux-2.6.14.3 as source for kernel
if [ -f /usr/src/linux-2.6.14.3/.config ]; then \
echo " /usr/src/linux-2.6.14.3 is not clean, please run 'make

mrproper'";\
       echo "  in the '/usr/src/linux-2.6.14.3' directory.";\
       /bin/false; \

fi;
if [ ! -d include2 ]; then mkdir -p include2; fi;
mkdir: cannot create directory `include2': Permission denied make[4]: *** [prepare3] Error 1
make[3]: *** [all] Error 2
make[2]: *** [tarbz2-pkg] Error 2
make[1]: *** [tarbz2-pkg] Error 2
make: *** [tarbz2-pkg] Error 2
==========

Briefly, the make rpm-pagage options call "make install" and "make modules_install". Both attempt to install the kernel for the "target" system onto the the "Build" system. This isn't desired as the "target" kernel won't even run on the "build" system.

I could supply error listings, but it seems unneeded as it is so easily reproduced.

I see JanB is on the making "tar" image, but how long ago was building of the rpm stuff done? Is it the case that the designer/writer of that hasn't had time to make it work with the "O=" param? Is someone still actively maintaining those targets?

Ideally it would be good to be able to make an installable kernel package as a non-root user. The idea is to not need root-privs except to install, right? :-)

Linda

#3 lore raw
Quoted 3 lines · Linda Walsh

Ideally it would be good to be able to make an installable kernel package as a non-root user. The idea is to not need root-privs except to install, right? :-)

That's how it is ment to be. Actually, for in-tree builds, you get the tarball without first becoming root. I'll fix that soon, as I find some spare minute to hack it :)

MfG, JBG

Signature · 4 lines
Jan-Benedict Glaw       [email protected]    . +49-172-7608481             _ O _
"Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg  _ _ O
 für einen Freien Staat voll Freier Bürger"  | im Internet! |   im Irak!   O O O
ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));
#4 lore raw
Quoted 3 lines · Jan-Benedict Glaw

That's how it is ment to be. Actually, for in-tree builds, you get the tarball without first becoming root.

---
So I see. Slight minor possible "nit".
I have a suffix (CONFIG_LOCALVERSION) set.
While the tar picks up the /lib/modules files in <ver>-suffix, the map and vm files all have suffixes w/o the "localversion" appended. This doesn't seem desirable, eh?:

ishtar:/usr/src/ast-26144> tar tf linux-2.6.14.4.tar |more ./
./boot/
./boot/System.map-2.6.14.4
./boot/config-2.6.14.4
./boot/vmlinux-2.6.14.4
./boot/vmlinuz-2.6.14.4
./lib/
./lib/modules/
./lib/modules/2.6.14.4-ast/
./lib/modules/2.6.14.4-ast/kernel/
./lib/modules/2.6.14.4-ast/kernel/crypto/
....

Quoted 3 lines

I'll fix that soon, as I find
some spare minute to hack it :)

---
S'okey...I can at least use the above to xfer files over for now...

tnx,
Linda

#8 lore raw
In reply to Jan-Benedict Glaw

On 19 Dec 2005, Jan-Benedict Glaw stated:

Quoted 3 lines

I've not really used out-of-tree building for Linux, so the tar* targets are probably not really tested with that regard. Though I'll check that and see if it works (of if a patch is needed.)

Given the existence of `cp -al', I'm not quite sure what the point of out-of-tree building *is*. (I'm equally mystified by srcdir!=objdir builds; I can see their utility for software with many generated files and multi-stage bootstraps, like GCC, 'cos they simplify the makefiles, but for normal software? What's the point?)

I mean, instead of

cd linux-2.6.blah
make blah O=/some/directory

you can always do

cp -al linux-2.6.blah /some/directory/
cd /some/directory
make blah

unless you're so short of space that you can't store the object files and the source files on the same disk, and that should be a *very* rare complaint with today's disk sizes.

Signature · 2 lines
`I must caution that dipping fingers into molten lead
 presents several serious dangers.' --- Jearl Walker
#9 lore raw
Quoted 4 lines · Nix

but for normal software? What's the point?)

I mean, instead of

Quoted 8 lines

cd linux-2.6.blah
make blah O=/some/directory

you can always do

cp -al linux-2.6.blah /some/directory/
cd /some/directory
make blah

Consider following use cases:
1) Src located on NFS mounted filesystem
2) Src on RO media
3) Src on another filesystem
4) Builds for several architectures from same source base
5) Builds for several different configurations etc.

It is convinient in many places. Maybe not for you but for others.

Sam

#17 lore raw
In reply to Linda Walsh
Quoted 12 lines · Linda Walsh

Unpacked 2.6.13.3 and made it read-only.

Using the "O=" param, built output tree for another machine as a non-root user.

I wanted to create an installable kernel & module package to copy to the new machine & install.

I noted new targets:
binrpm-pkg [& rpm-pkg], and
tarbz2-pkg [& targz-pkg, & tar-pkg].

Quoted 8 lines

Both seem to fail either for reasons that appear to be related to not honoring the "O=" param, or attempting to actually install into the root of my build-machine.

Should these targets work or have they not yet been converted to work within the "O=" framework? In cases where the Makefile is attempting to install into "<Root>/boot" or "<Root>/lib/modules" ,should I expect the output to appear in "$O/boot" and "$O/lib/modules/"?

Look at the "deb" target for how this was fixed for building Debian (-style) packages.

Specifically, you want to change:

	$(MAKE)
to
	$(MAKE) KBUILD_SRC=

At a glance, I don't see a similar problem in the binrpm-pkg target, and I don't understand the rpm target at all, so those may have other issues.

I am, however, looking at 2.6.15-rc{something}, not 2.6.13, but I think it's been a while since Sam sent the deb packages fixes upstream.

Signature · 2 lines
Ryan Anderson
  sometimes Pug Majere
#10 lore raw
In reply to Sam Ravnborg
Quoted 2 lines · Sam Ravnborg

Consider following use cases:
1) Src located on NFS mounted filesystem

Nothing wrong with that. NFS is fast for me.

(Admittedly, on heavily contended networks you're right.)

Quoted 1 line

2) Src on RO media

*Good* reason (one that unionfs would solve, but still...)

Quoted 1 line

4) Builds for several architectures from same source base

cp -al

Quoted 1 line

5) Builds for several different configurations

cp -al

Quoted 1 line

It is convinient in many places. Maybe not for you but for others.

There were a couple of compelling arguments in there. :)

Signature · 2 lines
`I must caution that dipping fingers into molten lead
 presents several serious dangers.' --- Jearl Walker
#11 lore raw
Quoted 1 line · Nix

cp -al

$ time cp -al linux-2.6 nix
real 0m5.360s

That was second time I did it. It took considerably longer the first time when nothing was in cache.

Sam

#12 lore raw
In reply to Nix
Quoted 7 lines · Nix
Quoted 1 line

4) Builds for several architectures from same source base

cp -al

Quoted 1 line

5) Builds for several different configurations

cp -al

... apply a patch and watch the resync hell. That really, really doesn't work well enough for use when doing any kind of development.

#13 lore raw
Quoted 11 lines · Al Viro
Quoted 7 lines · Nix
Quoted 1 line

4) Builds for several architectures from same source base

cp -al

Quoted 1 line

5) Builds for several different configurations

cp -al

... apply a patch and watch the resync hell. That really, really doesn't work well enough for use when doing any kind of development.

Well, personally I handle patch-application in cp -al'ed trees by doing cp -al via a script, and repatching all currently hardlinked trees (obviously if they are very divergent some patches will fail and I'll have to fix them up by hand).

It works for me well enough to keep hardlinked branches going for in some cases years without problems.

(On top of that, I've sometimes considered a switch to patch(1) that switches to truncate-and-rewrite rather than unlink-and-replace. Haven't implemented it yet though.)

(And if you're using this to maintain development branches, then you have resync and conflict-management problems *anyway*, which this makes no worse.)

Signature · 2 lines
`I must caution that dipping fingers into molten lead
 presents several serious dangers.' --- Jearl Walker
#14 lore raw
Quoted 4 lines · Nix

Well, personally I handle patch-application in cp -al'ed trees by doing cp -al via a script, and repatching all currently hardlinked trees (obviously if they are very divergent some patches will fail and I'll have to fix them up by hand).

... then you edit a file to fix a typo, and have a _nice_ day next Friday when you notice that stuff got out of sync.

Quoted 11 lines

It works for me well enough to keep hardlinked branches going for in some cases years without problems.

(On top of that, I've sometimes considered a switch to patch(1) that switches to truncate-and-rewrite rather than unlink-and-replace. Haven't implemented it yet though.)

(And if you're using this to maintain development branches, then you have resync and conflict-management problems *anyway*, which this makes no worse.)

Yes, but it's easier to deal with when the number of your repositories doesn't get multiplied by factor of 20 or so...

#16 lore raw
In reply to Nix
Quoted 7 lines · Nix

Well, personally I handle patch-application in cp -al'ed trees by doing cp -al via a script, and repatching all currently hardlinked trees (obviously if they are very divergent some patches will fail and I'll have to fix them up by hand).

It works for me well enough to keep hardlinked branches going for in some cases years without problems.

Well the O= syntax is much nicer when using Git. You symply checkout the
branch you're interested in and compile it; then switch to another, compile it, etc... Git makes checking out a brach lightning fast, it's
very slick. Using the O= notation means you only have one branch checked
out at a time and there's no need to have source files linked to a bunch of different directories.

Cheers,
Sean

#15 lore raw
In reply to Al Viro

On Wed, 21 Dec 2005, Al Viro stated:

Quoted 8 lines · Nix
Quoted 4 lines

Well, personally I handle patch-application in cp -al'ed trees by doing cp -al via a script, and repatching all currently hardlinked trees (obviously if they are very divergent some patches will fail and I'll have to fix them up by hand).

... then you edit a file to fix a typo, and have a _nice_ day next Friday when you notice that stuff got out of sync.

Your text editor is insufficiently flexible. Mine can snap hardlinks automatically when st_nlink > 1. :)

(But let's avoid editor wars. I'm sure there's a magic way vim can be coerced into doing things properly.)

What we really need is an FS that does behind-the-scenes block CoW and/or block merging anyway; then we could just cp -a the damn tree and keep the space savings.

Quoted 6 lines
Quoted 3 lines

(And if you're using this to maintain development branches, then you have resync and conflict-management problems *anyway*, which this makes no worse.)

Yes, but it's easier to deal with when the number of your repositories doesn't get multiplied by factor of 20 or so...

Er, I don't have a factor of 20 more repositories than anyone else. (I don't have the disk space for that!)

Signature · 2 lines
`I must caution that dipping fingers into molten lead
 presents several serious dangers.' --- Jearl Walker
#5 lore raw

patch for "scripts/package/buildtar" to pickup "localversion" on "/boot" file objects

In reply to Linda Walsh
Quoted 10 lines · Linda Walsh
Quoted 3 lines · Jan-Benedict Glaw

That's how it is ment to be. Actually, for in-tree builds, you get the tarball without first becoming root.

---
Slight minor possible "nit".
I have a suffix (CONFIG_LOCALVERSION) set.
While buildtar picks up the /lib/modules files in <ver>-suffix, the map and vm files all have suffixes w/o the "localversion" appended.

---
BTW -- here is a "quick" patch to pick up the "local version" to append to the files in "/boot" for the "tar[x]-pkg's"...

package/buildtar 2005-12-29 22:26:34.543513254 -0800 renamed from package/buildtar.orig 2005-11-24 14:10:21.000000000 -0800 +0 -0
@@ -15,7 +15,8 @@

#
# Some variables and settings used throughout the script
# -version="${VERSION}.${PATCHLEVEL}.${SUBLEVEL}${EXTRAVERSION}${EXTRANAME}" +version="$(grep UTS_RELEASE include/linux/version.h| \
+ sed 's/^#[^"]*"// ; s/"$// ')"
tmpdir="${objtree}/tar-install" tarball="${objtree}/linux-${version}.tar"

Quoted 1 line
#6 lore raw

Re: patch for "scripts/package/buildtar" to pickup "localversion" on "/boot" file objects

package/buildtar 2005-12-29 22:26:34.543513254 -0800 line 19

Quoted 12 lines · Linda A. Walsh

BTW -- here is a "quick" patch to pick up the "local version" to append to the files in "/boot" for the "tar[x]-pkg's"...

package/buildtar 2005-12-29 22:26:34.543513254 -0800 renamed from package/buildtar.orig 2005-11-24 14:10:21.000000000 -0800 +2 -1
@@ -15,7 +15,8 @@
 #
 # Some variables and settings used throughout the script
 #
-version="${VERSION}.${PATCHLEVEL}.${SUBLEVEL}${EXTRAVERSION}${EXTRANAME}"
+version="$(grep UTS_RELEASE include/linux/version.h| \
+       sed 's/^#[^"]*"// ; s/"$// ')"

A much easier way is to use:

  version="${KERNELRELEASE}"

A patch for this is already in -mm, I expect/hope Linus will pick it up after 2.6.15.

Erik

Signature · 2 lines
+-- Erik Mouw -- www.harddisk-recovery.com -- +31 70 370 12 90 --
| Lab address: Delftechpark 26, 2628 XH, Delft, The Netherlands
#7 lore raw

Re: patch for "scripts/package/buildtar" to pickup "localversion" on "/boot" file objects

Quoted 8 lines · Erik Mouw

A much easier way is to use:

version="${KERNELRELEASE}"

A patch for this is already in -mm, I expect/hope Linus will pick it up after 2.6.15.

---
Cool! Easier is good! :-)
-linda