kpatchup 0.02 kernel patching script
This is the first release of kpatchup, a script for managing switching between kernel releases via patches with some smarts:
- understands -pre and -rc version numbering
- aware of various external trees
- automatically patch between any tree in an x.y release
- automatically download and cache patches on demand
- automatically determine the latest patch in various series
- optionally print version strings or URLs for patches
Currently it knows about 2.4, 2.4-pre, 2.6, 2.6-pre, 2.6-bk, 2.6-mm, and 2.6-tiny.
Example usage:
$ head Makefile VERSION = 2 PATCHLEVEL = 6 SUBLEVEL = 2 EXTRAVERSION =-rc2 [...] $ kpatchup 2.6-mm 2.6.2-rc2 -> 2.6.4-rc1-mm1 Applying patch-2.6.2-rc2.bz2 -R Applying patch-2.6.2.bz2 Applying patch-2.6.3.bz2 Downloading patch-2.6.4-rc1.bz2... Applying patch-2.6.4-rc1.bz2 Downloading 2.6.4-rc1-mm1.bz2... Applying 2.6.4-rc1-mm1.bz2 $ head Makefile VERSION = 2 PATCHLEVEL = 6 SUBLEVEL = 4 EXTRAVERSION =-rc1-mm1 NAME=Feisty Dunnart [...] $ kpatchup -q 2.6.3-rc1 $ head Makefile VERSION = 2 PATCHLEVEL = 6 SUBLEVEL = 3 EXTRAVERSION =-rc1 NAME=Feisty Dunnart [...] $ kpatchup -s 2.6-bk 2.6.4-rc1-bk3 $ kpatchup -u 2.4-pre http://www.kernel.org/pub/linux/kernel/v2.4/testing/patch-2.4.26-pre1.bz2
This is an alpha release for people to experiment with. Feedback and patches encouraged. Grab your copy today at:
Signature · 1 line
Matt Mackall : http://www.selenic.com : Linux development and consulting
Quoted 10 lines · Matt Mackall
This is the first release of kpatchup, a script for managing switching between kernel releases via patches with some smarts:
- understands -pre and -rc version numbering
- aware of various external trees
- automatically patch between any tree in an x.y release
- automatically download and cache patches on demand
- automatically determine the latest patch in various series
- optionally print version strings or URLs for patches
Quoted 2 lines
Currently it knows about 2.4, 2.4-pre, 2.6, 2.6-pre, 2.6-bk, 2.6-mm, and 2.6-tiny.
Oh i definitely owe you one now, this is replacing the ugly shell script i had before, i'm mostly using this now to download and patch up trees before cvs import'ing them.
Thanks!
Quoted 2 lines · Matt Mackall
This is the first release of kpatchup, a script for managing switching between kernel releases via patches with some smarts:
Hi Matt!
Please find below my grab_kernel, latest-kernel-version and lkvercmp scripts, in case they are useful for you to merge. Kinda rough, like any unexposed code...
Rusty
Signature · 1 line
Anyone who quotes me in their signature is an idiot -- Rusty Russell
#! /bin/sh
# Compare multiple kernel versions
# Return more recent of two kernel versions. if [ $# -eq 0 ]; then
echo Usage: $0 kernelversion kernelversion ... >&2 echo Returns greatest of the versions. >&2 exit 1
fi
unknown()
{
# Does a -dj kernel come before a -ac kernel? Or a Linus kernel? # Undefined.
echo Can\'t compare extension \'"$1"\'. >&2
exit 1
}
firstpart()
{
echo $1 | sed 's/[-.].*//'
}lastpart()
{
echo $1 | sed 's/.*[-.]//'
}trimlastpart()
{
echo $1 | sed 's/[-.][^-.]*$//'
}trimfirstpart()
{
echo $1 | sed 's/^[^-.]*[-.]//'
}letters()
{
echo $1 | tr -d '[0-9]'
}numbers()
{
echo $1 | tr -dc '[0-9]'
}if [ $# -eq 1 ]; then
echo "$1"
exit 0fi
if [ $# -gt 2 ]; then
FIRST=$1
shift
set -- $FIRST $($0 "$@")fi
VER1=$1
VER2=$2
# Vanity kernels (at end): handle 2.6.3-mm2 vs 2.6.3-rc1-mm3 by comparing
# base first.
TAIL1=$(lastpart $VER1)
TAIL2=$(lastpart $VER2)
if [ x"$(letters $TAIL1)" = x"$(letters $TAIL2)" ]; then
case "$TAIL1" in
bk*|rc*|pre*) : ;;
*)
BASE1=$(trimlastpart $VER1)
BASE2=$(trimlastpart $VER2)
if [ x"$BASE1" = x"$BASE2" ]; then
if [ $(numbers $TAIL1) -gt $(numbers $TAIL2) ]; then
echo $1
else
echo $2
fi
else
if [ x$(lkvercmp $BASE1 $BASE2) = x$BASE1 ]; then
echo $1
else
echo $2
fi
fi
exit
;;
esac
fiwhile true; do
PART1=$(firstpart $VER1)
PART2=$(firstpart $VER2)
VER1=$(trimfirstpart $VER1)
VER2=$(trimfirstpart $VER2)
PART1LETTERS=$(letters $PART1)
PART2LETTERS=$(letters $PART2)
PART1NUMBERS=$(numbers $PART1)
PART2NUMBERS=$(numbers $PART2)
if [ "$PART1LETTERS" = "$PART2LETTERS" ]; then
if [ "$PART1NUMBERS" = "$PART2NUMBERS" ]; then
continue;
else
if [ "$PART1NUMBERS" -gt "$PART2NUMBERS" ]; then
echo $1; exit
else
echo $2; exit
fi
fi
fi
case x"$PART1LETTERS" in
xbk)
# bk snapshots come after (ie. 2.6.1-bk1 is > 2.6.1).
# So this always wins.
case x"$PART2LETTERS" in
x|xrc|xpre) echo $1; exit;;
esac
unknown $PART2LETTERS;;
x)
# Final beats everything except bk.
case x"$PART2LETTERS" in
xbk) echo $2; exit;;
xrc|xpre) echo $1; exit;;
esac
unknown $PART2LETTERS;;
xrc)
# Only a final or bk can win.
case x"$PART2LETTERS" in
x|xbk) echo $2; exit;;
xpre) echo $1; exit;;
esac
unknown $PART2LETTERS;;
xpre)
# Final, bk or rc can win.
case x"$PART2LETTERS" in
x|xbk|xrc) echo $2; exit;;
xpre) echo $1; exit;;
esac
unknown $PART2LETTERS;;
*)
unknown $PART1LETTERS;;
esacdone
#! /bin/sh -e
# Latest kernel version of this vintage.
KERNELDIR=~/devel/kernel
# Return the latest kernel versions.get_kernel_vers()
{
# if [ x"$GK_KERNS" = x ]; then
# GK_KERNS=$(finger @finger.kernel.org | tail +2 | cut -s -d: -f2- | sed 's/[ ]*//')
# fi
if [ x"$GK_KERNS" = x ]; then GK_KERNS=$(lynx -width=500 -dump http://www.kernel.org/kdist/finger_banner | tail +2 | cut -s -d: -f2- | sed 's/[ ]*//') fi if [ x"$GK_KERNS" = x ]; then echo Failed to get kernel list from kernel.org >&2 exit 1 fi # Vanity are there too: only take recognized patches. # Ignore 2.4-bk versions: I don't know where they're coming from 8( echo "$GK_KERNS" # | egrep -- "$1|-rc|-bk|-pre|^[0-9]*\.[0-9]*\.[0-9]*\$" | grep -v '2\.4\.[0-9]*-bk[0-9]*' }
PROBE=0
if [ x"$1" = x--probe ]; then
PROBE=1
shiftfi
case "$1" in --*)
echo "Usage: $0 [--probe] [kernel-base-version]" >&2
echo "Returns the latest version on the system, or with --probe," >&2 echo "returns the latest version on network." >&2
exit 1
;;
esac
if [ $# -eq 0 ]; then set .; fi
if [ $PROBE = 1 ]; then
CANDIDATES=$(get_kernel_vers)
else
CANDIDATES=$(cd $KERNELDIR && ls -d linux-* | fgrep -v '.tmp' | grep -v -- '-ppc' | sed 's/^linux-//')fi
for w; do
case "$w" in
# By default, ignore any vanity kernels.
.) FILTER="^[0-9.]*(-rc[0-9]*|-bk[0-9]*|-pre[0-9]*)*\$";;
# eg -mm.
-*) FILTER="^[0-9.]*(-rc[0-9]*|-bk[0-9]*|-pre[0-9]*)*$w[0-9]*\$";;
# eg. 2.4
*) FILTER="^$w[0-9.]*(-rc[0-9]*|-bk[0-9]*|-pre[0-9]*)*\$";;
esac
lkvercmp $(echo "$CANDIDATES" | egrep -- "$FILTER")done
#! /bin/sh -e
# Script to grab a given kernel version (stolen from Virtual Anton).
barf()
{
echo "$@" >&2
exit 1
}PATCHDIR=~/devel/kernel/kernel-patches
MIRRORS="rsync://kernel::kernel http://kernel.linux.pacific.net.au/linux/kernel/ rsync://master.kernel.org:/pub/linux/kernel" #MIRRORS="rsync.planetmirror.com::ftp.kernel.org/pub/linux/kernel master.kernel.org:/pub/linux/kernel"
# Takes destination directory, and filename, tries each mirror. get_file()
{
get_file_dir=$1
get_file_name=$2
# Patch there already?
if [ -f "$get_file_dir"/"$(basename "$get_file_name")" ]; then
echo " Already have patch..."
return 0
fi
for get_file_server in $MIRRORS; do
echo Trying $get_file_server...
case $get_file_server in
rsync://*) get_file_s=`echo $get_file_server | cut -c9-`;
rsync $get_file_s/$get_file_name $get_file_dir && return 0
;;
http://*) get_file_s=`echo $get_file_server | cut -c8-`;
wget -O $get_file_dir/`basename $get_file_name` $get_file_s/$get_file_name && return 0
# Leaves a zero-len file on failure 8(
rm -f $get_file_dir/`basename $get_file_name`
;;
esac
done
return 1
}if [ $# -ne 2 ]; then
echo Usage: $0 version kerneldir >&2
exit 1
fi
VERSION=$1
KDIR=$2
if [ -d $KDIR/linux-$VERSION ]; then exit 0; fi
MAJOR=echo $VERSION | cut -d. -f1-2
# Given vanity kernel name, major version, and previous version, produce pathname vanity_source()
{
case "$1" in
*-ac*)
echo people/alan/linux-$2/$3/patch-$1.bz2;;
*-dj*)
echo people/davej/patches/$2/$3/patch-$1.bz2;;
*-mm*)
echo people/akpm/patches/$2/$3/$1/$1.bz2;;
*)
barf "Unknown kernel series $1";;
esac
}# Given base name, new name and patch, do patch application. apply_patch()
{
rm -rf $KDIR/.linux-$2
cp -al $KDIR/linux-$1 $KDIR/.linux-$2 ||
barf "Couldn't clone $1 dir"
bunzip2 -t $3 || barf "Corrupt patch $3"
bzcat $3 | patch -s -d $KDIR/.linux-$2 -p1 ||
barf "Problem patching new kernel $2"
mv $KDIR/.linux-$2 $KDIR/linux-$2
}echo Grabbing $VERSION
case "$VERSION" in
# Bitkeeper snapshot
*-bk*)
if [ -d $KDIR/linux-$VERSION ]; then exit 0; fi
PREVIOUS=`echo "$VERSION" | sed 's/-bk.*//'`
$0 "$PREVIOUS" $KDIR || barf "Couldn't get $PREVIOUS to build $VERSION"
# Grab patch
get_file $PATCHDIR v$MAJOR/snapshots/patch-$VERSION.bz2 || barf "Couldn't get v$MAJOR/snapshots/patch-$VERSION.bz2"
apply_patch $PREVIOUS $VERSION $PATCHDIR/patch-$VERSION.bz2
;;
# Various vanity patches
*-mm[0-9]*|*-ac[0-9]*|*-dj[0-9]*)
# First make sure we have base tree
BASE_KERNEL=`echo "$VERSION" | sed -e 's/-[a-z][a-z][0-9]*$//'`
$0 "$BASE_KERNEL" $KDIR ||
barf "Couldn't get $BASE_KERNEL to build $VERSION"
VANITY=`vanity_source $VERSION $MAJOR $BASE_KERNEL`
get_file $PATCHDIR $VANITY ||
barf "Couldn't fetch $VANITY"
apply_patch $BASE_KERNEL $VERSION $PATCHDIR/`basename $VANITY`
;;
# *-pre*: This refers to the Linus or Marcelo pre-patches.
*-pre*|*-rc*)
# Linus or Marcelo pre-patch.
if [ -d $KDIR/linux-$VERSION ]; then exit 0; fi
PREVIOUS=`echo "$VERSION" | sed 's/-\(pre\|rc\).*//'`
LAST_VER=`echo "$PREVIOUS" | cut -d. -f3`
PREVIOUS=`echo "$PREVIOUS" | cut -d. -f1,2`.`expr $LAST_VER - 1 || true`
$0 "$PREVIOUS" $KDIR || barf "Couldn't get $PREVIOUS to build $VERSION"
# Grab patch
get_file $PATCHDIR v$MAJOR/testing/patch-$VERSION.bz2 ||
get_file $PATCHDIR v$MAJOR/testing/old/patch-$VERSION.bz2 ||
barf "Couldn't get v$MAJOR/testing/patch-$VERSION.bz2"
apply_patch $PREVIOUS $VERSION $PATCHDIR/patch-$VERSION.bz2
;;
# Linus 2.6.0 test patch.
2.6.0-test*)
VER=`echo "$VERSION" | sed 's/.*test//'`
PREVIOUS=2.6.0-test`expr $VER - 1 || true`
if [ $PREVIOUS = 2.6.0-test0 ]; then
PREVIOUS=2.5.75
fi
$0 "$PREVIOUS" $KDIR || barf "Couldn't get $PREVIOUS to build $VERSION"
# Grab patch
get_file $PATCHDIR v$MAJOR/patch-$VERSION.bz2 ||
get_file $PATCHDIR v$MAJOR/old/patch-$VERSION.bz2 ||
barf "Couldn't get v$MAJOR/patch-$VERSION.bz2"
apply_patch $PREVIOUS $VERSION $PATCHDIR/patch-$VERSION.bz2
;;
2.6.0)
PREVIOUS=2.6.0-test11
$0 "$PREVIOUS" $KDIR || barf "Couldn't get $PREVIOUS to build $VERSION"
# Grab patch
get_file $PATCHDIR v$MAJOR/patch-$VERSION.bz2 ||
get_file $PATCHDIR v$MAJOR/old/patch-$VERSION.bz2 ||
barf "Couldn't get v$MAJOR/patch-$VERSION.bz2"
apply_patch $PREVIOUS $VERSION $PATCHDIR/patch-$VERSION.bz2
;;
# *.0: This refers to the base of the Linus kernels, fetched whole.
*.0)
# Linus' root of all trees.
get_file $PATCHDIR v$MAJOR/linux-$VERSION.tar.bz2 || barf "Couldn't get v$MAJOR/linux-$VERSION.tar.bz2"
bzcat $PATCHDIR/linux-$VERSION.tar.bz2 | (cd $KDIR && tar xf -) ||
barf "Problem unpacking linux-$VERSION.tar.bz2"
mv $KDIR/linux $KDIR/linux-$VERSION
;;
# [1-9].*.*: This refers to the any other Linus kernel.
[1-9].*.*[0-9])
# Linus standard ?
if [ -d $KDIR/linux-$VERSION ]; then exit 0; fi
LAST_VER=`echo "$VERSION" | cut -d. -f3`
PREVIOUS=`echo "$VERSION" | cut -d. -f1,2`.`expr $LAST_VER - 1 || true`
$0 "$PREVIOUS" $KDIR || barf "Couldn't get $PREVIOUS to build $VERSION"
# Grab patch
get_file $PATCHDIR v$MAJOR/patch-$VERSION.bz2 || barf "Couldn't get v$MAJOR/patch-$VERSION.bz2"
trap "rm -rf $KDIR/.linux-$VERSION" 0
apply_patch $PREVIOUS $VERSION $PATCHDIR/patch-$VERSION.bz2
;;
*)
barf "Unknown kernel version $VERSION"
;;esac
Quoted 2 lines · Matt Mackall
This is an alpha release for people to experiment with. Feedback and patches encouraged. Grab your copy today at:
First of all, very nice script.
But, it doesn't look like it properly handles empty directories. I tried this command, this morning, and it blew up. I think it's because this directory http://www.kernel.org/pub/linux/kernel/v2.6/snapshots/ is empty because of last night's 2.6.4-rc2 release. I don't grok python very well but is the "return p[-1]" there just to cause a fault like this? Would it be better if it just returned a "no version of that patch right now" message and exited nicely?
[dave@nighthawk linux-2.6]$ kpatchup-0.02 2.6-bk "Traceback (most recent call last):
File "/home/dave/bin/kpatchup-0.02", line 283, in ?
b = find_ver(args[0])
File "/home/dave/bin/kpatchup-0.02", line 240, in find_ver
return v[0](os.path.dirname(v[1]), v[2])
File "/home/dave/bin/kpatchup-0.02", line 147, in latest_dir
return p[-1]IndexError: list index out of range
I think your script, combined with Rusty's latest-kernel-version could make me a very happy person.
-- dave
Quoted 6 lines · Dave Hansen
Quoted 2 lines · Matt Mackall
This is an alpha release for people to experiment with. Feedback and patches encouraged. Grab your copy today at:
First of all, very nice script.
Quoted 7 lines
But, it doesn't look like it properly handles empty directories. I tried this command, this morning, and it blew up. I think it's because this directory http://www.kernel.org/pub/linux/kernel/v2.6/snapshots/ is empty because of last night's 2.6.4-rc2 release. I don't grok python very well but is the "return p[-1]" there just to cause a fault like this? Would it be better if it just returned a "no version of that patch right now" message and exited nicely?
Python does a good job at falling over loudly whenever anything unexpected happens. I hadn't noticed that the snapshot directory got purged. Hmmm. The right thing is to make it fall back to checking old/ where the most recent -bk is to be found. Like this:
$ kpatchup -s 2.6-pre 2.6.4-rc2 $ kpatchup -s 2.6-bk 2.6.4-rc1-bk4
New version at http://selenic.com/kpatchup/kpatchup-0.03
I've added a couple other niceties for scripting purposes: a -p option which will report the "base" version for a given version, -m which will parse Makefile and print the version therein.
Quoted 2 lines
I think your script, combined with Rusty's latest-kernel-version could make me a very happy person.
I skimmed latest-kernel-version, is it doing something my -s option doesn't do yet?
Signature · 1 line
Matt Mackall : http://www.selenic.com : Linux development and consulting
Quoted 2 lines · Matt Mackall
I skimmed latest-kernel-version, is it doing something my -s option doesn't do yet?
$ ./kpatchup-0.03 -s 2.6-bk 2.6.4-rc1-bk4 $ ./kpatchup-0.03 -s 2.6 2.6.3 $ ./kpatchup-0.03 -s 2.6-pre 2.6.4-rc2 $ latest-kernel-version --probe 2.6.4-rc2
I might be misunderstanding the options, but 'kpatchup -s' I think is limited to giving the latest version of a single tree, while 'latest-kernel-version' will look at several different trees. This is a tiny problem for kpatchup because it treats 2.6, 2.6-bk, and 2.6-pre as separate trees. For my use, I just want the latest snapshot, whether it's a bk snapshot, or one of the real point releases.
What I'm doing to work around it is this:
kpatchup latest-kernel-version --probe
-- dave
Quoted 6 lines · Dave Hansen
Quoted 2 lines · Matt Mackall
I skimmed latest-kernel-version, is it doing something my -s option doesn't do yet?
$ ./kpatchup-0.03 -s 2.6-bk 2.6.4-rc1-bk4
This is actually broken at the moment as it won't know how to fetch stuff from old/ when it's given a non-current bk version. I have a design assumption that I can go from a version name to a URL without any lookups, which the snapshot dir purging breaks.
ISTR Jeff maintains the snapshot scripts. Jeff, could you possibly have it put a copy of the each snapshot in old/ at creation time rather than moving stuff there at a later point? This will make URLs pointing to old/ for -bk patches stable but shouldn't break anything else.
Quoted 7 lines
$ ./kpatchup-0.03 -s 2.6 2.6.3 $ ./kpatchup-0.03 -s 2.6-pre 2.6.4-rc2 $ latest-kernel-version --probe 2.6.4-rc2
Quoted 6 lines
I might be misunderstanding the options, but 'kpatchup -s' I think is limited to giving the latest version of a single tree, while 'latest-kernel-version' will look at several different trees. This is a tiny problem for kpatchup because it treats 2.6, 2.6-bk, and 2.6-pre as separate trees. For my use, I just want the latest snapshot, whether it's a bk snapshot, or one of the real point releases.
I suppose I could add a 2.6-tip, which will return the greatest of 2.6, 2.6-pre, and 2.6-bk. Like this:
$ kpatchup -s 2.6-tip 2.6.4-rc2 $ kpatchup -u 2.6-tip http://www.kernel.org/pub/linux/kernel/v2.6/testing/patch-2.6.4-rc2.bz2
Signature · 1 line
Matt Mackall : http://www.selenic.com : Linux development and consulting
Quoted 7 lines · Matt Mackall
I suppose I could add a 2.6-tip, which will return the greatest of 2.6, 2.6-pre, and 2.6-bk. Like this:
$ kpatchup -s 2.6-tip 2.6.4-rc2 $ kpatchup -u 2.6-tip http://www.kernel.org/pub/linux/kernel/v2.6/testing/patch-2.6.4-rc2.bz2
That would at least keep me from mixing different tools like I am now. It's not too big of a deal.
-- dave