xen.git#staging smoke tests
PATCH v3 1/1 xen.git#staging smoke tests
Thanks for the reviews.
My play cr-daily-branch run revealed that:
1. The cr-* branch plumbing was wrong. So I have redone that patch and dropped your ack.
2. test-amd64-amd64-xl-qemuu-debianhvm-i386 was broken due to problems mostly with ts-debian-hvm-install. This results in 9 new patches. (In principle we could deploy the smoke tests without fixing this test but this is probably not desirable.)
Patches v2 00-16 are in pretest now and not included in my series numbering here.
Patches v3 01-14 inclusive (v2 17-30) have been acked and are unchanged. So I have removed them from this patchbomb.
* 15 cr*: Support xen-unstable-smoke a 16 ap-*: Be able to fetch and push xen.git#smoke a 17 crontab: Enable xen-unstable-smoke branch every 3h at <1mod3>:51 UTC + 18 ts-debian-hvm-install, etc.: Do not hardcode in-iso path + 19 ts-debian-hvm-install: Cope with images containing only isolinux + 20 ts-debian-hvm-install: Set $gsuite after $gho + 21 ts-debian-hvm-install: Do not create EFI partition if EFI not in use + 22 ts-debian-hvm-install: Use /dev/sda for i386, not /dev/xvda + 23 Timeouts: Introduce target_adjust_timeout + 24 Timeouts: Honour guest-related timeout-adjustment runvars + 25 Debian i386 HVM tests: Increase installation timeout + 26 cri-common: Add a missing semicolon
Ian.
PATCH 15/26 cr*: Support xen-unstable-smoke
Add this branch to select_xenbranch. This works like xen-unstable in most respects.
There are only two places in osstest where xenbranch `xen-unstable' is treated specially and only one of them needs adjusting to match xen-unstable-smoke too.
The new branch xen-unstable-smoke' has a prev' branch of `xen-unstable' according to cri-getprevxenbranch, which is technically wrong, but this is not important because xen-unstable-smoke has no prev tests.
We are going to sort out the push gate ref plumbing in xen.git in the next osstest patch.
Also, use a branch-settings file to set the new branch's resource priority to -20 to make it run ahead of anything else automatic.
---
v3: Dropped ack from Ian Campbell
Set qemuubranch to `qemu-xen-unstable'
Set xenbranch to xen-unstable-smoke', not xen-unstable'
cr-daily-branch: Update $realtree case for selecting qemuu revision
--- README.planner | 1 + branch-settings.xen-unstable-smoke | 1 + cr-daily-branch | 2 +- cri-common | 1 + 4 files changed, 4 insertions(+), 1 deletion(-) create mode 100644 branch-settings.xen-unstable-smoke
README.planner +1 -0
@@ -87,6 +87,7 @@ in the more distant future. Values for OSSTEST_RESOURCE_PRIORITY: mg-allocate -1000000 mg-blockage -2000000 + xen-unstable-smoke cr-daily-branch -20 default Executive.pm with controlling terminal -10 mg-execute-flight -8 default 0
branch-settings.xen-unstable-smoke new file +1 -0
@@ -0,0 +1 @@ +export OSSTEST_RESOURCE_PRIORITY=-20
cr-daily-branch +1 -1
@@ -340,7 +340,7 @@ case "$NEW_REVISION/$OLD_REVISION" in "$treeurl#$OLD_REVISION-$NEW_REVISION" \ case "$realtree" in - xen-4*|xen-unstable) + xen-4*|xen-unstable*) oldqemu=`./ap-qemu-revision $realtree $OLD_REVISION` newqemu=`./ap-qemu-revision $realtree $NEW_REVISION` if [ "$oldqemu" ] && [ "$newqemu" ]; then
cri-common +1 -0
@@ -66,6 +66,7 @@ select_prevxenbranch () { select_xenbranch () { case "$branch" in + xen-unstable-smoke) tree=xen; xenbranch=$branch; qemuubranch=qemu-xen-unstable;; xen-*) tree=xen; xenbranch=$branch ;; qemu-mainline) tree=qemuu; xenbranch=xen-unstable qemuubranch=qemu-mainline;; qemu-upstream-*) tree=qemuu; xenbranch=xen-${branch#qemu-upstream-};;
Signature · 1 line
1.7.10.4
Re: [OSSTEST PATCH 15/26] cr*: Support xen-unstable-smoke
Quoted 6 lines · Ian Jackson
Add this branch to
select_xenbranch. This works like xen-unstable in most respects.There are only two places in osstest where xenbranch `xen-unstable' is treated specially and only one of them needs adjusting to match xen-unstable-smoke too.
The included one is the one which adds the qemu revisions?
And the other one is, I think, the thing which sets
REVISION_LINUX_OLD=disable which we want (or maybe don't care about) for
the smoked tests. So good.
Quoted 5 lines
The new branch
xen-unstable-smoke' has aprev' branch of `xen-unstable' according to cri-getprevxenbranch, which is technically wrong, but this is not important because xen-unstable-smoke has no prev tests.
Quoted 7 lines
We are going to sort out the push gate ref plumbing in xen.git in the next osstest patch.
Also, use a branch-settings file to set the new branch's resource priority to -20 to make it run ahead of anything else automatic.
Signed-off-by Ian Jackson <[email protected]>
Re: [OSSTEST PATCH 15/26] cr*: Support xen-unstable-smoke
Ian Campbell writes ("Re: [OSSTEST PATCH 15/26] cr*: Support xen-unstable-smoke"):
Quoted 9 lines · Ian Jackson
Quoted 6 lines
Add this branch to
select_xenbranch. This works like xen-unstable in most respects.There are only two places in osstest where xenbranch `xen-unstable' is treated specially and only one of them needs adjusting to match xen-unstable-smoke too.
The included one is the one which adds the qemu revisions?
No, this in cr-daily-branch:
if [ "x$branch" != "xxen-unstable" ]; then
export REVISION_LINUX_OLD=disable
fiQuoted 3 lines
And the other one is, I think, the thing which sets
REVISION_LINUX_OLD=disable which we want (or maybe don't care about) for
the smoked tests. So good.
Indeed.
I'm not sure what the qemu revisions thing is you're thinking of but I think all the places where the source code matches xen-unstable[^-]', the actual effect is to treat xen-unstable-smoke' the same way as `xen-unstable'.
Quoted 1 line
Acked-by Ian Campbell <[email protected]>
Thanks,
Ian.
Re: [OSSTEST PATCH 15/26] cr*: Support xen-unstable-smoke
Quoted 11 lines · Ian Jackson
Ian Campbell writes ("Re: [OSSTEST PATCH 15/26] cr*: Support xen-unstable -smoke"):
Quoted 9 lines · Ian Jackson
Quoted 7 lines
Add this branch to
select_xenbranch. This works like xen-unstable in most respects.There are only two places in osstest where xenbranch `xen-unstable' is
treated specially and only one of them needs adjusting to match xen-unstable-smoke too.
Quoted 7 lines
Quoted 1 line
The included one is the one which adds the qemu revisions?
No, this in cr-daily-branch:
if [ "x$branch" != "xxen-unstable" ]; then export REVISION_LINUX_OLD=disable fi
That's the one which didn't need adjusting I think.
Quoted 11 lines
Quoted 4 lines
And the other one is, I think, the thing which sets
REVISION_LINUX_OLD=disable which we want (or maybe don't care about)
for
the smoked tests. So good.Indeed.
I'm not sure what the qemu revisions thing is you're thinking of but I think all the places where the source code matches
xen-unstable[^-]', the actual effect is to treatxen-unstable-smoke' the same way as `xen-unstable'.
"qemu revisions thing" is:
@@ -340,7 +340,7 @@ case "$NEW_REVISION/$OLD_REVISION" in
"$treeurl#$OLD_REVISION-$NEW_REVISION" \
case "$realtree" in
- xen-4*|xen-unstable) + xen-4*|xen-unstable*)
oldqemu=./ap-qemu-revision $realtree $OLD_REVISION newqemu=./ap-qemu-revision $realtree $NEW_REVISION if [ "$oldqemu" ] && [ "$newqemu" ]; then
Quoted 5 lines
Quoted 2 lines
Acked-by Ian Campbell <[email protected]>Thanks,
Ian.
Re: [OSSTEST PATCH 15/26] cr*: Support xen-unstable-smoke
Ian Campbell writes ("Re: [OSSTEST PATCH 15/26] cr*: Support xen-unstable-smoke"):
Quoted 6 lines · Ian Jackson
Quoted 3 lines
if [ "x$branch" != "xxen-unstable" ]; then export REVISION_LINUX_OLD=disable fiThat's the one which didn't need adjusting I think.
Yes.
Quoted 8 lines
Quoted 4 lines
I'm not sure what the qemu revisions thing is you're thinking of but I think all the places where the source code matches
xen-unstable[^-]', the actual effect is to treatxen-unstable-smoke' the same way as `xen-unstable'."qemu revisions thing" is:
cr-daily-branch case "$NEW_REVISION/$OLD_REVISION" in · line 346
Quoted 8 lines
cr-daily-branch +1
-1@@ -340,7 +340,7 @@ case "$NEW_REVISION/$OLD_REVISION" in "$treeurl#$OLD_REVISION-$NEW_REVISION" \ case "$realtree" in - xen-4*|xen-unstable) + xen-4*|xen-unstable*) oldqemu=`./ap-qemu-revision $realtree $OLD_REVISION` newqemu=`./ap-qemu-revision $realtree $NEW_REVISION` if [ "$oldqemu" ] && [ "$newqemu" ]; then
Ah, OK. Yes. We agree.
Thanks,
Ian.
PATCH 16/26 ap-*: Be able to fetch and push xen.git#smoke
In reply to Ian JacksonThe branches and push gates are now:
xen.git#staging -[xen-unstable-smoke]-> #smoke -[xen-unstable]-> #master
Deployment note: When this passes the osstest self-push-gate, the main xen-unstable flight will start using smoke as an input. Therefore, until the new cronjob is installed to run the xen-unstable-smoke tests, an automatic process should keep xen.git#smoke up to date with xen.git#staging. Eg, running in screen in xen@xenbits:~/git/xen.git:
while sleep 1800; do git fetch . staging:smoke; done
---
v2: xen.git#smoked' branch name changed to #smoke'
--- ap-fetch-version | 3 +++ ap-fetch-version-old | 3 +++ ap-push | 4 ++++ 3 files changed, 10 insertions(+)
ap-fetch-version +3 -0
@@ -40,6 +40,9 @@ xen-3.*) ' x ;; xen-unstable) + repo_tree_rev_fetch_git xen $TREE_XEN smoke $LOCALREV_XEN + ;; +xen-unstable-smoke) repo_tree_rev_fetch_git xen $TREE_XEN staging $LOCALREV_XEN ;; xen-4.*-testing)
ap-fetch-version-old +3 -0
@@ -50,6 +50,9 @@ xen-3.*) xen-unstable) repo_tree_rev_fetch_git xen $TREE_XEN master $LOCALREV_XEN ;; +xen-unstable-smoke) + repo_tree_rev_fetch_git xen $TREE_XEN smoke $LOCALREV_XEN + ;; xen-4.*-testing) branchcore=${branch#xen-} branchcore=${branchcore%-testing}
ap-push +4 -0
@@ -54,6 +54,10 @@ xen-unstable) cd $repos/xen git push $TREE_XEN $revision:refs/heads/master ;; +xen-unstable-smoke) + cd $repos/xen + git push $TREE_XEN $revision:refs/heads/smoke + ;; xen-*-testing) cd $repos/xen xenversion=$branch
Signature · 1 line
1.7.10.4
PATCH 17/26 crontab: Enable xen-unstable-smoke branch every 3h at <1mod3>:51 UTC
In reply to Ian Jackson--- crontab | 1 + 1 file changed, 1 insertion(+)
crontab +1 -0
@@ -2,6 +2,7 @@ PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin [email protected],[email protected] # m h dom mon dow command 49 1 * * * cd testing.git && BRANCHES_ALWAYS=xen-unstable ./cr-for-branches branches -w "./cr-daily-branch --real" +51 1-23/3 * * * cd testing.git && BRANCHES=xen-unstable-smoke ./cr-for-branches branches -w "./cr-daily-branch --real" 4-59/30 * * * * cd testing.git && ./cr-for-branches branches -q "./cr-daily-branch --real" 18 9 * * 1,3,5 cd testing.git && BRANCHES=linux-next ./cr-for-branches branches -w "./cr-daily-branch --real" 18 4 * * * cd testing.git && BRANCHES='linux-linus linux-mingo-tip-master linux-3.0 libvirt rumpuserxen' ./cr-for-branches branches -w "./cr-daily-branch --real"
Signature · 1 line
1.7.10.4
PATCH 18/26 ts-debian-hvm-install, etc.: Do not hardcode in-iso path
In reply to Ian Jacksonts-debian-hvm-install hardcoded `install.amd' as the directory in the .iso in which to find the kernel and initrd. This is wrong for architectures other than amd64.
Instead, pass this information in runvars (as is done for the netinst installs in make-distros-flight), and honour it in ts-debian-hvm-install.
If the runvars are not set, default to the previous hardcoded values. (This arranges that clones of old flights still work with new osstest, eg for bisection.)
--- make-flight | 8 ++++++++ ts-debian-hvm-install | 25 +++++++++++++++++++------ 2 files changed, 27 insertions(+), 6 deletions(-)
make-flight +8 -0
@@ -264,6 +264,12 @@ do_hvm_debian_test_one () { local arch=$(branch_debianhvm_arch) + case "$arch" in + amd64) iso_dir='install.amd' ;; + i386) iso_dir='install.386' ;; + *) iso_dir="install.$arch" ;; + esac + stubdom_suffix="" stubdom_runvar="" if [ x$stubdom = xtrue ]; then
@@ -276,6 +282,8 @@ do_hvm_debian_test_one () { enable_xsm=$xsm \ $stubdom_runvar \ debianhvm_image=debian-7.2.0-$arch-CD-1.iso \ + debianhvm_iso_kernel=/$iso_dir/vmlinuz \ + debianhvm_iso_ramdisk=/$iso_dir/initrd.gz \ bios=$bios \ all_hostflags=$most_hostflags,hvm }
ts-debian-hvm-install +19 -6
@@ -48,6 +48,8 @@ our $disk_mb= 10000; our $guesthost= "$gn.guest.osstest"; our $gho; +our ($kernel, $ramdisk); + our $gsuite= guest_var($gho,'suite',$c{GuestDebianSuite}); sub preseed () {
@@ -130,14 +132,14 @@ set default="0" set timeout=5 menuentry 'debian guest auto Install' { - linux /install.amd/vmlinuz $cmdline - initrd /install.amd/initrd.gz + linux $kernel $cmdline + initrd $ramdisk } END } sub isolinux_cfg () { - my $cmdline = gcmdline("initrd=/install.amd/initrd.gz"); + my $cmdline = gcmdline("initrd=$ramdisk"); return <<"END"; default autoinstall
@@ -145,7 +147,7 @@ sub isolinux_cfg () { timeout 0 label autoinstall - kernel /install.amd/vmlinuz + kernel $kernel append $cmdline END }
@@ -156,9 +158,9 @@ sub prepare_initrd ($$$) { rm -rf $initrddir mkdir $initrddir cd $initrddir - gzip -d < $newiso/install.amd/initrd.gz | cpio --extract --make-directories --no-absolute-filename + gzip -d < $newiso$ramdisk | cpio --extract --make-directories --no-absolute-filename cp $preseed_file_path preseed.cfg - find . | cpio -H newc --create | gzip -9 > $newiso/install.amd/initrd.gz + find . | cpio -H newc --create | gzip -9 > $newiso$ramdisk cd - rm -rf $initrddir cd $newiso
@@ -169,6 +171,13 @@ END our $emptyiso= "/root/$flight.$job.$gn-empty.iso"; +sub iso_path ($$) { + my ($which, $deftail) = @_; + my $v = guest_var($gho, "iso_$which", "/install.amd/$deftail"); + die "$gho->{Guest} $which $v ?" unless $v =~ m#^/#; + return $v; +} + sub prep () { target_install_packages_norec($ho, qw(lvm2 rsync genisoimage));
@@ -177,6 +186,10 @@ sub prep () { $gho= prepareguest($ho, $gn, $guesthost, 22, $disk_mb + 1, 200); + + $kernel = iso_path('kernel', 'vmlinuz'); + $ramdisk = iso_path('ramdisk', 'initrd.gz'); + my $base = "/root/$flight.$job.$gn-"; my $newiso= $base . "newiso"; my $emptydir= $base . "empty-dir";
Signature · 1 line
1.7.10.4
Re: [OSSTEST PATCH 18/26] ts-debian-hvm-install, etc.: Do not hardcode in-iso path
Quoted 8 lines · Ian Jackson
ts-debian-hvm-install hardcoded `install.amd' as the directory in the .iso in which to find the kernel and initrd. This is wrong for architectures other than amd64.
Instead, pass this information in runvars (as is done for the netinst installs in make-distros-flight), and honour it in ts-debian-hvm-install.
Quoted 5 lines
If the runvars are not set, default to the previous hardcoded values. (This arranges that clones of old flights still work with new osstest, eg for bisection.)
Signed-off-by Ian Jackson <[email protected]>
PATCH 19/26 ts-debian-hvm-install: Cope with images containing only isolinux
In reply to Ian Jacksondebian-7.2.0-i386-CD-1.iso contains no grub, only isolinux.
If the specified EFI grub file does not exist, fall back to isolinux. This requires a -c option as well, according to
https://wiki.debian.org/DebianInstaller/Modify/CD
Only try to set up a grub config if we are booting grub. (The i386 image in question does not contain a [debian]/boot/grub directory.)
If boot/grub/efi.img _does_ exist (ie, for other existing tests), the only difference in behaviour is to reorder slightly the options to
genisoimage: -b boot/grub/efi.img' now occurs after -no-emul-boot -r' rather than before.
--- ts-debian-hvm-install | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-)
ts-debian-hvm-install +11 -2
@@ -197,9 +197,16 @@ sub prep () { my $preseed_file_path = $base . "preseed"; my @isogen_extra = qw(-eltorito-alt-boot - -b boot/grub/efi.img -no-emul-boot -r); + + my $bootfile = 'boot/grub/efi.img'; + if (!target_file_exists($ho, "$newiso/$bootfile")) { + $bootfile = "isolinux/isolinux.bin"; + push @isogen_extra, qw(-c isolinux/boot.cat); + } + push @isogen_extra, '-b', $bootfile; + my @isogen_opts = (iso_gen_flags_basic(), @isogen_extra); iso_create_empty($ho, $emptyiso, $emptydir);
@@ -226,8 +233,10 @@ sub prep () { my $cmds = iso_copy_content_from_image($gho, $newiso); $cmds .= prepare_initrd($initrddir,$newiso,$preseed_file_path); target_cmd_root($ho, $cmds, $isotimeout); + target_putfilecontents_root_stash($ho, 10, grub_cfg(), - "$newiso/debian/boot/grub/grub.cfg"); + "$newiso/debian/boot/grub/grub.cfg") + if $bootfile =~ m/grub/; target_putfilecontents_root_stash($ho, 10, isolinux_cfg(), "$newiso/isolinux/isolinux.cfg");
Signature · 1 line
1.7.10.4
Re: [OSSTEST PATCH 19/26] ts-debian-hvm-install: Cope with images containing only isolinux
Quoted 6 lines · Ian Jackson
debian-7.2.0-i386-CD-1.iso contains no grub, only isolinux.
If the specified EFI grub file does not exist, fall back to isolinux. This requires a -c option as well, according to
https://wiki.debian.org/DebianInstaller/Modify/CD
Quoted 7 lines
Only try to set up a grub config if we are booting grub. (The i386 image in question does not contain a [debian]/boot/grub directory.)
If boot/grub/efi.img _does_ exist (ie, for other existing tests), the only difference in behaviour is to reorder slightly the options to
genisoimage:-b boot/grub/efi.img' now occurs after-no-emul-boot -r' rather than before.
I'm happy to determine experimentally (i.e. by pushing to pretest) if there is any meaning to the order of these (the wiki has them the new way around, so I presume not).
Quoted 12 lines
Signed-off-by Ian Jackson <[email protected]>--- ts-debian-hvm-install | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-)ts-debian-hvm-install +0
-0@@ -197,9 +197,16 @@ sub prep () { my $preseed_file_path = $base . "preseed";
ts-debian-hvm-install sub prep () { · line 207
Quoted 10 lines
ts-debian-hvm-install +6
-1my @isogen_extra = qw(-eltorito-alt-boot - -b boot/grub/efi.img -no-emul-boot -r); + + my $bootfile = 'boot/grub/efi.img'; + if (!target_file_exists($ho, "$newiso/$bootfile")) { + $bootfile = "isolinux/isolinux.bin"; + push @isogen_extra, qw(-c isolinux/boot.cat); + }
My preference would have been to produce an iso which was bootable either via EFI (grub) or legacy (isolinux), but that would require more complex command lines and I'm sure neither of us wants to figure out what those are. So:
Quoted 20 lines
ts-debian-hvm-install +5
-1+ push @isogen_extra, '-b', $bootfile; + my @isogen_opts = (iso_gen_flags_basic(), @isogen_extra); iso_create_empty($ho, $emptyiso, $emptydir);@@ -226,8 +233,10 @@ sub prep () { my $cmds = iso_copy_content_from_image($gho, $newiso); $cmds .= prepare_initrd($initrddir,$newiso,$preseed_file_path); target_cmd_root($ho, $cmds, $isotimeout); + target_putfilecontents_root_stash($ho, 10, grub_cfg(), - "$newiso/debian/boot/grub/grub.cfg"); + "$newiso/debian/boot/grub/grub.cfg") + if $bootfile =~ m/grub/;target_putfilecontents_root_stash($ho, 10,
isolinux_cfg(),"$newiso/isolinux/isolinux.cfg");
Re: [OSSTEST PATCH 19/26] ts-debian-hvm-install: Cope with images containing only isolinux
Ian Campbell writes ("Re: [OSSTEST PATCH 19/26] ts-debian-hvm-install: Cope with images containing only isolinux"):
Quoted 2 lines · Ian Jackson
Quoted 1 line
debian-7.2.0-i386-CD-1.iso contains no grub, only isolinux.
...
Quoted 3 lines
I'm happy to determine experimentally (i.e. by pushing to pretest) if there is any meaning to the order of these (the wiki has them the new way around, so I presume not).
My tests (which I left running) have determined that this patch was wrong in the amd64 case: it would work, but by falling back to isolinux.
This is because $newiso is not populated at the time $bootfile is determined. I think this is fairly easily fixed by reordering things and I will send an update when I have got something that actually works in both cases.
(This affects the EFI partition creation too, but shouldn't involve any change to that patch except maybe to context.)
Quoted 10 lines
ts-debian-hvm-install sub prep () { · line 207
Quoted 5 lines
ts-debian-hvm-install +5
-0+ my $bootfile = 'boot/grub/efi.img'; + if (!target_file_exists($ho, "$newiso/$bootfile")) { + $bootfile = "isolinux/isolinux.bin"; + push @isogen_extra, qw(-c isolinux/boot.cat); + }My preference would have been to produce an iso which was bootable either via EFI (grub) or legacy (isolinux), but that would require more complex command lines and I'm sure neither of us wants to figure out what those are. So:
Yes.
Quoted 1 line
Acked-by Ian Campbell <[email protected]>
Thanks, but given the circmstances I am not going to apply this ack to whatever revised version I produce.
Ian.
Re: [OSSTEST PATCH 19/26] ts-debian-hvm-install: Cope with images containing only isolinux
Ian Jackson writes ("Re: [OSSTEST PATCH 19/26] ts-debian-hvm-install: Cope with images containing only isolinux"):
Quoted 5 lines
Ian Campbell writes ("Re: [OSSTEST PATCH 19/26] ts-debian-hvm-install: Cope with images containing only isolinux"):
Quoted 1 line
Acked-by Ian Campbell <[email protected]>Thanks, but given the circmstances I am not going to apply this ack to whatever revised version I produce.
FYI I have reviewed the interdiff between v3 and what is going to be v4, and there are no changes to changed lines, only to context. But of course that means the changes are now in a different place, following the new code motion patch which will precede this.
v4 will follow soon I think.
Ian.
PATCH 20/26 ts-debian-hvm-install: Set $gsuite after $gho
In reply to Ian Jackson$gsuite was set from guest_var, but before $gho was set, leading to an undefined value warning from Perl.
This would ignore any guest-specific suite runvars. AFAICT these are set by some of the jobs in make-distros-flight. I think the effect of this change is to apply workarounds for the intended suite, rather than for wheezy.
(Although there is another assignment to $gho later in ts-debian-hvm-install, for stage 2, the stage 2 code does some trivial TestSupport calls and does not need $gsuite. So there is no need to make arrangements to assign to $gsuite - or, for that matter, $kernel or $ramdisk, in that path.)
--- ts-debian-hvm-install | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
ts-debian-hvm-install +2 -1
@@ -50,7 +50,7 @@ our $gho; our ($kernel, $ramdisk); -our $gsuite= guest_var($gho,'suite',$c{GuestDebianSuite}); +our $gsuite; sub preseed () {
@@ -187,6 +187,7 @@ sub prep () { $disk_mb + 1, 200); + $gsuite = guest_var($gho,'suite',$c{GuestDebianSuite}); $kernel = iso_path('kernel', 'vmlinuz'); $ramdisk = iso_path('ramdisk', 'initrd.gz');
Signature · 1 line
1.7.10.4
Re: [OSSTEST PATCH 20/26] ts-debian-hvm-install: Set $gsuite after $gho
Quoted 8 lines · Ian Jackson
$gsuite was set from
guest_var, but before $gho was set, leading to an undefined value warning from Perl.This would ignore any guest-specific suite runvars. AFAICT these are set by some of the jobs in make-distros-flight. I think the effect of this change is to apply workarounds for the intended suite, rather than for wheezy.
Quoted 7 lines
(Although there is another assignment to $gho later in ts-debian-hvm-install, for stage 2, the stage 2 code does some trivial TestSupport calls and does not need $gsuite. So there is no need to make arrangements to assign to $gsuite - or, for that matter, $kernel or $ramdisk, in that path.)
Signed-off-by Ian Jackson <[email protected]>
PATCH 21/26 ts-debian-hvm-install: Do not create EFI partition if EFI not in use
In reply to Ian JacksonIf we are booting our install ISO using a non-EFI executable, don't try to provide an EFI for the installed system either.
--- ts-debian-hvm-install | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
ts-debian-hvm-install +5 -2
@@ -49,6 +49,7 @@ our $guesthost= "$gn.guest.osstest"; our $gho; our ($kernel, $ramdisk); +our $bootfile; our $gsuite;
@@ -56,7 +57,7 @@ sub preseed () { my $preseed_file = preseed_base($gho,$gsuite,'','',()); - $preseed_file .= (<<END); + $preseed_file .= (<<END.($bootfile =~ m/\.efi/ ? <<END : '').<<END); d-i netcfg/get_hostname string $gn d-i partman-auto/disk string /dev/xvda
@@ -64,12 +65,14 @@ d-i partman-auto/method string regular d-i partman-auto/expert_recipe string \\ boot-root :: \\ +END 512 50 512 vfat \\ \$primary{ } \$bootable{ } \\ method{ efi } format{ } \\ use_filesystem{ } filesystem{ vfat } \\ mountpoint{ /boot/efi } \\ . \\ +END 5000 50 5000 ext4 \\ method{ format } format{ } \\ use_filesystem{ } filesystem{ ext4 } \\
@@ -201,7 +204,7 @@ sub prep () { -no-emul-boot -r); - my $bootfile = 'boot/grub/efi.img'; + $bootfile = 'boot/grub/efi.img'; if (!target_file_exists($ho, "$newiso/$bootfile")) { $bootfile = "isolinux/isolinux.bin"; push @isogen_extra, qw(-c isolinux/boot.cat);
Signature · 1 line
1.7.10.4
Re: [OSSTEST PATCH 21/26] ts-debian-hvm-install: Do not create EFI partition if EFI not in use
Quoted 4 lines · Ian Jackson
If we are booting our install ISO using a non-EFI executable, don't try to provide an EFI for the installed system either.
Signed-off-by Ian Jackson <[email protected]>
FWIW the default recipes used by d-i use "$iflabel{ gpt }" to achieve the same thing. Actually I'm a bit surprised that this isn't the affect of "method { efi }" too.
Anyway, this approach is good enough for our purposes so:
Quoted 49 lines
--- ts-debian-hvm-install | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)ts-debian-hvm-install +5
-2@@ -49,6 +49,7 @@ our $guesthost= "$gn.guest.osstest"; our $gho; our ($kernel, $ramdisk); +our $bootfile; our $gsuite;@@ -56,7 +57,7 @@ sub preseed () { my $preseed_file = preseed_base($gho,$gsuite,'','',()); - $preseed_file .= (<<END); + $preseed_file .= (<<END.($bootfile =~ m/\.efi/ ? <<END : '').<<END); d-i netcfg/get_hostname string $gn d-i partman-auto/disk string /dev/xvda@@ -64,12 +65,14 @@ d-i partman-auto/method string regular d-i partman-auto/expert_recipe string \\ boot-root :: \\ +END 512 50 512 vfat \\ \$primary{ } \$bootable{ } \\ method{ efi } format{ } \\ use_filesystem{ } filesystem{ vfat } \\ mountpoint{ /boot/efi } \\ . \\ +END 5000 50 5000 ext4 \\ method{ format } format{ } \\ use_filesystem{ } filesystem{ ext4 } \\@@ -201,7 +204,7 @@ sub prep () { -no-emul-boot -r); - my $bootfile = 'boot/grub/efi.img'; + $bootfile = 'boot/grub/efi.img'; if (!target_file_exists($ho, "$newiso/$bootfile")) { $bootfile = "isolinux/isolinux.bin"; push @isogen_extra, qw(-c isolinux/boot.cat);
Re: [OSSTEST PATCH 21/26] ts-debian-hvm-install: Do not create EFI partition if EFI not in use
Ian Campbell writes ("Re: [OSSTEST PATCH 21/26] ts-debian-hvm-install: Do not create EFI partition if EFI not in use"):
Quoted 3 lines · Ian Jackson
Quoted 2 lines
If we are booting our install ISO using a non-EFI executable, don't try to provide an EFI for the installed system either.
...
Quoted 3 lines
FWIW the default recipes used by d-i use "$iflabel{ gpt }" to achieve the same thing. Actually I'm a bit surprised that this isn't the affect of "method { efi }" too.
I'm happy to do this some other way but the way I did it avoided me having to look up how d-i partman-auto works...
Quoted 3 lines
Anyway, this approach is good enough for our purposes so:
Acked-by Ian Campbell <[email protected]>
Thanks,
Ian.
Re: [OSSTEST PATCH 21/26] ts-debian-hvm-install: Do not create EFI partition if EFI not in use
In reply to Ian CampbellIan Campbell writes ("Re: [OSSTEST PATCH 21/26] ts-debian-hvm-install: Do not create EFI partition if EFI not in use"):
Quoted 9 lines · Ian Jackson
Quoted 4 lines
If we are booting our install ISO using a non-EFI executable, don't try to provide an EFI for the installed system either.
Signed-off-by Ian Jackson <[email protected]>FWIW the default recipes used by d-i use "$iflabel{ gpt }" to achieve the same thing. Actually I'm a bit surprised that this isn't the affect of "method { efi }" too.
...
Quoted 2 lines
ts-debian-hvm-install sub preseed () { · line 60
Quoted 2 lines
ts-debian-hvm-install +1
-1- $preseed_file .= (<<END); + $preseed_file .= (<<END.($bootfile =~ m/\.efi/ ? <<END : '').<<END);
My amd64 tests point out that this regexp does not (of course) match the actual filename `boot/grub/efi.img'. I will fix this.
Ian.
PATCH 22/26 ts-debian-hvm-install: Use /dev/sda for i386, not /dev/xvda
In reply to Ian JacksonEmpirically, the device shows up as /dev/sda.
--- make-flight | 6 ++++-- ts-debian-hvm-install | 4 +++- 2 files changed, 7 insertions(+), 3 deletions(-)
make-flight +4 -2
@@ -263,10 +263,12 @@ do_hvm_debian_test_one () { stubdom=$5 # '' (or unset) or 'true' local arch=$(branch_debianhvm_arch) + local testvars case "$arch" in amd64) iso_dir='install.amd' ;; - i386) iso_dir='install.386' ;; + i386) iso_dir='install.386' + testvars+=' debianhvm_diskdevice=/dev/sda' ;; *) iso_dir="install.$arch" ;; esac
@@ -280,7 +282,7 @@ do_hvm_debian_test_one () { job_create_test test-$xenarch$kern-$dom0arch-$toolstack$qemuu_suffix$stubdom_suffix-$testname-$arch\ test-debianhvm $toolstack $xenarch $dom0arch $qemuu_runvar \ enable_xsm=$xsm \ - $stubdom_runvar \ + $stubdom_runvar $testvars \ debianhvm_image=debian-7.2.0-$arch-CD-1.iso \ debianhvm_iso_kernel=/$iso_dir/vmlinuz \ debianhvm_iso_ramdisk=/$iso_dir/initrd.gz \
ts-debian-hvm-install +3 -1
@@ -57,10 +57,12 @@ sub preseed () { my $preseed_file = preseed_base($gho,$gsuite,'','',()); + my $disk = guest_var($gho,'diskdevice','/dev/xvda'); + $preseed_file .= (<<END.($bootfile =~ m/\.efi/ ? <<END : '').<<END); d-i netcfg/get_hostname string $gn -d-i partman-auto/disk string /dev/xvda +d-i partman-auto/disk string $disk d-i partman-auto/method string regular d-i partman-auto/expert_recipe string \\
Signature · 1 line
1.7.10.4
Re: [OSSTEST PATCH 22/26] ts-debian-hvm-install: Use /dev/sda for i386, not /dev/xvda
Quoted 1 line · Ian Jackson
Empirically, the device shows up as
/dev/sda.
This is because the plain i386 kernel used by the installer does not support Xen (which requires PAE) and this affects even the options needed for PVHVM. amd64 on the other hand doesn't have this restriction and PVHVM works with the installer kernel.
Because of this the i386 install ends up using the emulated sda device instead of switching to the pv devices as amd64 does.
I think it's probably worth explaining some of this in the commit log rather than "Empirically".
IIRC someone was working on enabling PVHVM even for non-PAE i386 kernels, not sure what the status of that is, but clearly it isn't going to change things for Wheezy.
If you wanted instead to enable PV support for these installs then the multi-arch (i386+amd64) isoimages contain the Xen enabled kernels for i386 (as /install.386/xen/*, n.b. /install.amd/xen/* are suitable symlinks, so you can aoid special cases.
e.g. this one:
http://cdimage.debian.org/cdimage/archive/7.9.0/multi-arch/iso-cd/
(sorry for forgetting about this when we spoke IRL last week).
Anyway, that's clearly an "if you can be bothered thing" so with an expanded commit message:
Re: [OSSTEST PATCH 22/26] ts-debian-hvm-install: Use /dev/sda for i386, not /dev/xvda
Ian Campbell writes ("Re: [OSSTEST PATCH 22/26] ts-debian-hvm-install: Use /dev/sda for i386, not /dev/xvda"):
Quoted 2 lines
Because of this the i386 install ends up using the emulated sda device instead of switching to the pv devices as amd64 does.
Indeed.
Quoted 2 lines
I think it's probably worth explaining some of this in the commit log rather than "Empirically".
Fair enough.
Quoted 8 lines
IIRC someone was working on enabling PVHVM even for non-PAE i386 kernels, not sure what the status of that is, but clearly it isn't going to change things for Wheezy.
If you wanted instead to enable PV support for these installs then the multi-arch (i386+amd64) isoimages contain the Xen enabled kernels for i386 (as /install.386/xen/*, n.b. /install.amd/xen/* are suitable symlinks, so you can aoid special cases.
I think testing both the emulated and PVHVM devices, in a single test job, is quite nice, actually. And it doesn't seem too slow. So I will keep it this way.
Quoted 3 lines
Anyway, that's clearly an "if you can be bothered thing" so with an expanded commit message:
Acked-by Ian Campbell < [email protected] >
Thanks.
Ian.
PATCH 23/26 Timeouts: Introduce target_adjust_timeout
In reply to Ian JacksonThis function is now called for almost every timeout.
Specifically, it is called in the cases in TestSupport where a guest- or host-related timeout is passed from code which has a $ho or $gho, to code which does not: all callers of poll_loop, and tcmd.
Currently the function is a no-op. Its existence and use will allow us to introduce various provocations for adjusting timeouts, of which I have one planned.
--- Osstest/TestSupport.pm | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)
Osstest/TestSupport.pm +13 -1
@@ -310,6 +310,10 @@ END return $value; } +sub target_adjust_timeout ($$) { + my ($ho,$timeoutref) = @_; # $ho might be a $gho +} + #---------- running commands eg on targets ---------- sub cmd {
@@ -608,6 +612,7 @@ sub target_ping_check_up ($) { return target_ping_check_core(@_,0); } sub target_await_down ($$) { my ($ho,$timeout) = @_; + target_adjust_timeout($ho,\$timeout); poll_loop($timeout,5,'reboot-down', sub { return target_ping_check_down($ho); });
@@ -616,6 +621,7 @@ sub target_await_down ($$) { sub tcmd { # $tcmd will be put between '' but not escaped my ($stdout,$user,$ho,$tcmd,$timeout,$extrasshopts) = @_; $timeout=30 if !defined $timeout; + target_adjust_timeout($ho,\$timeout); tcmdex($timeout,$stdout, 'ssh', sshopts(), @{ $extrasshopts || [] }, sshuho($user,$ho), $tcmd);
@@ -1018,7 +1024,9 @@ sub common_toolstack ($) { sub host_reboot ($) { my ($ho) = @_; target_reboot($ho); - poll_loop(40,2, 'reboot-confirm-booted', sub { + my $timeout = 40; + target_adjust_timeout($ho,\$timeout); + poll_loop($timeout,2, 'reboot-confirm-booted', sub { my $output; if (!eval { $output= target_cmd_output($ho, <<END, 40);
@@ -1397,6 +1405,7 @@ sub report_once ($$$) { sub guest_await_state ($$$$$) { my ($ho,$gho, $what,$wait_st,$timeout) = @_; + target_adjust_timeout($gho,\$timeout); poll_loop($timeout, 30, "await $what request from $gho->{Guest}", sub { my $st= guest_get_state($ho,$gho); return undef if $st eq $wait_st;
@@ -1850,6 +1859,7 @@ sub guest_checkrunning ($$) { sub guest_await_dhcp_tcp ($$) { my ($gho,$timeout) = @_; guest_find_tcpcheckport($gho); + target_adjust_timeout($gho,\$timeout); poll_loop($timeout,1, "guest $gho->{Name} ".visible_undef($gho->{Ether}). " $gho->{TcpCheckPort}".
@@ -1925,6 +1935,7 @@ sub target_tcp_check ($$) { sub await_tcp ($$$) { my ($maxwait,$interval,$ho) = @_; + target_adjust_timeout($ho,\$maxwait); poll_loop($maxwait,$interval, "await tcp $ho->{Name} $ho->{TcpCheckPort}", sub {
@@ -2106,6 +2117,7 @@ sub await_webspace_fetch_byleaf ($$$$$) { my ($maxwait,$interval,$logtailer, $ho, $url) = @_; my $leaf= $url; $leaf =~ s,.*/,,; + target_adjust_timeout($ho,\$maxwait); poll_loop($maxwait,$interval, "fetch $leaf", sub { my ($line, $last); $last= '(none)';
Signature · 1 line
1.7.10.4
Re: [OSSTEST PATCH 23/26] Timeouts: Introduce target_adjust_timeout
Quoted 11 lines · Ian Jackson
This function is now called for almost every timeout.
Specifically, it is called in the cases in TestSupport where a guest- or host-related timeout is passed from code which has a $ho or $gho, to code which does not: all callers of
poll_loop, and tcmd.Currently the function is a no-op. Its existence and use will allow us to introduce various provocations for adjusting timeouts, of which I have one planned.
Signed-off-by Ian Jackson <[email protected]>
PATCH 24/26 Timeouts: Honour guest-related timeout-adjustment runvars
In reply to Ian JacksonWe look up a guest runvar GUEST_CONTEXT_timeoutfactor (according to the usual rules for guest runvars).
Here CONTEXT can currently be general' or install'. (`install' applies when the guest is being installed.) If the runvar exists, all timeouts relating to that guest in that context are increased by the specified factor.
--- Osstest/TestSupport.pm | 6 ++++++ 1 file changed, 6 insertions(+)
Osstest/TestSupport.pm +6 -0
@@ -312,6 +312,10 @@ END sub target_adjust_timeout ($$) { my ($ho,$timeoutref) = @_; # $ho might be a $gho + if ($ho->{Guest}) { + my $context = $ho->{TimeoutContext} // 'general'; + $$timeoutref *= guest_var($ho,"${context}_timeoutfactor",1); + } } #---------- running commands eg on targets ----------
@@ -1604,6 +1608,8 @@ sub prepareguest ($$$$$$) { } } + $gho->{TimeoutContext} = 'install'; + guest_find_lv($gho); guest_find_diskimg($gho); guest_find_ether($gho);
Signature · 1 line
1.7.10.4
Re: [OSSTEST PATCH 24/26] Timeouts: Honour guest-related timeout-adjustment runvars
Quoted 9 lines · Ian Jackson
We look up a guest runvar
GUEST_CONTEXT_timeoutfactor(according to the usual rules for guest runvars).Here CONTEXT can currently be
general' orinstall'. (`install' applies when the guest is being installed.) If the runvar exists, all timeouts relating to that guest in that context are increased by the specified factor.Signed-off-by Ian Jackson <[email protected]>
PATCH 25/26 Debian i386 HVM tests: Increase installation timeout
In reply to Ian JacksonThe Debian i386 image boots a 32-bit non-PAE kernel which therefore cannot have any PV drivers on our 64-bit hypervisors. This makes it a bit slow: in my test on a machine under my desk it took 1400s out of the allowed 2000s.
With this change the timeout is 3000s instead.
--- make-flight | 1 + 1 file changed, 1 insertion(+)
make-flight +1 -0
@@ -268,6 +268,7 @@ do_hvm_debian_test_one () { case "$arch" in amd64) iso_dir='install.amd' ;; i386) iso_dir='install.386' + testvars+=' debianhvm_install_timeoutfactor=1.5' testvars+=' debianhvm_diskdevice=/dev/sda' ;; *) iso_dir="install.$arch" ;; esac
Signature · 1 line
1.7.10.4
Re: [OSSTEST PATCH 25/26] Debian i386 HVM tests: Increase installation timeout
Quoted 8 lines · Ian Jackson
The Debian i386 image boots a 32-bit non-PAE kernel which therefore cannot have any PV drivers on our 64-bit hypervisors. This makes it a bit slow: in my test on a machine under my desk it took 1400s out of the allowed 2000s.
With this change the timeout is 3000s instead.
Signed-off-by Ian Jackson <[email protected]>
(although see comments earlier regarding how you could enable PV drivers here...)
PATCH 26/26 cri-common: Add a missing semicolon
In reply to Ian JacksonThis is not technically needed as bash interprets `a=1 b=2' as settings of both a and b. But it's not idiomatic sh within osstest to use this syntax.
No functional change.
--- cri-common | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
cri-common +1 -1
@@ -68,7 +68,7 @@ select_xenbranch () { case "$branch" in xen-unstable-smoke) tree=xen; xenbranch=$branch; qemuubranch=qemu-xen-unstable;; xen-*) tree=xen; xenbranch=$branch ;; - qemu-mainline) tree=qemuu; xenbranch=xen-unstable qemuubranch=qemu-mainline;; + qemu-mainline) tree=qemuu; xenbranch=xen-unstable; qemuubranch=qemu-mainline;; qemu-upstream-*) tree=qemuu; xenbranch=xen-${branch#qemu-upstream-};; linux) tree=linux; xenbranch=xen-unstable ;; linux-*) tree=linux; xenbranch=xen-unstable ;;
Signature · 1 line
1.7.10.4
Re: [OSSTEST PATCH 26/26] cri-common: Add a missing semicolon
Quoted 7 lines · Ian Jackson
This is not technically needed as bash interprets `a=1 b=2' as settings of both a and b. But it's not idiomatic sh within osstest to use this syntax.
No functional change.
Signed-off-by Ian Jackson <[email protected]>