Use snapshot for jessie-backports
cover Use snapshot for jessie-backports
Things are being removed from jessie-backports, which we rely on for arm64 tests. Switch to using snapshot.debian.org.
In my moderately-formal test, this has got as far as booting one of the machines into Xen. It's currently doing the guest install, which I expect to be fine.
Impact on the Xen Project Massachusetts CI, and the Xen release:
This enables the new code in preseed_backports_packages when $suite is jessie. preseed_backports_packages has three call sites; two in preseed_create_guest but only where $suite is wheezy or stretch, and one in preseed_create in a call to di_special_kernel.
di_special_kernel looks for host flags like `need-kernel-deb-SUITE' (and a compat case for wheezy). In the Massachusetts osstest instance, such flags are need-kernel-deb-jessie-backports (for the laxtons) and need-kernel-deb-jessie-special (for a handful of x86 boxes). But the call to preseed_backports_packages only triggers if $kp eq 'backports', so only for the laxtons.
Accordingly, I have convinced myself that this code will only affect arm64 in the Xen Project CI lab.
Ian Jackson (4):
backports snapshot: Honour DebianSnapshotBackports_<suite> config var backports snapshot: Provide for $apt_insert and $extra_rune backports snapshot: Disable apt timestamp checking (sometimes) backports snapshot: Use 20190206T211314Z for jessie-backports
Osstest/Debian.pm | 21 ++++++++++++++++++++- production-config | 2 ++ 2 files changed, 22 insertions(+), 1 deletion(-)
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 1/4 backports snapshot: Honour DebianSnapshotBackports_<suite> config var
If this is set, use it instead of the usual DebianMirrorHost and Subpath. No functional change with configs that don't set it.
This is not sufficient to work right yet, because snapshots repositories have out-of-date signatures...
--- Osstest/Debian.pm | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
Osstest/Debian.pm +4 -1
@@ -946,6 +946,9 @@ sub preseed_backports_packages ($$$$@) { my ($ho, $sfx, $xopts, $suite, @pkgs) = @_; if (! $xopts->{BackportsSourcesAlreadyAdded}++) { + my $bp_url = $c{"DebianSnapshotBackports_$suite"}; + $bp_url ||= "http://$c{DebianMirrorHost}/$c{DebianMirrorSubpath}"; + preseed_hook_command($ho, 'late_command', $sfx, <<END); #!/bin/sh set -ex
@@ -953,7 +956,7 @@ set -ex cat <<EOF >>/target/etc/apt/sources.list # $suite backports -deb http://$c{DebianMirrorHost}/$c{DebianMirrorSubpath} $suite-backports main +deb $bp_url $suite-backports main EOF in-target apt-get update END
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 2/4 backports snapshot: Provide for $apt_insert and $extra_rune
In reply to Ian JacksonNo functional change.
--- Osstest/Debian.pm | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
Osstest/Debian.pm +4 -1
@@ -949,6 +949,8 @@ sub preseed_backports_packages ($$$$@) { my $bp_url = $c{"DebianSnapshotBackports_$suite"}; $bp_url ||= "http://$c{DebianMirrorHost}/$c{DebianMirrorSubpath}"; + my $apt_insert=''; + my $extra_rune=''; preseed_hook_command($ho, 'late_command', $sfx, <<END); #!/bin/sh set -ex
@@ -956,8 +958,9 @@ set -ex cat <<EOF >>/target/etc/apt/sources.list # $suite backports -deb $bp_url $suite-backports main +deb $apt_insert $bp_url $suite-backports main EOF +$extra_rune in-target apt-get update END }
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 3/4 backports snapshot: Disable apt timestamp checking (sometimes)
In reply to Ian JacksonIn jessie and earlier, this has to be done with a global option.
In later releases, it can be done by putting some options in [ ] in the relevant sources list entry.
--- Osstest/Debian.pm | 13 +++++++++++++ 1 file changed, 13 insertions(+)
Osstest/Debian.pm +13 -0
@@ -951,6 +951,19 @@ sub preseed_backports_packages ($$$$@) { my $apt_insert=''; my $extra_rune=''; + if ($suite =~ m/wheezy|jessie/) { + # this has global effect, unfortunately + $extra_rune = <<END; +d=/etc/apt/apt.conf.d +mkdir -p \$d +cat >\$d/50osstestsnapshot <<EOF +Acquire::Check-Valid-Until false; +EOF +END + } else { + $apt_insert = '[check-valid-until=no]'; + } + preseed_hook_command($ho, 'late_command', $sfx, <<END); #!/bin/sh set -ex
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [OSSTEST PATCH 3/4] backports snapshot: Disable apt timestamp checking (sometimes)
Ian Jackson writes ("[OSSTEST PATCH 3/4] backports snapshot: Disable apt timestamp checking (sometimes)"):
Quoted 4 lines
In jessie and earlier, this has to be done with a global option.
In later releases, it can be done by putting some options in [ ] in the relevant sources list entry.
I discover that:
1. This was not effective.
2. It worked in my test yesterday because 20190206T211314Z was less than 14 days ago. It does not work today.
Ian.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
[OSSTEST PATCH 5/4] backports snapshot: Disable apt timestamp checking in right place
In reply to Ian JacksonWe need to put this in /target or it does not take effect.
Yesterday's 4 patches worked yesterday but not today, because the snapshot in question in fact expired in between. With this additional patch they work today too.
--- Osstest/Debian.pm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Osstest/Debian.pm +1 -1
@@ -954,7 +954,7 @@ sub preseed_backports_packages ($$$$@) { if ($suite =~ m/wheezy|jessie/) { # this has global effect, unfortunately $extra_rune = <<END; -d=/etc/apt/apt.conf.d +d=/target/etc/apt/apt.conf.d mkdir -p \$d cat >\$d/50osstestsnapshot <<EOF Acquire::Check-Valid-Until false;
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [OSSTEST PATCH 5/4] backports snapshot: Disable apt timestamp checking in right place
Quoted 7 lines · Ian Jackson
We need to put this in /target or it does not take effect.
Yesterday's 4 patches worked yesterday but not today, because the snapshot in question in fact expired in between. With this additional patch they work today too.
Signed-off-by Ian Jackson <[email protected]>
in case my implicit ack from yesterday expired as well ;-)
Juergen
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [OSSTEST PATCH 5/4] backports snapshot: Disable apt timestamp checking in right place
Juergen Gross writes ("Re: [Xen-devel] [OSSTEST PATCH 5/4] backports snapshot: Disable apt timestamp checking in right place"):
Quoted 3 lines
Release-acked-by Juergen Gross <[email protected]>in case my implicit ack from yesterday expired as well ;-)
Thanks :-). I pushed it without waiting. It is now in production (as of 07mumble UTC this morning). I think we will get a push of xen-unstable-smoke shortly.
Ian.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 4/4 backports snapshot: Use 20190206T211314Z for jessie-backports
In reply to Ian JacksonSome time on 2019-02-07, Debian removed linux-base from jessie-backports. This caused everything to break: apt wasn't happy to get linux-base from jessie-security (because of our -t jessie-backports, probably) and that meant there was no linux-base suitable for linux-image-4.9.x on arm64. We ended up trying to boot the installed system with 3.16, which does not work on our two SoftIron arm64 test boxes.
Also, jessie-backports about to be completely removed.
--- production-config | 2 ++ production-config-cambridge | 2 ++ 2 files changed, 4 insertions(+)
production-config +2 -0
@@ -91,6 +91,8 @@ TftpNetbootGroup osstest TftpDiVersion_wheezy 2016-06-08 TftpDiVersion_jessie 2018-06-26 +DebianSnapshotBackports_jessie http://snapshot.debian.org/archive/debian/20190206T211314Z/ + # For ISO installs DebianImageVersion_wheezy 7.2.0 DebianImageVersion_jessie 8.2.0
production-config-cambridge +2 -0
@@ -78,6 +78,8 @@ TftpNetbootGroup osstest TftpDiVersion_wheezy 2016-06-08 TftpDiVersion_jessie 2018-06-26 +DebianSnapshotBackports_jessie http://snapshot.debian.org/archive/debian/20190206T211314Z/ + # For ISO installs DebianImageVersion_wheezy 7.2.0 DebianImageVersion_jessie 8.2.0
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Ian Jackson writes ("[OSSTEST PATCH 0/4] Use snapshot for jessie-backports"):
Quoted 6 lines
Things are being removed from jessie-backports, which we rely on for arm64 tests. Switch to using snapshot.debian.org.
In my moderately-formal test, this has got as far as booting one of the machines into Xen. It's currently doing the guest install, which I expect to be fine.
...
Quoted 2 lines
Accordingly, I have convinced myself that this code will only affect arm64 in the Xen Project CI lab.
I have pushed this to osstest pretest.
Juergen, I wasn't sure if this met the formal conditions for putting your release ack on it, but I felt you would want me to go ahead.
If you disagree, I can rewind pretest and kill the self-test flight, putting us back in the same situation (although having maybe wasted some machine effort).
BTW, in my semiformal test this has got as far as the guest repeat stop/start test so it is looking good.
Ian.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel