Skip to contentSkip to conversation

Use snapshot for jessie-backports

PATCH In review Ian Jackson 3 participants 10 messages Feb 13, 2019 – Feb 15, 2019

lore
Share & tools
lore
#0 lore raw

cover Use snapshot for jessie-backports

Things are being removed from jessie-backports, which we rely on for arm64 tests. Switch to using snapshot.debian.org.

In my moderately-formal test, this has got as far as booting one of the machines into Xen. It's currently doing the guest install, which I expect to be fine.

Impact on the Xen Project Massachusetts CI, and the Xen release:

This enables the new code in preseed_backports_packages when $suite is jessie. preseed_backports_packages has three call sites; two in preseed_create_guest but only where $suite is wheezy or stretch, and one in preseed_create in a call to di_special_kernel.

di_special_kernel looks for host flags like `need-kernel-deb-SUITE' (and a compat case for wheezy). In the Massachusetts osstest instance, such flags are need-kernel-deb-jessie-backports (for the laxtons) and need-kernel-deb-jessie-special (for a handful of x86 boxes). But the call to preseed_backports_packages only triggers if $kp eq 'backports', so only for the laxtons.

Accordingly, I have convinced myself that this code will only affect arm64 in the Xen Project CI lab.

Ian Jackson (4):
backports snapshot: Honour DebianSnapshotBackports_<suite> config var backports snapshot: Provide for $apt_insert and $extra_rune backports snapshot: Disable apt timestamp checking (sometimes) backports snapshot: Use 20190206T211314Z for jessie-backports

 Osstest/Debian.pm | 21 ++++++++++++++++++++-
 production-config |  2 ++
 2 files changed, 22 insertions(+), 1 deletion(-)
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#1 lore raw

PATCH 1/4 backports snapshot: Honour DebianSnapshotBackports_<suite> config var

If this is set, use it instead of the usual DebianMirrorHost and Subpath. No functional change with configs that don't set it.

This is not sufficient to work right yet, because snapshots repositories have out-of-date signatures...

---
 Osstest/Debian.pm | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
Osstest/Debian.pm +4 -1
index c6858253..ee7e03cf 100644
@@ -946,6 +946,9 @@ sub preseed_backports_packages ($$$$@) {
     my ($ho, $sfx, $xopts, $suite, @pkgs) = @_;
 
     if (! $xopts->{BackportsSourcesAlreadyAdded}++) {
+	my $bp_url = $c{"DebianSnapshotBackports_$suite"};
+	$bp_url ||= "http://$c{DebianMirrorHost}/$c{DebianMirrorSubpath}";
+
 	preseed_hook_command($ho, 'late_command', $sfx, <<END);
 #!/bin/sh
 set -ex
@@ -953,7 +956,7 @@ set -ex
 cat <<EOF >>/target/etc/apt/sources.list
 
 # $suite backports
-deb http://$c{DebianMirrorHost}/$c{DebianMirrorSubpath} $suite-backports main
+deb $bp_url $suite-backports main
 EOF
 in-target apt-get update
 END
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#2 lore raw

PATCH 2/4 backports snapshot: Provide for $apt_insert and $extra_rune

In reply to Ian Jackson

No functional change.

---
 Osstest/Debian.pm | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
Osstest/Debian.pm +4 -1
index ee7e03cf..5e74e86e 100644
@@ -949,6 +949,8 @@ sub preseed_backports_packages ($$$$@) {
 	my $bp_url = $c{"DebianSnapshotBackports_$suite"};
 	$bp_url ||= "http://$c{DebianMirrorHost}/$c{DebianMirrorSubpath}";
 
+	my $apt_insert='';
+	my $extra_rune='';
 	preseed_hook_command($ho, 'late_command', $sfx, <<END);
 #!/bin/sh
 set -ex
@@ -956,8 +958,9 @@ set -ex
 cat <<EOF >>/target/etc/apt/sources.list
 
 # $suite backports
-deb $bp_url $suite-backports main
+deb $apt_insert $bp_url $suite-backports main
 EOF
+$extra_rune
 in-target apt-get update
 END
     }
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#3 lore raw

PATCH 3/4 backports snapshot: Disable apt timestamp checking (sometimes)

In reply to Ian Jackson

In jessie and earlier, this has to be done with a global option.

In later releases, it can be done by putting some options in [ ] in the relevant sources list entry.

---
 Osstest/Debian.pm | 13 +++++++++++++
 1 file changed, 13 insertions(+)
Osstest/Debian.pm +13 -0
index 5e74e86e..c1e75020 100644
@@ -951,6 +951,19 @@ sub preseed_backports_packages ($$$$@) {
 
 	my $apt_insert='';
 	my $extra_rune='';
+	if ($suite =~ m/wheezy|jessie/) {
+	    # this has global effect, unfortunately
+	    $extra_rune = <<END;
+d=/etc/apt/apt.conf.d
+mkdir -p \$d
+cat >\$d/50osstestsnapshot <<EOF
+Acquire::Check-Valid-Until false;
+EOF
+END
+	} else {
+	    $apt_insert = '[check-valid-until=no]';
+	}
+
 	preseed_hook_command($ho, 'late_command', $sfx, <<END);
 #!/bin/sh
 set -ex
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#4 lore raw

Re: [OSSTEST PATCH 3/4] backports snapshot: Disable apt timestamp checking (sometimes)

Ian Jackson writes ("[OSSTEST PATCH 3/4] backports snapshot: Disable apt timestamp checking (sometimes)"):

Quoted 4 lines

In jessie and earlier, this has to be done with a global option.

In later releases, it can be done by putting some options in [ ] in the relevant sources list entry.

I discover that:

1. This was not effective.
2. It worked in my test yesterday because 20190206T211314Z was less than 14 days ago. It does not work today.

Ian.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#5 lore raw

[OSSTEST PATCH 5/4] backports snapshot: Disable apt timestamp checking in right place

In reply to Ian Jackson

We need to put this in /target or it does not take effect.

Yesterday's 4 patches worked yesterday but not today, because the snapshot in question in fact expired in between. With this additional patch they work today too.

---
 Osstest/Debian.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Osstest/Debian.pm +1 -1
index c1e75020..17dfef54 100644
@@ -954,7 +954,7 @@ sub preseed_backports_packages ($$$$@) {
 	if ($suite =~ m/wheezy|jessie/) {
 	    # this has global effect, unfortunately
 	    $extra_rune = <<END;
-d=/etc/apt/apt.conf.d
+d=/target/etc/apt/apt.conf.d
 mkdir -p \$d
 cat >\$d/50osstestsnapshot <<EOF
 Acquire::Check-Valid-Until false;
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#6 lore raw

Re: [OSSTEST PATCH 5/4] backports snapshot: Disable apt timestamp checking in right place

Quoted 7 lines · Ian Jackson

We need to put this in /target or it does not take effect.

Yesterday's 4 patches worked yesterday but not today, because the snapshot in question in fact expired in between. With this additional patch they work today too.

in case my implicit ack from yesterday expired as well ;-)

Juergen

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#7 lore raw

Re: [OSSTEST PATCH 5/4] backports snapshot: Disable apt timestamp checking in right place

Juergen Gross writes ("Re: [Xen-devel] [OSSTEST PATCH 5/4] backports snapshot: Disable apt timestamp checking in right place"):

Quoted 3 lines

in case my implicit ack from yesterday expired as well ;-)

Thanks :-). I pushed it without waiting. It is now in production (as of 07mumble UTC this morning). I think we will get a push of xen-unstable-smoke shortly.

Ian.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#8 lore raw

PATCH 4/4 backports snapshot: Use 20190206T211314Z for jessie-backports

In reply to Ian Jackson

Some time on 2019-02-07, Debian removed linux-base from jessie-backports. This caused everything to break: apt wasn't happy to get linux-base from jessie-security (because of our -t jessie-backports, probably) and that meant there was no linux-base suitable for linux-image-4.9.x on arm64. We ended up trying to boot the installed system with 3.16, which does not work on our two SoftIron arm64 test boxes.

Also, jessie-backports about to be completely removed.

---
 production-config           | 2 ++
 production-config-cambridge | 2 ++
 2 files changed, 4 insertions(+)
production-config +2 -0
index 6b743d4f..59c74cca 100644
@@ -91,6 +91,8 @@ TftpNetbootGroup osstest
 TftpDiVersion_wheezy 2016-06-08
 TftpDiVersion_jessie 2018-06-26
 
+DebianSnapshotBackports_jessie http://snapshot.debian.org/archive/debian/20190206T211314Z/
+
 # For ISO installs
 DebianImageVersion_wheezy 7.2.0
 DebianImageVersion_jessie 8.2.0
production-config-cambridge +2 -0
index 8e2eadd2..ce6239bd 100644
@@ -78,6 +78,8 @@ TftpNetbootGroup osstest
 TftpDiVersion_wheezy 2016-06-08
 TftpDiVersion_jessie 2018-06-26
 
+DebianSnapshotBackports_jessie http://snapshot.debian.org/archive/debian/20190206T211314Z/
+
 # For ISO installs
 DebianImageVersion_wheezy 7.2.0
 DebianImageVersion_jessie 8.2.0
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#9 lore raw
In reply to Ian Jackson

Ian Jackson writes ("[OSSTEST PATCH 0/4] Use snapshot for jessie-backports"):

Quoted 6 lines

Things are being removed from jessie-backports, which we rely on for arm64 tests. Switch to using snapshot.debian.org.

In my moderately-formal test, this has got as far as booting one of the machines into Xen. It's currently doing the guest install, which I expect to be fine.

...

Quoted 2 lines

Accordingly, I have convinced myself that this code will only affect arm64 in the Xen Project CI lab.

I have pushed this to osstest pretest.

Juergen, I wasn't sure if this met the formal conditions for putting your release ack on it, but I felt you would want me to go ahead.

If you disagree, I can rewind pretest and kill the self-test flight, putting us back in the same situation (although having maybe wasted some machine effort).

BTW, in my semiformal test this has got as far as the guest repeat stop/start test so it is looking good.

Ian.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel