Initial support for ARM64

The following makes a start on support for arm64 systems.

Since arm64 was only added in Debian Jessie this requires Wei's "Debian Jessie patches" as a base line, I'm using v3 of those (note that these have picked up some of the patches which I previously included in this series, since they belong there more).

The main development here is support for UEFI PXE booting, which is not particular to arm64, but is more prominent there. Hopefully this work would be useful for UEFI only x86 systems too.

Since last time the Debian kernel has been updated and now reboots, so build-* works. I've also added support for test-*. This mainly involved plumbing in UEFI chainloading into the grub handling, which is was bit fiddly but worked out ok in the end, I think.

test-arm64-arm64-xl works ok.

test-arm64-arm64-libvirt needs a hack to work on Jessie (differingly library name) for which I don't yet have a clean solution. And then libvirtd SEGVs, like it seems to on armhf at the moment too.

Ian.

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xen.org/xen-devel

The story for PXE booting via UEFI (at least on arm64) is not so straightforward as with pxelinux on x86. There seems to no good bootloader to launch via UEFI+pxe, in fact all I could find was grub (syslinux, and by extension pxelinux.efi, is x86 only).

Add mg-pxe-loader-update modelled on mg-debian-installer-update which will download the necessary grub binaries and produce a grub image which can be used to pxe boot.

grub lacks the convenient ability to search for config file based on (substrings of) the MAC or IP address. So we arrange for the grub.cfg in TftpGrubBase to chain load another config file from TftpTmpDir/'$net_default_mac' where $net_default_mac is a grub variable which is substituted at boot time.

Actually using this requires that bootp/dhcp provide a next-file so UEFI knows what to boot (usually this would be pxelinux.0 which we can't use here). Locally we have configured this as $name/pxe.img, so we can use different loaders.

---
 Osstest.pm           |  3 ++
 README               |  4 ++-
 mg-pxe-loader-update | 89 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 production-config    |  5 +++
 4 files changed, 100 insertions(+), 1 deletion(-)
 create mode 100755 mg-pxe-loader-update

index e8bd77b..e668b3c 100644
@@ -199,6 +199,9 @@ sub readglobalconfig () {
     $c{TftpDiBase} ||= "$c{TftpPlayDir}debian-installer";
     $c{TftpDiVersion} ||= 'current';
 
+    $c{TftpGrubBase} ||= "$c{TftpPlayDir}grub";
+    $c{TftpGrubVersion} ||= 'current';
+
     $c{WebspaceFile} ||= "$ENV{'HOME'}/public_html/";
     $c{WebspaceUrl} ||= "http://$myfqdn/~$whoami/";
     $c{WebspaceCommon} ||= 'osstest/';

index 44e2989..503d15d 100644
@@ -206,7 +206,9 @@ To run osstest in standalone mode:
      netcat
      chiark-utils-bin
 
- - Optional: ipmitool
+ - Optional:
+     ipmitool -- for hosts which use IPMI for power control
+     grub-common -- for mg-pxe-loader-update
 
  - Write a config file
     ~/.xen-osstest/config

index 47c0c4c..46fedc9 100644
@@ -85,12 +85,17 @@ TftpPxeTemplates %name%/pxelinux.cfg
 TftpPxeTemplatesReal pxelinux.cfg/%ipaddrhex%
 
 TftpPxeGroup osstest
+# Update with ./mg-debian-installer-update(-all)
 TftpDiVersion 2015-01-10
 
 # These should normally be the same.
+# Update with ./mg-cpu-microcode-update
 MicrocodeUpdateAmd64 microcode.x86.2015-06-12.cpio
 MicrocodeUpdateI386 microcode.x86.2015-06-12.cpio
 
+# Update with ./mg-pxe-loader-update
+TftpGrubVersion XXXX-XX-XX
+
 XenUsePath /usr/groups/xencore/systems/bin/xenuse
 XenUseUser osstest
 

2.1.4

Ian Campbell writes ("[PATCH OSSTEST v2 01/19] Introduce mg-pxe-loader-update"):

...

I'll try and come up with mgi-debian (by analogy with cr-* using cri-*) with some helper routines for fetch packages etc.

I suppose we do, yes.

Ian.

AIUI the runes used will only result in an ESP if the system was booted via UEFI. IOW I don't think there should be any change for existing systems.

---
 Osstest.pm        | 1 +
 Osstest/Debian.pm | 6 ++++++
 2 files changed, 7 insertions(+)

index e668b3c..172f904 100644
@@ -67,6 +67,7 @@ our %c = qw(
     TestHostKeypairPath id_rsa_osstest
     HostProp_GenEtherPrefixBase 5e:36:0e:f5
 
+    HostDiskESP    300
     HostDiskBoot   300
     HostDiskRoot 10000
     HostDiskSwap  2000

index 929d22e..ded8575 100644
@@ -987,6 +987,12 @@ d-i partman-basicmethods/method_only boolean false
 
 d-i partman-auto/expert_recipe string					\\
 	boot-root ::							\\
+		$c{HostDiskESP} 60 $c{HostDiskESP} free			\\
+			\$iflabel{ gpt }				\\
+			reusemethod{ }					\\
+			method{ efi }					\\
+			format{ }					\\
+		.							\\
 		$c{HostDiskBoot} 50 $c{HostDiskBoot} ext3		\\
 			\$primary{ } \$bootable{ }			\\
 			method{ format } format{ }			\\

2.1.4

Ian Campbell writes ("[PATCH OSSTEST v2 02/19] Debian: Preseed a EFI system partition during host install"):

OK, good.

Other bootloaders handle this with an explicit separate option rather than parsing it out of the command line as pxelinux does. Prepare for supporting these.

---
 ts-host-install | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

index 04d5487..04f065f 100755
@@ -265,9 +265,7 @@ END
     my $initrd= "$ho->{Tftp}{TmpDir}$ho->{Name}--initrd.gz";
     system_checked("cat -- @initrds >$ho->{Tftp}{Path}$initrd");
     
-    push @installcmdline, ("initrd=/$initrd",
-                           "domain=$c{TestHostDomain}",
-                           );
+    push @installcmdline, "domain=$c{TestHostDomain}";
     push @installcmdline,
         get_host_property($ho, "install-append $ho->{Suite}", ''),
         get_host_property($ho, "install-append $ho->{Suite} $r{arch}", '');

@@ -295,7 +293,7 @@ label overwrite
 	menu label ^Overwrite
 	menu default
 	kernel $kernel
-	append $installcmdline
+	append initrd=/$initrd $installcmdline
 	ipappend $ipappend
 	$dtbs
 default overwrite

2.1.4

Ian Campbell writes ("[PATCH OSSTEST v2 03/19] ts-host-install: Split initrd out of @installcmdline"):

(i.e. the bit before/after the -- marker). When abstracting over different bootloaders in a future patch this will be convenient since it allows the code to add to either.

---
 ts-host-install | 21 +++++++++++----------
 1 file changed, 11 insertions(+), 10 deletions(-)

index 04f065f..57a6b3f 100755
@@ -175,8 +175,8 @@ sub setup_pxeboot_firstboot($) {
     
     my $d_i= $ho->{Tftp}{DiBase}.'/'.$r{arch}.'/'.$c{TftpDiVersion}.'-'.$ho->{Suite};
     
-    my @installcmdline= qw(vga=normal);
-    push @installcmdline, di_installcmdline_core($ho, $ps_url, %xopts);
+    my @dicmdline= qw(vga=normal);
+    push @dicmdline, di_installcmdline_core($ho, $ps_url, %xopts);
 
     my $src_initrd= "$d_i/initrd.gz";
     my @initrds= "$ho->{Tftp}{Path}/$src_initrd";

@@ -265,26 +265,27 @@ END
     my $initrd= "$ho->{Tftp}{TmpDir}$ho->{Name}--initrd.gz";
     system_checked("cat -- @initrds >$ho->{Tftp}{Path}$initrd");
     
-    push @installcmdline, "domain=$c{TestHostDomain}";
-    push @installcmdline,
+    push @dicmdline, "domain=$c{TestHostDomain}";
+    push @dicmdline,
         get_host_property($ho, "install-append $ho->{Suite}", ''),
         get_host_property($ho, "install-append $ho->{Suite} $r{arch}", '');
 
     my $console = get_host_native_linux_console($ho);
 
-    push @installcmdline, "console=$console" unless $console eq "NONE";
+    push @dicmdline, "console=$console" unless $console eq "NONE";
 
-    push @installcmdline, qw(--);
+    my @hocmdline;
 
     # See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=762007 for
     # why this is repeated.
-    push @installcmdline, "console=$console" unless $console eq "NONE";
+    push @hocmdline, "console=$console" unless $console eq "NONE";
 
-    push @installcmdline,
+    push @hocmdline,
         get_host_property($ho, "linux-boot-append $ho->{Suite}", ''),
         get_host_property($ho, "linux-boot-append $ho->{Suite} $r{arch}", '');
 
-    my $installcmdline= join ' ', @installcmdline;
+    my $dicmd= join ' ', @dicmdline;
+    my $hocmd= join ' ', @hocmdline;
 
     setup_pxeboot($ho, <<END);
 serial 0 $c{Baud}

@@ -293,7 +294,7 @@ label overwrite
 	menu label ^Overwrite
 	menu default
 	kernel $kernel
-	append initrd=/$initrd $installcmdline
+	append initrd=/$initrd $dicmd -- $hocmd
 	ipappend $ipappend
 	$dtbs
 default overwrite

2.1.4

Ian Campbell writes ("[PATCH OSSTEST v2 04/19] ts-host-install: split the "di" from the "host" command line"):

The mechanism used to PXE boot can differ depending on the firmware type. Therefore refactor into Osstest::TestSupport and key off a new host property "firmware".

Currently supported is "bios" (the default) and "uboot", both of which use pxelinux.cfg style files.

The default for the firmware property is "bios", hence no change for any existing system (including those which use uboot, regardless of whether they are now configured with the firmware proprty or not)

---
 Osstest/TestSupport.pm | 44 +++++++++++++++++++++++++++++++++++++++++---
 ts-host-install        | 19 +++++--------------
 2 files changed, 46 insertions(+), 17 deletions(-)

index 66a0d81..622f44d 100644
@@ -2098,16 +2098,34 @@ sub host_pxefile ($;$) {
         (join ",", sort keys %v)." ?";
 }
 
-sub setup_pxeboot ($$) {
+sub setup_pxelinux_bootcfg ($$) {
     my ($ho, $bootfile) = @_;
     my $f= host_pxefile($ho);
     file_link_contents("$ho->{Tftp}{Path}$ho->{Tftp}{PxeDir}$f", $bootfile,
 	"$ho->{Name}-pxelinux.cfg");
 }
 
-sub setup_pxeboot_local ($) {
+# Systems using BIOS are configured to use pxelinux
+sub setup_bios_pxeboot ($$$$$;%) {
+    my ($ho,$kern,$initrd,$dicmd,$hocmd,%xopts) = @_;
+    my $dtbs = "fdtdir $xopts{dtbs}" if $xopts{dtbs};
+    setup_pxelinux_bootcfg($ho, <<END);
+    serial 0 $c{Baud}
+timeout 5
+label overwrite
+	menu label ^Overwrite
+	menu default
+	kernel $kern
+	append $dicmd initrd=$initrd -- $hocmd
+	ipappend $xopts{ipappend}
+	$dtbs
+default overwrite
+END
+}
+
+sub setup_bios_pxeboot_local ($) {
     my ($ho) = @_;
-    setup_pxeboot($ho, <<END);
+    setup_pxelinux_bootcfg($ho, <<END);
 serial 0 $c{Baud}
 timeout 5
 label local

@@ -2116,6 +2134,26 @@ default local
 END
 }
 
+# uboot emulates pxelinux, so reuse BIOS stuff
+sub setup_uboot_pxeboot ($$$$$;%) { return &setup_bios_pxeboot; }
+sub setup_uboot_pxeboot_local ($) { return &setup_bios_pxeboot_local; }
+
+sub setup_pxeboot_local ($) {
+    my ($ho) = @_;
+    my $firmware = get_host_property($ho, "firmware", "bios");
+    $firmware =~ s/-/_/g;
+    no strict qw(refs);
+    return &{"setup_${firmware}_pxeboot_local"}($ho);
+}
+
+sub setup_pxeboot ($$$$$;%) {
+    my ($ho,$kern,$initrd,$dicmd,$hocmd,%xopts) = @_;
+    my $firmware = get_host_property($ho, "firmware", "bios");
+    $firmware =~ s/-/_/g;
+    no strict qw(refs);
+    return &{"setup_${firmware}_pxeboot"}($ho,$kern,$initrd,$dicmd,$hocmd,%xopts);
+}
+
 #---------- ISO images ----------
 sub iso_create_genisoimage ($$$$;@) {
     my ($ho,$iso,$dir,$isotimeout,@xopts) = @_;

index 57a6b3f..4e60e6e 100755
@@ -244,13 +244,14 @@ SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="$ho->{Ether}", A
 END
     }
 
-    my $dtbs = "";
+    my %xopts;
+
     foreach my $kp (keys %{ $ho->{Flags} }) {
 	# Backwards compatibility
 	$kp = "need-kernel-deb-wheezy-backports" if $kp eq "need-kernel-deb-armmp";
 	$kp =~ s/need-kernel-deb-$ho->{Suite}-// or next;
 
-	$dtbs = "fdtdir /$d_i/$kp-dtbs"
+	$xopts{dtbs} = "/$d_i/$kp-dtbs"
 	    if -e "$ho->{Tftp}{Path}/$d_i/$kp-dtbs";
     }
 

@@ -287,18 +288,8 @@ END
     my $dicmd= join ' ', @dicmdline;
     my $hocmd= join ' ', @hocmdline;
 
-    setup_pxeboot($ho, <<END);
-serial 0 $c{Baud}
-timeout 5
-label overwrite
-	menu label ^Overwrite
-	menu default
-	kernel $kernel
-	append initrd=/$initrd $dicmd -- $hocmd
-	ipappend $ipappend
-	$dtbs
-default overwrite
-END
+    $xopts{ipappend} = $ipappend;
+    setup_pxeboot($ho, $kernel, "/$initrd", $dicmd, $hocmd, %xopts);
 }
 
 install();

2.1.4

Ian Campbell writes ("[PATCH OSSTEST v2 05/19] Refactor pxelinux configuration"):

Are you expecting to introduce setup_FIRMWARE_SOMETHINGELSE ?

AFIACT not, in which case this name is quite confusing. Putting the variable part at the end would be much clearer:
setup_pxeboot_${firmware}.

...

I think initially I was initially expecting to but ended up not needing to.

I will assume you would also prefer "setup_${firmware}_pxeboot_local" to become "setup_pxeboot_local_${firmware}" (rather than "setup_pxeboot_ ${firmware}_local").

Good idea.

Do you think setup_pxeboot maybe ought to be setup_pxeboot_di or something along those lines?

Ian.

Ian Campbell writes ("Re: [PATCH OSSTEST v2 05/19] Refactor pxelinux configuration"):

Yes. (Do we mean pxeboot' or netboot'?)

Quite possibly, yes. Or maybe setup_netboot_di.

Ian.

Both BIOS and UEFI call it PXE, but I don't know if the UEFI case is actually "PXE".

I'll go for setup_pxeboot_di for now. I intend to leave it in TestSupport.pm with the pxe stuff rather than moving to Debian.pm with the Debian stuff.

Ian.

---
 Osstest/TestSupport.pm | 37 ++++++++++++++++++++++++++++++++++++-
 README                 |  1 +
 2 files changed, 37 insertions(+), 1 deletion(-)

index 622f44d..1164a11 100644
@@ -898,7 +898,7 @@ sub selecthost ($) {
     $ho->{Tftp} = { };
     $ho->{Tftp}{$_} = $c{"Tftp${_}_${tftpscope}"} || $c{"Tftp${_}"}
         foreach qw(Path TmpDir PxeDir PxeGroup PxeTemplates PxeTemplatesReal
-                   DiBase);
+                   DiBase GrubBase);
 
     #----- finalise -----
 

@@ -2138,6 +2138,41 @@ END
 sub setup_uboot_pxeboot ($$$$$;%) { return &setup_bios_pxeboot; }
 sub setup_uboot_pxeboot_local ($) { return &setup_bios_pxeboot_local; }
 
+sub setup_grub_efi_bootcfg ($$) {
+    my ($ho, $bootfile) = @_;
+    my $f = "grub.cfg-$ho->{Ether}";
+    my $grub= $ho->{Tftp}{Path}.'/'.$ho->{Tftp}{GrubBase}.'/'.$c{TftpGrubVersion}.
+	"/pxegrub-$r{arch}.efi";
+    my $pxe=$ho->{Tftp}{Path}.'/'.$ho->{Name}.'/pxe.img';
+
+    logm("Copy $grub => $pxe");
+    copy($grub, $pxe) or die "Copy $grub to $pxe failed: $!";
+
+    logm("grub_efi bootcfg into $f");
+    file_link_contents("$ho->{Tftp}{Path}$ho->{Tftp}{TmpDir}$f", $bootfile,
+		       "$ho->{Name}-pxegrub.cfg");
+}
+
+# UEFI systems PXE boot using grub.efi
+sub setup_uefi_pxeboot ($$$$$;%) {
+    my ($ho,$kern,$initrd,$dicmd,$hocmd,%xopts) = @_;
+    setup_grub_efi_bootcfg($ho, <<END);
+set default=0
+set timeout=5
+menuentry 'overwrite' {
+  linux $kern $dicmd -- $hocmd
+  initrd $initrd
+}
+END
+}
+
+sub setup_uefi_pxeboot_local ($) {
+    my ($ho) = @_;
+    setup_grub_efi_bootcfg($ho, <<END);
+echo Should do a local boot here, somehow
+END
+}
+
 sub setup_pxeboot_local ($) {
     my ($ho) = @_;
     my $firmware = get_host_property($ho, "firmware", "bios");

index 503d15d..1351f49 100644
@@ -455,6 +455,7 @@ TftpFoo_<scope> and TftpFoo
 
         DiBase        The path under `Path' to the root of the debian installer
                       images.
+        GrubBase      The path under `Path' to the root of the grub EFI PXE images.
 
     For hosts in scope "default", TftpFoo_default (if set) takes
     precedence over TftpFoo.  TftpFoo is used when the setting Foo is

2.1.4

Ian Campbell writes ("[PATCH OSSTEST v2 06/19] ts-host-install: Support UEFI PXE boot using grub.efi"):

First arrange for bootloader to be installed to removable media path, by using a new in Jessie preseed option. Then use that to chainload a bootloader from the disk.

The removable media path is well known (part of the UEFI spec) which saves us having to worry about which OS is on the host (so long as the OS can be configured to populate the removable media path)

---
 Osstest/Debian.pm      |  2 ++
 Osstest/TestSupport.pm | 18 +++++++++++++++++-
 2 files changed, 19 insertions(+), 1 deletion(-)

index ded8575..9e1e69e 100644
@@ -757,6 +757,8 @@ d-i apt-setup/contrib boolean false
 
 d-i pkgsel/include string openssh-server, ntp, ntpdate, ethtool, chiark-utils-bin, $extra_packages
 
+d-i grub-installer/force-efi-extra-removable boolean true
+
 $xopts{ExtraPreseed}
 
 END

index 1164a11..5eeb907 100644
@@ -2168,8 +2168,24 @@ END
 
 sub setup_uefi_pxeboot_local ($) {
     my ($ho) = @_;
+    my %efi_archs = qw(amd64 X64
+                       arm32 ARM
+                       arm64 AARCH64
+                       i386  IA32);
+    die "EFI arch" unless $efi_archs{ $r{arch} };
+    my $efi = $efi_archs{ $r{arch} };
     setup_grub_efi_bootcfg($ho, <<END);
-echo Should do a local boot here, somehow
+set default=0
+set timeout=5
+menuentry 'local' {
+  insmod chain
+  insmod part_gpt
+  insmod part_msdos
+  set root=(hd0,gpt1)
+  echo "Chainloading (\${root})/EFI/BOOT/BOOTAA64.EFI"
+  chainloader (\${root})/EFI/BOOT/BOOTAA64.EFI
+  boot
+}
 END
 }
 

2.1.4

Ian Campbell writes ("[PATCH OSSTEST v2 07/19] Enable chain loading to local disk for UEFI PXE systems."):

The `removable media path' is a UEFI path that refers to part of the fixed disk drive ?

Ian.

Yes, it is the UEFI standardized path (per arch) to an EFI executable which the firmware can/should boot by default when told to boot from a removable device (CD, USB stick, etc). (AIUI, etc). Somewhat like the UEFI equivalent of a stage 1 bootloader in the MBR on a floppy or something.

It's a fairly common BIOS bug to default to using this path (rather than the data base of boot options) for non-removable disks too, which is why Jessie gained an option to install a bootloader there too. Which was serendipitous for us here...

Ian.

Runvars:

	build-arm64                                   arch                        arm64
	build-arm64                                   build_lvextend_max          50
	build-arm64                                   enable_ovmf                 true
	build-arm64                                   enable_xend                 false
	build-arm64                                   enable_xsm                  false
	build-arm64                                   host_hostflags              share-build-jessie-arm64,arch-arm64,suite-jessie,purpose-build
	build-arm64                                   revision_ovmf
	build-arm64                                   revision_qemu
	build-arm64                                   revision_qemuu              b2da824bc5ad35fb9f1e74a203d7be96a7b0345e
	build-arm64                                   revision_seabios
	build-arm64                                   revision_xen                e13013dbf1d5997915548a3b5f1c39594d8c1d7b
	build-arm64                                   tree_ovmf
	build-arm64                                   tree_qemu                   git://xenbits.xen.org/staging/qemu-xen-unstable.git
	build-arm64                                   tree_qemuu                  git://xenbits.xen.org/staging/qemu-upstream-unstable.git
	build-arm64                                   tree_seabios
	build-arm64                                   tree_xen                    git://xenbits.xen.org/xen.git
	build-arm64-libvirt                           arch                        arm64
	build-arm64-libvirt                           build_lvextend_max          50
	build-arm64-libvirt                           buildjob                    build-arm64
	build-arm64-libvirt                           host_hostflags              share-build-jessie-arm64,arch-arm64,suite-jessie,purpose-build
	build-arm64-libvirt                           revision_libvirt            fd74e231751334b64af0934b680c5cc62f652453
	build-arm64-libvirt                           tree_libvirt                git://xenbits.xen.org/libvirt.git
	build-arm64-libvirt                           tree_xen                    git://xenbits.xen.org/xen.git
	build-arm64-pvops                             arch                        arm64
	build-arm64-pvops                             build_lvextend_max          50
	build-arm64-pvops                             host_hostflags              share-build-jessie-arm64,arch-arm64,suite-jessie,purpose-build
	build-arm64-pvops                             kconfig_override_y          CONFIG_EXT4_FS
	build-arm64-pvops                             kconfighow                  xen-enable-xen-config
	build-arm64-pvops                             revision_linux              ec18c9fc039971041d854e0d58551f1f1a32ff8f
	build-arm64-pvops                             revision_linuxfirmware      c530a75c1e6a472b0eb9558310b518f0dfcd8860
	build-arm64-pvops                             tree_linux                  git://xenbits.xen.org/linux-pvops.git
	build-arm64-pvops                             tree_linuxfirmware          git://xenbits.xen.org/osstest/linux-firmware.git
	build-arm64-pvops                             treevcs_linux               git
	build-arm64-xsm                               arch                        arm64
	build-arm64-xsm                               build_lvextend_max          50
	build-arm64-xsm                               enable_ovmf                 true
	build-arm64-xsm                               enable_xend                 false
	build-arm64-xsm                               enable_xsm                  true
	build-arm64-xsm                               host_hostflags              share-build-jessie-arm64,arch-arm64,suite-jessie,purpose-build
	build-arm64-xsm                               revision_ovmf
	build-arm64-xsm                               revision_qemu
	build-arm64-xsm                               revision_qemuu              b2da824bc5ad35fb9f1e74a203d7be96a7b0345e
	build-arm64-xsm                               revision_seabios
	build-arm64-xsm                               revision_xen                e13013dbf1d5997915548a3b5f1c39594d8c1d7b
	build-arm64-xsm                               tree_ovmf
	build-arm64-xsm                               tree_qemu                   git://xenbits.xen.org/staging/qemu-xen-unstable.git
	build-arm64-xsm                               tree_qemuu                  git://xenbits.xen.org/staging/qemu-upstream-unstable.git
	build-arm64-xsm                               tree_seabios
	build-arm64-xsm                               tree_xen                    git://xenbits.xen.org/xen.git
	test-arm64-arm64-libvirt                      all_hostflags               arch-arm64,arch-xen-arm64,suite-jessie,purpose-test
	test-arm64-arm64-libvirt                      arch                        arm64
	test-arm64-arm64-libvirt                      buildjob                    build-arm64
	test-arm64-arm64-libvirt                      debian_arch                 arm64
	test-arm64-arm64-libvirt                      debian_kernkind             pvops
	test-arm64-arm64-libvirt                      enable_xsm                  false
	test-arm64-arm64-libvirt                      kernbuildjob                build-arm64-pvops
	test-arm64-arm64-libvirt                      kernkind                    pvops
	test-arm64-arm64-libvirt                      libvirtbuildjob             build-arm64-libvirt
	test-arm64-arm64-libvirt                      toolstack                   libvirt
	test-arm64-arm64-libvirt                      xenbuildjob                 build-arm64
	test-arm64-arm64-libvirt-xsm                  all_hostflags               arch-arm64,arch-xen-arm64,suite-jessie,purpose-test
	test-arm64-arm64-libvirt-xsm                  arch                        arm64
	test-arm64-arm64-libvirt-xsm                  buildjob                    build-arm64-xsm
	test-arm64-arm64-libvirt-xsm                  debian_arch                 arm64
	test-arm64-arm64-libvirt-xsm                  debian_kernkind             pvops
	test-arm64-arm64-libvirt-xsm                  enable_xsm                  true
	test-arm64-arm64-libvirt-xsm                  kernbuildjob                build-arm64-pvops
	test-arm64-arm64-libvirt-xsm                  kernkind                    pvops
	test-arm64-arm64-libvirt-xsm                  libvirtbuildjob             build-arm64-libvirt
	test-arm64-arm64-libvirt-xsm                  toolstack                   libvirt
	test-arm64-arm64-libvirt-xsm                  xenbuildjob                 build-arm64-xsm
	test-arm64-arm64-xl                           all_hostflags               arch-arm64,arch-xen-arm64,suite-jessie,purpose-test
	test-arm64-arm64-xl                           arch                        arm64
	test-arm64-arm64-xl                           buildjob                    build-arm64
	test-arm64-arm64-xl                           debian_arch                 arm64
	test-arm64-arm64-xl                           debian_kernkind             pvops
	test-arm64-arm64-xl                           enable_xsm                  false
	test-arm64-arm64-xl                           kernbuildjob                build-arm64-pvops
	test-arm64-arm64-xl                           kernkind                    pvops
	test-arm64-arm64-xl                           toolstack                   xl
	test-arm64-arm64-xl                           xenbuildjob                 build-arm64
	test-arm64-arm64-xl-credit2                   all_hostflags               arch-arm64,arch-xen-arm64,suite-jessie,purpose-test
	test-arm64-arm64-xl-credit2                   arch                        arm64
	test-arm64-arm64-xl-credit2                   buildjob                    build-arm64
	test-arm64-arm64-xl-credit2                   debian_arch                 arm64
	test-arm64-arm64-xl-credit2                   debian_kernkind             pvops
	test-arm64-arm64-xl-credit2                   guests_vcpus                4
	test-arm64-arm64-xl-credit2                   kernbuildjob                build-arm64-pvops
	test-arm64-arm64-xl-credit2                   kernkind                    pvops
	test-arm64-arm64-xl-credit2                   toolstack                   xl
	test-arm64-arm64-xl-credit2                   xen_boot_append             sched=credit2
	test-arm64-arm64-xl-credit2                   xenbuildjob                 build-arm64
	test-arm64-arm64-xl-multivcpu                 all_hostflags               arch-arm64,arch-xen-arm64,suite-jessie,purpose-test
	test-arm64-arm64-xl-multivcpu                 arch                        arm64
	test-arm64-arm64-xl-multivcpu                 buildjob                    build-arm64
	test-arm64-arm64-xl-multivcpu                 debian_arch                 arm64
	test-arm64-arm64-xl-multivcpu                 debian_kernkind             pvops
	test-arm64-arm64-xl-multivcpu                 guests_vcpus                4
	test-arm64-arm64-xl-multivcpu                 kernbuildjob                build-arm64-pvops
	test-arm64-arm64-xl-multivcpu                 kernkind                    pvops
	test-arm64-arm64-xl-multivcpu                 toolstack                   xl
	test-arm64-arm64-xl-multivcpu                 xenbuildjob                 build-arm64
	test-arm64-arm64-xl-sedf                      all_hostflags               arch-arm64,arch-xen-arm64,suite-jessie,purpose-test
	test-arm64-arm64-xl-sedf                      arch                        arm64
	test-arm64-arm64-xl-sedf                      buildjob                    build-arm64
	test-arm64-arm64-xl-sedf                      debian_arch                 arm64
	test-arm64-arm64-xl-sedf                      debian_kernkind             pvops
	test-arm64-arm64-xl-sedf                      guests_vcpus                4
	test-arm64-arm64-xl-sedf                      kernbuildjob                build-arm64-pvops
	test-arm64-arm64-xl-sedf                      kernkind                    pvops
	test-arm64-arm64-xl-sedf                      linux_boot_append           loglevel=9 debug
	test-arm64-arm64-xl-sedf                      toolstack                   xl
	test-arm64-arm64-xl-sedf                      xen_boot_append             sched=sedf loglvl=all
	test-arm64-arm64-xl-sedf                      xenbuildjob                 build-arm64
	test-arm64-arm64-xl-sedf-pin                  all_hostflags               arch-arm64,arch-xen-arm64,suite-jessie,purpose-test
	test-arm64-arm64-xl-sedf-pin                  arch                        arm64
	test-arm64-arm64-xl-sedf-pin                  buildjob                    build-arm64
	test-arm64-arm64-xl-sedf-pin                  debian_arch                 arm64
	test-arm64-arm64-xl-sedf-pin                  debian_kernkind             pvops
	test-arm64-arm64-xl-sedf-pin                  guests_vcpus                4
	test-arm64-arm64-xl-sedf-pin                  kernbuildjob                build-arm64-pvops
	test-arm64-arm64-xl-sedf-pin                  kernkind                    pvops
	test-arm64-arm64-xl-sedf-pin                  linux_boot_append           loglevel=9 debug
	test-arm64-arm64-xl-sedf-pin                  toolstack                   xl
	test-arm64-arm64-xl-sedf-pin                  xen_boot_append             sched=sedf loglvl=all dom0_vcpus_pin
	test-arm64-arm64-xl-sedf-pin                  xenbuildjob                 build-arm64
	test-arm64-arm64-xl-xsm                       all_hostflags               arch-arm64,arch-xen-arm64,suite-jessie,purpose-test
	test-arm64-arm64-xl-xsm                       arch                        arm64
	test-arm64-arm64-xl-xsm                       buildjob                    build-arm64-xsm
	test-arm64-arm64-xl-xsm                       debian_arch                 arm64
	test-arm64-arm64-xl-xsm                       debian_kernkind             pvops
	test-arm64-arm64-xl-xsm                       enable_xsm                  true
	test-arm64-arm64-xl-xsm                       kernbuildjob                build-arm64-pvops
	test-arm64-arm64-xl-xsm                       kernkind                    pvops
	test-arm64-arm64-xl-xsm                       toolstack                   xl
	test-arm64-arm64-xl-xsm                       xenbuildjob                 build-arm64-xsm

---

---
 make-flight     |  4 ++--
 mfi-common      | 23 ++++++++++++++++++-----
 ts-kernel-build |  1 +
 3 files changed, 21 insertions(+), 7 deletions(-)

index 78ace52..c821833 100755
@@ -107,7 +107,7 @@ job_create_test_filter_callback () {
 
 test_matrix_branch_filter_callback () {
   case "$xenarch" in
-  armhf)
+  armhf|arm64)
         case "$branch" in
         linux-3.0) return 1;;
         linux-3.4) return 1;;

@@ -366,7 +366,7 @@ test_matrix_do_one () {
   do_credit2_tests
 
   # No further arm tests at the moment
-  if [ $dom0arch = armhf ]; then
+  if [ $dom0arch = armhf ] || [ $dom0arch = arm64 ]; then
       return
   fi
 

index a9e966f..ffb6775 100644
@@ -67,12 +67,12 @@ create_build_jobs () {
      BUILD_RUNVARS+=" build_lvextend_max=$BUILD_LVEXTEND_MAX "
   fi
 
-  for arch in ${BUILD_ARCHES- i386 amd64 armhf }; do
+  for arch in ${BUILD_ARCHES- i386 amd64 armhf arm64 }; do
 
     if [ "x$arch" = xdisable ]; then continue; fi
 
     case "$arch" in
-    armhf)
+    armhf|arm64)
       case "$branch" in
       linux-3.0) continue;;
       linux-3.4) continue;;

@@ -247,7 +247,7 @@ create_build_jobs () {
     fi
 
     case "$arch" in
-    armhf) continue;; # don't do any other kernel builds
+    armhf|arm64) continue;; # don't do any other kernel builds
     esac
 
     if [ "x$REVISION_LINUX_OLD" != xdisable ]; then

@@ -319,13 +319,25 @@ test_matrix_iterate () {
   *)                    onetoolstack=xl ;;
   esac
 
-  for xenarch in ${TEST_ARCHES- i386 amd64 armhf } ; do
+  for xenarch in ${TEST_ARCHES- i386 amd64 armhf arm64 } ; do
 
     if [ "x$xenarch" = xdisable ]; then continue; fi
 
     test_matrix_branch_filter_callback || continue
 
     case "$xenarch" in
+    arm64)
+          # Arm64 from 4.5 onwards only
+          case "$xenbranch" in
+          xen-3.*-testing) continue;;
+          xen-4.0-testing) continue;;
+          xen-4.1-testing) continue;;
+          xen-4.2-testing) continue;;
+          xen-4.3-testing) continue;;
+          xen-4.4-testing) continue;;
+          *) ;;
+          esac
+          ;;
     armhf)
           # Arm from 4.3 onwards only
           case "$xenbranch" in

@@ -370,13 +382,14 @@ test_matrix_iterate () {
       *)          echo >&2 "kernkind ?  $kern"; exit 1 ;;
       esac
 
-      for dom0arch in i386 amd64 armhf; do
+      for dom0arch in i386 amd64 armhf arm64; do
 
         case ${xenarch}_${dom0arch} in
             amd64_amd64) ;;
             amd64_i386) ;;
             i386_i386) ;;
             armhf_armhf) ;;
+            arm64_arm64) ;;
             *) continue ;;
         esac
 

index e7cbdc1..47ddf6f 100755
@@ -27,6 +27,7 @@ builddirsprops();
 
 my $archparms = {
     'armhf' => { DefConfig => 'multi_v7_defconfig', KernelImage => 'arch/arm/boot/zImage' },
+    'arm64' => { DefConfig => 'defconfig',          KernelImage => 'arch/arm64/boot/Image' },
     'i386'  => { DefConfig => 'defconfig',          KernelImage => 'arch/x86/boot/bzImage' },
     'amd64' => { DefConfig => 'defconfig',          KernelImage => 'arch/x86/boot/bzImage' }
 };

2.1.4

XXX We probably need a newer kernel to actually be useful.

---
 ts-kernel-build | 12 ++++++++++++
 1 file changed, 12 insertions(+)

index 47ddf6f..4014a6c 100755
@@ -526,6 +526,18 @@ setopt CONFIG_FUSION_LOGGING n
 
 setopt CONFIG_BLK_CPQ_CISS_DA m
 
+setopt CONFIG_FHANDLE y
+
+# APM XGENE
+setopt CONFIG_PHY_XGENE y
+#setopt CONFIG_NET_XGENE y
+setopt CONFIG_AHCI_XGENE y
+setopt CONFIG_POWER_RESET_XGENE y
+setopt CONFIG_RTC_DRV_XGENE y
+
+#arm64 crashes without -- determine why
+setopt CONFIG_SPARSEMEM_VMEMMAP y
+
 case ${XEN_TARGET_ARCH} in
     x86_32) setopt CONFIG_64BIT n ;;
     x86_64)

2.1.4

Ian Campbell writes ("[PATCH OSSTEST v2 09/19] ts-kernel-build: Additional kernel options for Mustang"):

This doesn't seem quite finished ?

What does this do and do we care about it on other platforms ?

Sorry, yes, I forgot about this...

Locally I've set {TREE,REVISION}_LINUX(_ARM) to v4.1-rcN + one upstream fix (NB: FWIW I heard that 4.1 is due to be the next LTSI version) which is what I'm testing with. I need to revisit how we will get this in osstest -- I think I'm going to propose winding the linux-xen-arm tree to something of that era (Stefano, does that sound ok?).

First I should retest with v3.18 and make sure that really doesn't work (I had some trouble with it, they may have been my own making though). If v3.18 works that would be best since I think we will want to wind the main kernel osstest uses up to that sooner or later.

This is a rebasing mistake, should actually be a separate patch which Wei already sent "[PATCH OSSTEST v3 11/11] ts-kernel-build: enable CONFIG_FHANDLE".

I suspect this won't be needed with the newer kernel (originally I was using 3.16 or so), I'll remove it and see.

Ian.

XXX fold somewhere?

---
 ts-host-install | 3 +++
 1 file changed, 3 insertions(+)

index 4e60e6e..6b18800 100755
@@ -255,6 +255,9 @@ END
 	    if -e "$ho->{Tftp}{Path}/$d_i/$kp-dtbs";
     }
 
+    $xopts{dtbs} = "/$d_i/dtbs"
+	if !$xopts{dtbs} && -e "$ho->{Tftp}{Path}/$d_i/dtbs";
+
     file_simple_write_contents("$initrd_overlay.cpio", sub {
         contents_make_cpio($_[0], 'newc', "$initrd_overlay.d");
     });

2.1.4

Ian Campbell writes ("[PATCH OSSTEST v2 10/19] ts-host-install: Set dtbs in the non-special kernel case too."):

I don't mind it being in a patch by itself, but I do want a commit message that tells me why, not just what.

Ian.

I really should have reviewed my commit messages more carefully, sorry.

I have made this:
ts-host-install: Set dtbs in the non-special kernel case too.

When DTB support was added for armhf it required a backports kernel for use with Wheezy and therefore DTBs were only collected for that special case. However as we move forward to newer kernels it should be possible to run using the proper kernel (e.g. Jessie should work for armhf). Therefore we will need the DTBs for those kernels too.

Ian.

Ian Campbell writes ("Re: [PATCH OSSTEST v2 10/19] ts-host-install: Set dtbs in the non-special kernel case too."):

Thanks.

Debian inserts itself before any existing entries, including the PXE one, meaning we otherwise cannot remotely regroove the box. Preseed some commands to reset the boot order to BootCurrent i.e. how we booted (so the PXE entry).

There is still a window between the Debian entry being added (by grub-installer.udeb) and us removing it, a failure (e.g. power cycle) in that interval would require manual action to fixup.

This is triggered by a new quirk-uefi-bootorder-reset how flag.

---

This seems to just be normal/expected behaviour for UEFI installs (not just of Debian), so perhaps the quirk is unneeded.

---
 Osstest/Debian.pm | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

index 9e1e69e..c6687db 100644
@@ -1018,6 +1018,21 @@ END
 
     preseed_microcode($ho,$sfx);
 
+    if (get_host_property($ho, "firmware") eq "uefi" &&
+        $ho->{Flags}{'quirk-uefi-bootorder-reset'}) {
+	# Disable any new Debian boot entry, so we reboot from PXE.
+        preseed_hook_command($ho, 'late_command', $sfx, <<'END');
+#!/bin/sh
+set -ex
+
+entry=`in-target --pass-stdout bash -c 'efibootmgr -v | sed -ne "s/BootCurrent: \([0-9]\+\)/\1/p"'` #/
+in-target efibootmgr -o $entry
+
+#in-target bash -c 'efibootmgr -o `efibootmgr -v | sed -ne "s/BootCurrent: \([0-9]\+\)/\1/p"`' #/
+
+END
+    }
+
     $preseed_file .= preseed_hook_cmds();
 
     if ($ho->{Flags}{'no-di-kernel'}) {

2.1.4

Ian Campbell writes ("[PATCH OSSTEST v2 11/19] Debian: Fixup UEFI boot order during install"):

This seems a pretty serious bug. Is there a way to avoid it ?

Ian.

Unfortunately not as far as I can tell, it seems to be a major shortcoming of the way UEFI boot order is managed both from the UEFI UI and via the Linux command line tools. FWIW I was inspired by the way XenRT has to do this too (so it is a problem for x86 too).

grub-installer happens pretty late in the install, so the gap until the late command which repairs things is short, but not ideal I agree.

I think the path to a fix would be something like:

efibootmgr to gain an option to add an entry but not add it to the bootloader (or to not put the new entry at the head of the list).

grub-install (from main grub package) to then gain an option causing it to pass the new efibootmgr option.

grub-installer udeb (i.e. the grub part of debian installer) to gain a presseded way of setting that option when calling grub-install.

:-/

Ian.

Ian Campbell writes ("Re: [PATCH OSSTEST v2 11/19] Debian: Fixup UEFI boot order during install"):

The real risk is, I think, that a marginal timeout results in the machine being powered off at the wrong moment.

We could reduce the risk of this by having the installer report back to the controller saying "I'm about to enter the critical region" and "I have now exited the critical region and am about to reboot", thus applying a separate timeout to the critical region.

Ian.

Hrm, yes.

Exiting the critical region is easy, it's the same script as I added here.

Entering the critical region I'm not sure how to hook into, grub-installer doesn't seem to provide any hooks of that sort. early_hook seems like a rather too early lower bound.

Ian.

This is very useful when iterating over kernel configurations, since it avoids blowing away the build tree and all the existing built objects. The Linux build system does the right thing when .config changes and only rebuilds the affected bits.

---
 ts-kernel-build | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

index 4014a6c..55c8acf 100755
@@ -22,6 +22,18 @@ use Osstest::TestSupport;
 use Osstest::BuildSupport;
 
 tsreadconfig();
+
+our $reuse = 0;
+while (@ARGV and $ARGV[0] =~ m/^-/) {
+    $_= shift @ARGV;
+    last if m/^--$/;
+    if (m/^--reuse$/) {
+	$reuse = 1;
+    } else {
+	die "$_ ?";
+    }
+}
+
 selectbuildhost(\@ARGV);
 builddirsprops();
 

@@ -35,6 +47,8 @@ my $archparms = {
 sub enable_xen_config ();
 
 sub checkout () {
+    return if $reuse;
+
     prepbuilddirs();
 
     build_clone($ho, 'linux', $builddir, 'linux');

2.1.4

Ian Campbell writes ("[PATCH OSSTEST v2 12/19] ts-kernel-build: Support --reuse to keep same build tree"):

This one should perhaps be considered for bundling into some other bunch of stuff, as AFAICT there is no reason for it to wait.

Ian.

Thanks.

Yes, it was really just something incidental which I happened to write to make the last but one patch easier (adjusting .config) to come up with.

It could go in alone or with some other miscellanea I think.

Ian

OSSTEST_CONFIG still trumps both.

---
 standalone | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

index 17fa40c..ad12bad 100755
@@ -68,7 +68,13 @@ TEMP=$(getopt -o c:f:h:rRs --long config:,flight:,host:,reuse,noreuse,reinstall,
 
 eval set -- "$TEMP"
 
-config=${OSSTEST_CONFIG-$HOME/.xen-osstest/config}
+if [ -n "${OSSTEST_CONFIG}" ]; then
+    config=${OSSTEST_CONFIG}
+elif [ -f standalone.config ]; then
+    config=standalone.config
+else
+    config=$HOME/.xen-osstest/config
+fi
 flight="standalone"
 host=
 reuse=1 # Don't blow away machines by default

2.1.4

Ian Campbell writes ("[PATCH OSSTEST v2 13/19] standalone: Prefer ./standalone.config to $HOME/.xen-osstest/config"):

This results in us having
standalone-config-example
production-config
production-config-cambridge
but
standalone.config
is actually read by default.

Perhaps the latter should be
local-config
?

Ian.

That's ok by me.

But standalone.config can from standalone-reset in the first place.standalone, I'm happy to change both but just wanted to mention since that's been around for a while (so I'm not just changing something new).

Don't know if we care about that though.

Ian.

Ian Campbell writes ("Re: [PATCH OSSTEST v2 13/19] standalone: Prefer ./standalone.config to $HOME/.xen-osstest/config"):

Your sentence seems mangled, but AFAICT you are saying `we already have standalone.config'. But AFAICT standalone.config is something entirely different. standalone.config is a shell script sourced by standalone-reset.

Ian.

It is rather.

Yes, you are right, they are entirely separate, so using standalone.config is extra wrong here then!

Ian.

Is useful when debugging.

---
 Osstest/Debian.pm | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

index c6687db..d2a7be9 100644
@@ -412,8 +412,7 @@ sub setupboot_grub2 ($$$$) {
             if (m/^\s*\}\s*$/) {
                 die unless $entry || $submenu;
                 if (!defined $entry && defined $submenu) {
-                    logm("Met end of a submenu starting from ".
-                        "$submenu->{StartLine}. ".
+                    logm("Met end of a submenu $submenu->{StartLine}..$.. ".
                         "Our want kern is $want_kernver");
                     $submenu=undef;
                     pop @offsets;

@@ -426,14 +425,14 @@ sub setupboot_grub2 ($$$$) {
 			 ? qw(Title Hv KernDom0 KernVer)
 			 : qw(Title Hv KernOnly KernVer));
 		if (@missing) {
-		    logm("(skipping entry at $entry->{StartLine};".
+		    logm("(skipping entry at $entry->{StartLine}..$.;".
 			 " no @missing)");
 		} elsif (defined $want_kernver &&
 			 $entry->{KernVer} ne $want_kernver) {
-		    logm("(skipping entry at $entry->{StartLine};".
+		    logm("(skipping entry at $entry->{StartLine}..$.;".
 			 " kernel $entry->{KernVer}, not $want_kernver)");
 		} elsif ($want_xsm && !defined $entry->{Xenpolicy}) {
-		    logm("(skipping entry at $entry->{StartLine};".
+		    logm("(skipping entry at $entry->{StartLine}..$.;".
 			 " XSM policy file not present)");
 		} else {
 		    # yes!

2.1.4

Ian Campbell writes ("[PATCH OSSTEST v2 14/19] Debian: grub2: Log full line range of menuentry and submenu entries"):

This overrides GRUB_CMDLINE_LINUX(_DEFAULT) which we were previously editing but only for the Xen entries, meaning that we don't switch to console=hvc0 for the native cases (i.e. don't break them).

We do however want to edit GRUB_CMDLINE_LINUX(_DEFAULT) to remove "quiet" if present, since it is useful to people, especially those using standalone mode as a provisioning tool, wanting to e.g. compare a boot under Xen with the native case.

This has been supported since Wheezy but in any case we supply our own 20_linux_xen grub generator based on Wheezy which supports this.

---
 Osstest/Debian.pm | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

index d2a7be9..31aa1e6 100644
@@ -507,7 +507,7 @@ sub setupboot_grub2 ($$$$) {
                     $v =~ s/^\s*([\'\"])(.*)\1\s*$/$2/;
                     $k{$k}= $v;
                 }
-                next if m/^GRUB_CMDLINE_(?:XEN|LINUX).*\=|^GRUB_DEFAULT.*\=/;
+                next if m/^GRUB_CMDLINE_(?:XEN|LINUX(?:_XEN_REPLACE)?(?:_DEFAULT)?).*\=|^GRUB_DEFAULT.*\=/;
                 print ::EO;
             }
             print ::EO <<END or die $!;

@@ -521,11 +521,20 @@ GRUB_CMDLINE_XEN="$xenhopt"
 END
             foreach my $k (qw(GRUB_CMDLINE_LINUX GRUB_CMDLINE_LINUX_DEFAULT)) {
                 my $v= $k{$k};
+
+                # Tailor native case
                 $v =~ s/\bquiet\b//;
+                print ::EO "$k=\"$v\"\n" or die $!;
+
+                # Xen overrides
                 $v =~ s/\b(?:console|xencons)=[0-9A-Za-z,]+//;
                 $v .= " $xenkopt" if $k eq 'GRUB_CMDLINE_LINUX';
-                print ::EO "$k=\"$v\"\n" or die $!;
-            }
+                # Ensure variable isn't zero length, or 20_linux_xen will ignore it.
+                $v .= " ";
+                my $rk = $k;
+                $rk =~ s/LINUX/LINUX_XEN_REPLACE/;
+                print ::EO "$rk=\"$v\"\n" or die $!;
+	    }
         });
     };
 

2.1.4

Ian Campbell writes ("[PATCH OSSTEST v2 15/19] Debian: grub2: Use GRUB_CMDLINE_LINUX_XEN_REPLACE(_DEFAULT)"):

Previously these binaries would have been included in the regular ("tools") dist file, whereas they really belong in the xen one.

Install into /boot/efi/EFI/osstest ready for use when chainloading.

Note that /boot/efi is (or should be) a VFAT filesystem. So a bit of care is needed WRT symlinks etc. This is also what prevents us from just including /boot/efi/EFI/osstest/xen.efi in the dist tarball since untarring over a VFAT needs a little care WRT the case of directories etc.

---
 ts-xen-build   | 7 +++++++
 ts-xen-install | 7 +++++++
 2 files changed, 14 insertions(+)

index 353a82c..cebfaf3 100755
@@ -172,6 +172,13 @@ sub divide () {
                 mv \$mvfiles xeninstall/boot/.
             fi
         fi
+	if test -d install/usr/lib64/efi/; then
+            if test -f install/usr/lib64/efi/xen.efi; then
+                mkdir -p xeninstall/usr/lib64/efi
+                mvfiles=`find install/usr/lib64/efi -name 'xen[a-z]*' -prune -o -name 'xen*' -print`
+                mv \$mvfiles xeninstall/usr/lib64/efi/.
+            fi
+	fi
 END
 }
 

index d55401c..83fa143 100755
@@ -74,6 +74,13 @@ sub extract () {
         target_extract_jobdistpath($ho, $part, "path_${part}dist",
 				   $r{"${part}buildjob"}, \%distpath);
     }
+    if (target_file_exists($ho, "/usr/lib64/efi/xen.efi")) {
+	target_cmd_root($ho,<<END);
+	    mkdir -p /boot/efi/EFI/osstest
+	    # /boot/efi is VFAT, so dereference the symlink
+	    cp -vL /usr/lib64/efi/xen.efi /boot/efi/EFI/osstest
+END
+    }
     target_cmd_root($ho, '/sbin/ldconfig');
 }
 

2.1.4

Ian Campbell writes ("[PATCH OSSTEST v2 16/19] Collect xen.efi into xendist and install in appropriate place"):

These can't (in general?) actually be booted.

---
 Osstest/Debian.pm | 3 +++
 1 file changed, 3 insertions(+)

index 31aa1e6..8f4c4ea 100644
@@ -427,6 +427,9 @@ sub setupboot_grub2 ($$$$) {
 		if (@missing) {
 		    logm("(skipping entry at $entry->{StartLine}..$.;".
 			 " no @missing)");
+		} elsif ($entry->{Hv} =~ m/xen-syms/) {
+		    logm("(skipping entry at $entry->{StartLine}..$.;".
+			 " is xen-syms)");
 		} elsif (defined $want_kernver &&
 			 $entry->{KernVer} ne $want_kernver) {
 		    logm("(skipping entry at $entry->{StartLine}..$.;".

2.1.4

Ian Campbell writes ("[PATCH OSSTEST v2 17/19] Debian: Ignore xen-syms entries in grub.cfg."):

They can't be booted at all. That they are included in the menu is a bug in something.

Yes. I can't decide if it is grub'2 20_linux_xen for not ignoring them or xen.git's Makefiles for putting them in /boot rather than /usr/lib/something/debug.

Thanks.

I'm going to want it in a subsequent patch

---
 Osstest/Debian.pm | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

index 8f4c4ea..d342103 100644
@@ -460,15 +460,17 @@ sub setupboot_grub2 ($$$$) {
                 die unless $entry;
                 $entry->{Hv}= $1;
             }
-            if (m/^\s*multiboot\s*(?:\/boot)?\/(vmlinu[xz]-(\S+))/) {
+            if (m/^\s*multiboot\s*(?:\/boot)?\/(vmlinu[xz]-(\S+))\s+(.*)/) {
                 die unless $entry;
                 $entry->{KernOnly}= $1;
                 $entry->{KernVer}= $2;
+                $entry->{KernOpts}= $3;
             }
-            if (m/^\s*module\s*(?:\/boot)?\/(vmlinu[xz]-(\S+))/) {
+            if (m/^\s*module\s*(?:\/boot)?\/(vmlinu[xz]-(\S+))\s+(.*)/) {
                 die unless $entry;
                 $entry->{KernDom0}= $1;
                 $entry->{KernVer}= $2;
+                $entry->{KernOpts}= $3;
             }
             if (m/^\s*module\s*(?:\/boot)?\/(initrd\S+)/) {
                 $entry->{Initrd}= $1;

2.1.4

Ian Campbell writes ("[PATCH OSSTEST v2 18/19] Debian: Collect kernel command line from grub.cfg"):

Xen cannot (currently) be booted directly via the usual multiboot path on EFI systems of any arch. Instead it is necessary to either launch xen.efi direct from the UEFI shell or to chainload it from grub. In both cases the Xen command line as well as what would normally be the multiboot modules (kernel+command line, XSM policy, initrd) must be configured in a Xen configuration file.

By patching overlay/etc/grub.d/20_linux_xen that if a suitable xen.efi is found in the EFI System Partition (as arrange by a previous patch) a suitable entry is created in grub.cfg as well.

When parsing the grub.cfg look for such an entry into addition to the regular/multiboot one and when necessary write the configuration file based on the regular entry and return the chainload one such that it gets booted.

This is currently enabled only for Jessie ARM64 systems.

Note that the 20_linux_xen change here is a bit specific to us and not really generic enough to go upstream IMHO, hence I haven't.

---
 Osstest/Debian.pm               | 75 +++++++++++++++++++++++++++++++++++++++--
 overlay/etc/grub.d/20_linux_xen | 19 +++++++++++
 2 files changed, 92 insertions(+), 2 deletions(-)

index d342103..c89a362 100644
@@ -400,12 +400,18 @@ sub setupboot_grub2 ($$$$) {
 
     my $rmenu= '/boot/grub/grub.cfg';
     my $kernkey= (defined $xenhopt ? 'KernDom0' : 'KernOnly');
- 
+
+    # Grub2 on Jessie/arm* doesn't do multiboot, so we must chainload.
+    my $need_uefi_chainload =
+        get_host_property($ho, "firmware") eq "uefi" &&
+        $ho->{Suite} =~ m/jessie/ && $r{arch} =~ m/^arm/;
+
     my $parsemenu= sub {
         my $f= bl_getmenu_open($ho, $rmenu, "$stash/$ho->{Name}--grub.cfg.1");
     
         my @offsets = (0);
         my $entry;
+        my $chainentry;
         my $submenu;
         while (<$f>) {
             next if m/^\s*\#/ || !m/\S/;

@@ -424,7 +430,13 @@ sub setupboot_grub2 ($$$$) {
 		        (defined $xenhopt
 			 ? qw(Title Hv KernDom0 KernVer)
 			 : qw(Title Hv KernOnly KernVer));
-		if (@missing) {
+		if ($need_uefi_chainload && $entry->{Chainload}) {
+		    # Needs to be before check of @missing, since a
+		    # chained entry doesn't have anything useful in it
+		    logm("Found chainload entry at $entry->{StartLine}..$.");
+		    die "already got one" if $chainentry;
+		    $chainentry = $entry;
+		} elsif (@missing) {
 		    logm("(skipping entry at $entry->{StartLine}..$.;".
 			 " no @missing)");
 		} elsif ($entry->{Hv} =~ m/xen-syms/) {

@@ -456,9 +468,15 @@ sub setupboot_grub2 ($$$$) {
                 $submenu={ StartLine =>$., MenuEntryPath => join ">", @offsets };
                 push @offsets,(0);
             }
+            if (m/^\s*chainloader\s*\/EFI\/osstest\/xen.efi/) {
+                die unless $entry;
+                $entry->{Hv}= $1;
+                $entry->{Chainload} = 1;
+            }
             if (m/^\s*multiboot\s*(?:\/boot)?\/(xen\S+)/) {
                 die unless $entry;
                 $entry->{Hv}= $1;
+                $entry->{Chainload} = 0;
             }
             if (m/^\s*multiboot\s*(?:\/boot)?\/(vmlinu[xz]-(\S+))\s+(.*)/) {
                 die unless $entry;

@@ -490,13 +508,66 @@ sub setupboot_grub2 ($$$$) {
 	    die unless $entry->{Hv};
 	}
 
+	if ($need_uefi_chainload) {
+	    die 'chainload entry not found' unless $chainentry;
+
+            # Propagate relevant fields of the main entry over to the
+            # chain entry for use of subsequent code.
+            foreach (qw(KernVer KernDom0 KernOnly KernOpts Initrd Xenpolicy)) {
+		next unless $entry->{$_};
+		die if $chainentry->{$_};
+		$chainentry->{$_} = $entry->{$_};
+            }
+
+            $entry = $chainentry;
+	}
+
         return $entry;
     };
 
 
     $bl->{UpdateConfig}= sub {
 	my ( $ho ) = @_;
+
+        target_editfile_root($ho, '/etc/default/grub', sub {
+            while (<::EI>) {
+                next if m/^export GRUB_ENABLE_XEN_UEFI_CHAINLOAD\=/;
+                print ::EO;
+            }
+	    print ::EO "export GRUB_ENABLE_XEN_UEFI_CHAINLOAD=\"osstest\"\n"
+		if $need_uefi_chainload;
+	});
+
 	target_cmd_root($ho, "update-grub");
+
+	if ($need_uefi_chainload) {
+	    my $entry= $parsemenu->();
+	    my $xencfg = <<END;
+[global]
+default=osstest
+
+[osstest]
+options=$xenhopt
+kernel=vmlinuz $entry->{KernOpts}
+END
+            $xencfg .= "ramdisk=initrd.gz\n" if $entry->{Initrd};
+            $xencfg .= "xsm=xenpolicy\n" if $entry->{Xenpolicy};
+
+            target_putfilecontents_root_stash($ho,30,$xencfg,"/boot/efi/EFI/osstest/xen.cfg");
+
+            # /boot/efi should be a mounted EFI system partition, and
+            # /boot/efi/EFI/osstest/xen.efi should already exist. Hence no mkdir
+            # here.
+            target_cmd_root($ho,
+	        <<END.($entry->{Initrd}?<<END:"").($entry->{Xenpolicy}?<<END:""));
+                set -ex
+                cp -vL /boot/$entry->{KernDom0} /boot/efi/EFI/osstest/vmlinuz #/
+END
+                cp -vL /boot/$entry->{Initrd} /boot/efi/EFI/osstest/initrd.gz #/
+END
+                cp -vL /boot/$entry->{Xenpolicy} /boot/efi/EFI/osstest/xenpolicy #/
+END
+	}
     };
 
     $bl->{GetBootKern}= sub { return $parsemenu->()->{$kernkey}; };

index aaead1b..7bee1a3 100755
@@ -142,6 +142,25 @@ EOF
 }
 EOF
 }
+chain_xen_entry () {
+    vendor="$1"
+    if [ ! -f /boot/efi/EFI/$vendor/xen.efi ] ; then
+	return
+    fi
+    title=$(gettext_quoted "Chainload $vendor Xen")
+    xmessage="$(gettext_printf "Chainloading $vendor Xen ...")"
+    printf "menuentry '${title}' ${CLASS} {\n"
+    prepare_grub_to_access_device `${grub_probe} --target=device /boot/efi/EFI/$vendor/xen.efi` | grub_add_tab
+    cat <<EOF
+	echo	'$xmessage'
+	chainloader	/EFI/$vendor/xen.efi
+}
+EOF
+}
+
+for i in ${GRUB_ENABLE_XEN_UEFI_CHAINLOAD} ; do
+    chain_xen_entry $i
+done
 
 linux_list=`for i in /boot/vmlinu[xz]-* /vmlinu[xz]-* /boot/kernel-*; do
     if grub_file_is_not_garbage "$i"; then

2.1.4

Ian Campbell writes ("[PATCH OSSTEST v2 19/19] Debian: Arrange to be able to chainload a xen.efi from grub2"):

Urgh. How horrible. I don't object to your patch, except that there's some linewrap or reformatting needed.