Use the DTB provided by the kernel we are trying to boot
v2 cover Use the DTB provided by the kernel we are trying to boot
On ARM we have been booting using the DTBS provided by the host Debian kernel package (i.e. the ones from 3.16) even when we are booting a Xen+kernel which we have built ourselves.
Recent Linux kernels (at least linux-next and linux-linus, perhaps mingo -tip too) have changes which are incompatible with these older kernels which stop the kernel from booting on Cubietruck (even natively).
Despite what I thought there is apparently[0] no ABI here. In any case testing the newer DTBs is desirable so here is a series to switch to doing so.
This is all acked but there were some changes suggested along with the acks, which I've made and am posting here. It was all pretty minor so I've kept the acks, I hope that is ok.
Ian.
[0] http://lists.infradead.org/pipermail/linux-arm-kernel/2015-October/3739
22.html
PATCH v2 1/8 Osstest: Add %arch_debian2linux (and inverse)
Modelled after %arch_debian2xen.
Will be used in ts-kernel-build.
---
v2: Wrapped long line
--- Osstest.pm | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-)
Osstest.pm +10 -1
@@ -40,7 +40,9 @@ BEGIN { db_begin_work db_prepare ensuredir get_filecontents_core_quiet system_checked nonempty visible_undef show_abs_time - %arch_debian2xen %arch_xen2debian $cfgvar_re + %arch_debian2xen %arch_xen2debian + %arch_debian2linux %arch_linux2debian + $cfgvar_re ); %EXPORT_TAGS = ( );
@@ -62,6 +64,13 @@ our %arch_debian2xen = qw(i386 x86_32 our %arch_xen2debian; $arch_xen2debian{$arch_debian2xen{$_}} = $_ foreach keys %arch_debian2xen; +our %arch_debian2linux = qw(i386 x86 + amd64 x86 + armhf arm); +our %arch_linux2debian; +$arch_linux2debian{$arch_debian2linux{$_}} = $_ + foreach keys %arch_debian2linux; + our $cfgvar_re = '[A-Z][0-9a-zA-Z-_]*'; #---------- static default config settings ----------
Signature · 1 line
2.5.3
PATCH v2 2/8 ts-kernel-build: Introduce global $parms
In reply to Ian CampbellWhich contains the relevant details from %archparms, making the use sites simpler.
--- ts-kernel-build | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
ts-kernel-build +3 -2
@@ -46,6 +46,7 @@ my $archparms = { 'i386' => { DefConfig => 'defconfig', KernelImage => 'arch/x86/boot/bzImage' }, 'amd64' => { DefConfig => 'defconfig', KernelImage => 'arch/x86/boot/bzImage' } }; +my $parms = $archparms->{ $r{arch} }; sub enable_xen_config ();
@@ -333,7 +334,7 @@ END ed <$edscript enable-xen-config chmod +x enable-xen-config cd linux - make $archparms->{ $r{arch} }->{DefConfig} + make $parms->{DefConfig} # ARM defconfig omits modules for some reason... sed -e "s/^# CONFIG_MODULES is not set\$/CONFIG_MODULES=y/g" -i .config ../enable-xen-config .config
@@ -376,7 +377,7 @@ END sub kinstall () { my $kernfile= $r{kimagefile}; - $kernfile= $archparms->{$r{arch}}->{KernelImage} if !defined $kernfile; + $kernfile= $parms->{KernelImage} if !defined $kernfile; target_cmd_build($ho, 300, $builddir, <<END); mkdir -p dist/boot dist/lib/modules
Signature · 1 line
2.5.3
PATCH v2 3/8 ts-kernel-build: Refactor $archparams::DefConfig selection
In reply to Ian CampbellThis is always either "foo_defconfig" or "defconfig". Record only "foo" or undef and construct the name.
This makes the $archparams less verbose.
--- ts-kernel-build | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
ts-kernel-build +6 -4
@@ -42,9 +42,9 @@ selectbuildhost(\@ARGV); builddirsprops(); my $archparms = { - 'armhf' => { DefConfig => 'multi_v7_defconfig', KernelImage => 'arch/arm/boot/zImage' }, - 'i386' => { DefConfig => 'defconfig', KernelImage => 'arch/x86/boot/bzImage' }, - 'amd64' => { DefConfig => 'defconfig', KernelImage => 'arch/x86/boot/bzImage' } + 'armhf' => { DefConfig => 'multi_v7', KernelImage => 'arch/arm/boot/zImage' }, + 'i386' => { KernelImage => 'arch/x86/boot/bzImage' }, + 'amd64' => { KernelImage => 'arch/x86/boot/bzImage' } }; my $parms = $archparms->{ $r{arch} };
@@ -330,11 +330,13 @@ END target_putfilecontents_stash($ho,100,enable_xen_config(), "$builddir/enable-xen-config"); + my $defconfig = $parms->{DefConfig} ? + $parms->{DefConfig}."_defconfig" : "defconfig"; target_cmd_build($ho, 1000, $builddir, <<END); ed <$edscript enable-xen-config chmod +x enable-xen-config cd linux - make $parms->{DefConfig} + make $defconfig # ARM defconfig omits modules for some reason... sed -e "s/^# CONFIG_MODULES is not set\$/CONFIG_MODULES=y/g" -i .config ../enable-xen-config .config
Signature · 1 line
2.5.3
PATCH v2 4/8 ts-kernel-build: Refactor $archparams KernelImage handling
In reply to Ian CampbellThis is always arch/$karch/boot/$img. Store $img in %archparms and use %arch_debian2linux to construct the full path as needed.
This makes the $archparams less verbose.
--- ts-kernel-build | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
ts-kernel-build +6 -4
@@ -42,9 +42,9 @@ selectbuildhost(\@ARGV); builddirsprops(); my $archparms = { - 'armhf' => { DefConfig => 'multi_v7', KernelImage => 'arch/arm/boot/zImage' }, - 'i386' => { KernelImage => 'arch/x86/boot/bzImage' }, - 'amd64' => { KernelImage => 'arch/x86/boot/bzImage' } + 'armhf' => { DefConfig => 'multi_v7', Image => 'zImage' }, + 'i386' => { Image => 'bzImage' }, + 'amd64' => { Image => 'bzImage' } }; my $parms = $archparms->{ $r{arch} };
@@ -379,7 +379,9 @@ END sub kinstall () { my $kernfile= $r{kimagefile}; - $kernfile= $parms->{KernelImage} if !defined $kernfile; + my $karch = $arch_debian2linux{$r{arch}}; + $kernfile = "arch/$karch/boot/$parms->{Image}" + if !defined $kernfile; target_cmd_build($ho, 300, $builddir, <<END); mkdir -p dist/boot dist/lib/modules
Signature · 1 line
2.5.3
PATCH v2 5/8 ts-kernel-build: Add arm64 support
In reply to Ian CampbellAnd to Osstest::%arch_debian2*.
Untested (but hopefully pretty obvious).
---
Note that this replaces a hunk in "Add arm64 build and test jobs"
--- Osstest.pm | 6 ++++-- ts-kernel-build | 1 + 2 files changed, 5 insertions(+), 2 deletions(-)
Osstest.pm +4 -2
@@ -60,13 +60,15 @@ scalar *main::DEBUG; our %arch_debian2xen = qw(i386 x86_32 amd64 x86_64 - armhf armhf); + armhf armhf + arm64 arm64); our %arch_xen2debian; $arch_xen2debian{$arch_debian2xen{$_}} = $_ foreach keys %arch_debian2xen; our %arch_debian2linux = qw(i386 x86 amd64 x86 - armhf arm); + armhf arm + arm64 arm64); our %arch_linux2debian; $arch_linux2debian{$arch_debian2linux{$_}} = $_ foreach keys %arch_debian2linux;
ts-kernel-build +1 -0
@@ -42,6 +42,7 @@ selectbuildhost(\@ARGV); builddirsprops(); my $archparms = { + 'arm64' => { Image => 'Image' }, 'armhf' => { DefConfig => 'multi_v7', Image => 'zImage' }, 'i386' => { Image => 'bzImage' }, 'amd64' => { Image => 'bzImage' }
Signature · 1 line
2.5.3
PATCH v2 6/8 ts-kernel-build: Include dtbs in dist file
In reply to Ian CampbellThese are installed to $(INSTALL_PATH)/dtbs/$(KERNEL_RELEASE) where $(INSTALL_PATH) defaults to /boot but we override it to our staging /boot.
Note that ts-host-install will install the OS dtbs directly into /boot/dtbs without the subdirectory, so this won't clash and could be considered a fallback hence I don't propose to move those ones.
The install_dtbs target has been available since v3.14, wherease we only test v3.16 onwards on ARM, hence no arrangements are needed to conditionalise this installation over and above the per-arch arrangements made here.
Having now set $(INSTALL_PATH) I think the "install" target could now take over the installation of System.map, vmlinux and .config into /boot but I've not checked this with all historical kernel versions and don't intend to make this change now.
Remove any previous dist dir on install, otherwise the kernel tends to create dist/boot/dtbs.old with the previous contents on repeated use. Seems like good hygiene anyway.
---
v2: Move modules_install to next line so it isn't lost after the INSTALL_MOD_PATH
--- ts-kernel-build | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-)
ts-kernel-build +12 -5
@@ -42,10 +42,10 @@ selectbuildhost(\@ARGV); builddirsprops(); my $archparms = { - 'arm64' => { Image => 'Image' }, - 'armhf' => { DefConfig => 'multi_v7', Image => 'zImage' }, - 'i386' => { Image => 'bzImage' }, - 'amd64' => { Image => 'bzImage' } + 'arm64' => { Image => 'Image', Dtbs => 1 }, + 'armhf' => { DefConfig => 'multi_v7', Image => 'zImage', Dtbs => 1 }, + 'i386' => { Image => 'bzImage' }, + 'amd64' => { Image => 'bzImage' } }; my $parms = $archparms->{ $r{arch} };
@@ -384,10 +384,17 @@ sub kinstall () { $kernfile = "arch/$karch/boot/$parms->{Image}" if !defined $kernfile; + my $dtbs_install = ''; + $dtbs_install = 'dtbs_install' if $parms->{Dtbs}; + target_cmd_build($ho, 300, $builddir, <<END); + rm -rf dist mkdir -p dist/boot dist/lib/modules cd linux - make $makeflags INSTALL_MOD_PATH=$builddir/dist modules_install + make $makeflags \\ + INSTALL_PATH=$builddir/dist/boot \\ + INSTALL_MOD_PATH=$builddir/dist \\ + modules_install $dtbs_install cp System.map $builddir/dist/boot/ cp $kernfile $builddir/dist/boot/vmlinuz cp .config $builddir/dist/boot/config
Signature · 1 line
2.5.3
PATCH v2 7/8 Debian: Enable interpolation in uboot_scr_load_dtb here doc
In reply to Ian CampbellBy switching <<'END' to <<END. A future patch is going to want to put a variable here which requires interpretation by the Perl.
Unfortunately this means lots of extra backslashes to escape things such that they pass through Perl and Shell and end up as ${foo} in the resulting u-boot script.
---
v2: Call a backslash a backslash.
--- Osstest/Debian.pm | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
Osstest/Debian.pm +5 -5
@@ -143,11 +143,11 @@ sub uboot_common_kernel_bootargs ($) } sub uboot_scr_load_dtb () { - return <<'END'; -if test -z "\${fdt_addr}" && test -n "\${fdtfile}" ; then - echo Loading dtbs/\${fdtfile} - ext2load scsi 0 \${fdt_addr_r} dtbs/\${fdtfile} - setenv fdt_addr \${fdt_addr_r} + return <<END; +if test -z "\\\${fdt_addr}" && test -n "\\\${fdtfile}" ; then + echo Loading dtbs/\\\${fdtfile} + ext2load scsi 0 \\\${fdt_addr_r} dtbs/\\\${fdtfile} + setenv fdt_addr \\\${fdt_addr_r} fi END }
Signature · 1 line
2.5.3
PATCH v2 8/8 Debian: Use dtbs from kernel dist when booting that kernel
In reply to Ian CampbellThe kernel dist built by ts-kernel-build puts the corresponding dtbs into /boot/dtbs/$kvers.
The host installer's dtbs remain in /boot/dtbs and are used when booting the native kernel.
It's possible that this change will expose bugs which exist in the DTBs in previous kernel branches (3.18 and 4.1). I've not investigated, I think we should accept this possibility and deal with it via backport requests (and maybe some force pushes if appropriate) as necessary.
---
v2: Add some commentary about which dtbs is where to uboot_scr_load_dtb.
--- Osstest/Debian.pm | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-)
Osstest/Debian.pm +15 -4
@@ -142,11 +142,22 @@ sub uboot_common_kernel_bootargs ($) return @bootargs; } -sub uboot_scr_load_dtb () { +# ts-host-install will install the OS-supplied dtbs directly into +# /boot/dtbs without a subdirectory, while the kernel dist tarball +# produced by ts-kernel-build will install the DTBS from that kernel +# into /boot/dtbs/$version. +# +# Therefore the optional $kvers argument allows the selection of +# either the OS-supplied DTBS (when undef) or those of a specific +# kernel built by osstest. +sub uboot_scr_load_dtb (;$) { + my ($kvers) = @_; + $kvers ||= ''; + $kvers .= '/' if $kvers; return <<END; if test -z "\\\${fdt_addr}" && test -n "\\\${fdtfile}" ; then - echo Loading dtbs/\\\${fdtfile} - ext2load scsi 0 \\\${fdt_addr_r} dtbs/\\\${fdtfile} + echo Loading dtbs/$kvers\\\${fdtfile} + ext2load scsi 0 \\\${fdt_addr_r} dtbs/$kvers\\\${fdtfile} setenv fdt_addr \\\${fdt_addr_r} fi END
@@ -229,7 +240,7 @@ END my $early_commands = get_host_property($ho, 'UBootScriptEarlyCommands', ''); my $xen_addr_r = get_host_property($ho, 'UBootSetXenAddrR', undef); - my $load_dtb = uboot_scr_load_dtb(); + my $load_dtb = uboot_scr_load_dtb($want_kernver); my $set_xen_addr_r = $xen_addr_r ? "setenv xen_addr_r $xen_addr_r" : "";
Signature · 1 line
2.5.3
Ian Campbell writes ("[PATCH OSSTEST v2 0/8] Use the DTB provided by the kernel we are trying to boot"):
Quoted 3 lines
On ARM we have been booting using the DTBS provided by the host Debian kernel package (i.e. the ones from 3.16) even when we are booting a Xen+kernel which we have built ourselves.
...
Quoted 3 lines
This is all acked but there were some changes suggested along with the acks, which I've made and am posting here. It was all pretty minor so I've kept the acks, I hope that is ok.
Indeed, it's what I expected, thanks.
Ian.