Use the DTB provided by the kernel we are trying to boot
cover Use the DTB provided by the kernel we are trying to boot
On ARM we have been booting using the DTBS provided by the host Debian kernel package (i.e. the ones from 3.16) even when we are booting a Xen+kernel which we have built ourselves.
Recent Linux kernels (at least linux-next and linux-linus, perhaps mingo -tip too) have changes which are incompatible with these older kernels which stop the kernel from booting on Cubietruck (even natively).
Despite what I thought there is apparently[0] no ABI here. In any case testing the newer DTBs is desirable so here is a series to switch to doing so.
Ian.
[0] http://lists.infradead.org/pipermail/linux-arm-kernel/2015-October/3739
22.html
PATCH 1/8 Osstest: Add %arch_debian2linux (and inverse)
Modelled after %arch_debian2xen.
Will be used in ts-kernel-build.
--- Osstest.pm | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
Osstest.pm +9 -1
@@ -40,7 +40,9 @@ BEGIN { db_begin_work db_prepare ensuredir get_filecontents_core_quiet system_checked nonempty visible_undef show_abs_time - %arch_debian2xen %arch_xen2debian $cfgvar_re + %arch_debian2xen %arch_xen2debian + %arch_debian2linux %arch_linux2debian + $cfgvar_re ); %EXPORT_TAGS = ( );
@@ -62,6 +64,12 @@ our %arch_debian2xen = qw(i386 x86_32 our %arch_xen2debian; $arch_xen2debian{$arch_debian2xen{$_}} = $_ foreach keys %arch_debian2xen; +our %arch_debian2linux = qw(i386 x86 + amd64 x86 + armhf arm); +our %arch_linux2debian; +$arch_linux2debian{$arch_debian2linux{$_}} = $_ foreach keys %arch_debian2linux; + our $cfgvar_re = '[A-Z][0-9a-zA-Z-_]*'; #---------- static default config settings ----------
Signature · 1 line
2.5.3
Re: [PATCH OSSTEST 1/8] Osstest: Add %arch_debian2linux (and inverse)
Ian Campbell writes ("[PATCH OSSTEST 1/8] Osstest: Add %arch_debian2linux (and inverse)"):
Quoted 3 lines
Modelled after %arch_debian2xen.
Will be used in ts-kernel-build.
Quoted 2 lines
+our %arch_linux2debian; +$arch_linux2debian{$arch_debian2linux{$_}} = $_ foreach keys %arch_debia\
n2linux;
It would be nice to wrap that line, though.
Ian.
PATCH 2/8 ts-kernel-build: Introduce global $parms
In reply to Ian CampbellWhich contains the relevant details from %archparms, making the use sites simpler.
--- ts-kernel-build | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
ts-kernel-build +3 -2
@@ -46,6 +46,7 @@ my $archparms = { 'i386' => { DefConfig => 'defconfig', KernelImage => 'arch/x86/boot/bzImage' }, 'amd64' => { DefConfig => 'defconfig', KernelImage => 'arch/x86/boot/bzImage' } }; +my $parms = $archparms->{ $r{arch} }; sub enable_xen_config ();
@@ -333,7 +334,7 @@ END ed <$edscript enable-xen-config chmod +x enable-xen-config cd linux - make $archparms->{ $r{arch} }->{DefConfig} + make $parms->{DefConfig} # ARM defconfig omits modules for some reason... sed -e "s/^# CONFIG_MODULES is not set\$/CONFIG_MODULES=y/g" -i .config ../enable-xen-config .config
@@ -376,7 +377,7 @@ END sub kinstall () { my $kernfile= $r{kimagefile}; - $kernfile= $archparms->{$r{arch}}->{KernelImage} if !defined $kernfile; + $kernfile= $parms->{KernelImage} if !defined $kernfile; target_cmd_build($ho, 300, $builddir, <<END); mkdir -p dist/boot dist/lib/modules
Signature · 1 line
2.5.3
Re: [PATCH OSSTEST 2/8] ts-kernel-build: Introduce global $parms
Ian Campbell writes ("[PATCH OSSTEST 2/8] ts-kernel-build: Introduce global $parms"):
Quoted 2 lines
Which contains the relevant details from %archparms, making the use sites simpler.
PATCH 3/8 ts-kernel-build: Refactor $archparams::DefConfig selection
In reply to Ian CampbellThis is always either "foo_defconfig" or "defconfig". Record only "foo" or undef and construct the name.
This makes the $archparams less verbose.
--- ts-kernel-build | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
ts-kernel-build +6 -4
@@ -42,9 +42,9 @@ selectbuildhost(\@ARGV); builddirsprops(); my $archparms = { - 'armhf' => { DefConfig => 'multi_v7_defconfig', KernelImage => 'arch/arm/boot/zImage' }, - 'i386' => { DefConfig => 'defconfig', KernelImage => 'arch/x86/boot/bzImage' }, - 'amd64' => { DefConfig => 'defconfig', KernelImage => 'arch/x86/boot/bzImage' } + 'armhf' => { DefConfig => 'multi_v7', KernelImage => 'arch/arm/boot/zImage' }, + 'i386' => { KernelImage => 'arch/x86/boot/bzImage' }, + 'amd64' => { KernelImage => 'arch/x86/boot/bzImage' } }; my $parms = $archparms->{ $r{arch} };
@@ -330,11 +330,13 @@ END target_putfilecontents_stash($ho,100,enable_xen_config(), "$builddir/enable-xen-config"); + my $defconfig = $parms->{DefConfig} ? + $parms->{DefConfig}."_defconfig" : "defconfig"; target_cmd_build($ho, 1000, $builddir, <<END); ed <$edscript enable-xen-config chmod +x enable-xen-config cd linux - make $parms->{DefConfig} + make $defconfig # ARM defconfig omits modules for some reason... sed -e "s/^# CONFIG_MODULES is not set\$/CONFIG_MODULES=y/g" -i .config ../enable-xen-config .config
Signature · 1 line
2.5.3
Re: [PATCH OSSTEST 3/8] ts-kernel-build: Refactor $archparams::DefConfig selection
Ian Campbell writes ("[PATCH OSSTEST 3/8] ts-kernel-build: Refactor $archparams::DefConfig selection"):
Quoted 2 lines
This is always either "
foo_defconfig" or "defconfig". Record only "foo" or undef and construct the name.
PATCH 4/8 ts-kernel-build: Refactor $archparams KernelImage handling
In reply to Ian CampbellThis is always arch/$karch/boot/$img. Store $img in %archparms and use %arch_debian2linux to construct the full path as needed.
This makes the $archparams less verbose.
--- ts-kernel-build | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
ts-kernel-build +6 -4
@@ -42,9 +42,9 @@ selectbuildhost(\@ARGV); builddirsprops(); my $archparms = { - 'armhf' => { DefConfig => 'multi_v7', KernelImage => 'arch/arm/boot/zImage' }, - 'i386' => { KernelImage => 'arch/x86/boot/bzImage' }, - 'amd64' => { KernelImage => 'arch/x86/boot/bzImage' } + 'armhf' => { DefConfig => 'multi_v7', Image => 'zImage' }, + 'i386' => { Image => 'bzImage' }, + 'amd64' => { Image => 'bzImage' } }; my $parms = $archparms->{ $r{arch} };
@@ -379,7 +379,9 @@ END sub kinstall () { my $kernfile= $r{kimagefile}; - $kernfile= $parms->{KernelImage} if !defined $kernfile; + my $karch = $arch_debian2linux{$r{arch}}; + $kernfile = "arch/$karch/boot/$parms->{Image}" + if !defined $kernfile; target_cmd_build($ho, 300, $builddir, <<END); mkdir -p dist/boot dist/lib/modules
Signature · 1 line
2.5.3
Re: [PATCH OSSTEST 4/8] ts-kernel-build: Refactor $archparams KernelImage handling
Ian Campbell writes ("[PATCH OSSTEST 4/8] ts-kernel-build: Refactor $archparams KernelImage handling"):
Quoted 4 lines
This is always
arch/$karch/boot/$img. Store $img in %archparms and use %arch_debian2linux to construct the full path as needed.This makes the $archparams less verbose.
PATCH 5/8 ts-kernel-build: Add arm64 support
In reply to Ian CampbellAnd to Osstest::%arch_debian2*.
Untested (but hopefully pretty obvious).
---
Note that this replaces a hunk in "Add arm64 build and test jobs"
--- Osstest.pm | 6 ++++-- ts-kernel-build | 1 + 2 files changed, 5 insertions(+), 2 deletions(-)
Osstest.pm +4 -2
@@ -60,13 +60,15 @@ scalar *main::DEBUG; our %arch_debian2xen = qw(i386 x86_32 amd64 x86_64 - armhf armhf); + armhf armhf + arm64 arm64); our %arch_xen2debian; $arch_xen2debian{$arch_debian2xen{$_}} = $_ foreach keys %arch_debian2xen; our %arch_debian2linux = qw(i386 x86 amd64 x86 - armhf arm); + armhf arm + arm64 arm64); our %arch_linux2debian; $arch_linux2debian{$arch_debian2linux{$_}} = $_ foreach keys %arch_debian2linux;
ts-kernel-build +1 -0
@@ -42,6 +42,7 @@ selectbuildhost(\@ARGV); builddirsprops(); my $archparms = { + 'arm64' => { Image => 'Image' }, 'armhf' => { DefConfig => 'multi_v7', Image => 'zImage' }, 'i386' => { Image => 'bzImage' }, 'amd64' => { Image => 'bzImage' }
Signature · 1 line
2.5.3
Re: [PATCH OSSTEST 5/8] ts-kernel-build: Add arm64 support
Ian Campbell writes ("[PATCH OSSTEST 5/8] ts-kernel-build: Add arm64 support"):
Quoted 3 lines
And to Osstest::%arch_debian2*.
Untested (but hopefully pretty obvious).
Quoted 2 lines
my $archparms = {
+ 'arm64' => { Image => 'Image' },
IWBN to double-check that the generated filename is correct before we try to push this.
Ian.
Re: [PATCH OSSTEST 5/8] ts-kernel-build: Add arm64 support
Quoted 6 lines · Ian Jackson
Ian Campbell writes ("[PATCH OSSTEST 5/8] ts-kernel-build: Add arm64 support"):
Quoted 3 lines
And to Osstest::%arch_debian2*.
Untested (but hopefully pretty obvious).
Quoted 7 lines
Acked-by Ian Jackson <[email protected]>Quoted 2 lines
my $archparms = {
+ 'arm64' => { Image => 'Image' },IWBN to double-check that the generated filename is correct before we try to push this.
For arm64 specifically or for all arches?
FWIW I know that Image is correct for arm64.
Ian.
Re: [PATCH OSSTEST 5/8] ts-kernel-build: Add arm64 support
Ian Campbell writes ("Re: [PATCH OSSTEST 5/8] ts-kernel-build: Add arm64 support"):
Quoted 5 lines · Ian Jackson
Quoted 2 lines
IWBN to double-check that the generated filename is correct before we try to push this.
For arm64 specifically or for all arches?
Just for arm64. The other nfc changes in earlier patches looked `obviously correct' to me :-).
Quoted 1 line
FWIW I know that Image is correct for arm64.
Good, thanks.
Ian.
PATCH 6/8 ts-kernel-build: Include dtbs in dist file
In reply to Ian CampbellThese are installed to $(INSTALL_PATH)/dtbs/$(KERNEL_RELEASE) where $(INSTALL_PATH) defaults to /boot but we override it to our staging /boot.
Note that ts-host-install will install the OS dtbs directly into /boot/dtbs without the subdirectory, so this won't clash and could be considered a fallback hence I don't propose to move those ones.
The install_dtbs target has been available since v3.14, wherease we only test v3.16 onwards on ARM, hence no arrangements are needed to conditionalise this installation over and above the per-arch arrangements made here.
Having now set $(INSTALL_PATH) I think the "install" target could now take over the installation of System.map, vmlinux and .config into /boot but I've not checked this with all historical kernel versions and don't intend to make this change now.
Remove any previous dist dir on install, otherwise the kernel tends to create dist/boot/dtbs.old with the previous contents on repeated use. Seems like good hygiene anyway.
--- ts-kernel-build | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-)
ts-kernel-build +12 -5
@@ -42,10 +42,10 @@ selectbuildhost(\@ARGV); builddirsprops(); my $archparms = { - 'arm64' => { Image => 'Image' }, - 'armhf' => { DefConfig => 'multi_v7', Image => 'zImage' }, - 'i386' => { Image => 'bzImage' }, - 'amd64' => { Image => 'bzImage' } + 'arm64' => { Image => 'Image', Dtbs => 1 }, + 'armhf' => { DefConfig => 'multi_v7', Image => 'zImage', Dtbs => 1 }, + 'i386' => { Image => 'bzImage' }, + 'amd64' => { Image => 'bzImage' } }; my $parms = $archparms->{ $r{arch} };
@@ -384,10 +384,17 @@ sub kinstall () { $kernfile = "arch/$karch/boot/$parms->{Image}" if !defined $kernfile; + my $dtbs_install = ''; + $dtbs_install = 'dtbs_install' if $parms->{Dtbs}; + target_cmd_build($ho, 300, $builddir, <<END); + rm -rf dist mkdir -p dist/boot dist/lib/modules cd linux - make $makeflags INSTALL_MOD_PATH=$builddir/dist modules_install + make $makeflags \\ + INSTALL_PATH=$builddir/dist/boot \\ + INSTALL_MOD_PATH=$builddir/dist modules_install \\ + $dtbs_install cp System.map $builddir/dist/boot/ cp $kernfile $builddir/dist/boot/vmlinuz cp .config $builddir/dist/boot/config
Signature · 1 line
2.5.3
Re: [PATCH OSSTEST 6/8] ts-kernel-build: Include dtbs in dist file
Ian Campbell writes ("[PATCH OSSTEST 6/8] ts-kernel-build: Include dtbs in dist file"):
Quoted 3 lines
These are installed to $(INSTALL_PATH)/dtbs/$(KERNEL_RELEASE) where $(INSTALL_PATH) defaults to /boot but we override it to our staging /boot.
Thanks for the detailed explanation.
ts-kernel-build sub kinstall () { · line 397
Quoted 5 lines
ts-kernel-build +4
-1- make $makeflags INSTALL_MOD_PATH=$builddir/dist modules_install + make $makeflags \\ + INSTALL_PATH=$builddir/dist/boot \\ + INSTALL_MOD_PATH=$builddir/dist modules_install \\ + $dtbs_install
I tripped over the formatting here. It would be nicer to put `modules_install' on the next line, to avoid hiding it in INSTALL_BLAH=.
Thanks,
Ian.
Re: [PATCH OSSTEST 6/8] ts-kernel-build: Include dtbs in dist file
Quoted 10 lines · Ian Jackson
Ian Campbell writes ("[PATCH OSSTEST 6/8] ts-kernel-build: Include dtbs in dist file"):
Quoted 3 lines
These are installed to $(INSTALL_PATH)/dtbs/$(KERNEL_RELEASE) where $(INSTALL_PATH) defaults to /boot but we override it to our staging /boot.
Thanks for the detailed explanation.
Acked-by Ian Jackson <[email protected]>
Quoted 10 lines
ts-kernel-build sub kinstall () { · line 397
Quoted 6 lines
(quoted diff) +0
-1- make $makeflags INSTALL_MOD_PATH=$builddir/dist
modules_installts-kernel-build +4
-0+ make $makeflags \\ + INSTALL_PATH=$builddir/dist/boot \\ + INSTALL_MOD_PATH=$builddir/dist modules_install \\ + $dtbs_installI tripped over the formatting here. It would be nicer to put `modules_install' on the next line, to avoid hiding it in INSTALL_BLAH=.
I did it that way because INSTALL_MOD_PATH is the variable which controls where modules_install puts things, so they are in some sense related.
Next line is fine too, or I could move it to first on the line it is on now?
Ian.
Re: [PATCH OSSTEST 6/8] ts-kernel-build: Include dtbs in dist file
Quoted 22 lines · Ian Campbell
Quoted 10 lines · Ian Jackson
Ian Campbell writes ("[PATCH OSSTEST 6/8] ts-kernel-build: Include dtbs in dist file"):
Quoted 3 lines
These are installed to $(INSTALL_PATH)/dtbs/$(KERNEL_RELEASE) where $(INSTALL_PATH) defaults to /boot but we override it to our staging /boot.
Thanks for the detailed explanation.
Acked-by Ian Jackson <[email protected]>Quoted 10 lines
ts-kernel-build sub kinstall () { · line 397
Quoted 6 lines
(quoted diff) +0
-1- make $makeflags INSTALL_MOD_PATH=$builddir/dist
modules_installts-kernel-build +4
-0+ make $makeflags \\ + INSTALL_PATH=$builddir/dist/boot \\ + INSTALL_MOD_PATH=$builddir/dist modules_install \\ + $dtbs_installI tripped over the formatting here. It would be nicer to put `modules_install' on the next line, to avoid hiding it in INSTALL_BLAH=.
Quoted 5 lines
I did it that way because
INSTALL_MOD_PATHis the variable which controls wheremodules_installputs things, so they are in some sense related.Next line is fine too, or I could move it to first on the line it is on now?
I went with:
+ make $makeflags \\
+ INSTALL_PATH=$builddir/dist/boot \\ + INSTALL_MOD_PATH=$builddir/dist \\ + modules_install $dtbs_install
which did seem clearer.
Ian.
Re: [PATCH OSSTEST 6/8] ts-kernel-build: Include dtbs in dist file
Ian Campbell writes ("Re: [Xen-devel] [PATCH OSSTEST 6/8] ts-kernel-build: Include dtbs in dist file"):
Quoted 8 lines
I went with:
ts-kernel-build +4
-0+ make $makeflags \\ + INSTALL_PATH=$builddir/dist/boot \\ + INSTALL_MOD_PATH=$builddir/dist \\ + modules_install $dtbs_installwhich did seem clearer.
Thanks,
Ian.
Re: [PATCH OSSTEST 6/8] ts-kernel-build: Include dtbs in dist file
In reply to Ian CampbellIan Campbell writes ("Re: [PATCH OSSTEST 6/8] ts-kernel-build: Include dtbs in dist file"):
Quoted 6 lines · Ian Jackson
Quoted 2 lines
`modules_install' on the next line, to avoid hiding it in INSTALL_BLAH=.
I did it that way because
INSTALL_MOD_PATHis the variable which controls wheremodules_installputs things, so they are in some sense related.
Aha.
Quoted 2 lines
Next line is fine too, or I could move it to first on the line it is on now?
Whatever you like. You can keep it how it is if you prefer...
Ian.
PATCH 7/8 Debian: Enable interpolation in uboot_scr_load_dtb here doc
In reply to Ian CampbellBy switching <<'END' to <<END. A future patch is going to want to put a variable here which requires interpretation by the Perl.
Unfortunately this means lots of extra backticks to escape things such that they pass through Perl and Shell and end up as ${foo} in the resulting u-boot script.
--- Osstest/Debian.pm | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
Osstest/Debian.pm +5 -5
@@ -142,11 +142,11 @@ sub uboot_common_kernel_bootargs ($) } sub uboot_scr_load_dtb () { - return <<'END'; -if test -z "\${fdt_addr}" && test -n "\${fdtfile}" ; then - echo Loading dtbs/\${fdtfile} - ext2load scsi 0 \${fdt_addr_r} dtbs/\${fdtfile} - setenv fdt_addr \${fdt_addr_r} + return <<END; +if test -z "\\\${fdt_addr}" && test -n "\\\${fdtfile}" ; then + echo Loading dtbs/\\\${fdtfile} + ext2load scsi 0 \\\${fdt_addr_r} dtbs/\\\${fdtfile} + setenv fdt_addr \\\${fdt_addr_r} fi END }
Signature · 1 line
2.5.3
Re: [PATCH OSSTEST 7/8] Debian: Enable interpolation in uboot_scr_load_dtb here doc
Ian Campbell writes ("[PATCH OSSTEST 7/8] Debian: Enable interpolation in uboot_scr_load_dtb here doc"):
Quoted 6 lines
By switching <<'END' to <<END. A future patch is going to want to put a variable here which requires interpretation by the Perl.
Unfortunately this means lots of extra backticks to escape things such that they pass through Perl and Shell and end up as ${foo} in the resulting u-boot script.
(I think a backtick is ASCII 0x60 ` not ASCII 0x5C \. I would call \
a backslash and ` a backquote or open single quote.)
Ian.
Re: [PATCH OSSTEST 7/8] Debian: Enable interpolation in uboot_scr_load_dtb here doc
Quoted 5 lines · Ian Jackson
Ian Campbell writes ("[PATCH OSSTEST 7/8] Debian: Enable interpolation in
uboot_scr_load_dtbhere doc"):Quoted 3 lines
By switching <<'END' to <<END. A future patch is going to want to put a variable here which requires interpretation by the Perl.
Quoted 8 lines
Quoted 3 lines
Unfortunately this means lots of extra backticks to escape things such that they pass through Perl and Shell and end up as ${foo} in the resulting u-boot script.
Acked-by Ian Jackson <[email protected]>(I think a backtick is ASCII
0x60` not ASCII0x5C\. I would call \
a backslash and ` a backquote or open single quote.)
So would I, I don't know what I was on about there...
Ian.
PATCH 8/8 Debian: Use dtbs from kernel dist when booting that kernel
In reply to Ian CampbellThe kernel dist built by ts-kernel-build puts the corresponding dtbs into /boot/dtbs/$kvers.
The host installers dtbs remain in /boot/dtbs and are used when booting the native kernel.
It's possible that this change will expose bugs which exist in the DTBs in previous kernel branches (3.18 and 4.1). I've not investigated, I think we should accept this possibility and deal with it via backport requests (and maybe some force pushes if appropriate) as necessary.
--- Osstest/Debian.pm | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)
Osstest/Debian.pm +7 -4
@@ -141,11 +141,14 @@ sub uboot_common_kernel_bootargs ($) return @bootargs; } -sub uboot_scr_load_dtb () { +sub uboot_scr_load_dtb (;$) { + my ($kvers) = @_; + $kvers ||= ''; + $kvers .= '/' if $kvers; return <<END; if test -z "\\\${fdt_addr}" && test -n "\\\${fdtfile}" ; then - echo Loading dtbs/\\\${fdtfile} - ext2load scsi 0 \\\${fdt_addr_r} dtbs/\\\${fdtfile} + echo Loading dtbs/$kvers\\\${fdtfile} + ext2load scsi 0 \\\${fdt_addr_r} dtbs/$kvers\\\${fdtfile} setenv fdt_addr \\\${fdt_addr_r} fi END
@@ -228,7 +231,7 @@ END my $early_commands = get_host_property($ho, 'UBootScriptEarlyCommands', ''); my $xen_addr_r = get_host_property($ho, 'UBootSetXenAddrR', undef); - my $load_dtb = uboot_scr_load_dtb(); + my $load_dtb = uboot_scr_load_dtb($want_kernver); my $set_xen_addr_r = $xen_addr_r ? "setenv xen_addr_r $xen_addr_r" : "";
Signature · 1 line
2.5.3
Re: [PATCH OSSTEST 8/8] Debian: Use dtbs from kernel dist when booting that kernel
Ian Campbell writes ("[PATCH OSSTEST 8/8] Debian: Use dtbs from kernel dist when booting that kernel"):
Quoted 4 lines
The kernel dist built by ts-kernel-build puts the corresponding dtbs into
/boot/dtbs/$kvers.The host installers dtbs remain in
/boot/dtbsand are used when
^
Missing apostrophe "installer's".
Quoted 5 lines
It's possible that this change will expose bugs which exist in the DTBs in previous kernel branches (3.18 and 4.1). I've not investigated, I think we should accept this possibility and deal with it via backport requests (and maybe some force pushes if appropriate) as necessary.
OK.
Osstest/Debian.pm sub uboot_common_kernel_bootargs ($) · line 144
Quoted 2 lines
Osstest/Debian.pm +1
-1-sub uboot_scr_load_dtb () { +sub uboot_scr_load_dtb (;$) {
Some of the information about host installer dtbs being in /boot etc. (particularly that from "ts-kernel-build: Include dtbs in dist file"'s commit message) might benefit from being in a comment here.
Anyway, all of this is (like my other comments) cosmetic and minor, so:
Ian.