Improve README and enable Wheezy
cover Improve README and enable Wheezy
Hi Ian and Ian
With the changes to Debian.pm and ts-debian-fixup I can now use Wheezy to run build-* jobs and test-amd64-amd64-{pv,xl}.
Wei Liu (5):
README: add curl and netcat to requirement list
README: describe job dependence
Debian: install libc6-dev-i386 for Wheezy amd64
ts-debian-fixup: remove dead code
ts-debian-fixup: workaround Debian bug #691320
README | 8 ++++++++ ts-debian-fixup | 32 ++++++++++++++------------------ ts-xen-build-prep | 4 ++++ 3 files changed, 26 insertions(+), 18 deletions(-)
Signature · 1 line
1.7.10.4
RFC PATCH 1/5 README: add curl and netcat to requirement list
--- README | 2 ++ 1 file changed, 2 insertions(+)
README +2 -0
@@ -5,6 +5,8 @@ To run osstest in standalone mode: tcl8.4 tclx8.4 libsqlite3-tcl libdbi-perl libdbd-sqlite3-perl pax rsync + curl + netcat - Write a config file ~/.xen-osstest/config
Signature · 1 line
1.7.10.4
RFC PATCH 2/5 README: describe job dependence
In reply to Wei Liu--- README | 6 ++++++ 1 file changed, 6 insertions(+)
README +6 -0
@@ -33,6 +33,12 @@ To run osstest in standalone mode: (strictly, the final `_HOST' is the uppercase of the host's `ident' in the job) + However, as test-amd64-{i386,amd64}-xl and other tests depends on + some runtime variables generated by build-* jobs, you can run + build-* jobs before running actual test jobs. If you don't want to do + so you need to insert those missing runvars into standalone.db with + sqlite3. + - Don't forget to set the machine to pxeboot in the BIOS. - Currently you need a serial console at 115200 8n1; see TODO.
Signature · 1 line
1.7.10.4
RFC PATCH 3/5 Debian: install libc6-dev-i386 for Wheezy amd64
In reply to Wei LiuOtherwise build-amd64 fails with missing header predefs.h.
--- ts-xen-build-prep | 4 ++++ 1 file changed, 4 insertions(+)
ts-xen-build-prep +4 -0
@@ -175,6 +175,10 @@ sub prep () { target_cmd_root($ho, "chmod -R a+r /usr/share/git-core/templates"); # workaround for Debian #595728 + if ($r{arch} eq 'amd64' && $ho->{Suite} =~ m/wheezy/) { + target_install_packages($ho, 'libc6-dev-i386'); + } + if ($ho->{Suite} !~ m/lenny/) { target_putfilecontents_stash($ho,60,<<'END','/home/osstest/.hgrc'); [extensions]
Signature · 1 line
1.7.10.4
Re: [PATCH OSSTEST RFC 3/5] Debian: install libc6-dev-i386 for Wheezy amd64
Quoted 1 line · Wei Liu
Otherwise build-amd64 fails with missing header
predefs.h.
This package exists in Squeeze too, it might simplify things to make it unconditional?
I wonder why it apparently isn't need for a Squeeze build though. What 32-bit stuff is getting built?
Quoted 21 lines
Signed-off-by Wei Liu <[email protected]>--- ts-xen-build-prep | 4 ++++ 1 file changed, 4 insertions(+)ts-xen-build-prep +4
-0@@ -175,6 +175,10 @@ sub prep () { target_cmd_root($ho, "chmod -R a+r /usr/share/git-core/templates"); # workaround for Debian #595728 + if ($r{arch} eq 'amd64' && $ho->{Suite} =~ m/wheezy/) { + target_install_packages($ho, 'libc6-dev-i386'); + } + if ($ho->{Suite} !~ m/lenny/) { target_putfilecontents_stash($ho,60,<<'END','/home/osstest/.hgrc'); [extensions]
Re: [PATCH OSSTEST RFC 3/5] Debian: install libc6-dev-i386 for Wheezy amd64
Quoted 6 lines · Ian Campbell
Quoted 1 line · Wei Liu
Otherwise build-amd64 fails with missing header
predefs.h.This package exists in Squeeze too, it might simplify things to make it unconditional?
No, it doesn't exist in Lenny -- I see there's branches conditional on Lenny, do we still use that?
Quoted 3 lines
I wonder why it apparently isn't need for a Squeeze build though. What 32-bit stuff is getting built?
I guess that's because Wheezy introduces multiarch? The thing being built is QEMU.
Wei.
Quoted 22 lines
Quoted 21 lines
Signed-off-by Wei Liu <[email protected]>--- ts-xen-build-prep | 4 ++++ 1 file changed, 4 insertions(+)ts-xen-build-prep +4
-0@@ -175,6 +175,10 @@ sub prep () { target_cmd_root($ho, "chmod -R a+r /usr/share/git-core/templates"); # workaround for Debian #595728 + if ($r{arch} eq 'amd64' && $ho->{Suite} =~ m/wheezy/) { + target_install_packages($ho, 'libc6-dev-i386'); + } + if ($ho->{Suite} !~ m/lenny/) { target_putfilecontents_stash($ho,60,<<'END','/home/osstest/.hgrc'); [extensions]
Re: [PATCH OSSTEST RFC 3/5] Debian: install libc6-dev-i386 for Wheezy amd64
Quoted 10 lines · Wei Liu
Quoted 6 lines · Ian Campbell
Quoted 1 line · Wei Liu
Otherwise build-amd64 fails with missing header
predefs.h.This package exists in Squeeze too, it might simplify things to make it unconditional?
No, it doesn't exist in Lenny -- I see there's branches conditional on Lenny, do we still use that?
I think it is all Squeeze these days.
Quoted 6 lines
Quoted 3 lines
I wonder why it apparently isn't need for a Squeeze build though. What 32-bit stuff is getting built?
I guess that's because Wheezy introduces multiarch? The thing being built is QEMU.
We're building 64-bit qemu and failing because of a missing 32-bit header? That sounds like a bug somewhere, perhaps in the compiler/libc packaging or maybe in some part of our build system. How does the actual error look?
Ian.
Re: [PATCH OSSTEST RFC 3/5] Debian: install libc6-dev-i386 for Wheezy amd64
Quoted 5 lines · Ian Campbell
Quoted 4 lines · Wei Liu
Quoted 3 lines · Ian Campbell
Quoted 1 line · Wei Liu
Otherwise build-amd64 fails with missing header
predefs.h.
Quoted 9 lines
Quoted 6 lines
Quoted 3 lines
This package exists in Squeeze too, it might simplify things to make it unconditional?
No, it doesn't exist in Lenny -- I see there's branches conditional on Lenny, do we still use that?
I think it is all Squeeze these days.
OK.
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
I wonder why it apparently isn't need for a Squeeze build though. What 32-bit stuff is getting built?
Quoted 8 lines
Quoted 2 lines
I guess that's because Wheezy introduces multiarch? The thing being built is QEMU.
We're building 64-bit qemu and failing because of a missing 32-bit header? That sounds like a bug somewhere, perhaps in the compiler/libc packaging or maybe in some part of our build system. How does the actual error look?
Oh sorrrry, it's not QEMU, -j32 and interleaving output fooled me. :-(
XEN_TARGET_ARCH=x86_32 make -f blowfish.mk all
make[5]: Entering directory `/local/scratch/xen.git/tools/tests/x86_emulator'
gcc -O1 -fno-omit-frame-pointer -m32 -march=i686 -g -fno-strict-aliasing -std=gnu99 -Wall -Wstrict-prototypes -Wdeclaration-after-statement -Wno-unused-but-set-variable -Wno-unused-local-typedefs -D__XEN_TOOLS__ -MMD -MF .blowfish.bin.d -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -fno-optimize-sibling-calls -mno-tls-direct-seg-refs -fno-stack-protector -fno-exceptions -fno-builtin -msoft-float -c blowfish.c
In file included from /usr/include/stdint.h:26:0,
from /usr/lib/gcc/x86_64-linux-gnu/4.7/include/stdint.h:3,
from blowfish.c:19:/usr/include/features.h:323:26: fatal error: bits/predefs.h: No such file or directory compilation terminated.
Wei.
Quoted 1 line
Ian.
Re: [PATCH OSSTEST RFC 3/5] Debian: install libc6-dev-i386 for Wheezy amd64
Quoted 33 lines · Wei Liu
Quoted 5 lines · Ian Campbell
Quoted 4 lines · Wei Liu
Quoted 3 lines · Ian Campbell
Quoted 1 line · Wei Liu
Otherwise build-amd64 fails with missing header
predefs.h.Quoted 9 lines
Quoted 6 lines
Quoted 3 lines
This package exists in Squeeze too, it might simplify things to make it unconditional?
No, it doesn't exist in Lenny -- I see there's branches conditional on Lenny, do we still use that?
I think it is all Squeeze these days.
OK.
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
I wonder why it apparently isn't need for a Squeeze build though. What 32-bit stuff is getting built?
Quoted 8 lines
Quoted 2 lines
I guess that's because Wheezy introduces multiarch? The thing being built is QEMU.
We're building 64-bit qemu and failing because of a missing 32-bit header? That sounds like a bug somewhere, perhaps in the compiler/libc packaging or maybe in some part of our build system. How does the actual error look?
Oh sorrrry, it's not QEMU, -j32 and interleaving output fooled me. :-(
Quoted 8 lines
XEN_TARGET_ARCH=x86_32 make -f blowfish.mk all
make[5]: Entering directory `/local/scratch/xen.git/tools/tests/x86_emulator'
gcc -O1 -fno-omit-frame-pointer -m32 -march=i686 -g -fno-strict-aliasing -std=gnu99 -Wall -Wstrict-prototypes -Wdeclaration-after-statement -Wno-unused-but-set-variable -Wno-unused-local-typedefs -D__XEN_TOOLS__ -MMD -MF .blowfish.bin.d -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -fno-optimize-sibling-calls -mno-tls-direct-seg-refs -fno-stack-protector -fno-exceptions -fno-builtin -msoft-float -cblowfish.c
In file included from/usr/include/stdint.h:26:0,from /usr/lib/gcc/x86_64-linux-gnu/4.7/include/stdint.h:3, from blowfish.c:19:
/usr/include/features.h:323:26: fatal error:bits/predefs.h: No such file or directory compilation terminated.
Ah, OK, this is test code for the 32-bit instruction decoder/emulator. In principal this probably shouldn't be built if the compiler is not available, but in the meantime your fix looks sensible.
Ian
RFC PATCH 4/5 ts-debian-fixup: remove dead code
In reply to Wei LiuPer c7910040 ("Osstest.pm, toolstack()->; always use "cfg", not separate xl configs"), we only pick the config file with .cfg suffix.
--- ts-debian-fixup | 18 ------------------ 1 file changed, 18 deletions(-)
ts-debian-fixup +0 -18
@@ -163,23 +163,6 @@ sub writecfg () { target_putfile_root($ho,10, $cfgstash, $cfgfile); } -sub xlcfg () { - my $cfg_xend= target_var($gho,'cfgpath'); - my $cfg_xl= "/etc/xen/$gho->{Name}.xl"; - store_runvar("$gho->{Guest}_xlpath", $cfg_xl); - target_editfile_root($ho, $cfg_xend, undef, $cfg_xl, sub { - local ($/)= undef; - $_= <EI>; - s/^ [ \t]* memory [ \t]* \= [ \t]* ['"] (.+) ['"] [ \t]* - $/memory=$1/mx; - s/([[,])\s*\n/$1/g; - s/,\s*\]/\]/g; - s/([^#\n].*)\n/$1;\n/mg; - s/\n\n+/\n/g; - print EO or die $!; - }); -} - savecfg(); ether(); target_kernkind_check($gho);
@@ -188,5 +171,4 @@ console(); filesystems(); otherfixupcfg(); writecfg(); -xlcfg(); unmount();
Signature · 1 line
1.7.10.4
RFC PATCH 5/5 ts-debian-fixup: workaround Debian bug #691320
In reply to Wei Liu--- ts-debian-fixup | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
ts-debian-fixup +14 -0
@@ -118,6 +118,20 @@ sub otherfixupcfg () { $cfg =~ s/^vcpus.*//mg; $cfg .= "\nvcpus = $vcpus\n"; + # Debian bug #691320 + # Wheezy's xen-create-image writes incorrect memory= option in + # config file. This is fixed in newer version of xen-tools but it + # is not likely to go in in short term. + # On the other hand we should consider making xl support '512M', + # '2G' etc. But xend will still need this workaround until Wheezy + # is fixed. + # The default value 512 comes from ts-debian-install. + if ($ho->{Suite} =~ m/wheezy/) { + my $memory= guest_var($gho,'memory',512); + $cfg =~ s/^memory.*//mg; + $cfg .= "\nmemory = $memory\n"; + } + # PCI passthrough # Look for runvars <gn>_pcipassthrough_<devtype>=<hostident> # and pass through all matching devices from the specified host.
Signature · 1 line
1.7.10.4
Re: [PATCH OSSTEST RFC 5/5] ts-debian-fixup: workaround Debian bug #691320
Quoted 11 lines · Wei Liu
Signed-off-by Wei Liu <[email protected]>--- ts-debian-fixup | 14 ++++++++++++++ 1 file changed, 14 insertions(+)ts-debian-fixup +0
-0@@ -118,6 +118,20 @@ sub otherfixupcfg () { $cfg =~ s/^vcpus.*//mg;
ts-debian-fixup sub otherfixupcfg () { · line 128
Quoted 10 lines
ts-debian-fixup +8
-0$cfg .= "\nvcpus = $vcpus\n"; + # Debian bug #691320 + # Wheezy's xen-create-image writes incorrect memory= option in + # config file. This is fixed in newer version of xen-tools but it + # is not likely to go in in short term. + # On the other hand we should consider making xl support '512M', + # '2G' etc. But xend will still need this workaround until Wheezy + # is fixed. + # The default value 512 comes from ts-debian-install.
Wouldn't it be better to fix the invocation of xne-create-image to use a Mb suffix instead of M, AFAICT that would work on both Squeeze and Wheezy.
I think the fact that Squeeze's accepted just M was unintentional, although it's good that they are willing to fix it to be acceptable again...
Quoted 9 lines
ts-debian-fixup +6
-0+ if ($ho->{Suite} =~ m/wheezy/) { + my $memory= guest_var($gho,'memory',512); + $cfg =~ s/^memory.*//mg; + $cfg .= "\nmemory = $memory\n"; + } + # PCI passthrough # Look for runvars <gn>_pcipassthrough_<devtype>=<hostident> # and pass through all matching devices from the specified host.
Re: [PATCH OSSTEST RFC 5/5] ts-debian-fixup: workaround Debian bug #691320
Quoted 14 lines · Ian Campbell
Quoted 13 lines · Wei Liu
Signed-off-by Wei Liu <[email protected]>--- ts-debian-fixup | 14 ++++++++++++++ 1 file changed, 14 insertions(+)ts-debian-fixup +0
-0@@ -118,6 +118,20 @@ sub otherfixupcfg () { $cfg =~ s/^vcpus.*//mg; $cfg .= "\nvcpus = $vcpus\n";
Quoted 13 lines
ts-debian-fixup sub otherfixupcfg () { · line 128
Quoted 8 lines
ts-debian-fixup +8
-0+ # Debian bug #691320 + # Wheezy's xen-create-image writes incorrect memory= option in + # config file. This is fixed in newer version of xen-tools but it + # is not likely to go in in short term. + # On the other hand we should consider making xl support '512M', + # '2G' etc. But xend will still need this workaround until Wheezy + # is fixed. + # The default value 512 comes from ts-debian-install.Wouldn't it be better to fix the invocation of xne-create-image to use a Mb suffix instead of M, AFAICT that would work on both Squeeze and Wheezy.
OK. I tested passing 'Mb', it did work in Wheezy.
I will drop this one.
Wei.
Quoted 14 lines
I think the fact that Squeeze's accepted just M was unintentional, although it's good that they are willing to fix it to be acceptable again...
Quoted 9 lines
ts-debian-fixup +6
-0+ if ($ho->{Suite} =~ m/wheezy/) { + my $memory= guest_var($gho,'memory',512); + $cfg =~ s/^memory.*//mg; + $cfg .= "\nmemory = $memory\n"; + } + # PCI passthrough # Look for runvars <gn>_pcipassthrough_<devtype>=<hostident> # and pass through all matching devices from the specified host.
Quoted 2 lines · Wei Liu
With the changes to Debian.pm and ts-debian-fixup I can now use Wheezy to run build-* jobs and test-amd64-amd64-{pv,xl}.
So I guess you've just been through the process of setting up osstest in standalone mode? I wonder if you could put that into a short blog or wiki doc as an illustration for people?
Ian.
Quoted 8 lines · Ian Campbell
Quoted 2 lines · Wei Liu
With the changes to Debian.pm and ts-debian-fixup I can now use Wheezy to run build-* jobs and test-amd64-amd64-{pv,xl}.
So I guess you've just been through the process of setting up osstest in standalone mode? I wonder if you could put that into a short blog or wiki doc as an illustration for people?
I'm actually writing a blog post on this topic, but still far from complete.
Wei.
Quoted 1 line
Ian.
Quoted 5 lines · Wei Liu
Quoted 4 lines · Ian Campbell
Quoted 2 lines · Wei Liu
With the changes to Debian.pm and ts-debian-fixup I can now use Wheezy to run build-* jobs and test-amd64-amd64-{pv,xl}.
Quoted 7 lines
Quoted 4 lines
So I guess you've just been through the process of setting up osstest in standalone mode? I wonder if you could put that into a short blog or wiki doc as an illustration for people?
I'm actually writing a blog post on this topic, but still far from complete.
Perfect, thanks! ;-)