host: introduce modify_host
PATCH v5 1/5 host: introduce modify_host
Abstract the set_property checks and DB call into a helper.
No functional change.
---
Requested on IRC:
17:09:30 Diziet Also if it were me I would put the modify_host refactoring in its own nfc patch,
but I won't insist on that...
---
changes since v4:
- New in this version.
--- Osstest/HostDB/Executive.pm | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-)
Osstest/HostDB/Executive.pm +11 -5
@@ -51,6 +51,16 @@ END } } +sub modify_host ($$$) { + my ($hd, $ho, $query) = @_; + my $blessing = intended_blessing(); + + die "Attempting to modify host with intended blessing $blessing != real" + if $blessing ne "real"; + + db_retry($dbh_tests, [qw(resources)], $query); +} + sub set_property($$$$) { my ($hd, $ho, $prop, $val) = @_; my $rmq = $dbh_tests->prepare(<<END);
@@ -61,12 +71,8 @@ END INSERT INTO resource_properties (restype,resname,name,val) VALUES ('host', ?,?,?) END - my $blessing = intended_blessing(); - - die "Attempting to modify host props with intended blessing $blessing != real" - if $blessing ne "real"; - db_retry($dbh_tests, [qw(resources)], sub { + modify_host($hd, $ho, sub { $rmq->execute($ho->{Name}, $prop); if (length $val) { $addq->execute($ho->{Name}, $prop, $val);
Signature · 7 lines
2.25.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 2/5 host: introduce a helper to modify hostflags
Add a generic function to perform database changes related to a host flag and add a wrapper to TestSupport.
---
Changes since v4:
- Move addition of hostflag_putative_record to a different patch.
- Introduce a single helper in TestSupport: modify_host_flag.
Changes since v3:
- Introduce modify_flag instead of {set/remove}_flag.
- Introduce a generic modify_host helper.
- Split from patch 1.
---
Requested on IRC:
17:08:58 Diziet royger: I think your ts-examine-hostprops-save hunk in 2/ belongs in 1/ ? (Or in
a separate 1.5/ along with hostflag_putative_record.)
--- Osstest/HostDB/Executive.pm | 17 +++++++++++++++++ Osstest/HostDB/Static.pm | 7 +++++++ Osstest/TestSupport.pm | 8 +++++++- 3 files changed, 31 insertions(+), 1 deletion(-)
Osstest/HostDB/Executive.pm +17 -0
@@ -96,6 +96,23 @@ END return $flags; } +sub modify_flag ($$$$) { + my ($hd, $ho, $flag, $set) = @_; + my $rmq = $dbh_tests->prepare(<<END); + DELETE FROM hostflags WHERE hostname=? AND hostflag=? +END + my $addq = $dbh_tests->prepare(<<END); + INSERT INTO hostflags (hostname,hostflag) VALUES (?,?) +END + + modify_host($hd, $ho, sub { + $rmq->execute($ho->{Name}, $flag); + if ($set) { + $addq->execute($ho->{Name}, $flag); + } + }); +} + sub get_arch_platforms ($$$) { my ($hd, $blessing, $arch, $suite) = @_;
Osstest/HostDB/Static.pm +7 -0
@@ -72,6 +72,13 @@ sub get_flags ($$) { #method return $flags; } +sub modify_flag ($$$$) { + my ($hd, $ho, $flag, $set) = @_; + + die + "Cannot modify flags in standalone mode for $ho->{Name} $flag set: $set\n"; +} + sub get_arch_platforms ($$$) { my ($hd, $blessing, $arch, $suite) = @_;
Osstest/TestSupport.pm +7 -1
@@ -84,7 +84,7 @@ BEGIN { get_target_property get_host_native_linux_console hostnamepath hostnamepath_list set_runtime_hostflag power_state power_cycle power_reboot_attempts - serial_fetch_logs set_host_property + serial_fetch_logs set_host_property modify_host_flag propname_massage propname_check hostprop_putative_record
@@ -1411,6 +1411,12 @@ sub hostprop_putative_record ($$$) { store_runvar("hostprop/$ho->{Ident}/$prop", $val); } +sub modify_host_flag ($$$) { + my ($ho, $flag, $set) = @_; + + $mhostdb->modify_flag($ho, $flag, $set); +} + sub get_target_property ($$;$); sub get_target_property ($$;$) { my ($ho, $prop, $defval) = @_;
Signature · 7 lines
2.25.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH OSSTEST v5 2/5] host: introduce a helper to modify hostflags
Roger Pau Monne writes ("[PATCH OSSTEST v5 2/5] host: introduce a helper to modify hostflags"):
Quoted 2 lines
Add a generic function to perform database changes related to a host flag and add a wrapper to TestSupport.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 3/5 ts-examine-hostprops-save: record hostflags also
In reply to Roger Pau MonneCommit putative hotflags into the database if present on the runvars.
---
Changes since v4:
- New in this version.
---
Requested by Ian on IRC:
17:08:58 Diziet royger: I think your ts-examine-hostprops-save hunk in 2/ belongs in 1/ ? (Or in
a separate 1.5/ along with hostflag_putative_record.)
--- Osstest/TestSupport.pm | 8 +++++++- ts-examine-hostprops-save | 23 ++++++++++++++--------- 2 files changed, 21 insertions(+), 10 deletions(-)
Osstest/TestSupport.pm +7 -1
@@ -86,7 +86,7 @@ BEGIN { power_state power_cycle power_reboot_attempts serial_fetch_logs set_host_property modify_host_flag propname_massage propname_check - hostprop_putative_record + hostprop_putative_record hostflag_putative_record get_stashed open_unique_stashfile compress_stashed dir_identify_vcs
@@ -1417,6 +1417,12 @@ sub modify_host_flag ($$$) { $mhostdb->modify_flag($ho, $flag, $set); } +sub hostflag_putative_record ($$$) { + my ($ho, $prop, $set) = @_; + + store_runvar("hostflag/$ho->{Ident}/$prop", !!$set); +} + sub get_target_property ($$;$); sub get_target_property ($$;$) { my ($ho, $prop, $defval) = @_;
ts-examine-hostprops-save +14 -9
@@ -27,20 +27,25 @@ tsreadconfig(); our $blessing = intended_blessing(); -logm("setting host properties"); +logm("setting host properties and flags"); # NB: in order to aid debug only attempt to save the host props on flights # with intended real blessing, for the rest just do a dry run. our $dry_run = $blessing ne "real"; -logm("not saving host props with intended blessing $blessing != real") +logm("not saving host props/flags with intended blessing $blessing != real") if $dry_run; foreach my $k (sort keys %r) { - next unless $k =~ m/^hostprop\/([^\/]*)\/([^\/]*)$/; - my $ho = selecthost($1); - my $prop = $2; - - logm("recording for $ho->{Name} $prop=$r{$k}"); - - set_host_property($ho, $prop, $r{$k}) if !$dry_run; + next unless $k =~ m/^host(prop|flag)\/([^\/]*)\/([^\/]*)$/; + my $type = $1; + my $ho = selecthost($2); + my $prop = $3; + + if ($type eq "flag") { + logm("recording flag $prop set: $r{$k} for $ho->{Name}"); + modify_host_flag($ho, $prop, !!$r{$k}) if !$dry_run; + } else { + logm("recording prop for $ho->{Name} $prop=$r{$k}"); + set_host_property($ho, $prop, $r{$k}) if !$dry_run; + } }
Signature · 7 lines
2.25.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH OSSTEST v5 3/5] ts-examine-hostprops-save: record hostflags also
Roger Pau Monne writes ("[PATCH OSSTEST v5 3/5] ts-examine-hostprops-save: record hostflags also"):
Quoted 3 lines
Commit putative hotflags into the database if present on the runvars.
Signed-off-by Roger Pau Monné <[email protected]>
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 4/5 examine: detect IOMMU availability and add it as a hostflag
In reply to Roger Pau MonneIntroduce a new test to check for iommu availability and add it as a hostflag if found.
---
Changes since v4:
- Split out code into separate patches.
Changes since v3:
- Fail if xl info command fails.
Changes since v2:
- Allow flags to be removed.
- Fix set_flag addition to HostBD/Static.pm.
--- sg-run-job | 1 + ts-examine-iommu | 32 ++++++++++++++++++++++++++++++++ 2 files changed, 33 insertions(+) create mode 100755 ts-examine-iommu
sg-run-job +1 -0
@@ -679,6 +679,7 @@ proc examine-host-examine {install} { if {$ok} { run-ts -. = ts-examine-serial-post + host run-ts . = ts-examine-logs-save + host + run-ts . = ts-examine-iommu + host run-ts . = ts-examine-hostprops-save } }
ts-examine-iommu new file +32 -0
@@ -0,0 +1,32 @@ +#!/usr/bin/perl -w +# This is part of "osstest", an automated testing framework for Xen. +# Copyright (C) 2009-2020 Citrix Inc. +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU Affero General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. + +use strict qw(vars); +BEGIN { unshift @INC, qw(.); } +use Osstest; +use Osstest::TestSupport; + +tsreadconfig(); + +our ($whhost) = @ARGV; +$whhost ||= 'host'; +our $ho= selecthost($whhost); +our $info = target_cmd_output_root($ho, 'xl info', 10); +our $has_iommu = $info =~ /directio/; + +logm("$ho->{Ident} iommu: $has_iommu"); +hostflag_putative_record($ho, "iommu", $has_iommu);
Signature · 7 lines
2.25.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH OSSTEST v5 4/5] examine: detect IOMMU availability and add it as a hostflag
Roger Pau Monne writes ("[PATCH OSSTEST v5 4/5] examine: detect IOMMU availability and add it as a hostflag"):
Quoted 2 lines
Introduce a new test to check for iommu availability and add it as a hostflag if found.
...
Quoted 1 line
+our $has_iommu = $info =~ /directio/;
I think this regexp is too lax. For example, if the Xen command line had something mentioning directio' it would always say yes'...
This appears in virt_caps, right ? So I think you want
+our $has_iommu = $info =~ /^virt_caps\s*:.*\sdirectio\s/m;
or some such. (Untested.)
Sorry for only spotting this now. Otherwise this LGTM.
Ian.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH OSSTEST v5 4/5] examine: detect IOMMU availability and add it as a hostflag
In reply to Roger Pau MonneRoger Pau Monne writes ("[PATCH OSSTEST v5 4/5] examine: detect IOMMU availability and add it as a hostflag"):
Quoted 25 lines
Introduce a new test to check for iommu availability and add it as a hostflag if found.
Signed-off-by Roger Pau Monné <[email protected]>---
Changes since v4:
- Split out code into separate patches.Changes since v3:
- Fail ifxl infocommand fails.Changes since v2:
- Allow flags to be removed.
- Fixset_flagaddition to HostBD/Static.pm.--- sg-run-job | 1 + ts-examine-iommu | 32 ++++++++++++++++++++++++++++++++ 2 files changed, 33 insertions(+) create mode 100755 ts-examine-iommusg-run-job +0
-0
sg-run-job proc examine-host-examine {install} { · line 683
Quoted 5 lines
sg-run-job +1
-0@@ -679,6 +679,7 @@ proc examine-host-examine {install} { if {$ok} { run-ts -. = ts-examine-serial-post + host run-ts . = ts-examine-logs-save + host + run-ts . = ts-examine-iommu + host run-ts . = ts-examine-hostprops-save
Thanks. I looked at your test flight and that caused me to notice that this ts-examine-iommu is in the wrong place, at least logically speaking. Log capture should come last, just before -hostprops-save.
So can you swap -logs-save and -iommu please ?
Thanks,
Ian.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v6 4/5 examine: detect IOMMU availability and add it as a hostflag
In reply to Roger Pau MonneIntroduce a new test to check for iommu availability and add it as a hostflag if found.
---
Changes since v5:
- Use a more restrictive regex.
- Place iommu test before log collection.
Changes since v4:
- Split out code into separate patches.
Changes since v3:
- Fail if xl info command fails.
Changes since v2:
- Allow flags to be removed.
- Fix set_flag addition to HostBD/Static.pm.
--- sg-run-job | 1 + ts-examine-iommu | 32 ++++++++++++++++++++++++++++++++ 2 files changed, 33 insertions(+) create mode 100755 ts-examine-iommu
sg-run-job +1 -0
@@ -678,6 +678,7 @@ proc examine-host-examine {install} { run-ts !broken capture-logs ts-logs-capture + host if {$ok} { run-ts -. = ts-examine-serial-post + host + run-ts . = ts-examine-iommu + host run-ts . = ts-examine-logs-save + host run-ts . = ts-examine-hostprops-save }
ts-examine-iommu new file +32 -0
@@ -0,0 +1,32 @@ +#!/usr/bin/perl -w +# This is part of "osstest", an automated testing framework for Xen. +# Copyright (C) 2009-2020 Citrix Inc. +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU Affero General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. + +use strict qw(vars); +BEGIN { unshift @INC, qw(.); } +use Osstest; +use Osstest::TestSupport; + +tsreadconfig(); + +our ($whhost) = @ARGV; +$whhost ||= 'host'; +our $ho= selecthost($whhost); +our $info = target_cmd_output_root($ho, 'xl info', 10); +our $has_iommu = $info =~ /^virt_caps\s*:.*\s(hvm|pv)_directio\s.*/m; + +logm("$ho->{Ident} iommu: $has_iommu"); +hostflag_putative_record($ho, "iommu", $has_iommu);
Signature · 7 lines
2.25.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH OSSTEST v6 4/5] examine: detect IOMMU availability and add it as a hostflag
Roger Pau Monne writes ("[PATCH OSSTEST v6 4/5] examine: detect IOMMU availability and add it as a hostflag"):
Quoted 2 lines
Introduce a new test to check for iommu availability and add it as a hostflag if found.
Thanks,
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v5 5/5 make-flight: add dom0 PVH test
In reply to Roger Pau MonneAdd a dom0 PVH test, the test to be run is launching a PV guest (test-debian). Note the PVH dom0 tests are only enabled for Xen >= 4.13.
The runvar difference is:
+test-amd64-amd64-dom0pvh-xl-amd all_host_di_version 2020-02-10
+test-amd64-amd64-dom0pvh-xl-intel all_host_di_version 2020-02-10
+test-amd64-amd64-dom0pvh-xl-amd all_host_suite stretch +test-amd64-amd64-dom0pvh-xl-intel all_host_suite stretch +test-amd64-amd64-dom0pvh-xl-amd all_hostflags arch-amd64,arch-xen-amd64,suite-stretch,purpose-test,hvm-amd,iommu +test-amd64-amd64-dom0pvh-xl-intel all_hostflags arch-amd64,arch-xen-amd64,suite-stretch,purpose-test,hvm-intel,iommu +test-amd64-amd64-dom0pvh-xl-amd arch amd64 +test-amd64-amd64-dom0pvh-xl-intel arch amd64 +test-amd64-amd64-dom0pvh-xl-amd buildjob build-amd64 +test-amd64-amd64-dom0pvh-xl-intel buildjob build-amd64 +test-amd64-amd64-dom0pvh-xl-amd debian_arch amd64 +test-amd64-amd64-dom0pvh-xl-intel debian_arch amd64 +test-amd64-amd64-dom0pvh-xl-amd debian_kernkind pvops +test-amd64-amd64-dom0pvh-xl-intel debian_kernkind pvops +test-amd64-amd64-dom0pvh-xl-amd debian_suite stretch +test-amd64-amd64-dom0pvh-xl-intel debian_suite stretch +test-amd64-amd64-dom0pvh-xl-amd kernbuildjob build-amd64-pvops +test-amd64-amd64-dom0pvh-xl-intel kernbuildjob build-amd64-pvops +test-amd64-amd64-dom0pvh-xl-amd kernkind pvops +test-amd64-amd64-dom0pvh-xl-intel kernkind pvops +test-amd64-amd64-dom0pvh-xl-amd toolstack xl +test-amd64-amd64-dom0pvh-xl-intel toolstack xl +test-amd64-amd64-dom0pvh-xl-amd xen_boot_append dom0=pvh,verbose +test-amd64-amd64-dom0pvh-xl-intel xen_boot_append dom0=pvh,verbose
---
Changes since v1:
- Request hosts with iommu flag.
--- make-flight | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+)
make-flight +24 -0
@@ -753,6 +753,16 @@ test_matrix_do_one () { *) test_shim=y ;; esac + # PVH dom0 tests for versions >= 4.13 only + case "$xenbranch" in + xen-3.*-testing) test_dom0pvh=n ;; + xen-4.?-testing) test_dom0pvh=n ;; + xen-4.10-testing) test_dom0pvh=n ;; + xen-4.11-testing) test_dom0pvh=n ;; + xen-4.12-testing) test_dom0pvh=n ;; + *) test_dom0pvh=y ;; + esac + # xend PV guest test on x86 only if [ x$test_xend = xy -a \( $dom0arch = "i386" -o $dom0arch = "amd64" \) ]; then job_create_test test-$xenarch$kern-$dom0arch-pv test-debian xend \
@@ -861,6 +871,20 @@ test_matrix_do_one () { fi + if [ x$test_dom0pvh = xy -a $xenarch = amd64 -a $dom0arch = amd64 ]; then + + for cpuvendor in amd intel; do + + job_create_test test-$xenarch$kern-$dom0arch-dom0pvh-xl-$cpuvendor \ + test-debian xl $xenarch $dom0arch $debian_runvars \ + all_hostflags=$most_hostflags,hvm-$cpuvendor,iommu \ + xen_boot_append='dom0=pvh,verbose' + + done + + fi + + if [ x$test_shim = xy -a $xenarch = amd64 ]; then job_create_test test-$xenarch$kern-$dom0arch-xl-pvshim \
Signature · 7 lines
2.25.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Roger Pau Monne writes ("[PATCH OSSTEST v5 1/5] host: introduce modify_host"):
Quoted 5 lines
Abstract the
set_propertychecks and DB call into a helper.No functional change.
Signed-off-by Roger Pau Monné <[email protected]>
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel