Skip to contentSkip to conversation

Setting dom0-cpus to 0 doesn't use all cpus asdocumented

PATCH Applied Carb, Brian A 3 participants 4 messages May 9, 2006 – May 11, 2006

lore
Share & tools
lore
#0 lore raw

We discovered this when running the xm-test suite. One of the tests (01_enforce_dom0_cpus) changes the number of cpus by editing the xend-config.sxp file and restarting xend. Should this test be updated to use the method Ian suggested, to vary the number of cpus, rather than restarting xend?

brian carb (610-648-2437 or N2 385-2437)
unisys cmp technology lab
malvern, pa
[email protected]

THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.

-----Original Message-----
From: Ryan Harper [mailto:[email protected]]
Sent: Tuesday, May 09, 2006 11:45 AM
To: Ian Pratt
Cc: Carb, Brian A; [email protected]; Subrahmanian, Raj; Krysan, Susan; Puthiyaparambil, Aravindh; Vessey, Bruce A
Subject: Re: [Xen-devel] Setting dom0-cpus to 0 doesn't use all cpus asdocumented

Quoted 19 lines · Ian Pratt
Quoted 19 lines

The comments for dom0-cpus in the xend-config.sxp file indicate that

setting this value to 0 in an SMP system will cause dom0 to use all the available CPUs. However, it appears as if setting this value to 0 does not change the number of CPUs at all.

I'm running SLES10 Beta 11 and xen-unstable changeset 9960 on a Unisys ES7000 with 32 CPUs and 32GB memory. In xend-config.sxp, dom0-cpus is set to 0. When xen dom0 boots, it is using all 32 procs

(as reported by 'grep processor /proc/cpuinfo | wc -l').

If I edit the xend-config.sxp file, change the value of dom0-cpus to

1, and restart xend, dom0 correctly uses 1 CPUs.
If I then change dom0-cpus to 0 and restart xend, dom0 still uses 1 cpu. The only way to have dom0 use 32 cpus again is to either set the value of dom0-cpus to 32, or reboot xen.

Quoted 5 lines
Quoted 1 line

Has anyone else seen this behavior?

You can just use the normal xm commands to vary the number of CPUs. Restarting xend is a rather brutal way of trying to achieve the same thing...

Is it worth keeping dom0-cpus then?

--
Ryan Harper
Software Engineer; Linux Technology Center IBM Corp., Austin, Tx
(512) 838-9253 T/L: 678-9253
[email protected]

#1 lore raw
Quoted 5 lines · Carb, Brian A

We discovered this when running the xm-test suite. One of the tests (01_enforce_dom0_cpus) changes the number of cpus by editing the xend-config.sxp file and restarting xend. Should this test be updated to use the method Ian suggested, to vary the number of cpus, rather than restarting xend?

As Ian said, it is for setting the value when xend starts which is what the test is about. However, the test should bring dom0 vcpus to the max and then test enforcement down to 1 rather than assuming that dom0's current value can be reduced. Or, bail on the test if the current value of dom0's online vcpus isn't > 2.

Signature · 5 lines
Ryan Harper
Software Engineer; Linux Technology Center
IBM Corp., Austin, Tx
(512) 838-9253   T/L: 678-9253
[email protected]
#2 lore raw

PATCH 1/1 xm-test: update enforce_dom0_cpus testcase

I chose to update the test to SKIP if dom0 isn't proper configured which at this point means SMP host and dom0 with at least 2 VCPUs online. Also I noticed that the test was still using vcpu-enable which is no longer available via xm; we use vcpu-set. This patch fixes the issue where after successfully running the test, dom0's vcpu state was not restored. Updated some comments and lines that exceeded 80 columns.

Signature · 122 lines
Ryan Harper
Software Engineer; Linux Technology Center
IBM Corp., Austin, Tx
(512) 838-9253   T/L: 678-9253
[email protected]


diffstat output:
 01_enforce_dom0_cpus_basic_pos.py |   55 ++++++++++++++++++++++++--------------
 1 files changed, 36 insertions(+), 19 deletions(-)

Signed-off-by: Ryan Harper <[email protected]>
---
# HG changeset patch
# User Ryan Harper <[email protected]>
# Node ID 279b674363bc78a284284bd62b9a0e2b5c65acb6
# Parent  1e3977e029fddc1e53995a3502d590f8a07e5c62
Fix up enforce_dom0_cpu test case to check that dom0 has at least 2 vcpus
online.  Also fix bogus call to vcpu-enable; xm now uses vcpu-set as method for
enable/disable of vcpus.  Previously after a successful test run, this test
would not leave dom0 in the same state as it was when the test started.  Also
some comment language changes and 80 column formatting.

diff -r 1e3977e029fd -r 279b674363bc tools/xm-test/tests/enforce_dom0_cpus/01_enforce_dom0_cpus_basic_pos.py
--- a/tools/xm-test/tests/enforce_dom0_cpus/01_enforce_dom0_cpus_basic_pos.py	Mon May  8 18:21:41 2006
+++ b/tools/xm-test/tests/enforce_dom0_cpus/01_enforce_dom0_cpus_basic_pos.py	Tue May  9 19:19:47 2006
@@ -4,20 +4,23 @@
 # Authors: Dan Smith <[email protected]>
 #          Ryan Harper <[email protected]>
 
-# 1) Make sure we have a multi cpu system
+# 1) Make sure we have a multi cpu system and dom0 has at 
+#    least 2 vcpus online.
 # 2) clone standard config (/etc/xen/xend-config.sxp) 
 # 3) modify clone with enforce_dom0_cpus=X
 # 4) restart xend with modified config
 # 5) check /proc/cpuinfo for cpu count
-# 6) check xm list -v to see that only 1 cpu is online for dom0
-# 7) Restart xend with default config
+# 6) check xm info 'VCPUs' field to see that only 'enforce_dom0_cpus' 
+#    number of cpus are online in dom0
+# 7) Restore initial dom0 vcpu state
+# 8) Restart xend with default config
 
 import sys
 import re
 import time
 import os
 
-# what value should dom0_cpus be enforced?
+# what value should dom0_cpus enforce?
 enforce_dom0_cpus=1
 
 from XmTestLib import *
@@ -25,12 +28,19 @@
 check_status = 1
 max_tries = 10
 
-# 1) Make sure we have a multi cpu system
+# 1) Make sure we have a multi cpu system and dom0 has at least 2 vcpus online.
 
 if smpConcurrencyLevel() <= 1:
     print "*** NOTE: This machine does not have more than one physical"
     print "          or logical cpu.  The vcpu-disable test cannot be run!"
     SKIP("Host not capable of running test")
+
+# count number of online vcpus in dom0
+dom0_online_vcpus = int(getDomInfo("Domain-0", "VCPUs"))
+if dom0_online_vcpus <= 1:
+    print "*** NOTE: DOM0 needs at least 2 VCPUs online to run this test"
+    print "          Please enable additional vcpus if possible via xm vcpu-set"
+    SKIP("Host state not capable of running test")
     
 # 2) clone standard config (/etc/xen/xend-config.sxp) 
 # 3) modify clone with enforce_dom0_cpus=1
@@ -65,26 +75,33 @@
 if output != str(enforce_dom0_cpus):
     os.unsetenv("XEND_CONFIG")
     restartXend()
-    FAIL("/proc/cpuinfo says xend didn't enforce dom0_cpus (%s != %s)" %(output, enforce_dom0_cpus))
+    FAIL("/proc/cpuinfo says xend didn't enforce dom0_cpus (%s != %s)"%(output, 
+                                                             enforce_dom0_cpus))
 
-# 7) count number of online cpus and see that it matches enforce value
-dom0vcpus = getVcpuInfo("Domain-0")
-num_online = len(filter(lambda x: x >= 0, dom0vcpus.values()))
+# 6) count number of online cpus and see that it matches enforce value
+num_online = int(getDomInfo("Domain-0", "VCPUs"))
 if num_online != enforce_dom0_cpus:
     os.unsetenv("XEND_CONFIG")
     restartXend()
-    FAIL("xm says xend didn't enforce dom0_cpus (%s != %s)" %(num_online, enforce_dom0_cpus))
+    FAIL("xm says xend didn't enforce dom0_cpus (%s != %s)" %(num_online, 
+                                                             enforce_dom0_cpus))
 
-# restore dead processors 
-for (k,v) in zip(dom0vcpus.keys(),dom0vcpus.values()):
-    if v == -1:
-        status, output = traceCommand("xm vcpu-enable 0 %s"%(k))
-        if check_status and status != 0:
-            os.unsetenv("XEND_CONFIG")
-            restartXend()
-            FAIL("\"%s\" returned invalid %i != 0" %(cmd,status))
+# 7) restore dead processors 
+status, output = traceCommand("xm vcpu-set 0 %s"%(dom0_online_vcpus))
+if check_status and status != 0:
+    os.unsetenv("XEND_CONFIG")
+    restartXend()
+    FAIL("\"%s\" returned invalid %i != 0" %(cmd,status))
 
-# Restart xend with default config
+# check restore worked
+num_online = int(getDomInfo("Domain-0", "VCPUs"))
+if num_online != dom0_online_vcpus:
+    os.unsetenv("XEND_CONFIG")
+    restartXend()
+    FAIL("failed to restore dom0's VCPUs")
+
+
+# 8) Restart xend with default config
 os.unsetenv("XEND_CONFIG")
 restartXend()
Applied
#3 lore raw

Re: [PATCH] xm-test: update enforce_dom0_cpus testcase

Quoted 6 lines · Ryan Harper

I chose to update the test to SKIP if dom0 isn't proper configured which at this point means SMP host and dom0 with at least 2 VCPUs online. Also I noticed that the test was still using vcpu-enable which is no longer available via xm; we use vcpu-set. This patch fixes the issue where after successfully running the test, dom0's vcpu state was not restored. Updated some comments and lines that exceeded 80 columns.

Applied, thanks.

Ewan.