xend: Add multiple cpumasks support
PATCH 2/3 xend: Add multiple cpumasks support
This patch modifies xend to accept and parse multiple cpumask strings from the cpus parameter. The cpus string stays the same, but it now can parse either a python list of strings:
[ '2-5, '2-5', '2-5' ]
A regular string with ", " as the separator:
"2-5, 2-5, 2-5, 2-5"
or a mixture of both:
[ 2-5, '2-5', 2-5, '2-5' ]
all result in the same list of integers which is used to create the cpumask for each vcpu in the domain.
Without this patch the cpus parameter only enables one bit of the cpumask preventing the credit scheduler from balancing.
Signature · 177 lines
Ryan Harper Software Engineer; Linux Technology Center IBM Corp., Austin, Tx (512) 838-9253 T/L: 678-9253 [email protected] diffstat output: examples/xmexample.hvm | 3 + examples/xmexample.vti | 1 examples/xmexample1 | 3 + examples/xmexample2 | 3 + examples/xmexample3 | 3 + python/xen/xend/XendDomainInfo.py | 66 +++++++++++++++++++++++++------------- 6 files changed, 53 insertions(+), 26 deletions(-) Signed-off-by: Ryan Harper <[email protected]> --- diff -r e6de470f9287 tools/python/xen/xend/XendDomainInfo.py --- a/tools/python/xen/xend/XendDomainInfo.py Sat Aug 12 08:22:45 2006 -0500 +++ b/tools/python/xen/xend/XendDomainInfo.py Sat Aug 12 08:22:47 2006 -0500 @@ -328,27 +328,49 @@ def parseConfig(config): else: result['cpus'] = str(result['cpu']) - # convert 'cpus' string to list of ints - # 'cpus' supports a list of ranges (0-3), seperated by - # commas, and negation, (^1). - # Precedence is settled by order of the string: - # "0-3,^1" -> [0,2,3] - # "0-3,^1,1" -> [0,1,2,3] - if result['cpus']: - cpus = [] - for c in result['cpus'].split(','): - if c.find('-') != -1: - (x,y) = c.split('-') - for i in range(int(x),int(y)+1): - cpus.append(int(i)) - else: - # remove this element from the list - if c[0] == '^': - cpus = [x for x in cpus if x != int(c[1:])] + if result['cpus'] is not None: + # see if cpus string specifies multiple cpumasks, or just one + # e.g: "[ '2-5', '0-1', '0-3,^1', '5' ]" vs. "0-3,5" + if result['cpus'].startswith("["): + # the below was tested with the following sample string + # "[4, '2-5','2-5', '1-3,6,^2', '2-6','1' 1-7,^2 ]" and resulted in + # ['4', '2-5', '2-5', '1-3,6,^2', '2-6', '1'] + result['cpus'] = filter(lambda x: len(x), map(lambda x: x.strip(", "), + result['cpus'].replace('[',"").replace(']',"").split("'"))) + + # if user didn't use list bracket, convert to to list of strings + # cpus = "4, 2-5,^4, 1-3,5,^2, 1, ^2" -> + # ['4', '2-5,^4', '1-3,5,^2', '1', '^2'] + # also takes care of cpus = "4" + else: + result['cpus'] = map(lambda x: x.strip(", "), result['cpus'].split()) + + # convert 'cpus' list of strings into a list of list of ints + # 'cpus' supports a list of ranges (0-3), seperated by + # commas, and negation, (^1). + # Precedence is settled by order of the string: + # "0-3,^1" -> [0,2,3] + # "0-3,^1,1" -> [0,1,2,3] + + new_cpus = [] + for x in result['cpus']: + cpus = [] + for c in x.split(','): + if c.find('-') != -1: + (x,y) = c.split('-') + for i in range(int(x),int(y)+1): + cpus.append(int(i)) else: - cpus.append(int(c)) - - result['cpus'] = cpus + # remove this element from the list + if c[0] == '^': + cpus = [x for x in cpus if x != int(c[1:])] + else: + cpus.append(int(c)) + + new_cpus.append(cpus) + + + result['cpus'] = new_cpus except ValueError, exn: raise VmError( @@ -1275,8 +1297,8 @@ class XendDomainInfo: cpus = self.info['cpus'] if cpus is not None and len(cpus) > 0: for v in range(0, self.info['max_vcpu_id']+1): - # pincpu takes a list of ints - cpu = [ int( cpus[v % len(cpus)] ) ] + # pincpu takes a list of ints, + cpu = map(lambda x: int(x), cpus[v % len(cpus)]) xc.vcpu_setaffinity(self.domid, v, cpu) # set domain maxmem in KiB diff -r e6de470f9287 tools/examples/xmexample.hvm --- a/tools/examples/xmexample.hvm Sat Aug 12 08:22:45 2006 -0500 +++ b/tools/examples/xmexample.hvm Sat Aug 12 08:34:13 2006 -0500 @@ -47,10 +47,11 @@ name = "ExampleHVMDomain" # enable/disable HVM guest APIC, default=0 (disabled) #apic=0 -# List of which CPUS this domain is allowed to use, default Xen picks +# List of which CPUS vcpus are allowed to use, default Xen picks #cpus = "" # leave to Xen to pick #cpus = "0" # all vcpus run on CPU0 #cpus = "0-3,5,^1" # run on cpus 0,2,3,5 +#cpus = "0-1, 2-3" # run VCPU0 on CPU0-1, VCPU1 on CPU2-3 # Optionally define mac and/or bridge for the network interfaces. # Random MACs are assigned if not given. diff -r e6de470f9287 tools/examples/xmexample.vti --- a/tools/examples/xmexample.vti Sat Aug 12 08:22:45 2006 -0500 +++ b/tools/examples/xmexample.vti Sat Aug 12 08:39:42 2006 -0500 @@ -34,6 +34,7 @@ name = "ExampleVTIDomain" #cpus = "" # leave to Xen to pick #cpus = "0" # all vcpus run on CPU0 #cpus = "0-3,5,^1" # run on cpus 0,2,3,5 +#cpus = "0-1, 2-3" # run VCPU0 on CPU0-1, VCPU1 on CPU2-3 # Optionally define mac and/or bridge for the network interfaces. # Random MACs are assigned if not given. diff -r e6de470f9287 tools/examples/xmexample1 --- a/tools/examples/xmexample1 Sat Aug 12 08:22:45 2006 -0500 +++ b/tools/examples/xmexample1 Sat Aug 12 08:27:02 2006 -0500 @@ -30,10 +30,11 @@ name = "ExampleDomain" # on each call to 'xm create'. #uuid = "06ed00fe-1162-4fc4-b5d8-11993ee4a8b9" -# List of which CPUS this domain is allowed to use, default Xen picks +# List of which CPUS vcpus are allowed to use, default Xen picks #cpus = "" # leave to Xen to pick #cpus = "0" # all vcpus run on CPU0 #cpus = "0-3,5,^1" # run on cpus 0,2,3,5 +#cpus = "0-1, 2-3" # run VCPU0 on CPU0-1, VCPU1 on CPU2-3 # Number of Virtual CPUS to use, default is 1 #vcpus = 1 diff -r e6de470f9287 tools/examples/xmexample2 --- a/tools/examples/xmexample2 Sat Aug 12 08:22:45 2006 -0500 +++ b/tools/examples/xmexample2 Sat Aug 12 08:27:47 2006 -0500 @@ -59,10 +59,11 @@ name = "VM%d" % vmid # on each call to 'xm create'. #uuid = "06ed00fe-1162-4fc4-b5d8-11993ee4a8b9" -# List of which CPUS this domain is allowed to use, default Xen picks +# List of which CPUS vcpus are allowed to use, default Xen picks #cpus = "" # leave to Xen to pick #cpus = "0" # all vcpus run on CPU0 #cpus = "0-3,5,^1" # run on cpus 0,2,3,5 +#cpus = "0-1, 2-3" # run VCPU0 on CPU0-1, VCPU1 on CPU2-3 #cpus = "%s" % vmid # set based on vmid (mod number of CPUs) # Number of Virtual CPUS to use, default is 1 diff -r e6de470f9287 tools/examples/xmexample3 --- a/tools/examples/xmexample3 Sat Aug 12 08:22:45 2006 -0500 +++ b/tools/examples/xmexample3 Sat Aug 12 08:33:35 2006 -0500 @@ -59,10 +59,11 @@ name = "VM%d" % vmid # on each call to 'xm create'. #uuid = "06ed00fe-1162-4fc4-b5d8-11993ee4a8b9" -# List of which CPUS this domain is allowed to use, default Xen picks +# List of which CPUS vcpus are allowed to use, default Xen picks #cpus = "" # leave to Xen to pick #cpus = "0" # all vcpus run on CPU0 #cpus = "0-3,5,^1" # run on cpus 0,2,3,5 +#cpus = "0-1, 2-3" # run VCPU0 on CPU0-1, VCPU1 on CPU2-3 cpus = "%s" % vmid # set based on vmid (mod number of CPUs) #----------------------------------------------------------------------------
Quoted 6 lines · Ryan Harper
This patch modifies xend to accept and parse multiple cpumask strings from the cpus parameter. The cpus string stays the same, but it now can parse either a python list of strings:
[ '2-5, '2-5', '2-5' ]
Quoted 7 lines
A regular string with ", " as the separator:
"2-5, 2-5, 2-5, 2-5"
or a mixture of both:
[ 2-5, '2-5', 2-5, '2-5' ]
This isn't really a mixture of both, is it? It looks syntactically incorrect (e.g., first 2-5 is unquoted so not a string).
I'm not sure about the use of ', ' as a delimiter. It would be less confusing to strictly require the use of the list form. I would imagine it's then clearest used as:
Cpus = []
Cpus[0] = '2-5'
Cpus[1] = '3-4'
...
Trying to read long lists of cpu constraints with spaces in will make people go cross-eyed!
-- Keir
Quoted 2 lines
all result in the same list of integers which is used to create the cpumask for each vcpu in the domain.
Quoted 10 lines · Keir Fraser
Quoted 8 lines · Ryan Harper
This patch modifies xend to accept and parse multiple cpumask strings from the cpus parameter. The cpus string stays the same, but it now can parse either a python list of strings:
[ '2-5, '2-5', '2-5' ]
A regular string with ", " as the separator:
Quoted 8 lines
Quoted 5 lines
"2-5, 2-5, 2-5, 2-5"
or a mixture of both:
[ 2-5, '2-5', 2-5, '2-5' ]
This isn't really a mixture of both, is it? It looks syntactically incorrect (e.g., first 2-5 is unquoted so not a string).
create.py ends up passing the whole thing as a string even if you mark it up as a list with strings embedded which is why I can mix it. If you like, we can drop support for dealing with the [ ... ] form and just use cpus = ""
Quoted 8 lines
I'm not sure about the use of ', ' as a delimiter. It would be less confusing to strictly require the use of the list form. I would imagine it's then clearest used as:
Cpus = []
Cpus[0] = '2-5'
Cpus[1] = '3-4'
...
I'd prefer to not have to use any language arrays. I'm not sure how this will map to the xml-based config files that Ewan was talking about, but I suppose he will have to come up with something since we have things like the disk parameter which is in python list format.
Quoted 3 lines
Trying to read long lists of cpu constraints with spaces in will make people go cross-eyed!
I agree. The point was that I'm attempting to parse as much as I can make sense of, not to indicate how we should tell the user to convey this information. I also like being able to retain the cpu=, cpus=, previous syntax while extending it to support multiple cpumasks.
I think the following (which I added in the example files) is good enough without having to resort to arrays/dictionaries.
cpus = "2-5, 2-5, 2-5, 2-5"
I'm open to other field delimiters if that is a point of contention.
Signature · 5 lines
Ryan Harper Software Engineer; Linux Technology Center IBM Corp., Austin, Tx (512) 838-9253 T/L: 678-9253 [email protected]
Quoted 9 lines · Ryan Harper
Quoted 8 lines
I'm not sure about the use of ', ' as a delimiter. It would be less confusing to strictly require the use of the list form. I would imagine it's then clearest used as:
Cpus = []
Cpus[0] = '2-5'
Cpus[1] = '3-4'
...
Quoted 4 lines
I'd prefer to not have to use any language arrays. I'm not sure how this will map to the xml-based config files that Ewan was talking about, but I suppose he will have to come up with something since we have things like the disk parameter which is in python list format.
I think that'll be a different format again. Perhaps a separate cpu-info section of the file for each VCPU, within which we specify the affinity info. I'm pretty sure we won't go for a single string with whitespace delimiters!
Anyhow, the current patch does have flexibility in its favour, since it supports the list format *and* the string format. So I guess it's okay really.
-- Keir