xend: fix best NUMA node allocation
PATCH 1/1 xend: fix best NUMA node allocation
Hi,
since we moved several NUMA info fields from physinfo into separate functions/structures, we must adapt the node picking algorithm, too. Currently xm create complains about undefined hash values.
The patch uses the new Python xc binding to get the information and create a reverse mapping for node_to_cpu, since we now only have a cpu_to_node field.
Signature · 3 lines
Andre Przywara AMD-Operating System Research Center (OSRC), Dresden, Germany Tel: +49 351 448-3567-12
tools/python/xen/xend/XendDomainInfo.py +17 -12
@@ -2711,7 +2711,7 @@ else: def find_relaxed_node(node_list): import sys - nr_nodes = info['max_node_id']+1 + nr_nodes = info['max_node_index'] + 1 if node_list is None: node_list = range(0, nr_nodes) nodeload = [0]
@@ -2724,35 +2724,40 @@ if sxp.child_value(vcpu, 'online') == 0: continue cpumap = list(sxp.child_value(vcpu,'cpumap')) for i in range(0, nr_nodes): - node_cpumask = info['node_to_cpu'][i] + node_cpumask = node_to_cpu[i] for j in node_cpumask: if j in cpumap: nodeload[i] += 1 break for i in range(0, nr_nodes): - if len(info['node_to_cpu'][i]) == 0: + if len(node_to_cpu[i]) == 0: nodeload[i] += 8 else: - nodeload[i] = int(nodeload[i] * 16 / len(info['node_to_cpu'][i])) + nodeload[i] = int(nodeload[i] * 16 / len(node_to_cpu[i])) if i not in node_list: nodeload[i] += 8 return map(lambda x: x[0], sorted(enumerate(nodeload), key=lambda x:x[1])) - info = xc.physinfo() - if info['nr_nodes'] > 1: - node_memory_list = info['node_to_memory'] + info = xc.numainfo() + if info['max_node_index'] > 0: + node_memory_list = info['node_memfree'] + node_to_cpu = [] + for i in range(0, info['max_node_index'] + 1): + node_to_cpu.append([]) + for cpu, node in enumerate(xc.topologyinfo()['cpu_to_node']): + node_to_cpu[node].append(cpu) needmem = self.image.getRequiredAvailableMemory(self.info['memory_dynamic_max']) / 1024 candidate_node_list = [] - for i in range(0, info['max_node_id']+1): - if node_memory_list[i] >= needmem and len(info['node_to_cpu'][i]) > 0: + for i in range(0, info['max_node_index'] + 1): + if node_memory_list[i] >= needmem and len(node_to_cpu[i]) > 0: candidate_node_list.append(i) best_node = find_relaxed_node(candidate_node_list)[0] - cpumask = info['node_to_cpu'][best_node] - best_nodes = find_relaxed_node(filter(lambda x: x != best_node, range(0,info['max_node_id']+1))) + cpumask = node_to_cpu[best_node] + best_nodes = find_relaxed_node(filter(lambda x: x != best_node, range(0,info['max_node_index']+1))) for node_idx in best_nodes: if len(cpumask) >= self.info['VCPUs_max']: break - cpumask = cpumask + info['node_to_cpu'][node_idx] + cpumask = cpumask + node_to_cpu[node_idx] log.debug("allocating additional NUMA node %d", node_idx) for v in range(0, self.info['VCPUs_max']): xc.vcpu_setaffinity(self.domid, v, cpumask)
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel
Even with this patch applied I'm still seeing the cpu_to_node key errors in the xen-unstable build:
[2010-05-28 02:14:42 3313] ERROR (xmlrpclib2:178) Internal error handling xend.domain.create
Traceback (most recent call last):
File "/usr/local/lib/python2.6/dist-packages/xen/util/xmlrpclib2.py",
line 131, in _marshaled_dispatch
response = self._dispatch(method, params)
File "/usr/lib/python2.6/SimpleXMLRPCServer.py", line 418, in _dispatch
return func(*params)
File "/usr/local/lib/python2.6/dist-packages/xen/xend/server/XMLRPCServer.py",
line 80, in domain_create
info = XendDomain.instance().domain_create(config)
File "/usr/local/lib/python2.6/dist-packages/xen/xend/XendDomain.py",
line 982, in domain_create
dominfo = XendDomainInfo.create(config)
File "/usr/local/lib/python2.6/dist-packages/xen/xend/XendDomainInfo.py",
line 106, in create
vm.start()
File "/usr/local/lib/python2.6/dist-packages/xen/xend/XendDomainInfo.py",
line 470, in start
XendTask.log_progress(0, 30, self._constructDomain)
File "/usr/local/lib/python2.6/dist-packages/xen/xend/XendTask.py",
line 209, in log_progress
retval = func(*args, **kwds)
File "/usr/local/lib/python2.6/dist-packages/xen/xend/XendDomainInfo.py",
line 2530, in _constructDomain
balloon.free(16*1024, self) # 16MB should be plenty
File "/usr/local/lib/python2.6/dist-packages/xen/xend/balloon.py",
line 187, in free
nodenum = xc.numainfo()['cpu_to_node'][cpu]KeyError: 'cpu_to_node'
xen_major : 4xen_minor : 1xen_extra : -unstablexen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32
hvm-3.0-x86_32p hvm-3.0-x86_64xen_scheduler : creditxen_pagesize : 4096platform_params : virt_start=0xffff800000000000xen_changeset : Sat May 22 06:36:41 2010 +0100 21446:93410e5e4ad8xen_commandline : dummy=dummy console=com1 115200,8n1
dom0_mem=512M dom0_max_vcpus=1 dom0_vcpus_pin=true iommu=1,passthrough,no-intremap loglvl=all loglvl_guest=all loglevl=10 debug acpi=force apic=on apic_verbosity=verbosecc_compiler : gcc version 4.3.3 (Ubuntu 4.3.3-5ubuntu4)cc_compile_by : bedge
cc_compile_domain : cc_compile_date : Tue May 25 14:51:02 PDT 2010 xend_config_format : 4
-Bruce
Quoted 19 lines · Andre Przywara
Hi,
since we moved several NUMA info fields from physinfo into separate functions/structures, we must adapt the node picking algorithm, too. Currently xm create complains about undefined hash values.
The patch uses the new Python xc binding to get the information and create a reverse mapping fornode_to_cpu, since we now only have acpu_to_nodefield.Signed-off-by Andre Przywara <[email protected]>--
Andre Przywara
AMD-Operating System Research Center (OSRC), Dresden, Germany
Tel: +49 351 448-3567-12_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel
Quoted 2 lines · Bruce Edge
Even with this patch applied I'm still seeing the
cpu_to_nodekey errors in the xen-unstable build:
This should be fixed by xen-unstable:21494.
Thanks,
Keir