Skip to contentSkip to conversation

dynamic nr online cpus

PATCH Applied Jes Sorensen 3 participants 5 messages Jan 13, 2009 – Jan 22, 2009

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 dynamic nr online cpus

Hi,

Working on allowing larger max vcpus I'd like to put in this patch that calculates the number of online cpus. It's a building block for allowing the larger number of max vcpus without crippling performance for the smaller cases.

Cheers,
Jes

Account for number of online cpus and use that in loops iterating over the list of vpus. This patch is a building block, in the work to allow for larger max number of vcpus.

A copy of the number of online cpus is stored in vcpu 0's private data area to allow the number to be visible from the KVM module.

---
 arch/ia64/include/asm/kvm_host.h |    3 +++
 arch/ia64/kvm/kvm-ia64.c         |   20 ++++++++++++++++----
 arch/ia64/kvm/vcpu.c             |    2 +-
 3 files changed, 20 insertions(+), 5 deletions(-)
arch/ia64/include/asm/kvm_host.h +3 -0
Index: linux-2.6.git/arch/ia64/include/asm/kvm_host.h
===================================================================
@@ -377,6 +377,7 @@
 	int last_run_cpu;
 	int vmm_tr_slot;
 	int vm_tr_slot;
+	int online_cpus;
 
 #define KVM_MP_STATE_RUNNABLE          0
 #define KVM_MP_STATE_UNINITIALIZED     1
@@ -470,6 +471,8 @@
 	unsigned long	metaphysical_rr4;
 	unsigned long	vmm_init_rr;
 
+	int		online_cpus;
+
 	struct kvm_ioapic *vioapic;
 	struct kvm_vm_stat stat;
 	struct kvm_sal_data rdv_sal_data;
arch/ia64/kvm/kvm-ia64.c +16 -4
Index: linux-2.6.git/arch/ia64/kvm/kvm-ia64.c
===================================================================
@@ -316,7 +316,7 @@
 	union ia64_lid lid;
 	int i;
 
-	for (i = 0; i < KVM_MAX_VCPUS; i++) {
+	for (i = 0; i < kvm->arch.online_cpus; i++) {
 		if (kvm->vcpus[i]) {
 			lid.val = VCPU_LID(kvm->vcpus[i]);
 			if (lid.id == id && lid.eid == eid)
@@ -390,7 +390,7 @@
 
 	call_data.ptc_g_data = p->u.ptc_g_data;
 
-	for (i = 0; i < KVM_MAX_VCPUS; i++) {
+	for (i = 0; i < kvm->arch.online_cpus; i++) {
 		if (!kvm->vcpus[i] || kvm->vcpus[i]->arch.mp_state ==
 						KVM_MP_STATE_UNINITIALIZED ||
 					vcpu == kvm->vcpus[i])
@@ -825,6 +825,8 @@
 		return ERR_PTR(-ENOMEM);
 	kvm_init_vm(kvm);
 
+	kvm->arch.online_cpus = 0; /* xxx hack me harder xxx */
+
 	return kvm;
 
 }
@@ -1186,7 +1188,7 @@
 
 		/*Initialize itc offset for vcpus*/
 		itc_offset = 0UL - ia64_getreg(_IA64_REG_AR_ITC);
-		for (i = 0; i < KVM_MAX_VCPUS; i++) {
+		for (i = 0; i < kvm->arch.online_cpus; i++) {
 			v = (struct kvm_vcpu *)((char *)vcpu +
 					sizeof(struct kvm_vcpu_data) * i);
 			v->arch.itc_offset = itc_offset;
@@ -1320,6 +1322,16 @@
 		goto fail;
 	}
 
+	kvm->arch.online_cpus++;
+	/*
+	 * For vcpu 0, kvm->vcpus hasn't been assigned yet, special case it
+	 */
+	if (!id)
+		vcpu->arch.online_cpus = kvm->arch.online_cpus;
+	else
+		kvm->vcpus[0]->arch.online_cpus = kvm->arch.online_cpus;
+
+	printk(KERN_INFO "arch.online_cpus %i\n", kvm->arch.online_cpus);
 	return vcpu;
 fail:
 	return ERR_PTR(r);
@@ -1766,7 +1778,7 @@
 	struct kvm_vcpu *lvcpu = kvm->vcpus[0];
 	int i;
 
-	for (i = 1; i < KVM_MAX_VCPUS; i++) {
+	for (i = 1; i < kvm->arch.online_cpus; i++) {
 		if (!kvm->vcpus[i])
 			continue;
 		if (lvcpu->arch.xtp > kvm->vcpus[i]->arch.xtp)
arch/ia64/kvm/vcpu.c +1 -1
Index: linux-2.6.git/arch/ia64/kvm/vcpu.c
===================================================================
@@ -816,7 +816,7 @@
 	unsigned long vitv = VCPU(vcpu, itv);
 
 	if (vcpu->vcpu_id == 0) {
-		for (i = 0; i < KVM_MAX_VCPUS; i++) {
+		for (i = 0; i < vcpu->arch.online_cpus; i++) {
 			v = (struct kvm_vcpu *)((char *)vcpu +
 					sizeof(struct kvm_vcpu_data) * i);
 			VMX(v, itc_offset) = itc_offset;
#1 lore raw

Hi, Jes
This patch is okay to me, and still have some minor comments below, and expect them to address in your final patch. Thanks!

1. For readability, I prefer to use online_vcpus intead of online_cpus
2. I don't think it is necessary to save a copy in vcpu's private data, just use the following code to get the kvm pointer and access its non-pointer fileds in vmm module,

    struct kvm *get_kvm() {
         return (struct kvm*) VM_BASE; 
    }

    After getting kvm pointer, you can use it for vcpu.c's change.

Xiantao

Quoted 9 lines · Jes Sorensen

Hi,

Working on allowing larger max vcpus I'd like to put in this patch that calculates the number of online cpus. It's a building block for allowing the larger number of max vcpus without crippling performance for the smaller cases.

Cheers,
Jes

#2 lore raw
In reply to Jes Sorensen

Hi Xiantao,

Here's an update version of the online_vcpus patch. I agree with you that the name online_vcpus is more appropriate and I used your trick to get to struct kvm, so it's a lot cleaner than the old version.

I renamed VM_BASE to KVM_VM_BASE as it seems a more correct name to me and has less chance of clashing with something. In addition the parenthesis I added are required to avoid unexpected results from CPP when you typecast it to struct kvm *.

Cheers,
Jes

Quoted 5 lines · Xiantao

Hi, Jes
This patch is okay to me, and still have some minor comments below, and expect them to address in your final patch. Thanks!

1. For readability, I prefer to use online_vcpus intead of online_cpus
2. I don't think it is necessary to save a copy in vcpu's private data, just use the following code to get the kvm pointer and access its non-pointer fileds in vmm module,

Quoted 17 lines
    struct kvm *get_kvm() {
         return (struct kvm*) VM_BASE; 
    }

    After getting kvm pointer, you can use it for vcpu.c's change.

Xiantao

Quoted 9 lines · Jes Sorensen

Hi,

Working on allowing larger max vcpus I'd like to put in this patch that calculates the number of online cpus. It's a building block for allowing the larger number of max vcpus without crippling performance for the smaller cases.

Cheers,
Jes

Account for number of online cpus and use that in loops iterating over the list of vpus instead of scanning the full array unconditionally. This patch is a building block to facilitate allowing to bump up the size of MAX_VCPUS significantly.

---
 arch/ia64/include/asm/kvm_host.h |   10 ++++++----
 arch/ia64/kvm/kvm-ia64.c         |   12 ++++++++----
 arch/ia64/kvm/vcpu.c             |    5 ++++-
 3 files changed, 18 insertions(+), 9 deletions(-)
arch/ia64/include/asm/kvm_host.h +6 -4
Index: linux-2.6.git/arch/ia64/include/asm/kvm_host.h
===================================================================
@@ -163,10 +163,10 @@
 	struct kvm_vcpu_data vcpu_data[KVM_MAX_VCPUS];
 };
 
-#define VCPU_BASE(n)	KVM_VM_DATA_BASE + \
-				offsetof(struct kvm_vm_data, vcpu_data[n])
-#define VM_BASE		KVM_VM_DATA_BASE + \
-				offsetof(struct kvm_vm_data, kvm_vm_struct)
+#define VCPU_BASE(n)	(KVM_VM_DATA_BASE + \
+				offsetof(struct kvm_vm_data, vcpu_data[n]))
+#define KVM_VM_BASE	(KVM_VM_DATA_BASE + \
+				offsetof(struct kvm_vm_data, kvm_vm_struct))
 #define KVM_MEM_DIRTY_LOG_BASE	KVM_VM_DATA_BASE + \
 				offsetof(struct kvm_vm_data, kvm_mem_dirty_log)
 
@@ -479,6 +479,8 @@
 	unsigned long	metaphysical_rr4;
 	unsigned long	vmm_init_rr;
 
+	int		online_vcpus;
+
 	struct kvm_ioapic *vioapic;
 	struct kvm_vm_stat stat;
 	struct kvm_sal_data rdv_sal_data;
arch/ia64/kvm/kvm-ia64.c +8 -4
Index: linux-2.6.git/arch/ia64/kvm/kvm-ia64.c
===================================================================
@@ -316,7 +316,7 @@
 	union ia64_lid lid;
 	int i;
 
-	for (i = 0; i < KVM_MAX_VCPUS; i++) {
+	for (i = 0; i < kvm->arch.online_vcpus; i++) {
 		if (kvm->vcpus[i]) {
 			lid.val = VCPU_LID(kvm->vcpus[i]);
 			if (lid.id == id && lid.eid == eid)
@@ -390,7 +390,7 @@
 
 	call_data.ptc_g_data = p->u.ptc_g_data;
 
-	for (i = 0; i < KVM_MAX_VCPUS; i++) {
+	for (i = 0; i < kvm->arch.online_vcpus; i++) {
 		if (!kvm->vcpus[i] || kvm->vcpus[i]->arch.mp_state ==
 						KVM_MP_STATE_UNINITIALIZED ||
 					vcpu == kvm->vcpus[i])
@@ -825,6 +825,8 @@
 		return ERR_PTR(-ENOMEM);
 	kvm_init_vm(kvm);
 
+	kvm->arch.online_vcpus = 0;
+
 	return kvm;
 
 }
@@ -1186,7 +1188,7 @@
 
 		/*Initialize itc offset for vcpus*/
 		itc_offset = 0UL - ia64_getreg(_IA64_REG_AR_ITC);
-		for (i = 0; i < KVM_MAX_VCPUS; i++) {
+		for (i = 0; i < kvm->arch.online_vcpus; i++) {
 			v = (struct kvm_vcpu *)((char *)vcpu +
 					sizeof(struct kvm_vcpu_data) * i);
 			v->arch.itc_offset = itc_offset;
@@ -1320,6 +1322,8 @@
 		goto fail;
 	}
 
+	kvm->arch.online_vcpus++;
+
 	return vcpu;
 fail:
 	return ERR_PTR(r);
@@ -1766,7 +1770,7 @@
 	struct kvm_vcpu *lvcpu = kvm->vcpus[0];
 	int i;
 
-	for (i = 1; i < KVM_MAX_VCPUS; i++) {
+	for (i = 1; i < kvm->arch.online_vcpus; i++) {
 		if (!kvm->vcpus[i])
 			continue;
 		if (lvcpu->arch.xtp > kvm->vcpus[i]->arch.xtp)
arch/ia64/kvm/vcpu.c +4 -1
Index: linux-2.6.git/arch/ia64/kvm/vcpu.c
===================================================================
@@ -811,12 +811,15 @@
 static void vcpu_set_itc(struct kvm_vcpu *vcpu, u64 val)
 {
 	struct kvm_vcpu *v;
+	struct kvm *kvm;
 	int i;
 	long itc_offset = val - ia64_getreg(_IA64_REG_AR_ITC);
 	unsigned long vitv = VCPU(vcpu, itv);
 
+	kvm = (struct kvm *)KVM_VM_BASE;
+
 	if (vcpu->vcpu_id == 0) {
-		for (i = 0; i < KVM_MAX_VCPUS; i++) {
+		for (i = 0; i < kvm->arch.online_vcpus; i++) {
 			v = (struct kvm_vcpu *)((char *)vcpu +
 					sizeof(struct kvm_vcpu_data) * i);
 			VMX(v, itc_offset) = itc_offset;
Acked-by on v1 1/1
#3 lore raw
In reply to Jes Sorensen

The version is Okay to me. Thanks for your work!:)

Avi,
Please help to applied it, Thanks!

-----Original Message-----
From: Jes Sorensen [mailto:[email protected]]
Sent: Wednesday, January 21, 2009 10:17 PM
To: Zhang, Xiantao
Cc: [email protected]
Subject: Re: [patch] dynamic nr online cpus

Hi Xiantao,

Here's an update version of the online_vcpus patch. I agree with you that the name online_vcpus is more appropriate and I used your trick to get to struct kvm, so it's a lot cleaner than the old version.

I renamed VM_BASE to KVM_VM_BASE as it seems a more correct name to me and has less chance of clashing with something. In addition the parenthesis I added are required to avoid unexpected results from CPP when you typecast it to struct kvm *.

Cheers,
Jes

Quoted 22 lines · Xiantao

Hi, Jes
This patch is okay to me, and still have some minor comments below, and expect them to address in your final patch. Thanks!

1. For readability, I prefer to use online_vcpus intead of online_cpus
2. I don't think it is necessary to save a copy in vcpu's private data, just use the following code to get the kvm pointer and access its non-pointer fileds in vmm module,

    struct kvm *get_kvm() {
         return (struct kvm*) VM_BASE; 
    }

    After getting kvm pointer, you can use it for vcpu.c's change.

Xiantao

Quoted 9 lines · Jes Sorensen

Hi,

Working on allowing larger max vcpus I'd like to put in this patch that calculates the number of online cpus. It's a building block for allowing the larger number of max vcpus without crippling performance for the smaller cases.

Cheers,
Jes

Applied
#4 lore raw
In reply to Jes Sorensen
Quoted 11 lines · Jes Sorensen

Hi Xiantao,

Here's an update version of the online_vcpus patch. I agree with you that the name online_vcpus is more appropriate and I used your trick to get to struct kvm, so it's a lot cleaner than the old version.

I renamed VM_BASE to KVM_VM_BASE as it seems a more correct name to me and has less chance of clashing with something. In addition the parenthesis I added are required to avoid unexpected results from CPP when you typecast it to struct kvm *.

Applied, thanks.

Signature · 1 line
error compiling committee.c: too many arguments to function