Skip to contentSkip to conversation

KVM: ARM: Cortex-A7 support and fixes

PATCH New Jonathan Austin 6 participants 17 messages Sep 26, 2013 – Oct 16, 2013 2 messages not in the archive Arm (32-bit)

lore
Share & tools
lore
#0 lore raw

cover KVM: ARM: Cortex-A7 support and fixes

This series adds support for using KVM on ARM Cortex-A7 CPUs. As the Cortex-A7 is architecturally identical to the A15 there is not a huge amount of code required, however there are a number of fixups necessary to core KVM code that testing on A7 has revealed.

I'm open to comments about other ways to share the functionality between A7/A15 while still maintaining the architectural/implementation defined split that existed between coproc.c and coproc_a15.c - The approach in this series avoids adding '3 levels' of abstraction and feels like a good compromise that doesn't require duplication of a lot of the A15 code for A7 but still permits implementation differences to be fully expressed in coproc_X files.

This patch series has been developed/tested on TC2 with the A15 cluster held in reset.

Jonathan Austin (3):
KVM: ARM: fix the size of TTBCR_{T0SZ,T1SZ} masks
KVM: ARM: Fix calculation of virtual CPU ID
KVM: ARM: Add support for Cortex-A7

 arch/arm/include/asm/kvm_arm.h  |    4 +-
 arch/arm/include/asm/kvm_asm.h  |    2 +-
 arch/arm/include/uapi/asm/kvm.h |    3 +-
 arch/arm/kvm/Makefile           |    2 +-
 arch/arm/kvm/coproc.c           |  114 ++++++++++++++++++++++++++++++++++++++
 arch/arm/kvm/coproc_a15.c       |  117 +--------------------------------------
 arch/arm/kvm/coproc_a7.c        |   54 ++++++++++++++++++
 arch/arm/kvm/guest.c            |    4 +-
 arch/arm/kvm/reset.c            |   15 ++---
 9 files changed, 186 insertions(+), 129 deletions(-)
 create mode 100644 arch/arm/kvm/coproc_a7.c
Signature · 1 line
1.7.9.5
#1 lore raw

PATCH 1/3 KVM: ARM: fix the size of TTBCR_{T0SZ,T1SZ} masks

The T{0,1}SZ fields of TTBCR are 3 bits wide when using the long descriptor format. Likewise, the T0SZ field of the HTCR is 3-bits. KVM currently defines TTBCR_T{0,1}SZ as 3, not 7.

The T0SZ mask is used to calculate the value for the HTCR, both to pick out TTBCR.T0SZ and mask off the equivalent field in the HTCR during read-modify-write. The incorrect mask size causes the (UNKNOWN) reset value of HTCR.T0SZ to leak in to the calculated HTCR value. Linux will hang when initializing KVM if HTCR's reset value has bit 2 set (sometimes the case on A7/TC2)

Fixing T0SZ allows A7 cores to boot and T1SZ is also fixed for completeness.

---
 arch/arm/include/asm/kvm_arm.h |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
arch/arm/include/asm/kvm_arm.h +2 -2
index 64e9696..d556f03 100644
@@ -95,12 +95,12 @@
 #define TTBCR_IRGN1	(3 << 24)
 #define TTBCR_EPD1	(1 << 23)
 #define TTBCR_A1	(1 << 22)
-#define TTBCR_T1SZ	(3 << 16)
+#define TTBCR_T1SZ	(7 << 16)
 #define TTBCR_SH0	(3 << 12)
 #define TTBCR_ORGN0	(3 << 10)
 #define TTBCR_IRGN0	(3 << 8)
 #define TTBCR_EPD0	(1 << 7)
-#define TTBCR_T0SZ	3
+#define TTBCR_T0SZ	(7 << 0)
 #define HTCR_MASK	(TTBCR_T0SZ | TTBCR_IRGN0 | TTBCR_ORGN0 | TTBCR_SH0)
 
 /* Hyp System Trap Register */
Signature · 1 line
1.7.9.5
#2 lore raw

PATCH 1/3 KVM: ARM: fix the size of TTBCR_{T0SZ,T1SZ} masks

Quoted 41 lines · Jonathan Austin

The T{0,1}SZ fields of TTBCR are 3 bits wide when using the long descriptor format. Likewise, the T0SZ field of the HTCR is 3-bits. KVM currently defines TTBCR_T{0,1}SZ as 3, not 7.

The T0SZ mask is used to calculate the value for the HTCR, both to pick out TTBCR.T0SZ and mask off the equivalent field in the HTCR during read-modify-write. The incorrect mask size causes the (UNKNOWN) reset value of HTCR.T0SZ to leak in to the calculated HTCR value. Linux will hang when initializing KVM if HTCR's reset value has bit 2 set (sometimes the case on A7/TC2)

Fixing T0SZ allows A7 cores to boot and T1SZ is also fixed for completeness.

---
 arch/arm/include/asm/kvm_arm.h |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
arch/arm/include/asm/kvm_arm.h +2 -2
index 64e9696..d556f03 100644
@@ -95,12 +95,12 @@
 #define TTBCR_IRGN1	(3 << 24)
 #define TTBCR_EPD1	(1 << 23)
 #define TTBCR_A1	(1 << 22)
-#define TTBCR_T1SZ	(3 << 16)
+#define TTBCR_T1SZ	(7 << 16)
 #define TTBCR_SH0	(3 << 12)
 #define TTBCR_ORGN0	(3 << 10)
 #define TTBCR_IRGN0	(3 << 8)
 #define TTBCR_EPD0	(1 << 7)
-#define TTBCR_T0SZ	3
+#define TTBCR_T0SZ	(7 << 0)
 #define HTCR_MASK	(TTBCR_T0SZ | TTBCR_IRGN0 | TTBCR_ORGN0 | TTBCR_SH0)
 
 /* Hyp System Trap Register */
Signature · 1 line
1.7.9.5

Good catch!

ack.

-Christoffer

#3 lore raw

PATCH 2/3 KVM: ARM: Fix calculation of virtual CPU ID

In reply to Jonathan Austin

KVM does not have a notion of multiple clusters for CPUs, just a linear array of CPUs. When using a system with cores in more than one cluster, the current method for calculating the virtual MPIDR will leak the (physical) cluster information into the virtual MPIDR. One effect of this is that Linux under KVM fails to boot multiple CPUs that aren't in the 0th cluster.

This patch does away with exposing the real MPIDR fields in favour of simply using the virtual CPU number (but preserving the U bit, as before).

---
 arch/arm/kvm/coproc_a15.c |   11 ++++-------
 1 file changed, 4 insertions(+), 7 deletions(-)
arch/arm/kvm/coproc_a15.c +4 -7
index cf93472..bbd4b88 100644
@@ -27,14 +27,11 @@
 static void reset_mpidr(struct kvm_vcpu *vcpu, const struct coproc_reg *r)
 {
 	/*
-	 * Compute guest MPIDR:
-	 * (Even if we present only one VCPU to the guest on an SMP
-	 * host we don't set the U bit in the MPIDR, or vice versa, as
-	 * revealing the underlying hardware properties is likely to
-	 * be the best choice).
+	 * Compute guest MPIDR. No need to mess around with different clusters
+	 * but we read the 'U' bit from the underlying hardware directly.
 	 */
-	vcpu->arch.cp15[c0_MPIDR] = (read_cpuid_mpidr() & ~MPIDR_LEVEL_MASK)
-		| (vcpu->vcpu_id & MPIDR_LEVEL_MASK);
+	vcpu->arch.cp15[c0_MPIDR] = (read_cpuid_mpidr() & MPIDR_SMP_BITMASK)
+					| vcpu->vcpu_id;
 }
 
 #include "coproc.h"
Signature · 1 line
1.7.9.5
#4 lore raw

PATCH 2/3 KVM: ARM: Fix calculation of virtual CPU ID

Quoted 42 lines · Jonathan Austin

KVM does not have a notion of multiple clusters for CPUs, just a linear array of CPUs. When using a system with cores in more than one cluster, the current method for calculating the virtual MPIDR will leak the (physical) cluster information into the virtual MPIDR. One effect of this is that Linux under KVM fails to boot multiple CPUs that aren't in the 0th cluster.

This patch does away with exposing the real MPIDR fields in favour of simply using the virtual CPU number (but preserving the U bit, as before).

---
 arch/arm/kvm/coproc_a15.c |   11 ++++-------
 1 file changed, 4 insertions(+), 7 deletions(-)
arch/arm/kvm/coproc_a15.c +4 -7
index cf93472..bbd4b88 100644
@@ -27,14 +27,11 @@
 static void reset_mpidr(struct kvm_vcpu *vcpu, const struct coproc_reg *r)
 {
 	/*
-	 * Compute guest MPIDR:
-	 * (Even if we present only one VCPU to the guest on an SMP
-	 * host we don't set the U bit in the MPIDR, or vice versa, as
-	 * revealing the underlying hardware properties is likely to
-	 * be the best choice).
+	 * Compute guest MPIDR. No need to mess around with different clusters
+	 * but we read the 'U' bit from the underlying hardware directly.
 	 */
-	vcpu->arch.cp15[c0_MPIDR] = (read_cpuid_mpidr() & ~MPIDR_LEVEL_MASK)
-		| (vcpu->vcpu_id & MPIDR_LEVEL_MASK);
+	vcpu->arch.cp15[c0_MPIDR] = (read_cpuid_mpidr() & MPIDR_SMP_BITMASK)
+					| vcpu->vcpu_id;
 }
 
 #include "coproc.h"
Signature · 1 line
1.7.9.5

Makes sense.

ack.

-Christoffer

#5 lore raw

PATCH 3/3 KVM: ARM: Add support for Cortex-A7

In reply to Jonathan Austin

This patch adds support for running Cortex-A7 guests on Cortex-A7 hosts.

As Cortex-A7 is architecturally compatible with A15, this patch is largely just generalising existing code. Areas where 'implementation defined' behaviour is identical for A7 and A15 is moved to allow it to be used by both cores.

The check to ensure that coprocessor register tables are sorted correctly is also moved in to 'common' code to avoid each new cpu doing its own check (and possibly forgetting to do so!)

---
 arch/arm/include/asm/kvm_asm.h  |    2 +-
 arch/arm/include/uapi/asm/kvm.h |    3 +-
 arch/arm/kvm/Makefile           |    2 +-
 arch/arm/kvm/coproc.c           |  114 +++++++++++++++++++++++++++++++++++++++
 arch/arm/kvm/coproc_a15.c       |  114 +--------------------------------------
 arch/arm/kvm/coproc_a7.c        |   54 +++++++++++++++++++
 arch/arm/kvm/guest.c            |    4 +-
 arch/arm/kvm/reset.c            |   15 +++---
 8 files changed, 184 insertions(+), 124 deletions(-)
 create mode 100644 arch/arm/kvm/coproc_a7.c
arch/arm/include/asm/kvm_asm.h +1 -1
index a2f43dd..661da11 100644
@@ -39,7 +39,7 @@
 #define c6_IFAR		17	/* Instruction Fault Address Register */
 #define c7_PAR		18	/* Physical Address Register */
 #define c7_PAR_high	19	/* PAR top 32 bits */
-#define c9_L2CTLR	20	/* Cortex A15 L2 Control Register */
+#define c9_L2CTLR	20	/* Cortex A15/A7 L2 Control Register */
 #define c10_PRRR	21	/* Primary Region Remap Register */
 #define c10_NMRR	22	/* Normal Memory Remap Register */
 #define c12_VBAR	23	/* Vector Base Address Register */
arch/arm/include/uapi/asm/kvm.h +2 -1
index c1ee007..c498b60 100644
@@ -63,7 +63,8 @@ struct kvm_regs {
 
 /* Supported Processor Types */
 #define KVM_ARM_TARGET_CORTEX_A15	0
-#define KVM_ARM_NUM_TARGETS		1
+#define KVM_ARM_TARGET_CORTEX_A7	1
+#define KVM_ARM_NUM_TARGETS		2
 
 /* KVM_ARM_SET_DEVICE_ADDR ioctl id encoding */
 #define KVM_ARM_DEVICE_TYPE_SHIFT	0
arch/arm/kvm/Makefile +1 -1
index d99bee4..789bca9 100644
@@ -19,6 +19,6 @@ kvm-arm-y = $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o
 
 obj-y += kvm-arm.o init.o interrupts.o
 obj-y += arm.o handle_exit.o guest.o mmu.o emulate.o reset.o
-obj-y += coproc.o coproc_a15.o mmio.o psci.o perf.o
+obj-y += coproc.o coproc_a15.o coproc_a7.o mmio.o psci.o perf.o
 obj-$(CONFIG_KVM_ARM_VGIC) += $(KVM)/arm/vgic.o
 obj-$(CONFIG_KVM_ARM_TIMER) += $(KVM)/arm/arch_timer.o
arch/arm/kvm/coproc.c +114 -0

148 lines not on this page · raw message

arch/arm/kvm/coproc_a15.c +1 -113

140 lines not on this page · raw message

arch/arm/kvm/coproc_a7.c new file +54 -0

54 lines not on this page · raw message

arch/arm/kvm/guest.c +3 -1
index 152d036..c01ae86 100644
@@ -190,6 +190,8 @@ int __attribute_const__ kvm_target_cpu(void)
 		return -EINVAL;
 
 	switch (part_number) {
+	case ARM_CPU_PART_CORTEX_A7:
+		return KVM_ARM_TARGET_CORTEX_A7;
 	case ARM_CPU_PART_CORTEX_A15:
 		return KVM_ARM_TARGET_CORTEX_A15;
 	default:
@@ -202,7 +204,7 @@ int kvm_vcpu_set_target(struct kvm_vcpu *vcpu,
 {
 	unsigned int i;
 
-	/* We can only do a cortex A15 for now. */
+	/* We can only cope with guest==host and only on A15/A7 (for now). */
 	if (init->target != kvm_target_cpu())
 		return -EINVAL;
 
arch/arm/kvm/reset.c +8 -7
index b7840e7..14b2f83 100644
@@ -30,16 +30,16 @@
 #include <kvm/arm_arch_timer.h>
 
 /******************************************************************************
- * Cortex-A15 Reset Values
+ * Cortex-A15 and Cortex-A7 Reset Values
  */
 
-static const int a15_max_cpu_idx = 3;
+static const int cortexa_max_cpu_idx = 3;
 
-static struct kvm_regs a15_regs_reset = {
+static struct kvm_regs cortexa_regs_reset = {
 	.usr_regs.ARM_cpsr = SVC_MODE | PSR_A_BIT | PSR_I_BIT | PSR_F_BIT,
 };
 
-static const struct kvm_irq_level a15_vtimer_irq = {
+static const struct kvm_irq_level cortexa_vtimer_irq = {
 	.irq = 27,
 	.level = 1,
 };
@@ -62,12 +62,13 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
 	const struct kvm_irq_level *cpu_vtimer_irq;
 
 	switch (vcpu->arch.target) {
+	case KVM_ARM_TARGET_CORTEX_A7:
 	case KVM_ARM_TARGET_CORTEX_A15:
-		if (vcpu->vcpu_id > a15_max_cpu_idx)
+		if (vcpu->vcpu_id > cortexa_max_cpu_idx)
 			return -EINVAL;
-		cpu_reset = &a15_regs_reset;
+		cpu_reset = &cortexa_regs_reset;
 		vcpu->arch.midr = read_cpuid_id();
-		cpu_vtimer_irq = &a15_vtimer_irq;
+		cpu_vtimer_irq = &cortexa_vtimer_irq;
 		break;
 	default:
 		return -ENODEV;
Signature · 1 line
1.7.9.5
#6 lore raw

PATCH 3/3 KVM: ARM: Add support for Cortex-A7

Quoted 6 lines · Jonathan Austin

This patch adds support for running Cortex-A7 guests on Cortex-A7 hosts.

As Cortex-A7 is architecturally compatible with A15, this patch is largely just generalising existing code. Areas where 'implementation defined' behaviour is identical for A7 and A15 is moved to allow it to be used by both cores.

Quoted 6 lines

The check to ensure that coprocessor register tables are sorted correctly is also moved in to 'common' code to avoid each new cpu doing its own check (and possibly forgetting to do so!)

The patch itself looks good, I'm wondering if we are creating potential confusion by not having a separate table for shared, but implementation defined, co-processors. One option would be to have multiple levels of 'inheritance' of the the coprocessor table, so you could load the generic one, overload it with the a15 one, and then overload it with the a7 one. That may be a bit over-engineerd though.

Then again, if A7/A15 end up being just two cores out of many many more, it's a bit weird to have their register definitions in the coproc.c files and everyone elses in their own files, sort of pushing the problem under the rug.

Unless of course we just agree that there are probably not going to be that many cores and they're all just going to look a lot like each other wrt. to the impdef registers anyhow, so it makes sense to provide this default implementation...

If nobody else objects, I'm ok with merging this code (once I have a chance to test it). We can always add more logic and move things around later, as there's no api breakage or anything like that.

Thanks for taking care of this!

-Christoffer

#7 lore raw

PATCH 3/3 KVM: ARM: Add support for Cortex-A7

Quoted 14 lines · Christoffer Dall
Quoted 6 lines · Jonathan Austin

This patch adds support for running Cortex-A7 guests on Cortex-A7 hosts.

As Cortex-A7 is architecturally compatible with A15, this patch is largely just generalising existing code. Areas where 'implementation defined' behaviour is identical for A7 and A15 is moved to allow it to be used by both cores.

Quoted 6 lines

The check to ensure that coprocessor register tables are sorted correctly is also moved in to 'common' code to avoid each new cpu doing its own check (and possibly forgetting to do so!)

Quoted 6 lines

The patch itself looks good, I'm wondering if we are creating potential confusion by not having a separate table for shared, but implementation defined, co-processors. One option would be to have multiple levels of 'inheritance' of the the coprocessor table, so you could load the generic one, overload it with the a15 one, and then overload it with the a7 one. That may be a bit over-engineerd though.

So my take on this is that it is OK to have default implementation defined at the generic level, as long as the handling is "benign enough". The core-specific code can always override it with something more fun.

Quoted 9 lines

Then again, if A7/A15 end up being just two cores out of many many more, it's a bit weird to have their register definitions in the coproc.c files and everyone elses in their own files, sort of pushing the problem under the rug.

Unless of course we just agree that there are probably not going to be that many cores and they're all just going to look a lot like each other wrt. to the impdef registers anyhow, so it makes sense to provide this default implementation...

There will probably be some Cortex-A12 code at one point, but I cannot foresee massive changes, as it is supposed to be fairly similar to both A7 and A15.

Quoted 3 lines

If nobody else objects, I'm ok with merging this code (once I have a chance to test it). We can always add more logic and move things around later, as there's no api breakage or anything like that.

Sounds good to me.

	M.
Signature · 1 line
Jazz is not dead. It just smells funny...
#8 lore raw

PATCH 3/3 KVM: ARM: Add support for Cortex-A7

In reply to Christoffer Dall
Quoted 26 lines · Christoffer Dall
Quoted 6 lines · Jonathan Austin

This patch adds support for running Cortex-A7 guests on Cortex-A7 hosts.

As Cortex-A7 is architecturally compatible with A15, this patch is largely just generalising existing code. Areas where 'implementation defined' behaviour is identical for A7 and A15 is moved to allow it to be used by both cores.

Quoted 6 lines

The check to ensure that coprocessor register tables are sorted correctly is also moved in to 'common' code to avoid each new cpu doing its own check (and possibly forgetting to do so!)

The patch itself looks good, I'm wondering if we are creating potential confusion by not having a separate table for shared, but implementation defined, co-processors. One option would be to have multiple levels of 'inheritance' of the the coprocessor table, so you could load the generic one, overload it with the a15 one, and then overload it with the a7 one. That may be a bit over-engineerd though.

Then again, if A7/A15 end up being just two cores out of many many more, it's a bit weird to have their register definitions in the coproc.c files and everyone elses in their own files, sort of pushing the problem under the rug.

Quoted 4 lines

Unless of course we just agree that there are probably not going to be that many cores and they're all just going to look a lot like each other wrt. to the impdef registers anyhow, so it makes sense to provide this default implementation...

Certainly for now it looks this way to me... As Marc says, A12 should look a lot like the other cores here.

Quoted 4 lines

If nobody else objects, I'm ok with merging this code (once I have a chance to test it). We can always add more logic and move things around later, as there's no api breakage or anything like that.

Cool - give me a shout if you need any help with the TC2 setup for testing, etc. I used boot-wrapper/mon.axf for hyp booting as you'd expect, and board.txt to hold the A15s in reset and select A7s as the boot cluster.

Jonny

Quoted 5 lines

Thanks for taking care of this!

-Christoffer

A message that is not in the archive (sent off-list or not yet received): CA+eFSM1YdhBDMAvyXknMA62Ao3CEntMptdi7RD=NXis1F-BmfQ@mail.gmail.com

#10 lore raw

PATCH 3/3 KVM: ARM: Add support for Cortex-A7

Quoted 1 line · Gavin Guo

Does it mean the currently qemu can't support A15 vcpu running on host A7?

No, that's a kernel restriction. That is why the patch description reads:
# This patch adds support for running Cortex-A7 guests on Cortex-A7 hosts.

(What QEMU is currently missing is A7 support, so you can't ask it to run an A7 guest CPU.)

-- PMM

A message that is not in the archive (sent off-list or not yet received): CA+eFSM0k3mA4uOiEvmE_tfVjnpq8DpXV-=nJXhucUDePJRTYXw@mail.gmail.com

#12 lore raw

PATCH 3/3 KVM: ARM: Add support for Cortex-A7

Quoted 16 lines · Gavin Guo

On Sun, Oct 6, 2013 at 10:56 PM, Peter Maydell <[email protected] <mailto:[email protected]>> wrote:

On 6 October 2013 23:03, Gavin Guo <[email protected] <mailto:[email protected]>> wrote:
> Does it mean the currently qemu can't support A15 vcpu running on host A7?

No, that's a kernel restriction. That is why the patch description reads:
# This patch adds support for running Cortex-A7 guests on Cortex-A7 hosts.

(What QEMU is currently missing is A7 support, so you can't ask it to run an A7 guest CPU.)

Quoted 6 lines

-- PMM

Thanks, I saw the checking in the kvm_vcpu_set_target(). I'm confused that what is the concern to run the same cpu type? Does it matter to run A15 on A7 or inverse?

It does. MIDR registers are different. Cache size is different. It may work, or it may not. And at the very least, you should present a consistent view of the underlying system to the unsuspecting OS.

	M.
Signature · 1 line
Jazz is not dead. It just smells funny...
#13 lore raw

PATCH 3/3 KVM: ARM: Add support for Cortex-A7

Quoted 25 lines · Marc Zyngier
Quoted 16 lines · Gavin Guo

On Sun, Oct 6, 2013 at 10:56 PM, Peter Maydell <[email protected] <mailto:[email protected]>> wrote:

On 6 October 2013 23:03, Gavin Guo <[email protected] <mailto:[email protected]>> wrote:
> Does it mean the currently qemu can't support A15 vcpu running on host A7?

No, that's a kernel restriction. That is why the patch description reads:
# This patch adds support for running Cortex-A7 guests on Cortex-A7 hosts.

(What QEMU is currently missing is A7 support, so you can't ask it to run an A7 guest CPU.)

Quoted 6 lines

-- PMM

Thanks, I saw the checking in the kvm_vcpu_set_target(). I'm confused that what is the concern to run the same cpu type? Does it matter to run A15 on A7 or inverse?

Quoted 8 lines

It does. MIDR registers are different. Cache size is different. It may work, or it may not. And at the very least, you should present a consistent view of the underlying system to the unsuspecting OS.

        M.

--
Jazz is not dead. It just smells funny...

Hi Marc,

How about the guest os with 4 x A15 vcpus run on a TC2 with 5 cores enabled (2 x A15 + 3 x A7). Does it mean that the 4 x A15 vcpus need to run on the 2 x A15 real cpus and can't be migrate to the 3 x A7? It seems that running 4 x A15 vcpus on the 2 x A15 and 2 x A7 core and leaving 1 A7 idle can take more computing power if all of the 4 x A15 guest os vcpus are full of loading. BTW, thanks for your reply, according to your reply I know the MIDR and cache size are different. Furthermore, I'd like to know if it is possible to make vcpu and real cpu in different type? If yes, I would like to study how to modify to make it work.

Thanks,
Gavin

#14 lore raw

PATCH 3/3 KVM: ARM: Add support for Cortex-A7

Quoted 35 lines · Gavin Guo
Quoted 25 lines · Marc Zyngier
Quoted 16 lines · Gavin Guo

On Sun, Oct 6, 2013 at 10:56 PM, Peter Maydell <[email protected] <mailto:[email protected]>> wrote:

On 6 October 2013 23:03, Gavin Guo <[email protected] <mailto:[email protected]>> wrote:
> Does it mean the currently qemu can't support A15 vcpu running on host A7?

No, that's a kernel restriction. That is why the patch description reads:
# This patch adds support for running Cortex-A7 guests on Cortex-A7 hosts.

(What QEMU is currently missing is A7 support, so you can't ask it to run an A7 guest CPU.)

Quoted 6 lines

-- PMM

Thanks, I saw the checking in the kvm_vcpu_set_target(). I'm confused that what is the concern to run the same cpu type? Does it matter to run A15 on A7 or inverse?

Quoted 8 lines

It does. MIDR registers are different. Cache size is different. It may work, or it may not. And at the very least, you should present a consistent view of the underlying system to the unsuspecting OS.

        M.

--
Jazz is not dead. It just smells funny...

Quoted 5 lines

Hi Marc,

How about the guest os with 4 x A15 vcpus run on a TC2 with 5 cores enabled (2 x A15 + 3 x A7). Does it mean that the 4 x A15 vcpus need to run on the 2 x A15 real cpus and can't be migrate to the 3 x A7? It

This isn't what I said. If you want to migrate your A15 vcpus to an A7, you need to make sure that your guest still sees an A15, despite running on an A7.

Quoted 7 lines

seems that running 4 x A15 vcpus on the 2 x A15 and 2 x A7 core and leaving 1 A7 idle can take more computing power if all of the 4 x A15 guest os vcpus are full of loading. BTW, thanks for your reply, according to your reply I know the MIDR and cache size are different. Furthermore, I'd like to know if it is possible to make vcpu and real cpu in different type? If yes, I would like to study how to modify to make it work.

If you want to run KVM in a heterogeneous configuration (big-little like), you need to be able to dynamically change the trapping of some CP15 accesses, depending on which physical CPU your vcpu is running on.

So we need an infrastructure to select a HCR/HSTR/HCPTR configuration based on the vcpu matching the CPU or not. And of course handling the additional trapping, if necessary.

For A7 vs A15, I'd expect the trapping to be fairly minimal, but until someone has spent quality time to write the above and work out what actually needs trapping, KVM on big-little system will stay a pipe dream.

	M.
Signature · 1 line
Jazz is not dead. It just smells funny...
#15 lore raw

PATCH 3/3 KVM: ARM: Add support for Cortex-A7

In reply to Peter Maydell
Quoted 7 lines · Peter Maydell
Quoted 1 line · Gavin Guo

Does it mean the currently qemu can't support A15 vcpu running on host A7?

No, that's a kernel restriction. That is why the patch description reads:
# This patch adds support for running Cortex-A7 guests on Cortex-A7 hosts.

(What QEMU is currently missing is A7 support, so you can't ask it to run an A7 guest CPU.)

Does anybody know if anybody is planning on adding support for A7 in QEMU?

Thanks,

Diana

#16 lore raw

cover KVM: ARM: Cortex-A7 support and fixes

In reply to Jonathan Austin
Quoted 37 lines · Jonathan Austin

This series adds support for using KVM on ARM Cortex-A7 CPUs. As the Cortex-A7 is architecturally identical to the A15 there is not a huge amount of code required, however there are a number of fixups necessary to core KVM code that testing on A7 has revealed.

I'm open to comments about other ways to share the functionality between A7/A15 while still maintaining the architectural/implementation defined split that existed between coproc.c and coproc_a15.c - The approach in this series avoids adding '3 levels' of abstraction and feels like a good compromise that doesn't require duplication of a lot of the A15 code for A7 but still permits implementation differences to be fully expressed in coproc_X files.

This patch series has been developed/tested on TC2 with the A15 cluster held in reset.

Jonathan Austin (3):
KVM: ARM: fix the size of TTBCR_{T0SZ,T1SZ} masks
KVM: ARM: Fix calculation of virtual CPU ID
KVM: ARM: Add support for Cortex-A7

 arch/arm/include/asm/kvm_arm.h  |    4 +-
 arch/arm/include/asm/kvm_asm.h  |    2 +-
 arch/arm/include/uapi/asm/kvm.h |    3 +-
 arch/arm/kvm/Makefile           |    2 +-
 arch/arm/kvm/coproc.c           |  114 ++++++++++++++++++++++++++++++++++++++
 arch/arm/kvm/coproc_a15.c       |  117 +--------------------------------------
 arch/arm/kvm/coproc_a7.c        |   54 ++++++++++++++++++
 arch/arm/kvm/guest.c            |    4 +-
 arch/arm/kvm/reset.c            |   15 ++---
 9 files changed, 186 insertions(+), 129 deletions(-)
 create mode 100644 arch/arm/kvm/coproc_a7.c

--
1.7.9.5

Quoted 4 lines

_______________________________________________
kvmarm mailing list
kvmarm at lists.cs.columbia.edu
https://lists.cs.columbia.edu/cucslists/listinfo/kvmarm

Hi Jonathan,

I'd like to help test the kvm on Cortex-A7. I've applied your 3 kernel patches, and download the Virtual Open Systems's kvmtools to test. But the guest os can't be brought up. Could you provide the steps about where to download kvmtools and the command to bring up the guest?

Thanks,
Gavin

#17 lore raw

cover KVM: ARM: Cortex-A7 support and fixes

Hi Gavin,

Quoted 8 lines · Gavin Guo
Quoted 5 lines · Jonathan Austin

This series adds support for using KVM on ARM Cortex-A7 CPUs. As the Cortex-A7 is architecturally identical to the A15 there is not a huge amount of code required, however there are a number of fixups necessary to core KVM code that testing on A7 has revealed.

Quoted 7 lines

Hi Jonathan,

I'd like to help test the kvm on Cortex-A7. I've applied your 3 kernel patches, and download the Virtual Open Systems's kvmtools to test. But the guest os can't be brought up. Could you provide the steps about where to download kvmtools and the command to bring up the guest?

Firstly, if you're on a TC2, you need to make sure the A15s are held in reset, and you need a shim to boot the kernel in hyp mode. For now I'll assume you're *not* in that situation.

Assuming your platform does enter the kernel in Hyp mode (it needs to!) then you need a couple of patches to kvmtool:

http://www.spinics.net/lists/kvm/msg96662.html

At the moment, I think you're still in the business of picking up individual patches from the list - Will (Deacon) has offered to host them in his kvm branch but is waiting until the core A7 support is in Christoffer's 'next'.

Once you've applied the patches, build lkvm and a kernel image, copy them over to your host, then try something like:

/lkvm run /Image -d /fs.img -m 512 -c 2 --console virtio -p "console=hvc0,38400"

Finally have a check in /proc/cpuinfo to be sure that you've got the kind of CPUs you were expecting!

Hope that helps,
Jonny

Quoted 3 lines

Thanks,
Gavin

#18 lore raw

cover KVM: ARM: Cortex-A7 support and fixes

In reply to Jonathan Austin
Quoted 12 lines · Jonathan Austin

This series adds support for using KVM on ARM Cortex-A7 CPUs. As the Cortex-A7 is architecturally identical to the A15 there is not a huge amount of code required, however there are a number of fixups necessary to core KVM code that testing on A7 has revealed.

I'm open to comments about other ways to share the functionality between A7/A15 while still maintaining the architectural/implementation defined split that existed between coproc.c and coproc_a15.c - The approach in this series avoids adding '3 levels' of abstraction and feels like a good compromise that doesn't require duplication of a lot of the A15 code for A7 but still permits implementation differences to be fully expressed in coproc_X files.

Quoted 3 lines

This patch series has been developed/tested on TC2 with the A15 cluster held in reset.

Tested on Cubieboard2 and applied to kvm-arm-next.

Thanks,
-Christoffer