Skip to contentSkip to conversation

ARMv7: Add HYP mode switching support

RFC PATCH New Andre Przywara 3 participants 16 messages Apr 26, 2013 – May 10, 2013

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover ARMv7: Add HYP mode switching support

ARM CPUs with the virtualization extension have a new mode called HYP mode, which allows hypervisors to safely control and monitor guests. The current hypervisor (KVM and Xen) implementations require the kernel to be entered in that HYP mode.

This patch series introduces a new U-Boot command called "hypmode" which can be used at any time at the U-Boot command prompt to switch the CPU into HYP mode - ideally this would be done before starting the kernel. Since U-Boot does not use the MMU, it runs fine in HYP mode, so you could as well enter HYP mode earlier.

The process of switching into HYP mode requires the CPU to be in non-secure state, which requires the GIC to be programmed properly first. Explanations about the details are in the commit messages of the respective patches.

The code aims to be as generic as possible, though currently it has only been tested on the Versatile Express TC-2 board. The last patch thus enables the feature for that board and relies on the Versatile Express updates patches sent out lately[1].

I would like to get some feedback on the patches, especially about:
1) Is the code in the right places? I used arch/arm/lib/ and arch/arm/cpu/armv7/ as directories for the ARM code and common/ for the actual command.
2) Is a command to switch to HYP actually the right thing? Should HYP be entered automatically instead? Or provide an option to skip HYP mode if requested by the user?
3) Does it make sense to provide a "nonsec" command also? I cannot name any usecases by now, but the implementation would be rather easy with basically all the functionality already there.

In general I appreciate any comments about coding style, patch layout or the actual implementation.

Thanks for watching!

Andre.

[1] http://lists.denx.de/pipermail/u-boot/2013-April/151366.html

Andre Przywara (6):
ARM: add secure monitor handler to switch to non-secure state
ARM: add assembly routine to switch to non-secure state
ARM: add U-Boot command "hypmode" to switch to non-secure state
ARM: add SMP support for non-secure switch
ARM: extend non-secure switch to also go into HYP mode
ARM: VExpress: enable ARMv7 virt support for VExpress A15

 arch/arm/cpu/armv7/start.S          | 116 ++++++++++++++++++++++++++++---
 arch/arm/include/asm/armv7.h        |   2 +
 arch/arm/lib/Makefile               |   2 +
 arch/arm/lib/virt-v7.c              | 132 ++++++++++++++++++++++++++++++++++++
 common/Makefile                     |   1 +
 common/cmd_virt.c                   |  65 ++++++++++++++++++
 include/configs/vexpress_ca15_tc2.h |   3 +
 7 files changed, 311 insertions(+), 10 deletions(-)
 create mode 100644 arch/arm/lib/virt-v7.c
 create mode 100644 common/cmd_virt.c
Signature · 1 line
1.7.12.1
#1 lore raw

RFC PATCH 1/6 ARM: add secure monitor handler to switch to non-secure state

A prerequisite for using virtualization is to be in HYP mode, which requires the CPU to be in non-secure state.
According to the ARM ARM this should not be done in SVC mode, so we have to setup a SMC handler for this. We reuse the current vector table for this and make sure that we only access the MVBAR register if the CPU supports the virtualization extensions.

Introduce a monitor handler routine which switches the CPU to non-secure state by setting the NS bit (and associated bits).

---
 arch/arm/cpu/armv7/start.S | 20 +++++++++++++++++---
 1 file changed, 17 insertions(+), 3 deletions(-)
arch/arm/cpu/armv7/start.S +17 -3
index e9e57e6..7bfb19d 100644
@@ -155,6 +155,10 @@ reset:
 	/* Set vector address in CP15 VBAR register */
 	ldr	r0, =_start
 	mcr	p15, 0, r0, c12, c0, 0	@Set VBAR
+
+	mrc	p15, 0, r1, c0, c1, 1	@ check for security extension
+	ands	r1, r1, #0x30
+	mcrne	p15, 0, r0, c12, c0, 1	@ Set secure monitor MVBAR
 #endif
 
 	/* the mask ROM code should have PLL and others stable */
@@ -256,6 +260,9 @@ ENTRY(c_runtime_cpu_setup)
 	/* Set vector address in CP15 VBAR register */
 	ldr     r0, =_start
 	mcr     p15, 0, r0, c12, c0, 0  @Set VBAR
+	mrc	p15, 0, r1, c0, c1, 1	@ check for security extension
+	ands	r1, r1, #0x30
+	mcrne	p15, 0, r0, c12, c0, 1  @ Set secure monitor MVBAR
 
 	bx	lr
 
@@ -492,9 +499,16 @@ undefined_instruction:
 
 	.align	5
 software_interrupt:
-	get_bad_stack_swi
-	bad_save_user_regs
-	bl	do_software_interrupt
+	mrc	p15, 0, r1, c1, c1, 0		@ read SCR
+	bic	r1, r1, #0x07f
+	orr	r1, r1, #0x31			@ enable NS, AW, FW
+
+	mrc	p15, 0, r0, c12, c0, 0		@ save secure copy of VBAR
+	mcr	p15, 0, r1, c1, c1, 0		@ write SCR, switch to non-sec
+	isb
+	mcr	p15, 0, r0, c12, c0, 0		@ write non-secure copy of VBAR
+
+	movs	pc, lr
 
 	.align	5
 prefetch_abort:
Signature · 1 line
1.7.12.1
#2 lore raw

RFC PATCH 1/6 ARM: add secure monitor handler to switch to non-secure state

Quoted 41 lines · Andre Przywara

A prerequisite for using virtualization is to be in HYP mode, which requires the CPU to be in non-secure state.
According to the ARM ARM this should not be done in SVC mode, so we have to setup a SMC handler for this. We reuse the current vector table for this and make sure that we only access the MVBAR register if the CPU supports the virtualization extensions.

Introduce a monitor handler routine which switches the CPU to non-secure state by setting the NS bit (and associated bits).

---
 arch/arm/cpu/armv7/start.S | 20 +++++++++++++++++---
 1 file changed, 17 insertions(+), 3 deletions(-)
arch/arm/cpu/armv7/start.S +7 -0
index e9e57e6..7bfb19d 100644
@@ -155,6 +155,10 @@ reset:
  /* Set vector address in CP15 VBAR register */
  ldr r0, =_start
  mcr p15, 0, r0, c12, c0, 0 @Set VBAR
+
+ mrc p15, 0, r1, c0, c1, 1 @ check for security extension
+ ands r1, r1, #0x30
+ mcrne p15, 0, r0, c12, c0, 1 @ Set secure monitor MVBAR
 #endif
 
  /* the mask ROM code should have PLL and others stable */
@@ -256,6 +260,9 @@ ENTRY(c_runtime_cpu_setup)
  /* Set vector address in CP15 VBAR register */
  ldr     r0, =_start
  mcr     p15, 0, r0, c12, c0, 0  @Set VBAR
+ mrc p15, 0, r1, c0, c1, 1 @ check for security extension
+ ands r1, r1, #0x30
+ mcrne p15, 0, r0, c12, c0, 1  @ Set secure monitor MVBAR
 
  bx lr
 

arch/arm/cpu/armv7/start.S undefined_instruction: · line 497

Quoted 6 lines
arch/arm/cpu/armv7/start.S +0 -3
@@ -492,9 +499,16 @@ undefined_instruction:
 
  .align 5
 software_interrupt:
- get_bad_stack_swi
- bad_save_user_regs
- bl do_software_interrupt

I think you should add a comment block here saying that this is handling an smc #0 call and the intent is to change to non-secure world.

Quoted 16 lines
arch/arm/cpu/armv7/start.S +10 -1
+ mrc p15, 0, r1, c1, c1, 0 @ read SCR
+ bic r1, r1, #0x07f
+ orr r1, r1, #0x31 @ enable NS, AW, FW
+
+ mrc p15, 0, r0, c12, c0, 0 @ save secure copy of VBAR
+ mcr p15, 0, r1, c1, c1, 0 @ write SCR, switch to non-sec
+ isb
+ mcr p15, 0, r0, c12, c0, 0 @ write non-secure copy of VBAR
+
+ movs pc, lr
 
  .align 5
 prefetch_abort:
--

1.7.12.1

#3 lore raw

RFC PATCH 2/6 ARM: add assembly routine to switch to non-secure state

In reply to Andre Przywara

While actually switching to non-secure state is one thing, the more important part of this process is to make sure that we still have full access to the interrupt controller (GIC).
The GIC is fully aware of secure vs. non-secure state, some registers are banked, others may be configured to be accessible from secure state only.
To be as generic as possible, we get the GIC memory mapped address based on the PERIPHBASE register. We check explicitly for ARM Cortex-A7 and A15 cores, assuming an A9 otherwise, as for those cores we know the offsets for the GIC CPU interface from the PERIPHBASE content. Other cores could be added as needed.

With the GIC accessible, we:
a) allow private interrupts to be delivered to the core (GICD_IGROUPR0 = 0xFFFFFFFF)
b) enable the CPU interface (GICC_CTLR[0] = 1)
c) set the priority filter to allow non-secure interrupts (GICC_PMR = 0x80)

After having switched to non-secure state, we also enable the non-secure GIC CPU interface, since this register is banked.

Also we allow access to all coprocessor interfaces from non-secure state by writing the appropriate bits in the NSACR register.

For reasons obvious later we only use caller saved registers r0-r3.

---
 arch/arm/cpu/armv7/start.S | 45 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 45 insertions(+)
arch/arm/cpu/armv7/start.S +45 -0
index 7bfb19d..401b0eb 100644
@@ -561,3 +561,48 @@ fiq:
 
 #endif /* CONFIG_USE_IRQ */
 #endif /* CONFIG_SPL_BUILD */
+
+/* Routine to initialize GIC CPU interface and switch to nonsecure state.
+ */
+.globl _nonsec_gic_switch
+_nonsec_gic_switch:
+	mrc	p15, 4, r2, c15, c0, 0		@ r2 = PERIPHBASE
+	add	r3, r2, #0x1000			@ GIC dist i/f offset
+	mvn	r1, #0
+	str	r1, [r3, #0x80]			@ allow private interrupts
+
+	mrc	p15, 0, r0, c0, c0, 0		@ MIDR
+	bfc	r0, #16, #8			@ mask out variant, arch
+	bfc	r0, #0, #4			@ and revision
+	movw	r1, #0xc070
+	movt	r1, #0x4100
+	cmp	r0, r1				@ check for Cortex-A7
+	orr	r1, #0xf0
+	cmpne	r0, r1				@ check for Cortex-A15
+	movne	r1, #0x100			@ GIC CPU offset for A9
+	moveq	r1, #0x2000			@ GIC CPU offset for A15/A7
+	add	r3, r2, r1			@ r3 = GIC CPU i/f addr
+
+	mov	r1, #1
+	str	r1, [r3, #0]			@ set GICC_CTLR[enable]
+	mov	r1, #0x80
+	str	r1, [r3, #4]			@ set GICC_PIMR[7]
+
+	movw	r1, #0x3fff
+	movt	r1, #0x0006
+	mcr	p15, 0, r1, c1, c1, 2		@ NSACR = all copros to non-sec
+
+	ldr	r1, =_start
+	mcr	p15, 0, r1, c12, c0, 0		@ set VBAR
+	mcr	p15, 0, r1, c12, c0, 1		@ set MVBAR
+
+	isb
+	smc	#0				@ call into MONITOR mode
+	isb					@ clobbers r0 and r1
+
+	mov	r1, #1
+	str	r1, [r3, #0]			@ set GICC_CTLR[enable]
+	add	r2, r2, #0x1000			@ GIC dist i/f offset
+	str	r1, [r2]			@ allow private interrupts
+
+	mov	pc, lr
Signature · 1 line
1.7.12.1
#4 lore raw

RFC PATCH 3/6 ARM: add U-Boot command "hypmode" to switch to non-secure state

In reply to Andre Przywara

In preparation for the actual HYP mode switch, we introduce a new U-Boot command called "hypmode". For now we only do the non-secure switch here.

Some part of the work is done in the assembly routine in start.S, introduced with the previous patch, but for the full glory we need to setup the GIC distributor interface once for the whole system, which is done in C here.
The routine is placed in arch/arm/lib to allow easy access from different boards or CPUs.

First we check for the availability of the security extensions.

The generic timer base frequency register is only accessible from secure state, so we have to program it now. Actually this should be done from primary firmware before, but some boards seems to omit this, so if needed we do this here with a board specific value.

Since we need a safe way to access the GIC, we use the PERIPHBASE registers on Cortex-A15 and A7 CPUs and do some sanity checks.

Then we actually do the GIC enablement:
a) enable the GIC distributor, both for non-secure and secure state (GICD_CTLR[1:0] = 11b)
b) allow all interrupts to be handled from non-secure state (GICD_IGROUPRn = 0xFFFFFFFF)
The core specific GIC setup is then done in the assembly routine.

The actual U-Boot command is pretty small: calling the routine and doing some error reporting. A return value of 1 will be added later.

To enable the whole code we introduce the CONFIG_CMD_VIRT variable.

---
 arch/arm/include/asm/armv7.h |   2 +
 arch/arm/lib/Makefile        |   2 +
 arch/arm/lib/virt-v7.c       | 116 +++++++++++++++++++++++++++++++++++++++++++
 common/Makefile              |   1 +
 common/cmd_virt.c            |  59 ++++++++++++++++++++++
 5 files changed, 180 insertions(+)
 create mode 100644 arch/arm/lib/virt-v7.c
 create mode 100644 common/cmd_virt.c
arch/arm/include/asm/armv7.h +2 -0
index a73630b..3567692 100644
@@ -74,4 +74,6 @@ void v7_outer_cache_inval_all(void);
 void v7_outer_cache_flush_range(u32 start, u32 end);
 void v7_outer_cache_inval_range(u32 start, u32 end);
 
+int armv7_switch_nonsec(void);
+
 #endif
arch/arm/lib/Makefile +2 -0
index 6ae161a..c5b6b69 100644
@@ -58,6 +58,8 @@ COBJS-y	+= reset.o
 COBJS-y	+= cache.o
 COBJS-y	+= cache-cp15.o
 
+COBJS-y	+= virt-v7.o
+
 SRCS	:= $(GLSOBJS:.o=.S) $(GLCOBJS:.o=.c) \
 	   $(SOBJS-y:.o=.S) $(COBJS-y:.o=.c)
 OBJS	:= $(addprefix $(obj),$(SOBJS-y) $(COBJS-y))
arch/arm/lib/virt-v7.c new file +116 -0

116 lines not on this page · raw message

common/Makefile +1 -0
index 0e0fff1..d9ba5ab 100644
@@ -129,6 +129,7 @@ COBJS-y += cmd_load.o
 COBJS-$(CONFIG_LOGBUFFER) += cmd_log.o
 COBJS-$(CONFIG_ID_EEPROM) += cmd_mac.o
 COBJS-$(CONFIG_CMD_MD5SUM) += cmd_md5sum.o
+COBJS-$(CONFIG_CMD_VIRT) += cmd_virt.o
 COBJS-$(CONFIG_CMD_MEMORY) += cmd_mem.o
 COBJS-$(CONFIG_CMD_IO) += cmd_io.o
 COBJS-$(CONFIG_CMD_MFSL) += cmd_mfsl.o
common/cmd_virt.c new file +59 -0

59 lines not on this page · raw message

Signature · 1 line
1.7.12.1
#5 lore raw

RFC PATCH 4/6 ARM: add SMP support for non-secure switch

In reply to Andre Przywara

Currently the non-secure switch is only done for the boot processor. To later allow full SMP support, we have to switch all secondary cores into non-secure state also.

So we add an entry point for secondary CPUs coming out of low-power state and make sure we put them into WFI again after having switched to non-secure state.
For this we acknowledge and EOI the wake-up IPI, then go into WFI. Once being kicked out of it later, we sanity check that the start address has actually been changed (since another attempt to switch to non-secure would block the core) and jump to the new address.

The actual CPU kick is done by sending an inter-processor interrupt via the GIC to all CPU interfaces except the requesting processor. The secondary cores will then setup their respective GIC CPU interface.

The address secondary cores jump to is board specific, we provide the value here for the Versatile Express board.

---
 arch/arm/cpu/armv7/start.S          | 27 ++++++++++++++++++++++++++-
 arch/arm/lib/virt-v7.c              | 10 +++++++++-
 include/configs/vexpress_ca15_tc2.h |  1 +
 3 files changed, 36 insertions(+), 2 deletions(-)
arch/arm/cpu/armv7/start.S +26 -1
index 401b0eb..2b47881 100644
@@ -563,8 +563,19 @@ fiq:
 #endif /* CONFIG_SPL_BUILD */
 
 /* Routine to initialize GIC CPU interface and switch to nonsecure state.
+ * Will be executed directly by secondary CPUs after coming out of
+ * WFI, or can be called directly by C code for CPU 0.
+ * Those two paths mandate to not use any stack and to only use registers
+ * r0-r3 to comply with both the C ABI and the requirement of SMP startup
+ * code.
  */
 .globl _nonsec_gic_switch
+.globl _smp_pen
+_smp_pen:
+	mrs	r0, cpsr
+	orr	r0, r0, #0xc0
+	msr	cpsr, r0			@ disable interrupts
+	mov	lr, #0				@ clear LR to mark secondary
 _nonsec_gic_switch:
 	mrc	p15, 4, r2, c15, c0, 0		@ r2 = PERIPHBASE
 	add	r3, r2, #0x1000			@ GIC dist i/f offset
@@ -605,4 +616,18 @@ _nonsec_gic_switch:
 	add	r2, r2, #0x1000			@ GIC dist i/f offset
 	str	r1, [r2]			@ allow private interrupts
 
-	mov	pc, lr
+	cmp	lr, #0
+	movne	pc, lr				@ CPU 0 to return
+						@ all others: go to sleep
+_ack_int:
+	ldr	r1, [r3, #0x0c]			@ read GICD acknowledge
+	str	r1, [r3, #0x10]			@ write GICD EOI
+
+	adr	r1, _smp_pen
+waitloop:
+	wfi
+	ldr	r0, =CONFIG_SYSFLAGS_ADDR	@ load start address
+	ldr	r0, [r0]
+	cmp	r0, r1			@ make sure we dont execute this code
+	beq	waitloop		@ again (due to a spurious wakeup)
+	mov	pc, r0
arch/arm/lib/virt-v7.c +9 -1
index 416ca29..5ca093a 100644
@@ -29,6 +29,7 @@
 
 /* the assembly routine doing the actual work in start.S */
 void _nonsec_gic_switch(void);
+void _smp_pen(void);
 
 #define GICD_CTLR	0x000
 #define GICD_TYPER	0x004
@@ -51,6 +52,7 @@ int armv7_switch_nonsec(void)
 	unsigned int reg;
 	volatile unsigned int *gicdptr;
 	unsigned itlinesnr, i;
+	unsigned int *sysflags;
 
 	/* check whether the CPU supports the security extensions */
 	asm("mrc p15, 0, %0, c0, c1, 1\n" : "=r"(reg));
@@ -109,7 +111,13 @@ int armv7_switch_nonsec(void)
 	for (i = 0; i <= itlinesnr; i++)
 		gicdptr[GICD_IGROUPR0 / 4 + i] = (unsigned)-1;
 
-	/* call the non-sec switching code on this CPU */
+	/* now kick all CPUs (expect this one) by writing to GICD_SIGR */
+	sysflags = (void *)CONFIG_SYSFLAGS_ADDR;
+	sysflags[1] = (unsigned)-1;
+	sysflags[0] = (uintptr_t)_smp_pen;
+	gicdptr[GICD_SGIR / 4] = 1U << 24;
+
+	/* call the non-sec switching code on this CPU also */
 	_nonsec_gic_switch();
 
 	return 0;
include/configs/vexpress_ca15_tc2.h +1 -0
index 9e230ad..210a27c 100644
@@ -32,5 +32,6 @@
 #define CONFIG_BOOTP_VCI_STRING     "U-boot.armv7.vexpress_ca15x2_tc2"
 
 #define CONFIG_SYS_CLK_FREQ 24000000
+#define CONFIG_SYSFLAGS_ADDR 0x1c010030
 
 #endif
Signature · 1 line
1.7.12.1
#6 lore raw

RFC PATCH 4/6 ARM: add SMP support for non-secure switch

Quoted 111 lines · Andre Przywara

Currently the non-secure switch is only done for the boot processor. To later allow full SMP support, we have to switch all secondary cores into non-secure state also.

So we add an entry point for secondary CPUs coming out of low-power state and make sure we put them into WFI again after having switched to non-secure state.
For this we acknowledge and EOI the wake-up IPI, then go into WFI. Once being kicked out of it later, we sanity check that the start address has actually been changed (since another attempt to switch to non-secure would block the core) and jump to the new address.

The actual CPU kick is done by sending an inter-processor interrupt via the GIC to all CPU interfaces except the requesting processor. The secondary cores will then setup their respective GIC CPU interface.

The address secondary cores jump to is board specific, we provide the value here for the Versatile Express board.

---
 arch/arm/cpu/armv7/start.S          | 27 ++++++++++++++++++++++++++-
 arch/arm/lib/virt-v7.c              | 10 +++++++++-
 include/configs/vexpress_ca15_tc2.h |  1 +
 3 files changed, 36 insertions(+), 2 deletions(-)
arch/arm/cpu/armv7/start.S +26 -1
index 401b0eb..2b47881 100644
@@ -563,8 +563,19 @@ fiq:
 #endif /* CONFIG_SPL_BUILD */
 
 /* Routine to initialize GIC CPU interface and switch to nonsecure state.
+ * Will be executed directly by secondary CPUs after coming out of
+ * WFI, or can be called directly by C code for CPU 0.
+ * Those two paths mandate to not use any stack and to only use registers
+ * r0-r3 to comply with both the C ABI and the requirement of SMP startup
+ * code.
  */
 .globl _nonsec_gic_switch
+.globl _smp_pen
+_smp_pen:
+       mrs     r0, cpsr
+       orr     r0, r0, #0xc0
+       msr     cpsr, r0                        @ disable interrupts
+       mov     lr, #0                          @ clear LR to mark secondary
 _nonsec_gic_switch:
        mrc     p15, 4, r2, c15, c0, 0          @ r2 = PERIPHBASE
        add     r3, r2, #0x1000                 @ GIC dist i/f offset
@@ -605,4 +616,18 @@ _nonsec_gic_switch:
        add     r2, r2, #0x1000                 @ GIC dist i/f offset
        str     r1, [r2]                        @ allow private interrupts
 
-       mov     pc, lr
+       cmp     lr, #0
+       movne   pc, lr                          @ CPU 0 to return
+                                               @ all others: go to sleep
+_ack_int:
+       ldr     r1, [r3, #0x0c]                 @ read GICD acknowledge
+       str     r1, [r3, #0x10]                 @ write GICD EOI
+
+       adr     r1, _smp_pen
+waitloop:
+       wfi
+       ldr     r0, =CONFIG_SYSFLAGS_ADDR       @ load start address
+       ldr     r0, [r0]
+       cmp     r0, r1                  @ make sure we dont execute this code
+       beq     waitloop                @ again (due to a spurious wakeup)
+       mov     pc, r0
arch/arm/lib/virt-v7.c +9 -1
index 416ca29..5ca093a 100644
@@ -29,6 +29,7 @@
 
 /* the assembly routine doing the actual work in start.S */
 void _nonsec_gic_switch(void);
+void _smp_pen(void);
 
 #define GICD_CTLR      0x000
 #define GICD_TYPER     0x004
@@ -51,6 +52,7 @@ int armv7_switch_nonsec(void)
        unsigned int reg;
        volatile unsigned int *gicdptr;
        unsigned itlinesnr, i;
+       unsigned int *sysflags;
 
        /* check whether the CPU supports the security extensions */
        asm("mrc p15, 0, %0, c0, c1, 1\n" : "=r"(reg));
@@ -109,7 +111,13 @@ int armv7_switch_nonsec(void)
        for (i = 0; i <= itlinesnr; i++)
                gicdptr[GICD_IGROUPR0 / 4 + i] = (unsigned)-1;
 
-       /* call the non-sec switching code on this CPU */
+       /* now kick all CPUs (expect this one) by writing to GICD_SIGR */
+       sysflags = (void *)CONFIG_SYSFLAGS_ADDR;
+       sysflags[1] = (unsigned)-1;
+       sysflags[0] = (uintptr_t)_smp_pen;
+       gicdptr[GICD_SGIR / 4] = 1U << 24;
+
+       /* call the non-sec switching code on this CPU also */
        _nonsec_gic_switch();
 
        return 0;
include/configs/vexpress_ca15_tc2.h +0 -0
index 9e230ad..210a27c 100644

include/configs/vexpress_ca15_tc2.h

Quoted 7 lines
include/configs/vexpress_ca15_tc2.h +1 -1
@@ -32,5 +32,6 @@
 #define CONFIG_BOOTP_VCI_STRING     "U-boot.armv7.vexpress_ca15x2_tc2"
 
 #define CONFIG_SYS_CLK_FREQ 24000000
+#define CONFIG_SYSFLAGS_ADDR 0x1c010030
 
 #endif
--

hmm, is all this likely to work across all armv7 cores? I imagined that the SMP boot protocol could be potentially vastly different on different implementations and this stuff would have to call into board specific code...?

#7 lore raw

RFC PATCH 4/6 ARM: add SMP support for non-secure switch

Quoted 115 lines · Christoffer Dall
Quoted 113 lines · Andre Przywara

Currently the non-secure switch is only done for the boot processor. To later allow full SMP support, we have to switch all secondary cores into non-secure state also.

So we add an entry point for secondary CPUs coming out of low-power state and make sure we put them into WFI again after having switched to non-secure state.
For this we acknowledge and EOI the wake-up IPI, then go into WFI. Once being kicked out of it later, we sanity check that the start address has actually been changed (since another attempt to switch to non-secure would block the core) and jump to the new address.

The actual CPU kick is done by sending an inter-processor interrupt via the GIC to all CPU interfaces except the requesting processor. The secondary cores will then setup their respective GIC CPU interface.

The address secondary cores jump to is board specific, we provide the value here for the Versatile Express board.

---
  arch/arm/cpu/armv7/start.S          | 27 ++++++++++++++++++++++++++-
  arch/arm/lib/virt-v7.c              | 10 +++++++++-
  include/configs/vexpress_ca15_tc2.h |  1 +
  3 files changed, 36 insertions(+), 2 deletions(-)
arch/arm/cpu/armv7/start.S +26 -1
index 401b0eb..2b47881 100644
@@ -563,8 +563,19 @@ fiq:
  #endif /* CONFIG_SPL_BUILD */
 
  /* Routine to initialize GIC CPU interface and switch to nonsecure state.
+ * Will be executed directly by secondary CPUs after coming out of
+ * WFI, or can be called directly by C code for CPU 0.
+ * Those two paths mandate to not use any stack and to only use registers
+ * r0-r3 to comply with both the C ABI and the requirement of SMP startup
+ * code.
   */
  .globl _nonsec_gic_switch
+.globl _smp_pen
+_smp_pen:
+       mrs     r0, cpsr
+       orr     r0, r0, #0xc0
+       msr     cpsr, r0                        @ disable interrupts
+       mov     lr, #0                          @ clear LR to mark secondary
  _nonsec_gic_switch:
         mrc     p15, 4, r2, c15, c0, 0          @ r2 = PERIPHBASE
         add     r3, r2, #0x1000                 @ GIC dist i/f offset
@@ -605,4 +616,18 @@ _nonsec_gic_switch:
         add     r2, r2, #0x1000                 @ GIC dist i/f offset
         str     r1, [r2]                        @ allow private interrupts
 
-       mov     pc, lr
+       cmp     lr, #0
+       movne   pc, lr                          @ CPU 0 to return
+                                               @ all others: go to sleep
+_ack_int:
+       ldr     r1, [r3, #0x0c]                 @ read GICD acknowledge
+       str     r1, [r3, #0x10]                 @ write GICD EOI
+
+       adr     r1, _smp_pen
+waitloop:
+       wfi
+       ldr     r0, =CONFIG_SYSFLAGS_ADDR       @ load start address
+       ldr     r0, [r0]
+       cmp     r0, r1                  @ make sure we dont execute this code
+       beq     waitloop                @ again (due to a spurious wakeup)
+       mov     pc, r0
arch/arm/lib/virt-v7.c +9 -1
index 416ca29..5ca093a 100644
@@ -29,6 +29,7 @@
 
  /* the assembly routine doing the actual work in start.S */
  void _nonsec_gic_switch(void);
+void _smp_pen(void);
 
  #define GICD_CTLR      0x000
  #define GICD_TYPER     0x004
@@ -51,6 +52,7 @@ int armv7_switch_nonsec(void)
         unsigned int reg;
         volatile unsigned int *gicdptr;
         unsigned itlinesnr, i;
+       unsigned int *sysflags;
 
         /* check whether the CPU supports the security extensions */
         asm("mrc p15, 0, %0, c0, c1, 1\n" : "=r"(reg));
@@ -109,7 +111,13 @@ int armv7_switch_nonsec(void)
         for (i = 0; i <= itlinesnr; i++)
                 gicdptr[GICD_IGROUPR0 / 4 + i] = (unsigned)-1;
 
-       /* call the non-sec switching code on this CPU */
+       /* now kick all CPUs (expect this one) by writing to GICD_SIGR */
+       sysflags = (void *)CONFIG_SYSFLAGS_ADDR;
+       sysflags[1] = (unsigned)-1;
+       sysflags[0] = (uintptr_t)_smp_pen;
+       gicdptr[GICD_SGIR / 4] = 1U << 24;
+
+       /* call the non-sec switching code on this CPU also */
         _nonsec_gic_switch();
 
         return 0;
include/configs/vexpress_ca15_tc2.h +0 -0
index 9e230ad..210a27c 100644
@@ -32,5 +32,6 @@
  #define CONFIG_BOOTP_VCI_STRING     "U-boot.armv7.vexpress_ca15x2_tc2"
 
Quoted 8 lines

include/configs/vexpress_ca15_tc2.h

Quoted 5 lines
include/configs/vexpress_ca15_tc2.h +1 -1
  #define CONFIG_SYS_CLK_FREQ 24000000
+#define CONFIG_SYSFLAGS_ADDR 0x1c010030
 
  #endif
--

hmm, is all this likely to work across all armv7 cores?

Probably not.

Quoted 3 lines

I imagined that the SMP boot protocol could be potentially vastly different on different implementations and this stuff would have to call into board specific code...?

Right. For now I am trying to get this code upstream. Support for other boards can then be merged in on demand. Looks like at least the Arndale board can be kicked very similarly, only the SYSFLAGS address is different. I am willing to re-architecture the SMP code as needed.

Regards,
Andre.

#8 lore raw

RFC PATCH 4/6 ARM: add SMP support for non-secure switch

Quoted 131 lines · Andre Przywara
Quoted 115 lines · Christoffer Dall
Quoted 113 lines · Andre Przywara

Currently the non-secure switch is only done for the boot processor. To later allow full SMP support, we have to switch all secondary cores into non-secure state also.

So we add an entry point for secondary CPUs coming out of low-power state and make sure we put them into WFI again after having switched to non-secure state.
For this we acknowledge and EOI the wake-up IPI, then go into WFI. Once being kicked out of it later, we sanity check that the start address has actually been changed (since another attempt to switch to non-secure would block the core) and jump to the new address.

The actual CPU kick is done by sending an inter-processor interrupt via the GIC to all CPU interfaces except the requesting processor. The secondary cores will then setup their respective GIC CPU interface.

The address secondary cores jump to is board specific, we provide the value here for the Versatile Express board.

---
 arch/arm/cpu/armv7/start.S          | 27 ++++++++++++++++++++++++++-
 arch/arm/lib/virt-v7.c              | 10 +++++++++-
 include/configs/vexpress_ca15_tc2.h |  1 +
 3 files changed, 36 insertions(+), 2 deletions(-)
arch/arm/cpu/armv7/start.S +26 -1
index 401b0eb..2b47881 100644
@@ -563,8 +563,19 @@ fiq:
 #endif /* CONFIG_SPL_BUILD */
 
 /* Routine to initialize GIC CPU interface and switch to nonsecure state.
+ * Will be executed directly by secondary CPUs after coming out of
+ * WFI, or can be called directly by C code for CPU 0.
+ * Those two paths mandate to not use any stack and to only use registers
+ * r0-r3 to comply with both the C ABI and the requirement of SMP startup
+ * code.
  */
 .globl _nonsec_gic_switch
+.globl _smp_pen
+_smp_pen:
+       mrs     r0, cpsr
+       orr     r0, r0, #0xc0
+       msr     cpsr, r0                        @ disable interrupts
+       mov     lr, #0                          @ clear LR to mark secondary
 _nonsec_gic_switch:
        mrc     p15, 4, r2, c15, c0, 0          @ r2 = PERIPHBASE
        add     r3, r2, #0x1000                 @ GIC dist i/f offset
@@ -605,4 +616,18 @@ _nonsec_gic_switch:
        add     r2, r2, #0x1000                 @ GIC dist i/f offset
        str     r1, [r2]                        @ allow private interrupts
 
-       mov     pc, lr
+       cmp     lr, #0
+       movne   pc, lr                          @ CPU 0 to return
+                                               @ all others: go to sleep
+_ack_int:
+       ldr     r1, [r3, #0x0c]                 @ read GICD acknowledge
+       str     r1, [r3, #0x10]                 @ write GICD EOI
+
+       adr     r1, _smp_pen
+waitloop:
+       wfi
+       ldr     r0, =CONFIG_SYSFLAGS_ADDR       @ load start address
+       ldr     r0, [r0]
+       cmp     r0, r1                  @ make sure we dont execute this code
+       beq     waitloop                @ again (due to a spurious wakeup)
+       mov     pc, r0
arch/arm/lib/virt-v7.c +9 -1
index 416ca29..5ca093a 100644
@@ -29,6 +29,7 @@
 
 /* the assembly routine doing the actual work in start.S */
 void _nonsec_gic_switch(void);
+void _smp_pen(void);
 
 #define GICD_CTLR      0x000
 #define GICD_TYPER     0x004
@@ -51,6 +52,7 @@ int armv7_switch_nonsec(void)
        unsigned int reg;
        volatile unsigned int *gicdptr;
        unsigned itlinesnr, i;
+       unsigned int *sysflags;
 
        /* check whether the CPU supports the security extensions */
        asm("mrc p15, 0, %0, c0, c1, 1\n" : "=r"(reg));
@@ -109,7 +111,13 @@ int armv7_switch_nonsec(void)
        for (i = 0; i <= itlinesnr; i++)
                gicdptr[GICD_IGROUPR0 / 4 + i] = (unsigned)-1;
 
-       /* call the non-sec switching code on this CPU */
+       /* now kick all CPUs (expect this one) by writing to GICD_SIGR */
+       sysflags = (void *)CONFIG_SYSFLAGS_ADDR;
+       sysflags[1] = (unsigned)-1;
+       sysflags[0] = (uintptr_t)_smp_pen;
+       gicdptr[GICD_SGIR / 4] = 1U << 24;
+
+       /* call the non-sec switching code on this CPU also */
        _nonsec_gic_switch();
 
        return 0;
include/configs/vexpress_ca15_tc2.h +0 -0
index 9e230ad..210a27c 100644
@@ -32,5 +32,6 @@
 #define CONFIG_BOOTP_VCI_STRING     "U-boot.armv7.vexpress_ca15x2_tc2"
 
Quoted 8 lines

include/configs/vexpress_ca15_tc2.h

Quoted 5 lines
include/configs/vexpress_ca15_tc2.h +1 -1
 #define CONFIG_SYS_CLK_FREQ 24000000
+#define CONFIG_SYSFLAGS_ADDR 0x1c010030
 
 #endif
--

hmm, is all this likely to work across all armv7 cores?

Probably not.

Quoted 3 lines

I imagined that the SMP boot protocol could be potentially vastly different on different implementations and this stuff would have to call into board specific code...?

Quoted 6 lines

Right. For now I am trying to get this code upstream. Support for other boards can then be merged in on demand. Looks like at least the Arndale board can be kicked very similarly, only the SYSFLAGS address is different. I am willing to re-architecture the SMP code as needed.

Sorry for the late response, I've been out sick.

So, I'm not very familiar with conventions and practices for the U-boot project, but it seems quite clear to me that since these changes go in arch/arm/lib/... paths those changes must be generic.

Therefore, I'm pretty sure that you need to factor out anything that's specific to vexpress into a part of the code that's specific to vexpress and thereby establish a board interface that generic code can call into for other boards too, for example Arndale; in fact it would be cool if you could show this working for both platforms at the same time, since the working code is already available.

-Christoffer

#9 lore raw

RFC PATCH 5/6 ARM: extend non-secure switch to also go into HYP mode

In reply to Andre Przywara

For the KVM and XEN hypervisors to be usable, we need to enter the kernel in HYP mode. Now that we already are in non-secure state, HYP mode switching is within short reach.

While doing the non-secure switch, we have to enable the HVC instruction and setup the HYP mode HVBAR (while still secure).

The actual switch is done by dropping back from a HYP mode handler without actually leaving HYP mode, so we introduce a new handler routine in the exception vector table.

In the assembly switching routine - which we rename to hyp_gic_switch on the way - we save and restore the banked LR and SP registers around the hypercall to do the actual HYP mode switch.

The C routine first checks whether we are in HYP mode already and also whether the virtualization extensions are available. It also checks whether the HYP mode switch was finally successful. The U-Boot command only adds and adjusts some error reporting.

---
 arch/arm/cpu/armv7/start.S   | 34 +++++++++++++++++++++++-----------
 arch/arm/include/asm/armv7.h |  2 +-
 arch/arm/lib/virt-v7.c       | 24 ++++++++++++++++--------
 common/cmd_virt.c            | 20 +++++++++++++-------
 4 files changed, 53 insertions(+), 27 deletions(-)
arch/arm/cpu/armv7/start.S +23 -11
index 2b47881..00890a3 100644
@@ -41,7 +41,7 @@ _start: b	reset
 	ldr	pc, _software_interrupt
 	ldr	pc, _prefetch_abort
 	ldr	pc, _data_abort
-	ldr	pc, _not_used
+	ldr	pc, _hyp_trap
 	ldr	pc, _irq
 	ldr	pc, _fiq
 #ifdef CONFIG_SPL_BUILD
@@ -49,7 +49,7 @@ _undefined_instruction: .word _undefined_instruction
 _software_interrupt:	.word _software_interrupt
 _prefetch_abort:	.word _prefetch_abort
 _data_abort:		.word _data_abort
-_not_used:		.word _not_used
+_hyp_trap:		.word _hyp_trap
 _irq:			.word _irq
 _fiq:			.word _fiq
 _pad:			.word 0x12345678 /* now 16*4=64 */
@@ -58,7 +58,7 @@ _undefined_instruction: .word undefined_instruction
 _software_interrupt:	.word software_interrupt
 _prefetch_abort:	.word prefetch_abort
 _data_abort:		.word data_abort
-_not_used:		.word not_used
+_hyp_trap:		.word hyp_trap
 _irq:			.word irq
 _fiq:			.word fiq
 _pad:			.word 0x12345678 /* now 16*4=64 */
@@ -502,12 +502,18 @@ software_interrupt:
 	mrc	p15, 0, r1, c1, c1, 0		@ read SCR
 	bic	r1, r1, #0x07f
 	orr	r1, r1, #0x31			@ enable NS, AW, FW
+	mrc	p15, 0, r0, c0, c1, 1		@ check for Virt ext
+	and	r0, r0, #0xf000
+	cmp	r0, #0x1000
+	orreq	r1, r1, #0x100			@ allow HVC instruction
 
 	mrc	p15, 0, r0, c12, c0, 0		@ save secure copy of VBAR
 	mcr	p15, 0, r1, c1, c1, 0		@ write SCR, switch to non-sec
 	isb
 	mcr	p15, 0, r0, c12, c0, 0		@ write non-secure copy of VBAR
 
+	mcreq	p15, 4, r0, c12, c0, 0		@ write HYP mode HVBAR
+
 	movs	pc, lr
 
 	.align	5
@@ -523,10 +529,9 @@ data_abort:
 	bl	do_data_abort
 
 	.align	5
-not_used:
-	get_bad_stack
-	bad_save_user_regs
-	bl	do_not_used
+hyp_trap:
+	.byte 0x00, 0xe3, 0x0e, 0xe1		@ mrs lr, elr_hyp
+	mov pc, lr
 
 #ifdef CONFIG_USE_IRQ
 
@@ -562,21 +567,21 @@ fiq:
 #endif /* CONFIG_USE_IRQ */
 #endif /* CONFIG_SPL_BUILD */
 
-/* Routine to initialize GIC CPU interface and switch to nonsecure state.
- * Will be executed directly by secondary CPUs after coming out of
+/* Routine to initialize GIC CPU interface, switch to nonsecure and to HYP
+ * mode. Will be executed directly by secondary CPUs after coming out of
  * WFI, or can be called directly by C code for CPU 0.
  * Those two paths mandate to not use any stack and to only use registers
  * r0-r3 to comply with both the C ABI and the requirement of SMP startup
  * code.
  */
-.globl _nonsec_gic_switch
+.globl _hyp_gic_switch
 .globl _smp_pen
 _smp_pen:
 	mrs	r0, cpsr
 	orr	r0, r0, #0xc0
 	msr	cpsr, r0			@ disable interrupts
 	mov	lr, #0				@ clear LR to mark secondary
-_nonsec_gic_switch:
+_hyp_gic_switch:
 	mrc	p15, 4, r2, c15, c0, 0		@ r2 = PERIPHBASE
 	add	r3, r2, #0x1000			@ GIC dist i/f offset
 	mvn	r1, #0
@@ -616,6 +621,13 @@ _nonsec_gic_switch:
 	add	r2, r2, #0x1000			@ GIC dist i/f offset
 	str	r1, [r2]			@ allow private interrupts
 
+	mov	r2, lr
+	mov	r1, sp
+	.byte 0x70, 0x00, 0x40, 0xe1		@ hvc #0
+	isb
+	mov	sp, r1
+	mov	lr, r2
+
 	cmp	lr, #0
 	movne	pc, lr				@ CPU 0 to return
 						@ all others: go to sleep
arch/arm/include/asm/armv7.h +1 -1
index 3567692..6c6955b 100644
@@ -74,6 +74,6 @@ void v7_outer_cache_inval_all(void);
 void v7_outer_cache_flush_range(u32 start, u32 end);
 void v7_outer_cache_inval_range(u32 start, u32 end);
 
-int armv7_switch_nonsec(void);
+int armv7_switch_hyp(void);
 
 #endif
arch/arm/lib/virt-v7.c +16 -8
index 5ca093a..72481c6 100644
@@ -3,6 +3,7 @@
  * Andre Przywara, Linaro
  *
  * routines to push ARMv7 processors from secure into non-secure state
+ * and from non-secure SVC into HYP mode
  * needed to enable ARMv7 virtualization for current hypervisors
  *
  * See file CREDITS for list of people who contributed to this
@@ -28,7 +29,7 @@
 #include <asm/armv7.h>
 
 /* the assembly routine doing the actual work in start.S */
-void _nonsec_gic_switch(void);
+void _hyp_gic_switch(void);
 void _smp_pen(void);
 
 #define GICD_CTLR	0x000
@@ -47,16 +48,20 @@ static inline unsigned int read_cpsr(void)
 	return reg;
 }
 
-int armv7_switch_nonsec(void)
+int armv7_switch_hyp(void)
 {
 	unsigned int reg;
 	volatile unsigned int *gicdptr;
 	unsigned itlinesnr, i;
 	unsigned int *sysflags;
 
-	/* check whether the CPU supports the security extensions */
+	/* check whether we are in HYP mode already */
+	if ((read_cpsr() & 0x1F) == 0x1a)
+		return 1;
+
+	/* check whether the CPU supports the virtualization extensions */
 	asm("mrc p15, 0, %0, c0, c1, 1\n" : "=r"(reg));
-	if ((reg & 0xF0) == 0)
+	if ((reg & 0xF000) != 0x1000)
 		return 2;
 
 	/* the timer frequency for the generic timer needs to be
@@ -77,8 +82,8 @@ int armv7_switch_nonsec(void)
 	 */
 
 	/* check whether we are an Cortex-A15 or A7.
-	 * The actual non-secure switch should work with all CPUs supporting
-	 * the security extension, but we need the GIC address,
+	 * The actual HYP switch should work with all CPUs supporting
+	 * the virtualization extension, but we need the GIC address,
 	 * which we know only for sure for those two CPUs.
 	 */
 	asm("mrc p15, 0, %0, c0, c0, 0\n" : "=r"(reg));
@@ -117,8 +122,11 @@ int armv7_switch_nonsec(void)
 	sysflags[0] = (uintptr_t)_smp_pen;
 	gicdptr[GICD_SGIR / 4] = 1U << 24;
 
-	/* call the non-sec switching code on this CPU also */
-	_nonsec_gic_switch();
+	/* call the HYP switching code on this CPU also */
+	_hyp_gic_switch();
+
+	if ((read_cpsr() & 0x1F) != 0x1a)
+		return 5;
 
 	return 0;
 }
common/cmd_virt.c +13 -7
index 132b6b1..78b1aa1 100644
@@ -2,8 +2,8 @@
  * (C) Copyright 2013
  * Andre Przywara, Linaro
  *
- * command to switch an ARMv7 CPU with security extensions into
- * non-secure state
+ * command to switch an ARMv7 CPU with virtualization extensions into
+ * HYP mode to allow hypervisors to install themselves
  *
  * See file CREDITS for list of people who contributed to this
  * project.
@@ -29,17 +29,20 @@
 
 #include <asm/armv7.h>
 
-static int do_nonsec(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[])
+static int do_hyp(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[])
 {
 	unsigned int ret;
 
-	ret = armv7_switch_nonsec();
+	ret = armv7_switch_hyp();
 
 	switch (ret) {
 	case 0:
 		break;
+	case 1:
+		printf("Already in HYP mode\n");
+		break;
 	case 2:
-		printf("Security extensions not implemented.\n");
+		printf("Virtualization extensions not implemented.\n");
 		break;
 	case 3:
 		printf("CPU not supported, must be either Cortex-A15 or A7.\n");
@@ -47,13 +50,16 @@ static int do_nonsec(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[])
 	case 4:
 		printf("PERIPHBASE is above 4 GB, cannot access this.\n");
 		break;
+	case 5:
+		printf("HYP mode switch not successful.\n");
+		break;
 	}
 
 	return ret;
 }
 
 U_BOOT_CMD(
-	hypmode, 1, 0, do_nonsec,
-	"switch ARM CPUs into non-secure state",
+	hypmode, 1, 0, do_hyp,
+	"switch ARM CPUs into HYP mode",
 	""
 );
Signature · 1 line
1.7.12.1
#10 lore raw

RFC PATCH 6/6 ARM: VExpress: enable ARMv7 virt support for VExpress A15

In reply to Andre Przywara

Enable the "hypmode" command for the Versatile Express TC2 board with the virtualization capable Cortex-A15 CPU to allow booting Xen or a KVM-enabled Linux kernel in HYP mode.

---
 include/configs/vexpress_ca15_tc2.h | 2 ++
 1 file changed, 2 insertions(+)
include/configs/vexpress_ca15_tc2.h +2 -0
index 210a27c..9df4400 100644
@@ -31,6 +31,8 @@
 #include "vexpress_common.h"
 #define CONFIG_BOOTP_VCI_STRING     "U-boot.armv7.vexpress_ca15x2_tc2"
 
+#define CONFIG_CMD_VIRT
+
 #define CONFIG_SYS_CLK_FREQ 24000000
 #define CONFIG_SYSFLAGS_ADDR 0x1c010030
 
Signature · 1 line
1.7.12.1
#11 lore raw

cover ARMv7: Add HYP mode switching support

In reply to Andre Przywara
Quoted 9 lines · Andre Przywara

ARM CPUs with the virtualization extension have a new mode called HYP mode, which allows hypervisors to safely control and monitor guests. The current hypervisor (KVM and Xen) implementations require the kernel to be entered in that HYP mode.

This patch series introduces a new U-Boot command called "hypmode" which can be used at any time at the U-Boot command prompt to switch the CPU into HYP mode - ideally this would be done before starting the kernel.

The obvious question here is "why do we need a new command?". The kernel booting specification says "boot the kernel in Hyp mode" so we should just always do that for booting Linux, surely?

thanks
-- PMM

#12 lore raw

cover ARMv7: Add HYP mode switching support

Quoted 11 lines · Peter Maydell
Quoted 9 lines · Andre Przywara

ARM CPUs with the virtualization extension have a new mode called HYP mode, which allows hypervisors to safely control and monitor guests. The current hypervisor (KVM and Xen) implementations require the kernel to be entered in that HYP mode.

This patch series introduces a new U-Boot command called "hypmode" which can be used at any time at the U-Boot command prompt to switch the CPU into HYP mode - ideally this would be done before starting the kernel.

Quoted 4 lines

The obvious question here is "why do we need a new command?". The kernel booting specification says "boot the kernel in Hyp mode" so we should just always do that for booting Linux, surely?

Because it avoids regressions. I kind of feel uneasy to do a lot of tinkering with secure state and the GIC unconditionally, especially if enabled on many boards with virt-capable CPUs.

As written in question 2) later in that mail, I can also live with a command to _dis_able the HYP mode switching in case it causes problems.

Regards,
Andre.

#13 lore raw

cover ARMv7: Add HYP mode switching support

Quoted 9 lines · Andre Przywara
Quoted 4 lines · Peter Maydell

The obvious question here is "why do we need a new command?". The kernel booting specification says "boot the kernel in Hyp mode" so we should just always do that for booting Linux, surely?

Because it avoids regressions. I kind of feel uneasy to do a lot of tinkering with secure state and the GIC unconditionally, especially if enabled on many boards with virt-capable CPUs.

There aren't exactly very many of those out there, so if we default to "boot in HYP mode" then (a) KVM will just work out of the box and (b) people doing u-boot ports to their board will find any issues and be able to submit fixes. If we don't turn it on by default then we'll end up stuck in a world where KVM doesn't work on half the virt capable boards out there.

I have no objection to a "turn this off" escape hatch if you think it's useful.

-- PMM

#14 lore raw

cover ARMv7: Add HYP mode switching support

Quoted 11 lines · Peter Maydell
Quoted 9 lines · Andre Przywara
Quoted 4 lines · Peter Maydell

The obvious question here is "why do we need a new command?". The kernel booting specification says "boot the kernel in Hyp mode" so we should just always do that for booting Linux, surely?

Because it avoids regressions. I kind of feel uneasy to do a lot of tinkering with secure state and the GIC unconditionally, especially if enabled on many boards with virt-capable CPUs.

Quoted 7 lines

There aren't exactly very many of those out there, so if we default to "boot in HYP mode" then (a) KVM will just work out of the box and (b) people doing u-boot ports to their board will find any issues and be able to submit fixes. If we don't turn it on by default then we'll end up stuck in a world where KVM doesn't work on half the virt capable boards out there.

OK, that's a point. I changed the code already and will include it in the next revision.

Thanks,
Andre.

Quoted 6 lines

I have no objection to a "turn this off" escape hatch if you think it's useful.

-- PMM

#15 lore raw

cover ARMv7: Add HYP mode switching support

Quoted 20 lines · Andre Przywara
Quoted 11 lines · Peter Maydell
Quoted 9 lines · Andre Przywara
Quoted 4 lines · Peter Maydell

The obvious question here is "why do we need a new command?". The kernel booting specification says "boot the kernel in Hyp mode" so we should just always do that for booting Linux, surely?

Because it avoids regressions. I kind of feel uneasy to do a lot of tinkering with secure state and the GIC unconditionally, especially if enabled on many boards with virt-capable CPUs.

Quoted 7 lines

There aren't exactly very many of those out there, so if we default to "boot in HYP mode" then (a) KVM will just work out of the box and (b) people doing u-boot ports to their board will find any issues and be able to submit fixes. If we don't turn it on by default then we'll end up stuck in a world where KVM doesn't work on half the virt capable boards out there.

Quoted 3 lines

OK, that's a point. I changed the code already and will include it in the next revision.

I strongly agree with Peter, and in fact I think it should only be a compile time option, not a run-time option, only for those debugging u-boot and a kernel on some new platform.

-Christoffer