armv7 BE kvm support
cover armv7 BE kvm support
Hi Guys,
Here is series that enables KVM support for V7 big endian kernels. Mostly it deals with BE KVM host support. Marc Zyngier showed before with his patches how BE guest could run on top LE host. With these patches BE guest runs on top of BE host. If Marc's kvmtool is used with few additional changes I tested that BE host could run LE guest. Also I verified that there were no regressions in BE guest on top of LE host case.
Note that posted series covers only kernel side changes. The changes were tested inside of bigger setup with additional changes in qemu and kvmtool. I will post those changes separately in proper aliases but for completeness sake Appendix A gives pointers to git repositories and branches with all needed changes.
Please note first patch is not related to BE KVM per se. I've run into an issue of conflicting 'push' identifier use while trying to include assembler.h into KVM .S files. Details of an issue I observed covered in Appendix B. The first patch is my take on solving it.
Victor Kamensky (5):
ARM: kvm: replace push and pop with stdmb and ldmia instrs to enable
assembler.h inclusion
ARM: fix KVM assembler files to work in BE case
ARM: kvm one_reg coproc set and get BE fixes
ARM: kvm vgic mmio should return data in BE format in BE case
ARM: kvm MMIO support BE host running LE code
arch/arm/include/asm/assembler.h | 7 +++ arch/arm/include/asm/kvm_asm.h | 4 +- arch/arm/include/asm/kvm_emulate.h | 22 +++++++-- arch/arm/kvm/coproc.c | 94 ++++++++++++++++++++++++++++---------- arch/arm/kvm/init.S | 7 ++- arch/arm/kvm/interrupts.S | 50 +++++++++++--------- arch/arm/kvm/interrupts_head.S | 61 +++++++++++++++---------- virt/kvm/arm/vgic.c | 4 +- 8 files changed, 168 insertions(+), 81 deletions(-)
Signature · 68 lines
1.8.1.4
Thanks,
Victor
Appendix A: Testing and Full Setup Description
----------------------------------------------
I) No mixed mode setup - i.e BE guest on BE host; and LE guest
on LE host tested to make sure no regressions.
KVM host and guest kernels:
TC2 on top of Linus 3.13-rc4 (this patch series):
git: git://git.linaro.org/people/victor.kamensky/linux-linaro-tracking-be.git
branch: armv7be-kvm-3.13-rc4
TC2 and Arndale on top of Linaro BE tree:
git: git://git.linaro.org/people/victor.kamensky/linux-linaro-tracking-be.git
branch: llct-be-20131216-kvm
- TC1 kernels used as guests
qemu:
git: git://git.linaro.org/people/victor.kamensky/qemu-be.git
branch: armv7be-v1
description: changes to run qemu on armeb target; and other
changes to work with be image on top of be host
kvmtool:
git: git://git.linaro.org/people/victor.kamensky/linux-linaro-tracking-be.git
branch: kvmtool-armv7be-v1
desciption: minimal changes to build kvmtool for armeb target; and
tiny change with virtio magic
II) Mixed mode setup all possible combinations within V7 (LE guest on BE host;
BE guest on LE host as Marc's setup tested to make sure no regressions) only
with kvmtool.
This work is based on Marc Zyngier's work that made BE guest to run on top
of LE host. For this setup special version of kvmtool should be used and
in addition I had to apply patch to guest kernel that would switch reading
virtio configs reads to be LE only, that is made on top of previous Rusty
Russell's changes. Effectively I just had to do very minor addition to make
LE guest to work on BE host, most of heavy lifting was done before by Marc.
KVM host kernels: as in previous setup
Guest TC1 kernels with LE virtio config patch:
git: git://git.linaro.org/people/victor.kamensky/linux-linaro-tracking-be.git
branch: virtio-leconfig-3.13-rc4
kvmtool:
git: git://git.linaro.org/people/victor.kamensky/linux-linaro-tracking-be.git
branch: kvmtool-mixed-v1
description: based on git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git
branch kvm-arm64/kvmtool-be-on-le; adds missing include fix; above armeb target
build patches; and one fix related to BE mode
qemu:
git: git://git.linaro.org/people/victor.kamensky/qemu-be.git
branch: armv7be-leconfig-v1
description: change virtio-blk that so qemu could work with guest image
where virtio leconfig is made; note it does not work in mixed mode; to do
so qemu would need bunch of similar changes that Marc did in kvmtool
Appendix B: kvm asm file and asm/assembler.h file issue
-------------------------------------------------------arch/arm/kvm/interrupts.S +1 -0
@@ -25,6 +25,7 @@ #include <asm/kvm_asm.h> #include <asm/kvm_arm.h> #include <asm/vfpmacros.h> +#include <asm/assembler.h> #include "interrupts_head.S" .text
produce the following compilation errors:
/run/media/kamensky/wd/linaro/linux-linaro-core-tracking/092913/linux-linaro-tracking-be/arch/arm/kvm/interrupts.S: Assembler messages:/run/media/kamensky/wd/linaro/linux-linaro-core-tracking/092913/linux-linaro-tracking-be/arch/arm/kvm/interrupts.S:51: Error: ARM register expected -- lsr {r2,r3}' /run/media/kamensky/wd/linaro/linux-linaro-core-tracking/092913/linux-linaro-tracking-be/arch/arm/kvm/interrupts.S:100: Error: ARM register expected -- lsr {r2}' /run/media/kamensky/wd/linaro/linux-linaro-core-tracking/092913/linux-linaro-tracking-be/arch/arm/kvm/interrupts.S:100: Error: ARM register expected -- `lsr {r4-r12}'
PATCH 1/5 ARM: kvm: replace push and pop with stdmb and ldmia instrs to enable assembler.h inclusion
Before fix kvm interrupt.S and interrupt_head.S used push and pop assembler instruction. It causes problem if <asm/assembler.h> file should be include. In assembler.h "push" is defined as macro so it causes compilation errors like this:
arch/arm/kvm/interrupts.S: Assembler messages:arch/arm/kvm/interrupts.S:51: Error: ARM register expected -- `lsr {r2,r3}'
Solution implemented by this patch replaces all 'push {...}' with 'stdmb sp!, {...}' instruction; and all 'pop {...}' with 'ldmia sp!, {...}'.
--- arch/arm/kvm/interrupts.S | 38 +++++++++++++++++++------------------- arch/arm/kvm/interrupts_head.S | 34 +++++++++++++++++----------------- 2 files changed, 36 insertions(+), 36 deletions(-)
arch/arm/kvm/interrupts.S +19 -19
@@ -47,7 +47,7 @@ __kvm_hyp_code_start: * instead, ignoring the ipa value. */ ENTRY(__kvm_tlb_flush_vmid_ipa) - push {r2, r3} + stmdb sp!, {r2, r3} dsb ishst add r0, r0, #KVM_VTTBR
@@ -62,7 +62,7 @@ ENTRY(__kvm_tlb_flush_vmid_ipa) mcrr p15, 6, r2, r3, c2 @ Back to VMID #0 isb @ Not necessary if followed by eret - pop {r2, r3} + ldmia sp!, {r2, r3} bx lr ENDPROC(__kvm_tlb_flush_vmid_ipa)
@@ -110,7 +110,7 @@ ENTRY(__kvm_vcpu_run) #ifdef CONFIG_VFPv3 @ Set FPEXC_EN so the guest doesn't trap floating point instructions VFPFMRX r2, FPEXC @ VMRS - push {r2} + stmdb sp!, {r2} orr r2, r2, #FPEXC_EN VFPFMXR FPEXC, r2 @ VMSR #endif
@@ -175,7 +175,7 @@ __kvm_vcpu_return: after_vfp_restore: @ Restore FPEXC_EN which we clobbered on entry - pop {r2} + ldmia sp!, {r2} VFPFMXR FPEXC, r2 #endif
@@ -260,7 +260,7 @@ ENTRY(kvm_call_hyp) /* Handle undef, svc, pabt, or dabt by crashing with a user notice */ .macro bad_exception exception_code, panic_str - push {r0-r2} + stmdb sp!, {r0-r2} mrrc p15, 6, r0, r1, c2 @ Read VTTBR lsr r1, r1, #16 ands r1, r1, #0xff
@@ -338,7 +338,7 @@ hyp_hvc: * Getting here is either becuase of a trap from a guest or from calling * HVC from the host kernel, which means "switch to Hyp mode". */ - push {r0, r1, r2} + stmdb sp!, {r0, r1, r2} @ Check syndrome register mrc p15, 4, r1, c5, c2, 0 @ HSR
@@ -361,11 +361,11 @@ hyp_hvc: bne guest_trap @ Guest called HVC host_switch_to_hyp: - pop {r0, r1, r2} + ldmia sp!, {r0, r1, r2} - push {lr} + stmdb sp!, {lr} mrs lr, SPSR - push {lr} + stmdb sp!, {lr} mov lr, r0 mov r0, r1
@@ -375,9 +375,9 @@ host_switch_to_hyp: THUMB( orr lr, #1) blx lr @ Call the HYP function - pop {lr} + ldmia sp!, {lr} msr SPSR_csxf, lr - pop {lr} + ldmia sp!, {lr} eret guest_trap:
@@ -418,7 +418,7 @@ guest_trap: /* Preserve PAR */ mrrc p15, 0, r0, r1, c7 @ PAR - push {r0, r1} + stmdb sp!, {r0, r1} /* Resolve IPA using the xFAR */ mcr p15, 0, r2, c7, c8, 0 @ ATS1CPR
@@ -431,7 +431,7 @@ guest_trap: orr r2, r2, r1, lsl #24 /* Restore PAR */ - pop {r0, r1} + ldmia sp!, {r0, r1} mcrr p15, 0, r0, r1, c7 @ PAR 3: load_vcpu @ Load VCPU pointer to r0
@@ -440,10 +440,10 @@ guest_trap: 1: mov r1, #ARM_EXCEPTION_HVC b __kvm_vcpu_return -4: pop {r0, r1} @ Failed translation, return to guest +4: ldmia sp!, {r0, r1} @ Failed translation, return to guest mcrr p15, 0, r0, r1, c7 @ PAR clrex - pop {r0, r1, r2} + ldmia sp!, {r0, r1, r2} eret /*
@@ -455,7 +455,7 @@ guest_trap: #ifdef CONFIG_VFPv3 switch_to_guest_vfp: load_vcpu @ Load VCPU pointer to r0 - push {r3-r7} + stmdb sp!, {r3-r7} @ NEON/VFP used. Turn on VFP access. set_hcptr vmexit, (HCPTR_TCP(10) | HCPTR_TCP(11))
@@ -467,15 +467,15 @@ switch_to_guest_vfp: add r7, r0, #VCPU_VFP_GUEST restore_vfp_state r7 - pop {r3-r7} - pop {r0-r2} + ldmia sp!, {r3-r7} + ldmia sp!, {r0-r2} clrex eret #endif .align hyp_irq: - push {r0, r1, r2} + stmdb sp!, {r0, r1, r2} mov r1, #ARM_EXCEPTION_IRQ load_vcpu @ Load VCPU pointer to r0 b __kvm_vcpu_return
arch/arm/kvm/interrupts_head.S +17 -17
@@ -63,7 +63,7 @@ vcpu .req r0 @ vcpu pointer always in r0 mrs r2, SP_\mode mrs r3, LR_\mode mrs r4, SPSR_\mode - push {r2, r3, r4} + stmdb sp!, {r2, r3, r4} .endm /*
@@ -73,13 +73,13 @@ vcpu .req r0 @ vcpu pointer always in r0 .macro save_host_regs /* Hyp regs. Only ELR_hyp (SPSR_hyp already saved) */ mrs r2, ELR_hyp - push {r2} + stmdb sp!, {r2} /* usr regs */ - push {r4-r12} @ r0-r3 are always clobbered + stmdb sp!, {r4-r12} @ r0-r3 are always clobbered mrs r2, SP_usr mov r3, lr - push {r2, r3} + stmdb sp!, {r2, r3} push_host_regs_mode svc push_host_regs_mode abt
@@ -95,11 +95,11 @@ vcpu .req r0 @ vcpu pointer always in r0 mrs r7, SP_fiq mrs r8, LR_fiq mrs r9, SPSR_fiq - push {r2-r9} + stmdb sp!, {r2-r9} .endm .macro pop_host_regs_mode mode - pop {r2, r3, r4} + ldmia sp!, {r2, r3, r4} msr SP_\mode, r2 msr LR_\mode, r3 msr SPSR_\mode, r4
@@ -110,7 +110,7 @@ vcpu .req r0 @ vcpu pointer always in r0 * Clobbers all registers, in all modes, except r0 and r1. */ .macro restore_host_regs - pop {r2-r9} + ldmia sp!, {r2-r9} msr r8_fiq, r2 msr r9_fiq, r3 msr r10_fiq, r4
@@ -125,12 +125,12 @@ vcpu .req r0 @ vcpu pointer always in r0 pop_host_regs_mode abt pop_host_regs_mode svc - pop {r2, r3} + ldmia sp!, {r2, r3} msr SP_usr, r2 mov lr, r3 - pop {r4-r12} + ldmia sp!, {r4-r12} - pop {r2} + ldmia sp!, {r2} msr ELR_hyp, r2 .endm
@@ -218,7 +218,7 @@ vcpu .req r0 @ vcpu pointer always in r0 add r2, vcpu, #VCPU_USR_REG(3) stm r2, {r3-r12} add r2, vcpu, #VCPU_USR_REG(0) - pop {r3, r4, r5} @ r0, r1, r2 + ldmia sp!, {r3, r4, r5} @ r0, r1, r2 stm r2, {r3, r4, r5} mrs r2, SP_usr mov r3, lr
@@ -258,7 +258,7 @@ vcpu .req r0 @ vcpu pointer always in r0 mrc p15, 2, r12, c0, c0, 0 @ CSSELR .if \store_to_vcpu == 0 - push {r2-r12} @ Push CP15 registers + stmdb sp!, {r2-r12} @ Push CP15 registers .else str r2, [vcpu, #CP15_OFFSET(c1_SCTLR)] str r3, [vcpu, #CP15_OFFSET(c1_CPACR)]
@@ -286,7 +286,7 @@ vcpu .req r0 @ vcpu pointer always in r0 mrc p15, 0, r12, c12, c0, 0 @ VBAR .if \store_to_vcpu == 0 - push {r2-r12} @ Push CP15 registers + stmdb sp!, {r2-r12} @ Push CP15 registers .else str r2, [vcpu, #CP15_OFFSET(c13_CID)] str r3, [vcpu, #CP15_OFFSET(c13_TID_URW)]
@@ -305,7 +305,7 @@ vcpu .req r0 @ vcpu pointer always in r0 mrrc p15, 0, r4, r5, c7 @ PAR .if \store_to_vcpu == 0 - push {r2,r4-r5} + stmdb sp!, {r2,r4-r5} .else str r2, [vcpu, #CP15_OFFSET(c14_CNTKCTL)] add r12, vcpu, #CP15_OFFSET(c7_PAR)
@@ -322,7 +322,7 @@ vcpu .req r0 @ vcpu pointer always in r0 */ .macro write_cp15_state read_from_vcpu .if \read_from_vcpu == 0 - pop {r2,r4-r5} + ldmia sp!, {r2,r4-r5} .else ldr r2, [vcpu, #CP15_OFFSET(c14_CNTKCTL)] add r12, vcpu, #CP15_OFFSET(c7_PAR)
@@ -333,7 +333,7 @@ vcpu .req r0 @ vcpu pointer always in r0 mcrr p15, 0, r4, r5, c7 @ PAR .if \read_from_vcpu == 0 - pop {r2-r12} + ldmia sp!, {r2-r12} .else ldr r2, [vcpu, #CP15_OFFSET(c13_CID)] ldr r3, [vcpu, #CP15_OFFSET(c13_TID_URW)]
@@ -361,7 +361,7 @@ vcpu .req r0 @ vcpu pointer always in r0 mcr p15, 0, r12, c12, c0, 0 @ VBAR .if \read_from_vcpu == 0 - pop {r2-r12} + ldmia sp!, {r2-r12} .else ldr r2, [vcpu, #CP15_OFFSET(c1_SCTLR)] ldr r3, [vcpu, #CP15_OFFSET(c1_CPACR)]
Signature · 1 line
1.8.1.4
PATCH 1/5 ARM: kvm: replace push and pop with stdmb and ldmia instrs to enable assembler.h inclusion
Quoted 22 lines · Victor Kamensky
Before fix kvm
interrupt.Sandinterrupt_head.Sused push and pop assembler instruction. It causes problem if <asm/assembler.h> file should be include. Inassembler.h"push" is defined as macro so it causes compilation errors like this:
arch/arm/kvm/interrupts.S: Assembler messages:arch/arm/kvm/interrupts.S:51: Error: ARM register expected -- `lsr {r2,r3}'Solution implemented by this patch replaces all 'push {...}' with 'stdmb sp!, {...}' instruction; and all 'pop {...}' with 'ldmia sp!, {...}'.
Signed-off-by Victor Kamensky <[email protected]>--- arch/arm/kvm/interrupts.S | 38 +++++++++++++++++++------------------- arch/arm/kvm/interrupts_head.S | 34 +++++++++++++++++----------------- 2 files changed, 36 insertions(+), 36 deletions(-)arch/arm/kvm/interrupts.S +0
-0
arch/arm/kvm/interrupts.S __kvm_hyp_code_start: · line 50
Quoted 5 lines
arch/arm/kvm/interrupts.S +1
-1@@ -47,7 +47,7 @@ __kvm_hyp_code_start: * instead, ignoring the ipa value. */ ENTRY(__kvm_tlb_flush_vmid_ipa) - push {r2, r3} + stmdb sp!, {r2, r3}
Minor nit: when pushing and popping the stack, the convention for kernel code is to write
stmfd sp!, ldmfd sp!,
instead of
stmdb sp!, ldmia sp!,
This will have no effect on the assembly, but may be slightly easier to read.
Quoted 13 lines
arch/arm/kvm/interrupts.S +1
-1dsb ishst add r0, r0, #KVM_VTTBR@@ -62,7 +62,7 @@ ENTRY(__kvm_tlb_flush_vmid_ipa) mcrr p15, 6, r2, r3, c2 @ Back to VMID #0 isb @ Not necessary if followed by eret - pop {r2, r3} + ldmia sp!, {r2, r3} bx lr ENDPROC(__kvm_tlb_flush_vmid_ipa)
arch/arm/kvm/interrupts.S ENTRY(__kvm_vcpu_run) · line 113
Quoted 5 lines
arch/arm/kvm/interrupts.S +1
-1@@ -110,7 +110,7 @@ ENTRY(__kvm_vcpu_run) #ifdef CONFIG_VFPv3 @ Set FPEXC_EN so the guest doesn't trap floating point instructions VFPFMRX r2, FPEXC @ VMRS - push {r2} + stmdb sp!, {r2}
For saving or restoring a single register, you may want to do:
str r2, [sp, #-4]! ... ldr r2, [sp], #4
This is the commonly used convention, and may be marginally more efficient. The "push" and "pop" assembler mnemonics generate these instead of stm/ldm if the list only contains a single register, whereas ldm/stm always generate a real load or store multiple even if there is only one register in the list.
This is a trivial issue, so only worth fixing if you are going to repost the patch anyway.
Quoted 256 lines
arch/arm/kvm/interrupts.S +16
-16orr r2, r2, #FPEXC_EN VFPFMXR FPEXC, r2 @ VMSR #endif@@ -175,7 +175,7 @@ __kvm_vcpu_return: after_vfp_restore: @ Restore FPEXC_EN which we clobbered on entry - pop {r2} + ldmia sp!, {r2} VFPFMXR FPEXC, r2 #endif@@ -260,7 +260,7 @@ ENTRY(kvm_call_hyp) /* Handle undef, svc, pabt, or dabt by crashing with a user notice */ .macro bad_exception exception_code, panic_str - push {r0-r2} + stmdb sp!, {r0-r2} mrrc p15, 6, r0, r1, c2 @ Read VTTBR lsr r1, r1, #16 ands r1, r1, #0xff@@ -338,7 +338,7 @@ hyp_hvc: * Getting here is either becuase of a trap from a guest or from calling * HVC from the host kernel, which means "switch to Hyp mode". */ - push {r0, r1, r2} + stmdb sp!, {r0, r1, r2} @ Check syndrome register mrc p15, 4, r1, c5, c2, 0 @ HSR@@ -361,11 +361,11 @@ hyp_hvc: bne guest_trap @ Guest called HVC host_switch_to_hyp: - pop {r0, r1, r2} + ldmia sp!, {r0, r1, r2} - push {lr} + stmdb sp!, {lr} mrs lr, SPSR - push {lr} + stmdb sp!, {lr} mov lr, r0 mov r0, r1@@ -375,9 +375,9 @@ host_switch_to_hyp: THUMB( orr lr, #1) blx lr @ Call the HYP function - pop {lr} + ldmia sp!, {lr} msr SPSR_csxf, lr - pop {lr} + ldmia sp!, {lr} eret guest_trap:@@ -418,7 +418,7 @@ guest_trap: /* Preserve PAR */ mrrc p15, 0, r0, r1, c7 @ PAR - push {r0, r1} + stmdb sp!, {r0, r1} /* Resolve IPA using the xFAR */ mcr p15, 0, r2, c7, c8, 0 @ ATS1CPR@@ -431,7 +431,7 @@ guest_trap: orr r2, r2, r1, lsl #24 /* Restore PAR */ - pop {r0, r1} + ldmia sp!, {r0, r1} mcrr p15, 0, r0, r1, c7 @ PAR 3: load_vcpu @ Load VCPU pointer to r0@@ -440,10 +440,10 @@ guest_trap: 1: mov r1, #ARM_EXCEPTION_HVC b __kvm_vcpu_return -4: pop {r0, r1} @ Failed translation, return to guest +4: ldmia sp!, {r0, r1} @ Failed translation, return to guest mcrr p15, 0, r0, r1, c7 @ PAR clrex - pop {r0, r1, r2} + ldmia sp!, {r0, r1, r2} eret /*@@ -455,7 +455,7 @@ guest_trap: #ifdef CONFIG_VFPv3 switch_to_guest_vfp: load_vcpu @ Load VCPU pointer to r0 - push {r3-r7} + stmdb sp!, {r3-r7} @ NEON/VFP used. Turn on VFP access. set_hcptr vmexit, (HCPTR_TCP(10) | HCPTR_TCP(11))@@ -467,15 +467,15 @@ switch_to_guest_vfp: add r7, r0, #VCPU_VFP_GUEST restore_vfp_state r7 - pop {r3-r7} - pop {r0-r2} + ldmia sp!, {r3-r7} + ldmia sp!, {r0-r2} clrex eret #endif .align hyp_irq: - push {r0, r1, r2} + stmdb sp!, {r0, r1, r2} mov r1, #ARM_EXCEPTION_IRQ load_vcpu @ Load VCPU pointer to r0 b __kvm_vcpu_returnarch/arm/kvm/interrupts_head.S +17
-17@@ -63,7 +63,7 @@ vcpu .req r0 @ vcpu pointer always in r0 mrs r2, SP_\mode mrs r3, LR_\mode mrs r4, SPSR_\mode - push {r2, r3, r4} + stmdb sp!, {r2, r3, r4} .endm /*@@ -73,13 +73,13 @@ vcpu .req r0 @ vcpu pointer always in r0 .macro save_host_regs /* Hyp regs. Only ELR_hyp (SPSR_hyp already saved) */ mrs r2, ELR_hyp - push {r2} + stmdb sp!, {r2} /* usr regs */ - push {r4-r12} @ r0-r3 are always clobbered + stmdb sp!, {r4-r12} @ r0-r3 are always clobbered mrs r2, SP_usr mov r3, lr - push {r2, r3} + stmdb sp!, {r2, r3} push_host_regs_mode svc push_host_regs_mode abt@@ -95,11 +95,11 @@ vcpu .req r0 @ vcpu pointer always in r0 mrs r7, SP_fiq mrs r8, LR_fiq mrs r9, SPSR_fiq - push {r2-r9} + stmdb sp!, {r2-r9} .endm .macro pop_host_regs_mode mode - pop {r2, r3, r4} + ldmia sp!, {r2, r3, r4} msr SP_\mode, r2 msr LR_\mode, r3 msr SPSR_\mode, r4@@ -110,7 +110,7 @@ vcpu .req r0 @ vcpu pointer always in r0 * Clobbers all registers, in all modes, except r0 and r1. */ .macro restore_host_regs - pop {r2-r9} + ldmia sp!, {r2-r9} msr r8_fiq, r2 msr r9_fiq, r3 msr r10_fiq, r4@@ -125,12 +125,12 @@ vcpu .req r0 @ vcpu pointer always in r0 pop_host_regs_mode abt pop_host_regs_mode svc - pop {r2, r3} + ldmia sp!, {r2, r3} msr SP_usr, r2 mov lr, r3 - pop {r4-r12} + ldmia sp!, {r4-r12} - pop {r2} + ldmia sp!, {r2} msr ELR_hyp, r2 .endm@@ -218,7 +218,7 @@ vcpu .req r0 @ vcpu pointer always in r0 add r2, vcpu, #VCPU_USR_REG(3) stm r2, {r3-r12} add r2, vcpu, #VCPU_USR_REG(0) - pop {r3, r4, r5} @ r0, r1, r2 + ldmia sp!, {r3, r4, r5} @ r0, r1, r2 stm r2, {r3, r4, r5} mrs r2, SP_usr mov r3, lr@@ -258,7 +258,7 @@ vcpu .req r0 @ vcpu pointer always in r0 mrc p15, 2, r12, c0, c0, 0 @ CSSELR .if \store_to_vcpu == 0 - push {r2-r12} @ Push CP15 registers + stmdb sp!, {r2-r12} @ Push CP15 registers .else str r2, [vcpu, #CP15_OFFSET(c1_SCTLR)] str r3, [vcpu, #CP15_OFFSET(c1_CPACR)]@@ -286,7 +286,7 @@ vcpu .req r0 @ vcpu pointer always in r0 mrc p15, 0, r12, c12, c0, 0 @ VBAR .if \store_to_vcpu == 0 - push {r2-r12} @ Push CP15 registers + stmdb sp!, {r2-r12} @ Push CP15 registers .else str r2, [vcpu, #CP15_OFFSET(c13_CID)] str r3, [vcpu, #CP15_OFFSET(c13_TID_URW)]@@ -305,7 +305,7 @@ vcpu .req r0 @ vcpu pointer always in r0 mrrc p15, 0, r4, r5, c7 @ PAR .if \store_to_vcpu == 0 - push {r2,r4-r5} + stmdb sp!, {r2,r4-r5} .else str r2, [vcpu, #CP15_OFFSET(c14_CNTKCTL)] add r12, vcpu, #CP15_OFFSET(c7_PAR)@@ -322,7 +322,7 @@ vcpu .req r0 @ vcpu pointer always in r0 */ .macro write_cp15_state read_from_vcpu .if \read_from_vcpu == 0 - pop {r2,r4-r5} + ldmia sp!, {r2,r4-r5} .else ldr r2, [vcpu, #CP15_OFFSET(c14_CNTKCTL)] add r12, vcpu, #CP15_OFFSET(c7_PAR)@@ -333,7 +333,7 @@ vcpu .req r0 @ vcpu pointer always in r0 mcrr p15, 0, r4, r5, c7 @ PAR .if \read_from_vcpu == 0 - pop {r2-r12} + ldmia sp!, {r2-r12} .else ldr r2, [vcpu, #CP15_OFFSET(c13_CID)] ldr r3, [vcpu, #CP15_OFFSET(c13_TID_URW)]@@ -361,7 +361,7 @@ vcpu .req r0 @ vcpu pointer always in r0 mcr p15, 0, r12, c12, c0, 0 @ VBAR .if \read_from_vcpu == 0 - pop {r2-r12} + ldmia sp!, {r2-r12} .else ldr r2, [vcpu, #CP15_OFFSET(c1_SCTLR)] ldr r3, [vcpu, #CP15_OFFSET(c1_CPACR)]Signature · 7 lines
1.8.1.4 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel at lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH 2/5 ARM: fix KVM assembler files to work in BE case
In reply to Victor KamenskyARM v7 KVM assembler files fixes to work in big endian mode:
vgic h/w registers are little endian; when asm code reads/writes from/to them, it needs to do byteswap after/before. Byte swap code uses ARM_BE8 wrapper to add swap only if BIG_ENDIAN kernel is configured
mcrr and mrrc instructions take couple 32 bit registers as argument, one is supposed to be high part of 64 bit value and another is low part of 64 bit. Typically those values are loaded/stored with ldrd and strd instructions and those will load high and low parts in opposite register depending on endianity. Introduce and use rr_lo_hi macro that swap registers in BE mode when they are passed to mcrr and mrrc instructions.
function that returns 64 bit result __kvm_vcpu_run in couple registers has to be adjusted for BE case.
--- arch/arm/include/asm/assembler.h | 7 +++++++ arch/arm/include/asm/kvm_asm.h | 4 ++-- arch/arm/kvm/init.S | 7 +++++-- arch/arm/kvm/interrupts.S | 12 +++++++++--- arch/arm/kvm/interrupts_head.S | 27 ++++++++++++++++++++------- 5 files changed, 43 insertions(+), 14 deletions(-)
arch/arm/include/asm/assembler.h +7 -0
@@ -60,6 +60,13 @@ #define ARM_BE8(code...)#endif +/* swap pair of registers position depending on current endianity */ +#ifdef CONFIG_CPU_ENDIAN_BE8 +#define rr_lo_hi (a1, a2) a2, a1 +#else +#define rr_lo_hi (a1, a2) a1, a2 +#endif +/* * Data preload for architectures that support it */
arch/arm/include/asm/kvm_asm.h +2 -2
@@ -26,9 +26,9 @@#define c1_ACTLR 4/* Auxilliary Control Register */ #define c1_CPACR 5/* Coprocessor Access Control */ #define c2_TTBR0 6/* Translation Table Base Register 0 */ -#define c2_TTBR0_high 7/* TTBR0 top 32 bits */ +#define c2_TTBR0_hilo 7/* TTBR0 top 32 bits in LE case, low 32 bits in BE case */ #define c2_TTBR1 8/* Translation Table Base Register 1 */ -#define c2_TTBR1_high 9/* TTBR1 top 32 bits */ +#define c2_TTBR1_hilo 9/* TTBR1 top 32 bits in LE case, low 32 bits in BE case */ #define c2_TTBCR 10/* Translation Table Base Control R. */ #define c3_DACR 11/* Domain Access Control Register */ #define c5_DFSR 12/* Data Fault Status Register */
arch/arm/kvm/init.S +5 -2
@@ -22,6 +22,7 @@ #include <asm/kvm_asm.h> #include <asm/kvm_arm.h> #include <asm/kvm_mmu.h> +#include <asm/assembler.h> /******************************************************************** * Hypervisor initialization
@@ -70,8 +71,10 @@ __do_hyp_init: cmp r0, #0 @ We have a SP? bne phase2 @ Yes, second stage init +ARM_BE8(setend be) @ Switch to Big Endian mode if needed + @ Set the HTTBR to point to the hypervisor PGD pointer passed - mcrr p15, 4, r2, r3, c2 + mcrr p15, 4, rr_lo_hi(r2, r3), c2 @ Set the HTCR and VTCR to the same shareability and cacheability @ settings as the non-secure TTBCR and with T0SZ == 0.
@@ -137,7 +140,7 @@ phase2: mov pc, r0 target: @ We're now in the trampoline code, switch page tables - mcrr p15, 4, r2, r3, c2 + mcrr p15, 4, rr_lo_hi(r2, r3), c2 isb @ Invalidate the old TLBs
arch/arm/kvm/interrupts.S +9 -3
@@ -25,6 +25,7 @@ #include <asm/kvm_asm.h> #include <asm/kvm_arm.h> #include <asm/vfpmacros.h> +#include <asm/assembler.h> #include "interrupts_head.S" .text
@@ -52,14 +53,14 @@ ENTRY(__kvm_tlb_flush_vmid_ipa) dsb ishst add r0, r0, #KVM_VTTBR ldrd r2, r3, [r0] - mcrr p15, 6, r2, r3, c2 @ Write VTTBR + mcrr p15, 6, rr_lo_hi(r2, r3), c2 @ Write VTTBR isb mcr p15, 0, r0, c8, c3, 0 @ TLBIALLIS (rt ignored) dsb ish isb mov r2, #0 mov r3, #0 - mcrr p15, 6, r2, r3, c2 @ Back to VMID #0 + mcrr p15, 6, rr_lo_hi(r2, r3), c2 @ Back to VMID #0 isb @ Not necessary if followed by eret ldmia sp!, {r2, r3}
@@ -135,7 +136,7 @@ ENTRY(__kvm_vcpu_run) ldr r1, [vcpu, #VCPU_KVM] add r1, r1, #KVM_VTTBR ldrd r2, r3, [r1] - mcrr p15, 6, r2, r3, c2 @ Write VTTBR + mcrr p15, 6, rr_lo_hi(r2, r3), c2 @ Write VTTBR @ We're all done, just restore the GPRs and go to the guest restore_guest_regs
@@ -199,8 +200,13 @@ after_vfp_restore: restore_host_regs clrex @ Clear exclusive monitor +#ifndef __ARMEB__ mov r0, r1 @ Return the return code mov r1, #0 @ Clear upper bits in return value +#else + @ r1 already has return code + mov r0, #0 @ Clear upper bits in return value +#endif /* __ARMEB__ */ bx lr @ return to IOCTL /********************************************************************
arch/arm/kvm/interrupts_head.S +20 -7
@@ -251,8 +251,8 @@ vcpu .req r0 @ vcpu pointer always in r0 mrc p15, 0, r3, c1, c0, 2 @ CPACR mrc p15, 0, r4, c2, c0, 2 @ TTBCR mrc p15, 0, r5, c3, c0, 0 @ DACR - mrrc p15, 0, r6, r7, c2 @ TTBR 0 - mrrc p15, 1, r8, r9, c2 @ TTBR 1 + mrrc p15, 0, rr_lo_hi(r6, r7), c2 @ TTBR 0 + mrrc p15, 1, rr_lo_hi(r8, r9), c2 @ TTBR 1 mrc p15, 0, r10, c10, c2, 0 @ PRRR mrc p15, 0, r11, c10, c2, 1 @ NMRR mrc p15, 2, r12, c0, c0, 0 @ CSSELR
@@ -380,8 +380,8 @@ vcpu .req r0 @ vcpu pointer always in r0 mcr p15, 0, r3, c1, c0, 2 @ CPACR mcr p15, 0, r4, c2, c0, 2 @ TTBCR mcr p15, 0, r5, c3, c0, 0 @ DACR - mcrr p15, 0, r6, r7, c2 @ TTBR 0 - mcrr p15, 1, r8, r9, c2 @ TTBR 1 + mcrr p15, 0, rr_lo_hi(r6, r7), c2 @ TTBR 0 + mcrr p15, 1, rr_lo_hi(r8, r9), c2 @ TTBR 1 mcr p15, 0, r10, c10, c2, 0 @ PRRR mcr p15, 0, r11, c10, c2, 1 @ NMRR mcr p15, 2, r12, c0, c0, 0 @ CSSELR
@@ -413,13 +413,21 @@ vcpu .req r0 @ vcpu pointer always in r0 ldr r9, [r2, #GICH_ELRSR1] ldr r10, [r2, #GICH_APR] +ARM_BE8(rev r3, r3 ) str r3, [r11, #VGIC_CPU_HCR] +ARM_BE8(rev r4, r4 ) str r4, [r11, #VGIC_CPU_VMCR] +ARM_BE8(rev r5, r5 ) str r5, [r11, #VGIC_CPU_MISR] +ARM_BE8(rev r6, r6 ) str r6, [r11, #VGIC_CPU_EISR] +ARM_BE8(rev r7, r7 ) str r7, [r11, #(VGIC_CPU_EISR + 4)] +ARM_BE8(rev r8, r8 ) str r8, [r11, #VGIC_CPU_ELRSR] +ARM_BE8(rev r9, r9 ) str r9, [r11, #(VGIC_CPU_ELRSR + 4)] +ARM_BE8(rev r10, r10 ) str r10, [r11, #VGIC_CPU_APR] /* Clear GICH_HCR */
@@ -431,6 +439,7 @@ vcpu .req r0 @ vcpu pointer always in r0 add r3, r11, #VGIC_CPU_LR ldr r4, [r11, #VGIC_CPU_NR_LR] 1: ldr r6, [r2], #4 +ARM_BE8(rev r6, r6 ) str r6, [r3], #4 subs r4, r4, #1 bne 1b
@@ -459,8 +468,11 @@ vcpu .req r0 @ vcpu pointer always in r0 ldr r4, [r11, #VGIC_CPU_VMCR] ldr r8, [r11, #VGIC_CPU_APR] +ARM_BE8(rev r3, r3 ) str r3, [r2, #GICH_HCR] +ARM_BE8(rev r4, r4 ) str r4, [r2, #GICH_VMCR] +ARM_BE8(rev r8, r8 ) str r8, [r2, #GICH_APR] /* Restore list registers */
@@ -468,6 +480,7 @@ vcpu .req r0 @ vcpu pointer always in r0 add r3, r11, #VGIC_CPU_LR ldr r4, [r11, #VGIC_CPU_NR_LR] 1: ldr r6, [r3], #4 +ARM_BE8(rev r6, r6 ) str r6, [r2], #4 subs r4, r4, #1 bne 1b
@@ -498,7 +511,7 @@ vcpu .req r0 @ vcpu pointer always in r0 mcr p15, 0, r2, c14, c3, 1 @ CNTV_CTL isb - mrrc p15, 3, r2, r3, c14 @ CNTV_CVAL + mrrc p15, 3, rr_lo_hi(r2, r3), c14 @ CNTV_CVAL ldr r4, =VCPU_TIMER_CNTV_CVAL add r5, vcpu, r4 strd r2, r3, [r5]
@@ -538,12 +551,12 @@ vcpu .req r0 @ vcpu pointer always in r0 ldr r2, [r4, #KVM_TIMER_CNTVOFF] ldr r3, [r4, #(KVM_TIMER_CNTVOFF + 4)] - mcrr p15, 4, r2, r3, c14 @ CNTVOFF + mcrr p15, 4, rr_lo_hi(r2, r3), c14 @ CNTVOFF ldr r4, =VCPU_TIMER_CNTV_CVAL add r5, vcpu, r4 ldrd r2, r3, [r5] - mcrr p15, 3, r2, r3, c14 @ CNTV_CVAL + mcrr p15, 3, rr_lo_hi(r2, r3), c14 @ CNTV_CVAL isb ldr r2, [vcpu, #VCPU_TIMER_CNTV_CTL]
Signature · 1 line
1.8.1.4
PATCH 3/5 ARM: kvm one_reg coproc set and get BE fixes
In reply to Victor KamenskyThis patch fixes issue of reading and writing
ARM V7 registers values from/to user land. Existing code was designed to work only in LE case.
struct kvm_one_reg
------------------
registers value passed through kvm_one_reg structure. It is used by KVM_SET_ONE_REG, KVM_GET_ONE_REG ioctls. Note by looking at structure itself we cannot tell what is size of register. Note that structure carries address of user memory, 'addr' where register should be read or written
Setting register (from user-land to kvm)
----------------------------------------
kvm_arm_set_reg takes vcpu and pointer to struct kvm_one_reg which already read from user space
kvm_arm_set_reg calls set_core_reg or kvm_arm_coproc_set_reg
set_core_reg deals only with 4 bytes registers, it just reads 4 bytes from user space and store it properly into vcpu->arch.regs
kvm_arm_coproc_set_reg deals with registers of different size. At certain point code reaches phase where it retrieves description of register by id and it knows register size, which could be either 4 or 8 bytes. Kernel code is ready to read values from user space, but destination type may vary. It could be pointer to 32 bit integer or it could be pointer to 64 bit integer. And all possible permutation of size and destination pointer are possible. Depending on destination pointer type, 4 bytes or 8 bytes, two new helper functions are introduced - reg_from_user32 and reg_from_user64. They are used instead of reg_from_user function which could work only in LE case.
Size sizeof(*DstInt) Function used to read from user 4 4 reg_from_user32 8 4 reg_from_user32 - read two registers 4 8 reg_from_user64 - need special handling for BE 8 8 reg_from_user64
Getting register (to user-land from kvm)
----------------------------------------
Situation with reading registers is similar to writing. Integer pointer type of register to be copied could be 4 or 8 bytes. And size passed in struct kvm_one_reg could be 4 or 8. And any permutation is possible. Depending on src pointer type, 4 bytes or 8 bytes, two new helper functions are introduced - reg_from_user32 and reg_to_user64. They are used instead of reg_to_user function, which could work only in LE case.
Size sizeof(*SrcInt) Function used to write to user 4 4 reg_to_user32 8 4 reg_to_user32 - writes two registers 4 8 reg_to_user64 - need special handleing for BE 8 8 reg_to_user64
Note code does assume that it can only deals with 4 or 8 byte registers.
--- arch/arm/kvm/coproc.c | 94 +++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 69 insertions(+), 25 deletions(-)
arch/arm/kvm/coproc.c +69 -25
@@ -634,17 +634,61 @@ static struct coproc_reg invariant_cp15[] = { { CRn( 0), CRm( 0), Op1( 1), Op2( 7), is32, NULL, get_AIDR }, }; -static int reg_from_user (void *val,const void __user *uaddr,u64 id) +static int reg_from_user64 (u64 *val,const void __user *uaddr,u64 id) +{ +unsigned long regsize =KVM_REG_SIZE (id); +union { +u32 word; +u64 dword; + } tmp = {0 }; + +if (copy_from_user (&tmp, uaddr, regsize) !=0 ) +return -EFAULT ; + +switch (regsize) { +case 4 : + *val = tmp.word; +break ; +case 8 : + *val = tmp.dword; +break ; + } +return 0 ; +} + +/* Note it may really copy two u32 registers */ +static int reg_from_user32 (u32 *val,const void __user *uaddr,u64 id) { -/* This Just Works because we are little endian. */ if (copy_from_user (val, uaddr,KVM_REG_SIZE (id)) !=0 )return -EFAULT ;return 0 ; } -static int reg_to_user (void __user *uaddr,const void *val,u64 id) +static int reg_to_user64 (void __user *uaddr,const u64 *val,u64 id) +{ +unsigned long regsize =KVM_REG_SIZE (id); +union { +u32 word; +u64 dword; + } tmp; + +switch (regsize) { +case 4 : + tmp.word = *val; +break ; +case 8 : + tmp.dword = *val; +break ; + } + +if (copy_to_user (uaddr, &tmp, regsize) !=0 ) +return -EFAULT ; +return 0 ; +} + +/* Note it may really copy two u32 registers */ +static int reg_to_user32 (void __user *uaddr,const u32 *val,u64 id) { -/* This Just Works because we are little endian. */ if (copy_to_user (uaddr, val,KVM_REG_SIZE (id)) !=0 )return -EFAULT ;return 0 ;
@@ -662,7 +706,7 @@ static int get_invariant_cp15(u64 id, void __user *uaddr)if (!r)return -ENOENT ; -return reg_to_user (uaddr, &r->val, id); +return reg_to_user64 (uaddr, &r->val, id); }static int set_invariant_cp15 (u64 id,void __user *uaddr)
@@ -678,7 +722,7 @@ static int set_invariant_cp15(u64 id, void __user *uaddr)if (!r)return -ENOENT ; - err =reg_from_user (&val, uaddr, id); + err =reg_from_user64 (&val, uaddr, id);if (err)return err;
@@ -846,7 +890,7 @@ static int vfp_get_reg(const struct kvm_vcpu *vcpu, u64 id, void __user *uaddr)if (vfpid <num_fp_regs ()) {if (KVM_REG_SIZE (id) !=8 )return -ENOENT ; -return reg_to_user (uaddr, &vcpu->arch.vfp_guest.fpregs[vfpid], +return reg_to_user64 (uaddr, &vcpu->arch.vfp_guest.fpregs[vfpid], id); }
@@ -856,22 +900,22 @@ static int vfp_get_reg(const struct kvm_vcpu *vcpu, u64 id, void __user *uaddr)switch (vfpid) {case KVM_REG_ARM_VFP_FPEXC : -return reg_to_user (uaddr, &vcpu->arch.vfp_guest.fpexc, id); +return reg_to_user32 (uaddr, &vcpu->arch.vfp_guest.fpexc, id);case KVM_REG_ARM_VFP_FPSCR : -return reg_to_user (uaddr, &vcpu->arch.vfp_guest.fpscr, id); +return reg_to_user32 (uaddr, &vcpu->arch.vfp_guest.fpscr, id);case KVM_REG_ARM_VFP_FPINST : -return reg_to_user (uaddr, &vcpu->arch.vfp_guest.fpinst, id); +return reg_to_user32 (uaddr, &vcpu->arch.vfp_guest.fpinst, id);case KVM_REG_ARM_VFP_FPINST2 : -return reg_to_user (uaddr, &vcpu->arch.vfp_guest.fpinst2, id); +return reg_to_user32 (uaddr, &vcpu->arch.vfp_guest.fpinst2, id);case KVM_REG_ARM_VFP_MVFR0 : val =fmrx (MVFR0 ); -return reg_to_user (uaddr, &val, id); +return reg_to_user32 (uaddr, &val, id);case KVM_REG_ARM_VFP_MVFR1 : val =fmrx (MVFR1 ); -return reg_to_user (uaddr, &val, id); +return reg_to_user32 (uaddr, &val, id);case KVM_REG_ARM_VFP_FPSID : val =fmrx (FPSID ); -return reg_to_user (uaddr, &val, id); +return reg_to_user32 (uaddr, &val, id);default :return -ENOENT ; }
@@ -890,8 +934,8 @@ static int vfp_set_reg(struct kvm_vcpu *vcpu, u64 id, const void __user *uaddr)if (vfpid <num_fp_regs ()) {if (KVM_REG_SIZE (id) !=8 )return -ENOENT ; -return reg_from_user (&vcpu->arch.vfp_guest.fpregs[vfpid], - uaddr, id); +return reg_from_user64 (&vcpu->arch.vfp_guest.fpregs[vfpid], + uaddr, id); }/* FP control registers are all 32 bit. */
@@ -900,28 +944,28 @@ static int vfp_set_reg(struct kvm_vcpu *vcpu, u64 id, const void __user *uaddr)switch (vfpid) {case KVM_REG_ARM_VFP_FPEXC : -return reg_from_user (&vcpu->arch.vfp_guest.fpexc, uaddr, id); +return reg_from_user32 (&vcpu->arch.vfp_guest.fpexc, uaddr, id);case KVM_REG_ARM_VFP_FPSCR : -return reg_from_user (&vcpu->arch.vfp_guest.fpscr, uaddr, id); +return reg_from_user32 (&vcpu->arch.vfp_guest.fpscr, uaddr, id);case KVM_REG_ARM_VFP_FPINST : -return reg_from_user (&vcpu->arch.vfp_guest.fpinst, uaddr, id); +return reg_from_user32 (&vcpu->arch.vfp_guest.fpinst, uaddr, id);case KVM_REG_ARM_VFP_FPINST2 : -return reg_from_user (&vcpu->arch.vfp_guest.fpinst2, uaddr, id); +return reg_from_user32 (&vcpu->arch.vfp_guest.fpinst2, uaddr, id);/* These are invariant. */ case KVM_REG_ARM_VFP_MVFR0 : -if (reg_from_user (&val, uaddr, id)) +if (reg_from_user32 (&val, uaddr, id))return -EFAULT ;if (val !=fmrx (MVFR0 ))return -EINVAL ;return 0 ;case KVM_REG_ARM_VFP_MVFR1 : -if (reg_from_user (&val, uaddr, id)) +if (reg_from_user32 (&val, uaddr, id))return -EFAULT ;if (val !=fmrx (MVFR1 ))return -EINVAL ;return 0 ;case KVM_REG_ARM_VFP_FPSID : -if (reg_from_user (&val, uaddr, id)) +if (reg_from_user32 (&val, uaddr, id))return -EFAULT ;if (val !=fmrx (FPSID ))return -EINVAL ;
@@ -968,7 +1012,7 @@ int kvm_arm_coproc_get_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)return get_invariant_cp15 (reg->id, uaddr);/* Note: copies two regs if size is 64 bit. */ -return reg_to_user (uaddr, &vcpu->arch.cp15[r->reg], reg->id); +return reg_to_user32 (uaddr, &vcpu->arch.cp15[r->reg], reg->id); }int kvm_arm_coproc_set_reg (struct kvm_vcpu *vcpu,const struct kvm_one_reg *reg)
@@ -987,7 +1031,7 @@ int kvm_arm_coproc_set_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)return set_invariant_cp15 (reg->id, uaddr);/* Note: copies two regs if size is 64 bit */ -return reg_from_user (&vcpu->arch.cp15[r->reg], uaddr, reg->id); +return reg_from_user32 (&vcpu->arch.cp15[r->reg], uaddr, reg->id); }static unsigned int num_demux_regs (void )
Signature · 1 line
1.8.1.4
PATCH 4/5 ARM: kvm vgic mmio should return data in BE format in BE case
In reply to Victor KamenskyKVM mmio in BE case assumes that data it recieves is in BE format. Vgic operates in LE, so need byteswap data in BE case.
--- virt/kvm/arm/vgic.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
virt/kvm/arm/vgic.c +2 -2
@@ -236,12 +236,12 @@ static void vgic_cpu_irq_clear(struct kvm_vcpu *vcpu, int irq)static u32 mmio_data_read (struct kvm_exit_mmio *mmio,u32 mask) { -return *((u32 *)mmio->data) & mask; +return le32_to_cpu (*((u32 *)mmio->data)) & mask; }static void mmio_data_write (struct kvm_exit_mmio *mmio,u32 mask,u32 value) { - *((u32 *)mmio->data) = value & mask; + *((u32 *)mmio->data) =cpu_to_le32 (value) & mask; }/**
Signature · 1 line
1.8.1.4
PATCH 5/5 ARM: kvm MMIO support BE host running LE code
In reply to Victor KamenskyIn case of status register E bit is not set (LE mode) and host runs in BE mode we need byteswap data, so read/write is emulated correctly.
--- arch/arm/include/asm/kvm_emulate.h | 22 ++++++++++++++++++---- 1 file changed, 18 insertions(+), 4 deletions(-)
arch/arm/include/asm/kvm_emulate.h +18 -4
@@ -185,9 +185,16 @@ static inline unsigned long vcpu_data_guest_to_host(struct kvm_vcpu *vcpu,default :return be32_to_cpu (data); } + }else { +switch (len) { +case 1 : +return data &0xff ; +case 2 : +return le16_to_cpu (data &0xffff ); +default : +return le32_to_cpu (data); + } } - -return data;/* Leave LE untouched */ } static inline unsigned long vcpu_data_host_to_guest(struct kvm_vcpu *vcpu,
@@ -203,9 +210,16 @@ static inline unsigned long vcpu_data_host_to_guest(struct kvm_vcpu *vcpu,default : return cpu_to_be32(data); } + }else { +switch (len) { +case 1 : +return data &0xff ; +case 2 : +return cpu_to_le16 (data &0xffff ); +default : +return cpu_to_le32 (data); + } } - -return data;/* Leave LE untouched */ }#endif /* __ARM_KVM_EMULATE_H__ */
Signature · 1 line
1.8.1.4
cover armv7 BE kvm support
In reply to Victor KamenskyQuoted 1 line · Victor Kamensky
Hi Guys,
Hi Victor,
Quoted 7 lines
Here is series that enables KVM support for V7 big endian kernels. Mostly it deals with BE KVM host support. Marc Zyngier showed before with his patches how BE guest could run on top LE host. With these patches BE guest runs on top of BE host. If Marc's kvmtool is used with few additional changes I tested that BE host could run LE guest. Also I verified that there were no regressions in BE guest on top of LE host case.
Can you please cc the kvmarm list as per the MAINTAINERS files for patches to kvm/arm in the future?
Looking forward to reading your code!
-Christoffer
Quoted 120 lines
Note that posted series covers only kernel side changes. The changes were tested inside of bigger setup with additional changes in qemu and kvmtool. I will post those changes separately in proper aliases but for completeness sake Appendix A gives pointers to git repositories and branches with all needed changes.
Please note first patch is not related to BE KVM per se. I've run into an issue of conflicting 'push' identifier use while trying to include
assembler.hinto KVM .S files. Details of an issue I observed covered in Appendix B. The first patch is my take on solving it.Victor Kamensky (5):
ARM: kvm: replace push and pop with stdmb and ldmia instrs to enable
assembler.hinclusion
ARM: fix KVM assembler files to work in BE case
ARM: kvmone_regcoproc set and get BE fixes
ARM: kvm vgic mmio should return data in BE format in BE case
ARM: kvm MMIO support BE host running LE codearch/arm/include/asm/assembler.h | 7 +++ arch/arm/include/asm/kvm_asm.h | 4 +- arch/arm/include/asm/kvm_emulate.h | 22 +++++++-- arch/arm/kvm/coproc.c | 94 ++++++++++++++++++++++++++++---------- arch/arm/kvm/init.S | 7 ++- arch/arm/kvm/interrupts.S | 50 +++++++++++--------- arch/arm/kvm/interrupts_head.S | 61 +++++++++++++++---------- virt/kvm/arm/vgic.c | 4 +- 8 files changed, 168 insertions(+), 81 deletions(-)Signature · 68 lines
1.8.1.4 Thanks, Victor Appendix A: Testing and Full Setup Description ---------------------------------------------- I) No mixed mode setup - i.e BE guest on BE host; and LE guest on LE host tested to make sure no regressions. KVM host and guest kernels: TC2 on top of Linus 3.13-rc4 (this patch series): git: git://git.linaro.org/people/victor.kamensky/linux-linaro-tracking-be.git branch: armv7be-kvm-3.13-rc4 TC2 and Arndale on top of Linaro BE tree: git: git://git.linaro.org/people/victor.kamensky/linux-linaro-tracking-be.git branch: llct-be-20131216-kvm - TC1 kernels used as guests qemu: git: git://git.linaro.org/people/victor.kamensky/qemu-be.git branch: armv7be-v1 description: changes to run qemu on armeb target; and other changes to work with be image on top of be host kvmtool: git: git://git.linaro.org/people/victor.kamensky/linux-linaro-tracking-be.git branch: kvmtool-armv7be-v1 desciption: minimal changes to build kvmtool for armeb target; and tiny change with virtio magic II) Mixed mode setup all possible combinations within V7 (LE guest on BE host; BE guest on LE host as Marc's setup tested to make sure no regressions) only with kvmtool. This work is based on Marc Zyngier's work that made BE guest to run on top of LE host. For this setup special version of kvmtool should be used and in addition I had to apply patch to guest kernel that would switch reading virtio configs reads to be LE only, that is made on top of previous Rusty Russell's changes. Effectively I just had to do very minor addition to make LE guest to work on BE host, most of heavy lifting was done before by Marc. KVM host kernels: as in previous setup Guest TC1 kernels with LE virtio config patch: git: git://git.linaro.org/people/victor.kamensky/linux-linaro-tracking-be.git branch: virtio-leconfig-3.13-rc4 kvmtool: git: git://git.linaro.org/people/victor.kamensky/linux-linaro-tracking-be.git branch: kvmtool-mixed-v1 description: based on git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git branch kvm-arm64/kvmtool-be-on-le; adds missing include fix; above armeb target build patches; and one fix related to BE mode qemu: git: git://git.linaro.org/people/victor.kamensky/qemu-be.git branch: armv7be-leconfig-v1 description: change virtio-blk that so qemu could work with guest image where virtio leconfig is made; note it does not work in mixed mode; to do so qemu would need bunch of similar changes that Marc did in kvmtool Appendix B: kvm asm file and asm/assembler.h file issue -------------------------------------------------------arch/arm/kvm/interrupts.S +1
-0@@ -25,6 +25,7 @@ #include <asm/kvm_asm.h> #include <asm/kvm_arm.h> #include <asm/vfpmacros.h> +#include <asm/assembler.h> #include "interrupts_head.S" .textproduce the following compilation errors:
/run/media/kamensky/wd/linaro/linux-linaro-core-tracking/092913/linux-linaro-tracking-be/arch/arm/kvm/interrupts.S: Assembler messages:/run/media/kamensky/wd/linaro/linux-linaro-core-tracking/092913/linux-linaro-tracking-be/arch/arm/kvm/interrupts.S:51: Error: ARM register expected --lsr {r2,r3}' /run/media/kamensky/wd/linaro/linux-linaro-core-tracking/092913/linux-linaro-tracking-be/arch/arm/kvm/interrupts.S:100: Error: ARM register expected --lsr {r2}'/run/media/kamensky/wd/linaro/linux-linaro-core-tracking/092913/linux-linaro-tracking-be/arch/arm/kvm/interrupts.S:100: Error: ARM register expected -- `lsr {r4-r12}'
Signature · 1 line
Christoffer