arm/arm64: KVM: introduce new mapping API for percpu mappings
Subsystems (3)
PATCH 1/1 arm/arm64: KVM: introduce new mapping API for percpu mappings
Using virt_to_phys on percpu mappings is horribly wrong (my own bad). Thankfully, the kernel offers a way to obtain the physical address of such a mapping.
Add a new create_hyp_percpu_mappings function to deal with those.
---
Santosh, can you please give this new patch a spin on your HW?
Thanks,
M.
arch/arm/include/asm/kvm_mmu.h | 1 + arch/arm/kvm/arm.c | 2 +- arch/arm/kvm/mmu.c | 32 ++++++++++++++++++++++++++++++++ arch/arm64/include/asm/kvm_mmu.h | 1 + 4 files changed, 35 insertions(+), 1 deletion(-)
arch/arm/include/asm/kvm_mmu.h +1 -0
@@ -43,6 +43,7 @@#include <asm/pgalloc.h> int create_hyp_mappings (void *from,void *to); +int create_hyp_percpu_mappings (void *from,void *to);int create_hyp_io_mappings (void *from,void *to,phys_addr_t );void free_boot_hyp_pgd (void );void free_hyp_pgds (void );
arch/arm/kvm/arm.c +1 -1
@@ -911,7 +911,7 @@ static int init_hyp_mode(void)kvm_cpu_context_t *cpu_ctxt; cpu_ctxt =per_cpu_ptr (kvm_host_cpu_state, cpu); - err =create_hyp_mappings (cpu_ctxt, cpu_ctxt +1 ); + err =create_hyp_percpu_mappings (cpu_ctxt, cpu_ctxt +1 );if (err) {kvm_err ("Cannot map host CPU state: %d\n" , err);
arch/arm/kvm/mmu.c +32 -0
@@ -331,6 +331,38 @@ int create_hyp_mappings(void *from, void *to) }/** +* create_hyp_percpu_mappings - duplicate a percpu kernel virtual address +* range in Hyp mode +* @from: The virtual kernel start address of the range +* @to: The virtual kernel end address of the range (exclusive) +* +* The same virtual address as the kernel virtual address is also used +* in Hyp-mode mapping (modulo HYP_PAGE_OFFSET) to the same underlying +* physical pages. It *has* to be a percpu region. +*/ +int create_hyp_percpu_mappings (void *from,void *to) +{ +unsigned long phys_addr; +unsigned long virt_addr; +unsigned long start =KERN_TO_HYP ((unsigned long )from); +unsigned long end =KERN_TO_HYP ((unsigned long )to); + +for (virt_addr = start; virt_addr < end; virt_addr +=PAGE_SIZE ) { +int err; + + phys_addr =per_cpu_ptr_to_phys (from + virt_addr - start); + err =__create_hyp_mappings (hyp_pgd, virt_addr, + virt_addr +PAGE_SIZE , +__phys_to_pfn (phys_addr), +PAGE_HYP ); +if (err) +return err; + } + +return 0 ; +} + +/** * create_hyp_io_mappings - duplicate a kernel IO mapping into Hyp mode * @from: The kernel start VA of the range * @to: The kernel end VA of the range (exclusive)
arch/arm64/include/asm/kvm_mmu.h +1 -0
@@ -71,6 +71,7 @@#define S2_PGD_ORDER get_order(PTRS_PER_S2_PGD * sizeof(pgd_t))int create_hyp_mappings (void *from,void *to); +int create_hyp_percpu_mappings (void *from,void *to);int create_hyp_io_mappings (void *from,void *to,phys_addr_t );void free_boot_hyp_pgd (void );void free_hyp_pgds (void );
Signature · 1 line
1.8.2.3
PATCH 1/1 arm/arm64: KVM: introduce new mapping API for percpu mappings
Quoted 4 lines · Marc Zyngier
Using
virt_to_physon percpu mappings is horribly wrong (my own bad). Thankfully, the kernel offers a way to obtain the physical address of such a mapping.
Quoted 8 lines
Add a new
create_hyp_percpu_mappingsfunction to deal with those.Reported-by Santosh Shilimkar <[email protected]>Cc Christoffer Dall <[email protected]>Signed-off-by Marc Zyngier <[email protected]>---
Santosh, can you please give this new patch a spin on your HW?
Works except to make guest boot on my machine, phys_addr_t update is needed as discussed ;-)
Quoted 7 lines
arch/arm/include/asm/kvm_mmu.h | 1 + arch/arm/kvm/arm.c | 2 +- arch/arm/kvm/mmu.c | 32 ++++++++++++++++++++++++++++++++ arch/arm64/include/asm/kvm_mmu.h | 1 + 4 files changed, 35 insertions(+), 1 deletion(-)
[..]
Quoted 10 lines
arch/arm/kvm/mmu.c +2
-0@@ -331,6 +331,38 @@ int create_hyp_mappings(void *from, void *to) }/** +* create_hyp_percpu_mappings - duplicate a percpu kernel virtual address +* range in Hyp mode
arch/arm/kvm/mmu.c int create_hyp_mappings(void *from, void *to) · line 345
Quoted 10 lines
arch/arm/kvm/mmu.c +10
-0+* @from: The virtual kernel start address of the range +* @to: The virtual kernel end address of the range (exclusive) +* +* The same virtual address as the kernel virtual address is also used +* in Hyp-mode mapping (modulo HYP_PAGE_OFFSET) to the same underlying +* physical pages. It *has* to be a percpu region. +*/ +int create_hyp_percpu_mappings (void *from,void *to) +{ +unsigned long phys_addr;
s/unsigned long/phys_addr_t
regards,
Santosh
PATCH 1/1 arm/arm64: KVM: introduce new mapping API for percpu mappings
In reply to Marc ZyngierQuoted 10 lines · Marc Zyngier
Using
virt_to_physon percpu mappings is horribly wrong (my own bad). Thankfully, the kernel offers a way to obtain the physical address of such a mapping.Add a new
create_hyp_percpu_mappingsfunction to deal with those.Reported-by Santosh Shilimkar <[email protected]>Cc Christoffer Dall <[email protected]>Signed-off-by Marc Zyngier <[email protected]>---
So, I find this nicer, somehow, what do you think:
arch/arm/kvm/mmu.c +23 -6
@@ -334,6 +334,15 @@ out:return err; } +static phys_addr_t kvm_kaddr_to_phys (void *kaddr) +{ +if (!is_vmalloc_addr (kaddr)) +return __pa (kaddr); +else +return page_to_phys (vmalloc_to_page (kaddr)) + +offset_in_page (kaddr); +} +/** * create_hyp_mappings - duplicate a kernel virtual address range in Hyp mode * @from: The virtual kernel start address of the range
@@ -345,16 +354,24 @@ out:*/ int create_hyp_mappings (void *from,void *to) { -unsigned long phys_addr =virt_to_phys (from); +phys_addr_t phys_addr; +unsigned long virt_addr;unsigned long start =KERN_TO_HYP ((unsigned long )from);unsigned long end =KERN_TO_HYP ((unsigned long )to); -/* Check for a valid kernel memory mapping */ -if (!virt_addr_valid (from) || !virt_addr_valid (to -1 )) -return -EINVAL ; +for (virt_addr = start; virt_addr < end; virt_addr +=PAGE_SIZE ) { +int err; -return __create_hyp_mappings (hyp_pgd, start, end, -__phys_to_pfn (phys_addr),PAGE_HYP ); + phys_addr =kvm_kaddr_to_phys (from + virt_addr - start); + err =__create_hyp_mappings (hyp_pgd, virt_addr, + virt_addr +PAGE_SIZE , +__phys_to_pfn (phys_addr), +PAGE_HYP ); +if (err) +return err; + } + +return 0 ; }/**
PATCH 1/1 arm/arm64: KVM: introduce new mapping API for percpu mappings
Quoted 47 lines · Christoffer Dall
Quoted 10 lines · Marc Zyngier
Using
virt_to_physon percpu mappings is horribly wrong (my own bad). Thankfully, the kernel offers a way to obtain the physical address of such a mapping.Add a new
create_hyp_percpu_mappingsfunction to deal with those.Reported-by Santosh Shilimkar <[email protected]>Cc Christoffer Dall <[email protected]>Signed-off-by Marc Zyngier <[email protected]>---
So, I find this nicer, somehow, what do you think:
arch/arm/kvm/mmu.c +11
-2@@ -334,6 +334,15 @@ out:return err; } +static phys_addr_t kvm_kaddr_to_phys (void *kaddr) +{ +if (!is_vmalloc_addr (kaddr)) +return __pa (kaddr); +else +return page_to_phys (vmalloc_to_page (kaddr)) + +offset_in_page (kaddr); +} +/** * create_hyp_mappings - duplicate a kernel virtual address range in Hyp mode * @from: The virtual kernel start address of the range @@ -345,16 +354,24 @@ out:*/ int create_hyp_mappings (void *from,void *to) { -unsigned long phys_addr =virt_to_phys (from); +phys_addr_t phys_addr; +unsigned long virt_addr;unsigned long start =KERN_TO_HYP ((unsigned long )from);unsigned long end =KERN_TO_HYP ((unsigned long )to); -/* Check for a valid kernel memory mapping */
arch/arm/kvm/mmu.c out: · line 367
Quoted 10 lines
arch/arm/kvm/mmu.c +5
-4-if (!virt_addr_valid (from) || !virt_addr_valid (to -1 )) -return -EINVAL ; +for (virt_addr = start; virt_addr < end; virt_addr +=PAGE_SIZE ) { +int err; -return __create_hyp_mappings (hyp_pgd, start, end, -__phys_to_pfn (phys_addr),PAGE_HYP ); + phys_addr =kvm_kaddr_to_phys (from + virt_addr - start); + err = __create_hyp_mappings(hyp_pgd, virt_addr, + virt_addr + PAGE_SIZE,
I think I've introduced a bug here. It probably should read:
err = __create_hyp_mappings(hyp_pgd, virt_addr & PAGE_MASK,
(virt_addr + PAGE_SIZE) & PAGE_MASK, [...]
arch/arm/kvm/mmu.c out: · line 377
Quoted 11 lines
arch/arm/kvm/mmu.c +7
-0+__phys_to_pfn (phys_addr), + PAGE_HYP); +if (err) +return err; + } + +return 0 ; }/**
So that would work, but I'm slightly uncomfortable with what is basically an open-coded version of per_cpu_ptr_to_phys, and I think there is some value in having an explicit function for dealing with percpu mappings, at least for educational purpose.
Also, we loose the virt_addr_valid() check, which has been a valuable debugging tool for me in the past.
But maybe that's just me being a chicken... ;-)
M.
Signature · 1 line
Jazz is not dead. It just smells funny...
PATCH 1/1 arm/arm64: KVM: introduce new mapping API for percpu mappings
Quoted 74 lines · Marc Zyngier
Quoted 47 lines · Christoffer Dall
Quoted 10 lines · Marc Zyngier
Using
virt_to_physon percpu mappings is horribly wrong (my own bad). Thankfully, the kernel offers a way to obtain the physical address of such a mapping.Add a new
create_hyp_percpu_mappingsfunction to deal with those.Reported-by Santosh Shilimkar <[email protected]>Cc Christoffer Dall <[email protected]>Signed-off-by Marc Zyngier <[email protected]>---
So, I find this nicer, somehow, what do you think:
arch/arm/kvm/mmu.c +11
-2@@ -334,6 +334,15 @@ out:return err; } +static phys_addr_t kvm_kaddr_to_phys (void *kaddr) +{ +if (!is_vmalloc_addr (kaddr)) +return __pa (kaddr); +else +return page_to_phys (vmalloc_to_page (kaddr)) + +offset_in_page (kaddr); +} +/** * create_hyp_mappings - duplicate a kernel virtual address range in Hyp mode * @from: The virtual kernel start address of the range @@ -345,16 +354,24 @@ out:*/ int create_hyp_mappings (void *from,void *to) { -unsigned long phys_addr =virt_to_phys (from); +phys_addr_t phys_addr; +unsigned long virt_addr;unsigned long start =KERN_TO_HYP ((unsigned long )from);unsigned long end =KERN_TO_HYP ((unsigned long )to); -/* Check for a valid kernel memory mapping */ arch/arm/kvm/mmu.c out: · line 367
Quoted 10 lines
arch/arm/kvm/mmu.c +5
-4-if (!virt_addr_valid (from) || !virt_addr_valid (to -1 )) -return -EINVAL ; +for (virt_addr = start; virt_addr < end; virt_addr +=PAGE_SIZE ) { +int err; -return __create_hyp_mappings (hyp_pgd, start, end, -__phys_to_pfn (phys_addr),PAGE_HYP ); + phys_addr =kvm_kaddr_to_phys (from + virt_addr - start); + err = __create_hyp_mappings(hyp_pgd, virt_addr, + virt_addr + PAGE_SIZE,I think I've introduced a bug here. It probably should read:
err = __create_hyp_mappings(hyp_pgd,
virt_addr&PAGE_MASK,
(virt_addr+PAGE_SIZE) &PAGE_MASK, [...]Quoted 9 lines
arch/arm/kvm/mmu.c +7
-0+__phys_to_pfn (phys_addr), + PAGE_HYP); +if (err) +return err; + } + +return 0 ; }
Quoted 7 lines
arch/arm/kvm/mmu.c out: · line 377
Quoted 2 lines
arch/arm/kvm/mmu.c +0
-0/** So that would work, but I'm slightly uncomfortable with what is basically an open-coded version of
per_cpu_ptr_to_phys, and I think there is some value in having an explicit function for dealing with percpu mappings, at least for educational purpose.
I'm not concerned about the open-coded; this is a pretty fundamental functionality to distinguish between a vmalloc and a kmalloc and do the virt_to_phys translation accordingly - that's not specific to percpu allocated memory - they only add more complexity as an optimization. After all, this is a single if-statement that I'm sure we can master.
I think it's slightly more strange to have a call to map memory that depends on how you allocated it, and would prefer having something that just works, regardless. But maybe that's utopian. However, would we end up potentially having a create_vmalloc_hyp_mappings as well then?
Another concern with your proposal is that it duplicates more code and makes it a bit harder to track what's going on (who calls what when to allocate something, etc.).
Quoted 6 lines
Also, we loose the
virt_addr_valid()check, which has been a valuable debugging tool for me in the past.But maybe that's just me being a chicken... ;-)
Of course it is. No, but really, virt_addr_valid just checks that it's linearly mapped mapped memory. So we're checking that it's not highmeme, and assuming it's linearly mapped now. We can add a BUG_ON(!virt_addr_valid(x)) in the (!is_vmalloc_addr(kaddr)) case to make sure nobody is passing module addresses or DMA memory or something else crazy here, but I doubt that's a bug we need to concerne ourselves about.
-Christoffer
PATCH 1/1 arm/arm64: KVM: introduce new mapping API for percpu mappings
Quoted 83 lines · Christoffer Dall
Quoted 74 lines · Marc Zyngier
Quoted 47 lines · Christoffer Dall
Quoted 10 lines · Marc Zyngier
Using
virt_to_physon percpu mappings is horribly wrong (my own bad). Thankfully, the kernel offers a way to obtain the physical address of such a mapping.Add a new
create_hyp_percpu_mappingsfunction to deal with those.Reported-by Santosh Shilimkar <[email protected]>Cc Christoffer Dall <[email protected]>Signed-off-by Marc Zyngier <[email protected]>---
So, I find this nicer, somehow, what do you think:
arch/arm/kvm/mmu.c +11
-2@@ -334,6 +334,15 @@ out:return err; } +static phys_addr_t kvm_kaddr_to_phys (void *kaddr) +{ +if (!is_vmalloc_addr (kaddr)) +return __pa (kaddr); +else +return page_to_phys (vmalloc_to_page (kaddr)) + +offset_in_page (kaddr); +} +/** * create_hyp_mappings - duplicate a kernel virtual address range in Hyp mode * @from: The virtual kernel start address of the range @@ -345,16 +354,24 @@ out:*/ int create_hyp_mappings (void *from,void *to) { -unsigned long phys_addr =virt_to_phys (from); +phys_addr_t phys_addr; +unsigned long virt_addr;unsigned long start =KERN_TO_HYP ((unsigned long )from);unsigned long end =KERN_TO_HYP ((unsigned long )to); -/* Check for a valid kernel memory mapping */ arch/arm/kvm/mmu.c out: · line 367
Quoted 10 lines
arch/arm/kvm/mmu.c +5
-4-if (!virt_addr_valid (from) || !virt_addr_valid (to -1 )) -return -EINVAL ; +for (virt_addr = start; virt_addr < end; virt_addr +=PAGE_SIZE ) { +int err; -return __create_hyp_mappings (hyp_pgd, start, end, -__phys_to_pfn (phys_addr),PAGE_HYP ); + phys_addr =kvm_kaddr_to_phys (from + virt_addr - start); + err = __create_hyp_mappings(hyp_pgd, virt_addr, + virt_addr + PAGE_SIZE,I think I've introduced a bug here. It probably should read:
err = __create_hyp_mappings(hyp_pgd,
virt_addr&PAGE_MASK,
(virt_addr+PAGE_SIZE) &PAGE_MASK, [...]Quoted 9 lines
arch/arm/kvm/mmu.c +7
-0+__phys_to_pfn (phys_addr), + PAGE_HYP); +if (err) +return err; + } + +return 0 ; }Quoted 7 lines
arch/arm/kvm/mmu.c out: · line 377
Quoted 2 lines
arch/arm/kvm/mmu.c +0
-0/** So that would work, but I'm slightly uncomfortable with what is basically an open-coded version of
per_cpu_ptr_to_phys, and I think there is some value in having an explicit function for dealing with percpu mappings, at least for educational purpose.
Quoted 5 lines
I'm not concerned about the open-coded; this is a pretty fundamental functionality to distinguish between a vmalloc and a kmalloc and do the
virt_to_phystranslation accordingly - that's not specific to percpu allocated memory - they only add more complexity as an optimization. After all, this is a single if-statement that I'm sure we can master.
Hey, I've been seen messing up that kind of code very easily! ;-)
Quoted 15 lines
I think it's slightly more strange to have a call to map memory that depends on how you allocated it, and would prefer having something that just works, regardless. But maybe that's utopian. However, would we end up potentially having a
create_vmalloc_hyp_mappingsas well then?Another concern with your proposal is that it duplicates more code and makes it a bit harder to track what's going on (who calls what when to allocate something, etc.).
Quoted 6 lines
Also, we loose the
virt_addr_valid()check, which has been a valuable debugging tool for me in the past.But maybe that's just me being a chicken... ;-)
Quoted 7 lines
Of course it is. No, but really,
virt_addr_validjust checks that it's linearly mapped mapped memory. So we're checking that it's not highmeme, and assuming it's linearly mapped now. We can add a BUG_ON(!virt_addr_valid(x)) in the (!is_vmalloc_addr(kaddr)) case to make sure nobody is passing module addresses or DMA memory or something else crazy here, but I doubt that's a bug we need to concerne ourselves about.
Fair enough. Do you write the patch, or do I update mine? Don't mind either way.
M.
Signature · 1 line
Jazz is not dead. It just smells funny...
PATCH 1/1 arm/arm64: KVM: introduce new mapping API for percpu mappings
Quoted 115 lines · Marc Zyngier
Quoted 83 lines · Christoffer Dall
Quoted 74 lines · Marc Zyngier
Quoted 47 lines · Christoffer Dall
Quoted 10 lines · Marc Zyngier
Using
virt_to_physon percpu mappings is horribly wrong (my own bad). Thankfully, the kernel offers a way to obtain the physical address of such a mapping.Add a new
create_hyp_percpu_mappingsfunction to deal with those.Reported-by Santosh Shilimkar <[email protected]>Cc Christoffer Dall <[email protected]>Signed-off-by Marc Zyngier <[email protected]>---
So, I find this nicer, somehow, what do you think:
arch/arm/kvm/mmu.c +11
-2@@ -334,6 +334,15 @@ out:return err; } +static phys_addr_t kvm_kaddr_to_phys (void *kaddr) +{ +if (!is_vmalloc_addr (kaddr)) +return __pa (kaddr); +else +return page_to_phys (vmalloc_to_page (kaddr)) + +offset_in_page (kaddr); +} +/** * create_hyp_mappings - duplicate a kernel virtual address range in Hyp mode * @from: The virtual kernel start address of the range @@ -345,16 +354,24 @@ out:*/ int create_hyp_mappings (void *from,void *to) { -unsigned long phys_addr =virt_to_phys (from); +phys_addr_t phys_addr; +unsigned long virt_addr;unsigned long start =KERN_TO_HYP ((unsigned long )from);unsigned long end =KERN_TO_HYP ((unsigned long )to); -/* Check for a valid kernel memory mapping */ arch/arm/kvm/mmu.c out: · line 367
Quoted 10 lines
arch/arm/kvm/mmu.c +5
-4-if (!virt_addr_valid (from) || !virt_addr_valid (to -1 )) -return -EINVAL ; +for (virt_addr = start; virt_addr < end; virt_addr +=PAGE_SIZE ) { +int err; -return __create_hyp_mappings (hyp_pgd, start, end, -__phys_to_pfn (phys_addr),PAGE_HYP ); + phys_addr =kvm_kaddr_to_phys (from + virt_addr - start); + err = __create_hyp_mappings(hyp_pgd, virt_addr, + virt_addr + PAGE_SIZE,I think I've introduced a bug here. It probably should read:
err = __create_hyp_mappings(hyp_pgd,
virt_addr&PAGE_MASK,
(virt_addr+PAGE_SIZE) &PAGE_MASK, [...]Quoted 9 lines
arch/arm/kvm/mmu.c +7
-0+__phys_to_pfn (phys_addr), + PAGE_HYP); +if (err) +return err; + } + +return 0 ; }Quoted 7 lines
arch/arm/kvm/mmu.c out: · line 377
Quoted 2 lines
arch/arm/kvm/mmu.c +0
-0/** So that would work, but I'm slightly uncomfortable with what is basically an open-coded version of
per_cpu_ptr_to_phys, and I think there is some value in having an explicit function for dealing with percpu mappings, at least for educational purpose.Quoted 5 lines
I'm not concerned about the open-coded; this is a pretty fundamental functionality to distinguish between a vmalloc and a kmalloc and do the
virt_to_phystranslation accordingly - that's not specific to percpu allocated memory - they only add more complexity as an optimization. After all, this is a single if-statement that I'm sure we can master.Hey, I've been seen messing up that kind of code very easily! ;-)
Quoted 15 lines
I think it's slightly more strange to have a call to map memory that depends on how you allocated it, and would prefer having something that just works, regardless. But maybe that's utopian. However, would we end up potentially having a
create_vmalloc_hyp_mappingsas well then?Another concern with your proposal is that it duplicates more code and makes it a bit harder to track what's going on (who calls what when to allocate something, etc.).
Quoted 6 lines
Also, we loose the
virt_addr_valid()check, which has been a valuable debugging tool for me in the past.But maybe that's just me being a chicken... ;-)
Quoted 7 lines
Of course it is. No, but really,
virt_addr_validjust checks that it's linearly mapped mapped memory. So we're checking that it's not highmeme, and assuming it's linearly mapped now. We can add a BUG_ON(!virt_addr_valid(x)) in the (!is_vmalloc_addr(kaddr)) case to make sure nobody is passing module addresses or DMA memory or something else crazy here, but I doubt that's a bug we need to concerne ourselves about.
Quoted 3 lines
Fair enough. Do you write the patch, or do I update mine? Don't mind either way.
I'll write it and send it out.
Thanks,
-Christoffer