powerpc/pseries/svm: Allocate SWIOTLB buffer anywhere in memory
PATCH v3 1/1 powerpc/pseries/svm: Allocate SWIOTLB buffer anywhere in memory
POWER secure guests (i.e., guests which use the Protection Execution Facility) need to use SWIOTLB to be able to do I/O with the hypervisor, but they don't need the SWIOTLB memory to be in low addresses since the hypervisor doesn't have any addressing limitation.
This solves a SWIOTLB initialization problem we are seeing in secure guests with 128 GB of RAM: they are configured with 4 GB of crashkernel reserved memory, which leaves no space for SWIOTLB in low addresses.
To do this, we use mostly the same code as swiotlb_init(), but allocate the buffer using memblock_alloc() instead of memblock_alloc_low().
--- arch/powerpc/include/asm/svm.h | 4 ++++ arch/powerpc/mm/mem.c | 6 +++++- arch/powerpc/platforms/pseries/svm.c | 26 ++++++++++++++++++++++++++ 3 files changed, 35 insertions(+), 1 deletion(-)
Changes from v2:
- Panic if unable to allocate buffer, as suggested by Christoph.
Changes from v1:
- Open-code swiotlb_init() in arch-specific code, as suggested by Christoph.
arch/powerpc/include/asm/svm.h +4 -0
@@ -15,6 +15,8 @@ static inline bool is_secure_guest(void)return mfmsr () &MSR_S ; } +void __init svm_swiotlb_init (void ); +void dtl_cache_ctor (void *addr);#define get_dtl_cache_ctor () (is_secure_guest() ? dtl_cache_ctor : NULL)
@@ -25,6 +27,8 @@ static inline bool is_secure_guest(void) return false; } +static inline void svm_swiotlb_init (void ) {} +#define get_dtl_cache_ctor () NULL#endif /* CONFIG_PPC_SVM */
arch/powerpc/mm/mem.c +5 -1
@@ -50,6 +50,7 @@#include <asm/swiotlb.h> #include <asm/rtas.h> #include <asm/kasan.h> +#include <asm/svm.h> #include <mm/mmu_decl.h>
@@ -290,7 +291,10 @@ void __init mem_init(void)* back to to-down. */ memblock_set_bottom_up (true); -swiotlb_init (0 ); +if (is_secure_guest ()) +svm_swiotlb_init (); +else +swiotlb_init (0 );#endif high_memory = (void *)__va (max_low_pfn *PAGE_SIZE );
arch/powerpc/platforms/pseries/svm.c +26 -0
@@ -7,6 +7,7 @@*/ #include <linux/mm.h> +#include <linux/memblock.h> #include <asm/machdep.h> #include <asm/svm.h> #include <asm/swiotlb.h>
@@ -34,6 +35,31 @@ static int __init init_svm(void) }machine_early_initcall (pseries, init_svm); +/* +* Initialize SWIOTLB. Essentially the same as swiotlb_init(), except that it +* can allocate the buffer anywhere in memory. Since the hypervisor doesn't have +* any addressing limitation, we don't need to allocate it in low addresses. +*/ +void __init svm_swiotlb_init (void ) +{ +unsigned char *vstart; +unsigned long bytes, io_tlb_nslabs; + + io_tlb_nslabs = (swiotlb_size_or_default () >>IO_TLB_SHIFT ); + io_tlb_nslabs =ALIGN (io_tlb_nslabs,IO_TLB_SEGSIZE ); + + bytes = io_tlb_nslabs <<IO_TLB_SHIFT ; + + vstart =memblock_alloc (PAGE_ALIGN (bytes),PAGE_SIZE ); +if (vstart && !swiotlb_init_with_tbl (vstart, io_tlb_nslabs, false)) +return ; + +if (io_tlb_start) +memblock_free_early (io_tlb_start, +PAGE_ALIGN (io_tlb_nslabs <<IO_TLB_SHIFT )); +panic ("SVM: Cannot allocate SWIOTLB buffer" ); +} +int set_memory_encrypted (unsigned long addr,int numpages) {if (!PAGE_ALIGNED (addr))
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Quoted 5 lines · Thiago Jung Bauermann
POWER secure guests (i.e., guests which use the Protection Execution Facility) need to use SWIOTLB to be able to do I/O with the hypervisor, but they don't need the SWIOTLB memory to be in low addresses since the hypervisor doesn't have any addressing limitation.
Quoted 8 lines
This solves a SWIOTLB initialization problem we are seeing in secure guests with 128 GB of RAM: they are configured with 4 GB of crashkernel reserved memory, which leaves no space for SWIOTLB in low addresses.
To do this, we use mostly the same code as
swiotlb_init(), but allocate the buffer usingmemblock_alloc()instead ofmemblock_alloc_low().Signed-off-by Thiago Jung Bauermann <[email protected]>
Looks fine to me (except for the pointlessly long comment lines, but I've been told that's the powerpc way).
Quoted 10 lines · Christoph Hellwig
Quoted 9 lines · Thiago Jung Bauermann
POWER secure guests (i.e., guests which use the Protection Execution Facility) need to use SWIOTLB to be able to do I/O with the hypervisor, but they don't need the SWIOTLB memory to be in low addresses since the hypervisor doesn't have any addressing limitation.
This solves a SWIOTLB initialization problem we are seeing in secure guests with 128 GB of RAM: they are configured with 4 GB of crashkernel reserved memory, which leaves no space for SWIOTLB in low addresses.
Quoted 7 lines
Quoted 4 lines
To do this, we use mostly the same code as
swiotlb_init(), but allocate the buffer usingmemblock_alloc()instead ofmemblock_alloc_low().Signed-off-by Thiago Jung Bauermann <[email protected]>Looks fine to me (except for the pointlessly long comment lines, but I've been told that's the powerpc way).
Thanks! Do I have your Reviewed-by?
Signature · 6 lines
Thiago Jung Bauermann IBM Linux Technology Center _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Quoted 5 lines · Thiago Jung Bauermann
POWER secure guests (i.e., guests which use the Protection Execution Facility) need to use SWIOTLB to be able to do I/O with the hypervisor, but they don't need the SWIOTLB memory to be in low addresses since the hypervisor doesn't have any addressing limitation.
Quoted 8 lines
This solves a SWIOTLB initialization problem we are seeing in secure guests with 128 GB of RAM: they are configured with 4 GB of crashkernel reserved memory, which leaves no space for SWIOTLB in low addresses.
To do this, we use mostly the same code as
swiotlb_init(), but allocate the buffer usingmemblock_alloc()instead ofmemblock_alloc_low().Signed-off-by Thiago Jung Bauermann <[email protected]>
Quoted 93 lines
--- arch/powerpc/include/asm/svm.h | 4 ++++ arch/powerpc/mm/mem.c | 6 +++++- arch/powerpc/platforms/pseries/svm.c | 26 ++++++++++++++++++++++++++ 3 files changed, 35 insertions(+), 1 deletion(-)Changes from v2:
- Panic if unable to allocate buffer, as suggested by Christoph.Changes from v1:
- Open-codeswiotlb_init()in arch-specific code, as suggested by Christoph.arch/powerpc/include/asm/svm.h +4
-0@@ -15,6 +15,8 @@ static inline bool is_secure_guest(void)return mfmsr () &MSR_S ; } +void __init svm_swiotlb_init (void ); +void dtl_cache_ctor (void *addr);#define get_dtl_cache_ctor () (is_secure_guest() ? dtl_cache_ctor : NULL)@@ -25,6 +27,8 @@ static inline bool is_secure_guest(void) return false; } +static inline void svm_swiotlb_init (void ) {} +#define get_dtl_cache_ctor () NULL#endif /* CONFIG_PPC_SVM */ arch/powerpc/mm/mem.c +5
-1@@ -50,6 +50,7 @@#include <asm/swiotlb.h> #include <asm/rtas.h> #include <asm/kasan.h> +#include <asm/svm.h> #include <mm/mmu_decl.h> @@ -290,7 +291,10 @@ void __init mem_init(void)* back to to-down. */ memblock_set_bottom_up (true); -swiotlb_init (0 ); +if (is_secure_guest ()) +svm_swiotlb_init (); +else +swiotlb_init (0 );#endif high_memory = (void *)__va (max_low_pfn *PAGE_SIZE );arch/powerpc/platforms/pseries/svm.c +19
-0@@ -7,6 +7,7 @@*/ #include <linux/mm.h> +#include <linux/memblock.h> #include <asm/machdep.h> #include <asm/svm.h> #include <asm/swiotlb.h> @@ -34,6 +35,31 @@ static int __init init_svm(void) }machine_early_initcall (pseries, init_svm); +/* +* Initialize SWIOTLB. Essentially the same as swiotlb_init(), except that it +* can allocate the buffer anywhere in memory. Since the hypervisor doesn't have +* any addressing limitation, we don't need to allocate it in low addresses. +*/ +void __init svm_swiotlb_init (void ) +{ +unsigned char *vstart; +unsigned long bytes, io_tlb_nslabs; + + io_tlb_nslabs = (swiotlb_size_or_default () >>IO_TLB_SHIFT ); + io_tlb_nslabs =ALIGN (io_tlb_nslabs,IO_TLB_SEGSIZE ); + + bytes = io_tlb_nslabs <<IO_TLB_SHIFT ; + + vstart =memblock_alloc (PAGE_ALIGN (bytes),PAGE_SIZE ); +if (vstart && !swiotlb_init_with_tbl (vstart, io_tlb_nslabs, false)) +return ;
arch/powerpc/platforms/pseries/svm.c static int __init init_svm(void) · line 65
Quoted 10 lines
arch/powerpc/platforms/pseries/svm.c +7
-0+ +if (io_tlb_start) +memblock_free_early (io_tlb_start, +PAGE_ALIGN (io_tlb_nslabs <<IO_TLB_SHIFT )); +panic ("SVM: Cannot allocate SWIOTLB buffer" ); +} +int set_memory_encrypted (unsigned long addr,int numpages) {if (!PAGE_ALIGNED (addr))
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Quoted 10 lines · Konrad Rzeszutek Wilk
Quoted 9 lines · Thiago Jung Bauermann
POWER secure guests (i.e., guests which use the Protection Execution Facility) need to use SWIOTLB to be able to do I/O with the hypervisor, but they don't need the SWIOTLB memory to be in low addresses since the hypervisor doesn't have any addressing limitation.
This solves a SWIOTLB initialization problem we are seeing in secure guests with 128 GB of RAM: they are configured with 4 GB of crashkernel reserved memory, which leaves no space for SWIOTLB in low addresses.
Quoted 6 lines
Quoted 4 lines
To do this, we use mostly the same code as
swiotlb_init(), but allocate the buffer usingmemblock_alloc()instead ofmemblock_alloc_low().Signed-off-by Thiago Jung Bauermann <[email protected]>Reviewed-by Konrad Rzeszutek Wilk <[email protected]>
Thanks!
Signature · 6 lines
Thiago Jung Bauermann IBM Linux Technology Center _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Michael Ellerman
reviewerQuoted 10 lines · Christoph Hellwig
Quoted 9 lines · Thiago Jung Bauermann
POWER secure guests (i.e., guests which use the Protection Execution Facility) need to use SWIOTLB to be able to do I/O with the hypervisor, but they don't need the SWIOTLB memory to be in low addresses since the hypervisor doesn't have any addressing limitation.
This solves a SWIOTLB initialization problem we are seeing in secure guests with 128 GB of RAM: they are configured with 4 GB of crashkernel reserved memory, which leaves no space for SWIOTLB in low addresses.
Quoted 7 lines
Quoted 4 lines
To do this, we use mostly the same code as
swiotlb_init(), but allocate the buffer usingmemblock_alloc()instead ofmemblock_alloc_low().Signed-off-by Thiago Jung Bauermann <[email protected]>Looks fine to me (except for the pointlessly long comment lines, but I've been told that's the powerpc way).
They're 80 columns AFAICS?
cheers
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Michael Ellerman
AppliedQuoted 10 lines · Thiago Jung Bauermann
POWER secure guests (i.e., guests which use the Protection Execution Facility) need to use SWIOTLB to be able to do I/O with the hypervisor, but they don't need the SWIOTLB memory to be in low addresses since the hypervisor doesn't have any addressing limitation.
This solves a SWIOTLB initialization problem we are seeing in secure guests with 128 GB of RAM: they are configured with 4 GB of crashkernel reserved memory, which leaves no space for SWIOTLB in low addresses.
[...]
Applied to powerpc/next.
[1/1] powerpc/pseries/svm: Allocate SWIOTLB buffer anywhere in memory
https://git.kernel.org/powerpc/c/eae9eec476d13fad9af6da1f44a054ee02b7b161
cheers