Make PDX compression optional
v3 cover Make PDX compression optional
Currently there's a CONFIG_HAS_PDX Kconfig option, but it's impossible to disable it because the whole codebase performs unconditional compression/decompression operations on addresses. This has the unfortunate side effect that systems without a need for compression still have to pay the performance impact of juggling bits on every pfn<->pdx conversion (this requires reading several global variables). This series attempts to:
* Leave the state of pdx and pdx compression documented
* Factor out compression so it _can_ be removed through Kconfig
* Make it so compression is disabled on x86 and enabled on both Aarch32 and Aarch64 by default.
Series summary:
Patch 1 Moves hard-coded compression-related logic to helper functions Patch 2 Refactors all instances of regions being validated for pdx
compression conformance so it's done through a helper
Patch 3 Non-functional reorder in order to simplify the patch 8 diff Patch 4 Adds new Kconfig option to compile out PDX compression and removes
the old CONFIG_HAS_PDX, as it was non removable
Already committed:
v1/patch 1 documents the current general understanding of the pdx concept and
pdx compression in particular
v1/patch 3 Marks the pdx compression globals as ro_after_init v2/patch 1 Documents the differences between arm32 and arm64 directmaps
Alejandro Vallejo (4):
mm: Factor out the pdx compression logic in ma/va converters mm/pdx: Standardize region validation wrt pdx compression
pdx: Reorder pdx.[ch]
pdx: Add CONFIG_PDX_COMPRESSION as a common Kconfig option
xen/arch/arm/Kconfig | 1 - xen/arch/arm/include/asm/mm.h | 3 +- xen/arch/x86/Kconfig | 1 - xen/arch/x86/domain.c | 19 ++-- xen/arch/x86/include/asm/x86_64/page.h | 28 +++--- xen/arch/x86/x86_64/mm.c | 6 +- xen/common/Kconfig | 13 ++- xen/common/Makefile | 2 +- xen/common/efi/boot.c | 13 ++- xen/common/pdx.c | 76 +++++++++------ xen/include/xen/pdx.h | 127 +++++++++++++++++++------ 11 files changed, 193 insertions(+), 96 deletions(-)
Signature · 1 line
2.34.1
PATCH v3 1/4 mm: Factor out the pdx compression logic in ma/va converters
This patch factors out the pdx compression logic hardcoded in both ports for the maddr<->vaddr conversion functions.
Touches both x86 and arm ports.
---
v3:
* size_t -> unsigned long (Jan)
* uint64_t -> paddr_t (Jan)
--- xen/arch/arm/include/asm/mm.h | 3 +-- xen/arch/x86/include/asm/x86_64/page.h | 28 +++++++++++--------------- xen/include/xen/pdx.h | 25 +++++++++++++++++++++++ 3 files changed, 38 insertions(+), 18 deletions(-)
xen/arch/arm/include/asm/mm.h +1 -2
@@ -319,8 +319,7 @@ static inline void *maddr_to_virt(paddr_t ma) (DIRECTMAP_SIZE >> PAGE_SHIFT));return (void *)(XENHEAP_VIRT_START - (directmap_base_pdx <<PAGE_SHIFT ) + - ((ma & ma_va_bottom_mask) | - ((ma & ma_top_mask) >> pfn_pdx_hole_shift))); + maddr_to_directmapoff(ma)); }#endif
xen/arch/x86/include/asm/x86_64/page.h +12 -16
@@ -36,26 +36,22 @@ static inline unsigned long __virt_to_maddr(unsigned long va) {ASSERT (va <DIRECTMAP_VIRT_END );if ( va >=DIRECTMAP_VIRT_START ) - va -=DIRECTMAP_VIRT_START ; -else - { -BUILD_BUG_ON (XEN_VIRT_END -XEN_VIRT_START !=GB (1 )); -/* Signed, so ((long)XEN_VIRT_START >> 30) fits in an imm32. */ -ASSERT (((long )va >> (PAGE_ORDER_1G +PAGE_SHIFT )) == - ((long )XEN_VIRT_START >> (PAGE_ORDER_1G +PAGE_SHIFT ))); - - va += xen_phys_start -XEN_VIRT_START ; - } -return (va & ma_va_bottom_mask) | - ((va << pfn_pdx_hole_shift) & ma_top_mask); +return directmapoff_to_maddr (va -DIRECTMAP_VIRT_START ); + +BUILD_BUG_ON (XEN_VIRT_END -XEN_VIRT_START !=GB (1 )); +/* Signed, so ((long)XEN_VIRT_START >> 30) fits in an imm32. */ +ASSERT (((long )va >> (PAGE_ORDER_1G +PAGE_SHIFT )) == + ((long )XEN_VIRT_START >> (PAGE_ORDER_1G +PAGE_SHIFT ))); + +return xen_phys_start + va -XEN_VIRT_START ; }static inline void *__maddr_to_virt (unsigned long ma) { -ASSERT (pfn_to_pdx (ma >>PAGE_SHIFT ) < (DIRECTMAP_SIZE >>PAGE_SHIFT )); -return (void *)(DIRECTMAP_VIRT_START + - ((ma & ma_va_bottom_mask) | - ((ma & ma_top_mask) >> pfn_pdx_hole_shift))); +/* Offset in the direct map, accounting for pdx compression */ +unsigned long va_offset =maddr_to_directmapoff (ma); +ASSERT (va_offset <DIRECTMAP_SIZE ); +return (void *)(DIRECTMAP_VIRT_START + va_offset); }/* read access (should only be used for debug printk's) */
xen/include/xen/pdx.h +25 -0
@@ -160,6 +160,31 @@ static inline unsigned long pdx_to_pfn(unsigned long pdx)#define mfn_to_pdx (mfn) pfn_to_pdx(mfn_x(mfn))#define pdx_to_mfn (pdx) _mfn(pdx_to_pfn(pdx)) +/** +* Computes the offset into the direct map of an maddr +* +* @param ma Machine address +* @return Offset on the direct map where that +* machine address can be accessed +*/ +static inline unsigned long maddr_to_directmapoff (paddr_t ma) +{ +return ((ma & ma_top_mask) >> pfn_pdx_hole_shift) | + (ma & ma_va_bottom_mask); +} + +/** +* Computes a machine address given a direct map offset +* +* @param offset Offset into the direct map +* @return Corresponding machine address of that virtual location +*/ +static inline paddr_t directmapoff_to_maddr (unsigned long offset) +{ +return (((paddr_t )offset << pfn_pdx_hole_shift) & ma_top_mask) | + (offset & ma_va_bottom_mask); +} +/** * Initializes global variables with information about the compressible * range of the current memory regions.
Signature · 1 line
2.34.1
PATCH v3 2/4 mm/pdx: Standardize region validation wrt pdx compression
In reply to Alejandro VallejoRegions must be occasionally validated for pdx compression validity. That is, whether any of the machine addresses spanning the region have a bit set in the pdx "hole" (which is expected to always contain zeroes). There are a few such tests through the code, and they all check for different things.
This patch replaces all such occurrences with a call to a centralized function that checks a region for validity.
---
v3:
* Pack variable declarations on top of the function (Jan)
* Restrict variable scope by moving their declaration to the top of the loop in which they are used (Jan)
--- xen/arch/x86/x86_64/mm.c | 6 ++++-- xen/common/efi/boot.c | 13 ++++++++++--- xen/common/pdx.c | 10 ++++++++-- xen/include/xen/pdx.h | 9 +++++++++ 4 files changed, 31 insertions(+), 7 deletions(-)
xen/arch/x86/x86_64/mm.c +4 -2
@@ -1158,6 +1158,8 @@ static int transfer_pages_to_heap(struct mem_hotadd_info *info)static int mem_hotadd_check (unsigned long spfn,unsigned long epfn) {unsigned long s, e, length, sidx, eidx; +paddr_t mem_base =pfn_to_paddr (spfn); +unsigned long mem_npages = epfn - spfn;if ( (spfn >= epfn) )return 0 ;
@@ -1168,7 +1170,7 @@ static int mem_hotadd_check(unsigned long spfn, unsigned long epfn)if ( (spfn | epfn) & ((1UL <<PAGETABLE_ORDER ) -1 ) )return 0 ; -if ( (spfn | epfn) & pfn_hole_mask ) +if ( !pdx_is_region_compressible (mem_base, mem_npages) )return 0 ;/* Make sure the new range is not present now */
@@ -1207,7 +1209,7 @@ static int mem_hotadd_check(unsigned long spfn, unsigned long epfn) length += (e - s) *sizeof (struct page_info ); -if ((length >>PAGE_SHIFT ) > (epfn - spfn)) +if ((length >>PAGE_SHIFT ) > mem_npages)return 0 ;return 1 ;
xen/common/efi/boot.c +10 -3
@@ -14,6 +14,7 @@#include <xen/multiboot.h> #include <xen/param.h> #include <xen/pci_regs.h> +#include <xen/pdx.h> #include <xen/pfn.h> #if EFI_PAGE_SIZE !=PAGE_SIZE # error Cannot use xen/pfn.h here!
@@ -1645,9 +1646,11 @@ static __init void copy_mapping(unsigned long mfn, unsigned long end, static bool __init cf_check ram_range_valid(unsigned long smfn, unsigned long emfn) { +paddr_t ram_base =pfn_to_paddr (smfn); +unsigned long ram_npages = emfn - smfn;unsigned long sz =pfn_to_pdx (emfn -1 ) /PDX_GROUP_COUNT +1 ; -return !(smfn & pfn_hole_mask) && +return pdx_is_region_compressible (ram_base, ram_npages) &&find_next_bit (pdx_group_valid, sz,pfn_to_pdx (smfn) /PDX_GROUP_COUNT ) < sz; }
@@ -1681,6 +1684,8 @@ void __init efi_init_memory(void)u64 len = desc->NumberOfPages <<EFI_PAGE_SHIFT ;unsigned long smfn, emfn;unsigned int prot =PAGE_HYPERVISOR_RWX ; +paddr_t mem_base; +unsigned long mem_npages; printk(XENLOG_INFO " %013" PRIx64 "-%013" PRIx64 " type=%u attr=%016" PRIx64 "\n",
@@ -1732,6 +1737,9 @@ void __init efi_init_memory(void) smfn =PFN_DOWN (desc->PhysicalStart); emfn =PFN_UP (desc->PhysicalStart + len); + mem_base =pfn_to_paddr (smfn); + mem_npages = emfn - smfn; +if ( desc->Attribute &EFI_MEMORY_WB ) prot |= _PAGE_WB;else if ( desc->Attribute &EFI_MEMORY_WT )
@@ -1759,8 +1767,7 @@ void __init efi_init_memory(void) prot |= _PAGE_NX;if (pfn_to_pdx (emfn -1 ) < (DIRECTMAP_SIZE >>PAGE_SHIFT ) && - !(smfn & pfn_hole_mask) && - !((smfn ^ (emfn -1 )) & ~pfn_pdx_bottom_mask) ) +pdx_is_region_compressible (mem_base, mem_npages)) {if ( (unsigned long )mfn_to_virt (emfn -1 ) >=HYPERVISOR_VIRT_END ) prot &= ~_PAGE_GLOBAL;
xen/common/pdx.c +8 -2
@@ -88,7 +88,7 @@ bool __mfn_valid(unsigned long mfn) }/* Sets all bits from the most-significant 1-bit down to the LSB */ -static uint64_t __init fill_mask (uint64_t mask) +static uint64_t fill_mask (uint64_t mask) {while (mask & (mask +1 )) mask |= mask +1 ;
@@ -96,6 +96,12 @@ static uint64_t __init fill_mask(uint64_t mask)return mask; } +bool pdx_is_region_compressible (paddr_t base,unsigned long npages) +{ +return !(paddr_to_pfn (base) & pfn_hole_mask) && + !(pdx_region_mask (base, npages *PAGE_SIZE ) & ~ma_va_bottom_mask); +} +/* We don't want to compress the low MAX_ORDER bits of the addresses. */ uint64_t __init pdx_init_mask (uint64_t base_addr) {
@@ -103,7 +109,7 @@ uint64_t __init pdx_init_mask(uint64_t base_addr) (uint64_t)1 << (MAX_ORDER + PAGE_SHIFT)) - 1); } -uint64_t __init pdx_region_mask (uint64_t base,uint64_t len) +uint64_t pdx_region_mask (uint64_t base,uint64_t len) {/* * We say a bit "moves" in a range if there exist 2 addresses in that
xen/include/xen/pdx.h +9 -0
@@ -79,6 +79,15 @@ extern unsigned long pfn_top_mask, ma_top_mask; (sizeof(*frame_table) & -sizeof(*frame_table)))extern unsigned long pdx_group_valid[]; +/** +* Validate a region's compatibility with the current compression runtime +* +* @param base Base address of the region +* @param npages Number of PAGE_SIZE-sized pages in the region +* @return True iff the region can be used with the current compression +*/ +bool pdx_is_region_compressible (paddr_t base,unsigned long npages); +/** * Calculates a mask covering "moving" bits of all addresses of a region *
Signature · 1 line
2.34.1
PATCH v3 3/4 pdx: Reorder pdx.[ch]
In reply to Alejandro VallejoThe next patch compiles out compression-related chunks, and it's helpful to have them grouped together beforehand.
---
v3:
* No changes
--- xen/common/pdx.c | 58 +++++++++++++++++++++--------------------- xen/include/xen/pdx.h | 59 ++++++++++++++++++++++--------------------- 2 files changed, 59 insertions(+), 58 deletions(-)
xen/common/pdx.c +29 -29
@@ -20,6 +20,35 @@#include <xen/bitops.h> #include <xen/nospec.h> +/** +* Maximum (non-inclusive) usable pdx. Must be +* modifiable after init due to memory hotplug +*/ +unsigned long __read_mostly max_pdx; + +unsigned long __read_mostly pdx_group_valid[BITS_TO_LONGS ( + (FRAMETABLE_NR +PDX_GROUP_COUNT -1 ) /PDX_GROUP_COUNT )] = { [0 ] =1 }; + +bool __mfn_valid (unsigned long mfn) +{ +if (unlikely (evaluate_nospec (mfn >= max_page)) ) +return false; +return likely (!(mfn & pfn_hole_mask)) && +likely (test_bit (pfn_to_pdx (mfn) /PDX_GROUP_COUNT , + pdx_group_valid)); +} + +void set_pdx_range (unsigned long smfn,unsigned long emfn) +{ +unsigned long idx, eidx; + + idx =pfn_to_pdx (smfn) /PDX_GROUP_COUNT ; + eidx = (pfn_to_pdx (emfn -1 ) +PDX_GROUP_COUNT ) /PDX_GROUP_COUNT ; + +for ( ; idx < eidx; ++idx ) +__set_bit (idx, pdx_group_valid); +} +/* * Diagram to make sense of the following variables. The masks and shifts * are done on mfn values in order to convert to/from pdx:
@@ -47,12 +76,6 @@* ones. */ -/** -* Maximum (non-inclusive) usable pdx. Must be -* modifiable after init due to memory hotplug -*/ -unsigned long __read_mostly max_pdx; -/** Mask for the lower non-compressible bits of an mfn */ unsigned long __ro_after_init pfn_pdx_bottom_mask = ~0UL ;
@@ -75,18 +98,6 @@ unsigned long __ro_after_init pfn_hole_mask = 0;/** Number of bits of the "compressible" bit slice of an mfn */ unsigned int __ro_after_init pfn_pdx_hole_shift =0 ; -unsigned long __read_mostly pdx_group_valid[BITS_TO_LONGS ( - (FRAMETABLE_NR +PDX_GROUP_COUNT -1 ) /PDX_GROUP_COUNT )] = { [0 ] =1 }; - -bool __mfn_valid (unsigned long mfn) -{ -if (unlikely (evaluate_nospec (mfn >= max_page)) ) -return false; -return likely (!(mfn & pfn_hole_mask)) && -likely (test_bit (pfn_to_pdx (mfn) /PDX_GROUP_COUNT , - pdx_group_valid)); -} -/* Sets all bits from the most-significant 1-bit down to the LSB */ static uint64_t fill_mask (uint64_t mask) {
@@ -124,17 +135,6 @@ uint64_t pdx_region_mask(uint64_t base, uint64_t len)return fill_mask (base ^ (base + len -1 )); } -void set_pdx_range (unsigned long smfn,unsigned long emfn) -{ -unsigned long idx, eidx; - - idx =pfn_to_pdx (smfn) /PDX_GROUP_COUNT ; - eidx = (pfn_to_pdx (emfn -1 ) +PDX_GROUP_COUNT ) /PDX_GROUP_COUNT ; - -for ( ; idx < eidx; ++idx ) -__set_bit (idx, pdx_group_valid); -} -void __init pfn_pdx_hole_setup (unsigned long mask) {unsigned int i, j, bottom_shift =0 , hole_shift =0 ;
xen/include/xen/pdx.h +30 -29
82 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v3 4/4 pdx: Add CONFIG_PDX_COMPRESSION as a common Kconfig option
In reply to Alejandro VallejoAdds a new compile-time flag to allow disabling pdx compression and compiles out compression-related code/data. It also shorts the pdx<->pfn conversion macros and creates stubs for masking fucntions.
While at it, removes the old arch-defined CONFIG_HAS_PDX flag, as it was not removable in practice.
---
v3:
* Fixed a typo in Kconfig's help
* Gated CONFIG_PDX_COMPRESSION under EXPERT=y and left default=ARM
* Fixed commit message after renaming CONFIG_PDX_COMPRESSION (Julien)
* Changed #else comment to the reason it executes (Julien)
* Various blank-line related style changes (Jan)
--- xen/arch/arm/Kconfig | 1 - xen/arch/x86/Kconfig | 1 - xen/arch/x86/domain.c | 19 +++++++++++++------ xen/common/Kconfig | 13 ++++++++++--- xen/common/Makefile | 2 +- xen/common/pdx.c | 16 ++++++++++++---- xen/include/xen/pdx.h | 38 +++++++++++++++++++++++++++++++++++--- 7 files changed, 71 insertions(+), 19 deletions(-)
xen/arch/arm/Kconfig +0 -1
@@ -14,7 +14,6 @@ config ARM select HAS_ALTERNATIVE select HAS_DEVICE_TREE select HAS_PASSTHROUGH - select HAS_PDX select HAS_PMAP select HAS_UBSAN select IOMMU_FORCE_PT_SHARE
xen/arch/x86/Kconfig +0 -1
@@ -24,7 +24,6 @@ config X86 select HAS_PASSTHROUGH select HAS_PCI select HAS_PCI_MSI - select HAS_PDX select HAS_SCHED_GRANULARITY select HAS_UBSAN select HAS_VPCI if HVM
xen/arch/x86/domain.c +13 -6
@@ -458,7 +458,7 @@ void domain_cpu_policy_changed(struct domain *d) } } -#ifndef CONFIG_BIGMEM +#if !defined(CONFIG_BIGMEM ) && defined(CONFIG_PDX_COMPRESSION )/* * The hole may be at or above the 44-bit boundary, so we need to determine * the total bit count until reaching 32 significant (not squashed out) bits
@@ -485,13 +485,20 @@ static unsigned int __init noinline _domain_struct_bits(void)struct domain *alloc_domain_struct (void ) {struct domain *d; -#ifdef CONFIG_BIGMEM -const unsigned int bits =0 ; -#else +/* -* We pack the PDX of the domain structure into a 32-bit field within -* the page_info structure. Hence the MEMF_bits() restriction. +* Without CONFIG_BIGMEM, we pack the PDX of the domain structure into +* a 32-bit field within the page_info structure. Hence the MEMF_bits() +* restriction. With PDX compression in place the number of bits must +* be calculated at runtime, but it's fixed otherwise. +* +* On systems with CONFIG_BIGMEM there's no packing, and so there's no +* such restriction. */ +#if defined(CONFIG_BIGMEM ) || !defined(CONFIG_PDX_COMPRESSION ) +const unsigned int bits =IS_ENABLED (CONFIG_BIGMEM ) ?0 : +32 +PAGE_SHIFT ; +#else static unsigned int __read_mostly bits;if (unlikely (!bits) )
xen/common/Kconfig +10 -3
@@ -23,6 +23,16 @@ config GRANT_TABLE If unsure, say Y. +config PDX_COMPRESSION + bool "PDX (Page inDeX) compression support" if EXPERT + default ARM + help + PDX compression is a technique that allows the hypervisor to + represent physical addresses in a very space-efficient manner. + This is very helpful reducing memory wastage in systems with + memory banks with base addresses far from each other, but carries + a performance cost. + config ALTERNATIVE_CALL bool
@@ -53,9 +63,6 @@ config HAS_IOPORTS config HAS_KEXEC bool -config HAS_PDX - bool - config HAS_PMAP bool
xen/common/Makefile +1 -1
@@ -29,7 +29,7 @@ obj-y += multicall.o obj-y += notifier.o obj-$(CONFIG_NUMA) += numa.o obj-y += page_alloc.o -obj-$(CONFIG_HAS_PDX) += pdx.o +obj-y += pdx.o obj-$(CONFIG_PERF_COUNTERS) += perfc.o obj-bin-$(CONFIG_HAS_PMAP) += pmap.init.o obj-y += preempt.o
xen/common/pdx.c +12 -4
@@ -31,11 +31,16 @@ unsigned long __read_mostly pdx_group_valid[BITS_TO_LONGS(bool __mfn_valid (unsigned long mfn) { -if (unlikely (evaluate_nospec (mfn >= max_page)) ) +bool invalid = mfn >= max_page; + +#ifdef CONFIG_PDX_COMPRESSION + invalid |= mfn & pfn_hole_mask; +#endif + +if (unlikely (evaluate_nospec (invalid)) )return false; -return likely (!(mfn & pfn_hole_mask)) && -likely (test_bit (pfn_to_pdx (mfn) /PDX_GROUP_COUNT , - pdx_group_valid)); + +return test_bit (pfn_to_pdx (mfn) /PDX_GROUP_COUNT , pdx_group_valid); }void set_pdx_range (unsigned long smfn,unsigned long emfn)
@@ -49,6 +54,8 @@ void set_pdx_range(unsigned long smfn, unsigned long emfn)__set_bit (idx, pdx_group_valid); } +#ifdef CONFIG_PDX_COMPRESSION +/* * Diagram to make sense of the following variables. The masks and shifts * are done on mfn values in order to convert to/from pdx:
@@ -176,6 +183,7 @@ void __init pfn_pdx_hole_setup(unsigned long mask) ma_top_mask = pfn_top_mask << PAGE_SHIFT; } +#endif /* CONFIG_PDX_COMPRESSION */ /* * Local variables:
xen/include/xen/pdx.h +35 -3
@@ -67,8 +67,6 @@* region involved. */ -#ifdef CONFIG_HAS_PDX -extern unsigned long max_pdx;#define PDX_GROUP_COUNT ((1 << PDX_GROUP_SHIFT) / \
@@ -100,6 +98,8 @@ bool __mfn_valid(unsigned long mfn);#define mfn_to_pdx (mfn) pfn_to_pdx(mfn_x(mfn))#define pdx_to_mfn (pdx) _mfn(pdx_to_pfn(pdx)) +#ifdef CONFIG_PDX_COMPRESSION +extern unsigned long pfn_pdx_bottom_mask, ma_va_bottom_mask;extern unsigned int pfn_pdx_hole_shift;extern unsigned long pfn_hole_mask;
@@ -206,7 +206,39 @@ static inline paddr_t directmapoff_to_maddr(unsigned long offset)*/ void pfn_pdx_hole_setup (unsigned long mask); -#endif /* HAS_PDX */ +#else /* !CONFIG_PDX_COMPRESSION */ + +/* Without PDX compression we can skip some computations */ + +/* pdx<->pfn == identity */ +#define pdx_to_pfn (x) (x) +#define pfn_to_pdx (x) (x) + +/* directmap is indexed by by maddr */ +#define maddr_to_directmapoff (x) (x) +#define directmapoff_to_maddr (x) (x) + +static inline bool pdx_is_region_compressible (unsigned long smfn, +unsigned long emfn) +{ +return true; +} + +static inline uint64_t pdx_init_mask (uint64_t base_addr) +{ +return 0 ; +} + +static inline uint64_t pdx_region_mask (uint64_t base,uint64_t len) +{ +return 0 ; +} + +static inline void pfn_pdx_hole_setup (unsigned long mask) +{ +} + +#endif /* CONFIG_PDX_COMPRESSION */ #endif /* __XEN_PDX_H__ */ /*
Signature · 1 line
2.34.1
Re: [PATCH v3 4/4] pdx: Add CONFIG_PDX_COMPRESSION as a common Kconfig option
Several things.
First, Shawn: PPC has gained a HAS_PDX, the deletion of which needs merging into this patch.
It was added as part of 4a2f68f909304 which was "minimal to build". This series address the issue you presumably encountered where pdx.c appears to be optional but wasn't.
Do PPC platforms have (or potentially have) sparse RAM banks?
If like x86 the answer is definitely no, then you want to have PDX_COMPRESSION=n
If the answer is definitely yes always, then you want PDX_COMPRESSION=y
If the answer is system specific, then you want to offer users a choice.
xen/common/Kconfig config GRANT_TABLE · line 27
Quoted 10 lines · Alejandro Vallejo
xen/common/Kconfig +2
-0@@ -23,6 +23,16 @@ config GRANT_TABLE If unsure, say Y. +config PDX_COMPRESSION + bool "PDX (Page inDeX) compression support" if EXPERT
This still needs hiding on x86. The minimal hack for 4.18 is "if EXPERT && !X86".
Other adjustments needed depending on the PPC answer above.
xen/common/Kconfig config GRANT_TABLE · line 34
Quoted 7 lines
xen/common/Kconfig +7
-0+ default ARM + help + PDX compression is a technique that allows the hypervisor to + represent physical addresses in a very space-efficient manner. + This is very helpful reducing memory wastage in systems with + memory banks with base addresses far from each other, but carries + a performance cost.
This is still intractable for a non-Xen-maintainer, and inaccurate. Whether you get any benefit at all depends on how the sparseness happens to line up.
---
PDX compression is a technique designed to reduce the memory overhead of physical memory management on platforms with sparse RAM banks.
If your platform does have sparse RAM banks, enabling PDX compression may reduce the memory overhead of Xen, but does carry a runtime performance cost.
If your platform does not have sparse RAM banks, do not enable PDX compression.
---
~Andrew
Re: [PATCH v3 4/4] pdx: Add CONFIG_PDX_COMPRESSION as a common Kconfig option
Quoted 5 lines · Andrew Cooper
Quoted 4 lines · Alejandro Vallejo
xen/common/Kconfig +0
-0@@ -23,6 +23,16 @@ config GRANT_TABLE
Quoted 7 lines
xen/common/Kconfig config GRANT_TABLE · line 27
Quoted 4 lines
xen/common/Kconfig +2
-0If unsure, say Y. +config PDX_COMPRESSION + bool "PDX (Page inDeX) compression support" if EXPERTThis still needs hiding on x86. The minimal hack for 4.18 is "if EXPERT && !X86".
If you insist on complete hiding and I insist on allowing it to be used by people who want to play with exotic hardware, then this won't go anywhere. I think I've come far enough with accepting a compromise, and so I think it's your turn now to also take a step from your original position.
Jan
Re: [PATCH v3 4/4] pdx: Add CONFIG_PDX_COMPRESSION as a common Kconfig option
Quoted 14 lines · Jan Beulich
Quoted 5 lines · Andrew Cooper
Quoted 4 lines · Alejandro Vallejo
xen/common/Kconfig +0
-0@@ -23,6 +23,16 @@ config GRANT_TABLEQuoted 7 lines
xen/common/Kconfig config GRANT_TABLE · line 27
Quoted 4 lines
xen/common/Kconfig +2
-0If unsure, say Y. +config PDX_COMPRESSION + bool "PDX (Page inDeX) compression support" if EXPERTThis still needs hiding on x86. The minimal hack for 4.18 is "if EXPERT && !X86".
Quoted 4 lines
If you insist on complete hiding and I insist on allowing it to be used by people who want to play with exotic hardware, then this won't go anywhere. I think I've come far enough with accepting a compromise, and so I think it's your turn now to also take a step from your original position.
Just because I'm not familiar with this, is there any x86 hardware that has such sparse memory map that would benefit from PDX?
Wouldn't anyone doing bring up on such exotic hardware also likely need to perform other adjustments to Xen, and hence commenting out the !X86 in Kconfig be acceptable? (we would likely make it selectable at that point if such platforms exist).
Thanks, Roger.
Re: [PATCH v3 4/4] pdx: Add CONFIG_PDX_COMPRESSION as a common Kconfig option
Quoted 20 lines · Roger Pau Monné
Quoted 14 lines · Jan Beulich
Quoted 5 lines · Andrew Cooper
Quoted 4 lines · Alejandro Vallejo
xen/common/Kconfig +0
-0@@ -23,6 +23,16 @@ config GRANT_TABLEQuoted 7 lines
xen/common/Kconfig config GRANT_TABLE · line 27
Quoted 4 lines
xen/common/Kconfig +2
-0If unsure, say Y. +config PDX_COMPRESSION + bool "PDX (Page inDeX) compression support" if EXPERTThis still needs hiding on x86. The minimal hack for 4.18 is "if EXPERT && !X86".
Quoted 4 lines
If you insist on complete hiding and I insist on allowing it to be used by people who want to play with exotic hardware, then this won't go anywhere. I think I've come far enough with accepting a compromise, and so I think it's your turn now to also take a step from your original position.
Quoted 7 lines
Just because I'm not familiar with this, is there any x86 hardware that has such sparse memory map that would benefit from PDX?
Wouldn't anyone doing bring up on such exotic hardware also likely need to perform other adjustments to Xen, and hence commenting out the !X86 in
Kconfigbe acceptable? (we would likely make it selectable at that point if such platforms exist).
As mentioned before, the reason PDX was introduced was to actually make Xen work on such exotic x86 hardware. While I can't tell for sure, that hardware probably has never made it into production. Yet still things were known to work there after the original adjustments, so no, I would not expect other adjustments to be necessary (provided there was no bitrot).
Jan
Re: [PATCH v3 4/4] pdx: Add CONFIG_PDX_COMPRESSION as a common Kconfig option
In reply to Roger Pau MonnéQuoted 13 lines · Roger Pau Monné
Quoted 12 lines · Jan Beulich
Quoted 11 lines · Andrew Cooper
xen/common/Kconfig config GRANT_TABLE · line 27
Quoted 8 lines · Alejandro Vallejo
xen/common/Kconfig +2
-0@@ -23,6 +23,16 @@ config GRANT_TABLE If unsure, say Y. +config PDX_COMPRESSION + bool "PDX (Page inDeX) compression support" if EXPERTThis still needs hiding on x86. The minimal hack for 4.18 is "if EXPERT && !X86".
Quoted 6 lines
Quoted 4 lines
If you insist on complete hiding and I insist on allowing it to be used by people who want to play with exotic hardware, then this won't go anywhere. I think I've come far enough with accepting a compromise, and so I think it's your turn now to also take a step from your original position.
Just because I'm not familiar with this, is there any x86 hardware that has such sparse memory map that would benefit from PDX?
There is one known system which never shipped. Xen's implementation was from the anticipation of that system shipping. Nothing else known.
None of the other major kernels have facilities such as this, which is very likely a contributory factor to the system not shipping.
Quoted 4 lines
Wouldn't anyone doing bring up on such exotic hardware also likely need to perform other adjustments to Xen, and hence commenting out the !X86 in
Kconfigbe acceptable? (we would likely make it selectable at that point if such platforms exist).
People with bizarre hardware can cover the cost of bringing it up. And that includes tweaking Kconfig so they can select this option.
~Andrew
Re: [PATCH v3 4/4] pdx: Add CONFIG_PDX_COMPRESSION as a common Kconfig option
Quoted 18 lines · Andrew Cooper
Quoted 17 lines · Roger Pau Monné
Quoted 16 lines · Jan Beulich
Quoted 11 lines · Andrew Cooper
xen/common/Kconfig config GRANT_TABLE · line 27
Quoted 8 lines · Alejandro Vallejo
xen/common/Kconfig +2
-0@@ -23,6 +23,16 @@ config GRANT_TABLE If unsure, say Y. +config PDX_COMPRESSION + bool "PDX (Page inDeX) compression support" if EXPERTThis still needs hiding on x86. The minimal hack for 4.18 is "if EXPERT && !X86".
If you insist on complete hiding and I insist on allowing it to be used by people who want to play with exotic hardware, then this won't go anywhere. I think I've come far enough with accepting a compromise, and so I think it's your turn now to also take a step from your original position.
Quoted 8 lines
Quoted 2 lines
Just because I'm not familiar with this, is there any x86 hardware that has such sparse memory map that would benefit from PDX?
There is one known system which never shipped. Xen's implementation was from the anticipation of that system shipping. Nothing else known.
None of the other major kernels have facilities such as this, which is very likely a contributory factor to the system not shipping.
Possibly, yet there's one important difference (mentioned before): VA space that Xen can use (for directmap and frame_table[]) is far more constrained than for baremetal systems (Linux, Windows). Hence the guys who got in touch back at the time were not in need of Linux side changes at all (as far as I was told back then).
Jan
Re: [PATCH v3 4/4] pdx: Add CONFIG_PDX_COMPRESSION as a common Kconfig option
In reply to Andrew CooperQuoted 9 lines · Andrew Cooper
Several things.
First, Shawn: PPC has gained a
HAS_PDX, the deletion of which needs merging into this patch.It was added as part of
4a2f68f909304which was "minimal to build". This series address the issue you presumably encountered wherepdx.cappears to be optional but wasn't.
That's correct, build was broken without HAS_PDX.
Quoted 3 lines
Do PPC platforms have (or potentially have) sparse RAM banks?
Yes, they do. Especially on POWER9, it is very common for there to be large gaps in the physical address space between RAM banks.
Quoted 6 lines
If like x86 the answer is definitely no, then you want to have PDX_COMPRESSION=n
If the answer is definitely yes always, then you want PDX_COMPRESSION=y
If the answer is system specific, then you want to offer users a choice.
It looks like PDX_COMPRESSION=y would probably make the most sense for ppc, but I'm not against making it a choice.
Thanks,
Shawn
Re: [PATCH v3 4/4] pdx: Add CONFIG_PDX_COMPRESSION as a common Kconfig option
Quoted 17 lines · Shawn Anastasio
Quoted 9 lines · Andrew Cooper
Several things.
First, Shawn: PPC has gained a
HAS_PDX, the deletion of which needs merging into this patch.It was added as part of
4a2f68f909304which was "minimal to build". This series address the issue you presumably encountered wherepdx.cappears to be optional but wasn't.That's correct, build was broken without
HAS_PDX.Quoted 2 lines
Do PPC platforms have (or potentially have) sparse RAM banks?
Yes, they do. Especially on POWER9, it is very common for there to be large gaps in the physical address space between RAM banks.
Quoted 8 lines
Quoted 6 lines
If like x86 the answer is definitely no, then you want to have PDX_COMPRESSION=n
If the answer is definitely yes always, then you want PDX_COMPRESSION=y
If the answer is system specific, then you want to offer users a choice.
It looks like PDX_COMPRESSION=y would probably make the most sense for ppc, but I'm not against making it a choice.
Thanks. I'll make suitable adjustments.
~Andrew
Quoted 29 lines · Alejandro Vallejo
Currently there's a
CONFIG_HAS_PDXKconfigoption, but it's impossible to disable it because the whole codebase performs unconditional compression/decompression operations on addresses. This has the unfortunate side effect that systems without a need for compression still have to pay the performance impact of juggling bits on every pfn<->pdx conversion (this requires reading several global variables). This series attempts to:* Leave the state of pdx and pdx compression documented
* Factor out compression so it _can_ be removed throughKconfig
* Make it so compression is disabled on x86 and enabled on both Aarch32 and Aarch64 by default.Series summary:
Patch 1 Moves hard-coded compression-related logic to helper functions Patch 2 Refactors all instances of regions being validated for pdx
compression conformance so it's done through a helper
Patch 3 Non-functional reorder in order to simplify the patch 8 diff Patch 4 Adds newKconfigoption to compile out PDX compression and removes
the oldCONFIG_HAS_PDX, as it was non removableAlready committed:
v1/patch 1 documents the current general understanding of the pdx concept and
pdx compression in particular
v1/patch 3 Marks the pdx compression globals asro_after_initv2/patch 1 Documents the differences between arm32 and arm64 directmaps
Quoted 5 lines
Alejandro Vallejo (4):
mm: Factor out the pdx compression logic in ma/va converters mm/pdx: Standardize region validation wrt pdx compression
pdx: Reorder pdx.[ch]
pdx: AddCONFIG_PDX_COMPRESSIONas a commonKconfigoption
@Jan: Just making sure, are you generally ok with this series as-is?
Thanks,
Alejandro
Quoted 30 lines · Alejandro Vallejo
Quoted 29 lines · Alejandro Vallejo
Currently there's a
CONFIG_HAS_PDXKconfigoption, but it's impossible to disable it because the whole codebase performs unconditional compression/decompression operations on addresses. This has the unfortunate side effect that systems without a need for compression still have to pay the performance impact of juggling bits on every pfn<->pdx conversion (this requires reading several global variables). This series attempts to:* Leave the state of pdx and pdx compression documented
* Factor out compression so it _can_ be removed throughKconfig
* Make it so compression is disabled on x86 and enabled on both Aarch32 and Aarch64 by default.Series summary:
Patch 1 Moves hard-coded compression-related logic to helper functions Patch 2 Refactors all instances of regions being validated for pdx
compression conformance so it's done through a helper
Patch 3 Non-functional reorder in order to simplify the patch 8 diff Patch 4 Adds newKconfigoption to compile out PDX compression and removes
the oldCONFIG_HAS_PDX, as it was non removableAlready committed:
v1/patch 1 documents the current general understanding of the pdx concept and
pdx compression in particular
v1/patch 3 Marks the pdx compression globals asro_after_initv2/patch 1 Documents the differences between arm32 and arm64 directmaps
Quoted 7 lines
Quoted 5 lines
Alejandro Vallejo (4):
mm: Factor out the pdx compression logic in ma/va converters mm/pdx: Standardize region validation wrt pdx compression
pdx: Reorder pdx.[ch]
pdx: AddCONFIG_PDX_COMPRESSIONas a commonKconfigoption@Jan: Just making sure, are you generally ok with this series as-is?
Well, okay would be too strong; I still don't see why my runtime patching series isn't re-considered. But I don't mind it going in the way it is now. I won't ack any part of it, though (in case that wasn't obvious), so it'll be up to Andrew or Roger to supply the necessary x86 acks.
Jan
Hi Jan,
Quoted 37 lines · Jan Beulich
Quoted 36 lines · Alejandro Vallejo
Quoted 34 lines · Alejandro Vallejo
Currently there's a
CONFIG_HAS_PDXKconfigoption, but it's impossible to disable it because the whole codebase performs unconditional compression/decompression operations on addresses. This has the unfortunate side effect that systems without a need for compression still have to pay the performance impact of juggling bits on every pfn<->pdx conversion (this requires reading several global variables). This series attempts to:* Leave the state of pdx and pdx compression documented
* Factor out compression so it _can_ be removed throughKconfig
* Make it so compression is disabled on x86 and enabled on both Aarch32 and Aarch64 by default.Series summary:
Patch 1 Moves hard-coded compression-related logic to helper functions Patch 2 Refactors all instances of regions being validated for pdx
compression conformance so it's done through a helper
Patch 3 Non-functional reorder in order to simplify the patch 8 diff Patch 4 Adds newKconfigoption to compile out PDX compression and removes
the oldCONFIG_HAS_PDX, as it was non removableAlready committed:
v1/patch 1 documents the current general understanding of the pdx concept and
pdx compression in particular
v1/patch 3 Marks the pdx compression globals asro_after_initv2/patch 1 Documents the differences between arm32 and arm64 directmapsAlejandro Vallejo (4):
mm: Factor out the pdx compression logic in ma/va converters mm/pdx: Standardize region validation wrt pdx compression
pdx: Reorder pdx.[ch]
pdx: AddCONFIG_PDX_COMPRESSIONas a commonKconfigoption
Quoted 4 lines
Quoted 1 line
@Jan: Just making sure, are you generally ok with this series as-is?
Well, okay would be too strong; I still don't see why my runtime patching series isn't re-considered.
Do you have a pointer to the series? I would be interested to have a look.
That said... the problem with alt-patching is this is architectural specific. Right now, this seems to be a bit unnecessary given that we believe that virtually no-one will have a platform (I know we talked about a potential one...) where PDX is compressing.
Cheers,
Signature · 1 line
Julien Grall
Quoted 40 lines · Julien Grall
Hi Jan,
Quoted 37 lines · Jan Beulich
Quoted 36 lines · Alejandro Vallejo
Quoted 34 lines · Alejandro Vallejo
Currently there's a
CONFIG_HAS_PDXKconfigoption, but it's impossible to disable it because the whole codebase performs unconditional compression/decompression operations on addresses. This has the unfortunate side effect that systems without a need for compression still have to pay the performance impact of juggling bits on every pfn<->pdx conversion (this requires reading several global variables). This series attempts to:* Leave the state of pdx and pdx compression documented
* Factor out compression so it _can_ be removed throughKconfig
* Make it so compression is disabled on x86 and enabled on both Aarch32 and Aarch64 by default.Series summary:
Patch 1 Moves hard-coded compression-related logic to helper functions Patch 2 Refactors all instances of regions being validated for pdx
compression conformance so it's done through a helper
Patch 3 Non-functional reorder in order to simplify the patch 8 diff Patch 4 Adds newKconfigoption to compile out PDX compression and removes
the oldCONFIG_HAS_PDX, as it was non removableAlready committed:
v1/patch 1 documents the current general understanding of the pdx concept and
pdx compression in particular
v1/patch 3 Marks the pdx compression globals asro_after_initv2/patch 1 Documents the differences between arm32 and arm64 directmapsAlejandro Vallejo (4):
mm: Factor out the pdx compression logic in ma/va converters mm/pdx: Standardize region validation wrt pdx compression
pdx: Reorder pdx.[ch]
pdx: AddCONFIG_PDX_COMPRESSIONas a commonKconfigoption
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
@Jan: Just making sure, are you generally ok with this series as-is?
Well, okay would be too strong; I still don't see why my runtime patching series isn't re-considered.
Do you have a pointer to the series? I would be interested to have a look.
Sure, I can dig it out a 2nd time:
https://lists.xenproject.org/archives/html/xen-devel/2018-09/msg01616.html
Quoted 4 lines
That said... the problem with alt-patching is this is architectural specific. Right now, this seems to be a bit unnecessary given that we believe that virtually no-one will have a platform (I know we talked about a potential one...) where PDX is compressing.
But it defaults to enabled on other than x86 anyway. So it seems like it's generally wanted everywhere except on x86, and on x86 it can (could) be patched out.
Jan
Hi,
Quoted 43 lines · Jan Beulich
Quoted 42 lines · Julien Grall
Hi Jan,
Quoted 39 lines · Jan Beulich
Quoted 37 lines · Alejandro Vallejo
Quoted 29 lines · Alejandro Vallejo
Currently there's a
CONFIG_HAS_PDXKconfigoption, but it's impossible to disable it because the whole codebase performs unconditional compression/decompression operations on addresses. This has the unfortunate side effect that systems without a need for compression still have to pay the performance impact of juggling bits on every pfn<->pdx conversion (this requires reading several global variables). This series attempts to:* Leave the state of pdx and pdx compression documented
* Factor out compression so it _can_ be removed throughKconfig
* Make it so compression is disabled on x86 and enabled on both Aarch32 and Aarch64 by default.Series summary:
Patch 1 Moves hard-coded compression-related logic to helper functions Patch 2 Refactors all instances of regions being validated for pdx
compression conformance so it's done through a helper
Patch 3 Non-functional reorder in order to simplify the patch 8 diff Patch 4 Adds newKconfigoption to compile out PDX compression and removes
the oldCONFIG_HAS_PDX, as it was non removableAlready committed:
v1/patch 1 documents the current general understanding of the pdx concept and
pdx compression in particular
v1/patch 3 Marks the pdx compression globals asro_after_initv2/patch 1 Documents the differences between arm32 and arm64 directmapsQuoted 5 lines
Alejandro Vallejo (4):
mm: Factor out the pdx compression logic in ma/va converters mm/pdx: Standardize region validation wrt pdx compression
pdx: Reorder pdx.[ch]
pdx: AddCONFIG_PDX_COMPRESSIONas a commonKconfigoption@Jan: Just making sure, are you generally ok with this series as-is?
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
Well, okay would be too strong; I still don't see why my runtime patching series isn't re-considered.
Do you have a pointer to the series? I would be interested to have a look.
Sure, I can dig it out a 2nd time:
https://lists.xenproject.org/archives/html/xen-devel/2018-09/msg01616.html
Thanks! AFAIU, the optimization would only happen after the alternative has been patched. This is happening after initializing the heap. With Alejandro series, you will have some performance gain for the boot which I care about.
Quoted 9 lines
Quoted 5 lines
That said... the problem with alt-patching is this is architectural specific. Right now, this seems to be a bit unnecessary given that we believe that virtually no-one will have a platform (I know we talked about a potential one...) where PDX is compressing.
But it defaults to enabled on other than x86 anyway. So it seems like it's generally wanted everywhere except on x86, and on x86 it can (could) be patched out.
IIUC, you are saying that we would want both Alejandro series (to allow an admin to disable PDX at boot) and your series (to fully optimize the PDX at runtime). Is that correct?
If so, it is unclear to me why you want your series to be re-considered now. Can't this be done as a follow-up if there is a desire to further optimize?
Cheers,
Signature · 1 line
Julien Grall
Quoted 51 lines · Julien Grall
Hi,
Quoted 48 lines · Jan Beulich
Quoted 46 lines · Julien Grall
Hi Jan,
Quoted 37 lines · Jan Beulich
Quoted 36 lines · Alejandro Vallejo
Quoted 34 lines · Alejandro Vallejo
Currently there's a
CONFIG_HAS_PDXKconfigoption, but it's impossible to disable it because the whole codebase performs unconditional compression/decompression operations on addresses. This has the unfortunate side effect that systems without a need for compression still have to pay the performance impact of juggling bits on every pfn<->pdx conversion (this requires reading several global variables). This series attempts to:* Leave the state of pdx and pdx compression documented
* Factor out compression so it _can_ be removed throughKconfig
* Make it so compression is disabled on x86 and enabled on both Aarch32 and Aarch64 by default.Series summary:
Patch 1 Moves hard-coded compression-related logic to helper functions Patch 2 Refactors all instances of regions being validated for pdx
compression conformance so it's done through a helper
Patch 3 Non-functional reorder in order to simplify the patch 8 diff Patch 4 Adds newKconfigoption to compile out PDX compression and removes
the oldCONFIG_HAS_PDX, as it was non removableAlready committed:
v1/patch 1 documents the current general understanding of the pdx concept and
pdx compression in particular
v1/patch 3 Marks the pdx compression globals asro_after_initv2/patch 1 Documents the differences between arm32 and arm64 directmapsAlejandro Vallejo (4):
mm: Factor out the pdx compression logic in ma/va converters mm/pdx: Standardize region validation wrt pdx compression
pdx: Reorder pdx.[ch]
pdx: AddCONFIG_PDX_COMPRESSIONas a commonKconfigoptionQuoted 4 lines
Quoted 1 line
@Jan: Just making sure, are you generally ok with this series as-is?
Well, okay would be too strong; I still don't see why my runtime patching series isn't re-considered.
Do you have a pointer to the series? I would be interested to have a look.
Quoted 7 lines
Quoted 2 lines
Sure, I can dig it out a 2nd time:
https://lists.xenproject.org/archives/html/xen-devel/2018-09/msg01616.htmlThanks! AFAIU, the optimization would only happen after the alternative has been patched. This is happening after initializing the heap. With Alejandro series, you will have some performance gain for the boot which I care about.
Fair aspect.
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
That said... the problem with alt-patching is this is architectural specific. Right now, this seems to be a bit unnecessary given that we believe that virtually no-one will have a platform (I know we talked about a potential one...) where PDX is compressing.
Quoted 7 lines
Quoted 3 lines
But it defaults to enabled on other than x86 anyway. So it seems like it's generally wanted everywhere except on x86, and on x86 it can (could) be patched out.
IIUC, you are saying that we would want both Alejandro series (to allow an admin to disable PDX at boot) and your series (to fully optimize the PDX at runtime). Is that correct?
Not really, no. I don't view a build-time decision as necessary; I favor runtime decisions whenever possible. But as said I also don't mind this series going in.
Quoted 3 lines
If so, it is unclear to me why you want your series to be re-considered now. Can't this be done as a follow-up if there is a desire to further optimize?
In principle it could be, yes, but I'm afraid I know that no follow-up is going to happen (and me trying to revive the earlier work would be yet another case of pointlessly invested time).
Jan
Hi,
Quoted 71 lines · Jan Beulich
Quoted 51 lines · Julien Grall
Hi,
Quoted 48 lines · Jan Beulich
Quoted 46 lines · Julien Grall
Hi Jan,
Quoted 37 lines · Jan Beulich
Quoted 36 lines · Alejandro Vallejo
Quoted 34 lines · Alejandro Vallejo
Currently there's a
CONFIG_HAS_PDXKconfigoption, but it's impossible to disable it because the whole codebase performs unconditional compression/decompression operations on addresses. This has the unfortunate side effect that systems without a need for compression still have to pay the performance impact of juggling bits on every pfn<->pdx conversion (this requires reading several global variables). This series attempts to:* Leave the state of pdx and pdx compression documented
* Factor out compression so it _can_ be removed throughKconfig
* Make it so compression is disabled on x86 and enabled on both Aarch32 and Aarch64 by default.Series summary:
Patch 1 Moves hard-coded compression-related logic to helper functions Patch 2 Refactors all instances of regions being validated for pdx
compression conformance so it's done through a helper
Patch 3 Non-functional reorder in order to simplify the patch 8 diff Patch 4 Adds newKconfigoption to compile out PDX compression and removes
the oldCONFIG_HAS_PDX, as it was non removableAlready committed:
v1/patch 1 documents the current general understanding of the pdx concept and
pdx compression in particular
v1/patch 3 Marks the pdx compression globals asro_after_initv2/patch 1 Documents the differences between arm32 and arm64 directmapsAlejandro Vallejo (4):
mm: Factor out the pdx compression logic in ma/va converters mm/pdx: Standardize region validation wrt pdx compression
pdx: Reorder pdx.[ch]
pdx: AddCONFIG_PDX_COMPRESSIONas a commonKconfigoptionQuoted 4 lines
Quoted 1 line
@Jan: Just making sure, are you generally ok with this series as-is?
Well, okay would be too strong; I still don't see why my runtime patching series isn't re-considered.
Do you have a pointer to the series? I would be interested to have a look.
Quoted 7 lines
Quoted 2 lines
Sure, I can dig it out a 2nd time:
https://lists.xenproject.org/archives/html/xen-devel/2018-09/msg01616.htmlThanks! AFAIU, the optimization would only happen after the alternative has been patched. This is happening after initializing the heap. With Alejandro series, you will have some performance gain for the boot which I care about.
Fair aspect.
Quoted 9 lines
Quoted 8 lines
Quoted 4 lines
That said... the problem with alt-patching is this is architectural specific. Right now, this seems to be a bit unnecessary given that we believe that virtually no-one will have a platform (I know we talked about a potential one...) where PDX is compressing.
But it defaults to enabled on other than x86 anyway. So it seems like it's generally wanted everywhere except on x86, and on x86 it can (could) be patched out.
Quoted 6 lines
Quoted 3 lines
IIUC, you are saying that we would want both Alejandro series (to allow an admin to disable PDX at boot) and your series (to fully optimize the PDX at runtime). Is that correct?
Not really, no. I don't view a build-time decision as necessary; I favor runtime decisions whenever possible.
At least on Arm, we want to cater two different set of users. One set will want to tailor Xen to there platform and runtime decision may not be desirable. The other set (e.g. distros) will want to run Xen everywhere so runtime is preferable.
So I think we should be able to offer both build and runtime option when it makes sense. The PDX is one example where both could be interesting, yet at the moment there seem to be an appetite for build time only configuration.
Quoted 8 lines
Quoted 4 lines
If so, it is unclear to me why you want your series to be re-considered now. Can't this be done as a follow-up if there is a desire to further optimize?
In principle it could be, yes, but I'm afraid I know that no follow-up is going to happen (and me trying to revive the earlier work would be yet another case of pointlessly invested time).
Right which is why I wrote "desire". But my point was that Alejandro work is not a one-way door. It would still possible to have runtime patching on top of it.
Cheers,
Signature · 1 line
Julien Grall