Devmap cleanups + arm64 support
Subsystems (8)
v3 cover Devmap cleanups + arm64 support
Hi all,
Here's a refresh of my devmap stuff, now including the end goal as well. Patches #1 to #3 are just a rebase of v2, and I hope are ready to be picked up in the mm tree (#1 is currently doing double-duty in Dan's subsection series as well). Patch #4 could either go via mm if Will and Catalin agree, or could go via arm64 with a small tweak to let it build (but otherwise do nothing) until it meets up with #3 again.
Robin.
Robin Murphy (4):
mm/memremap: Rename and consolidate SECTION_SIZE
mm: clean up is_device_*_page() definitions
mm: introduce ARCH_HAS_PTE_DEVMAP
arm64: mm: Implement pte_devmap support
arch/arm64/Kconfig | 1 + arch/arm64/include/asm/pgtable-prot.h | 1 + arch/arm64/include/asm/pgtable.h | 19 ++++++++ arch/powerpc/Kconfig | 2 +- arch/powerpc/include/asm/book3s/64/pgtable.h | 1 - arch/x86/Kconfig | 2 +- arch/x86/include/asm/pgtable.h | 4 +- arch/x86/include/asm/pgtable_types.h | 1 - include/linux/mm.h | 47 +++++++------------- include/linux/mmzone.h | 1 + include/linux/pfn_t.h | 4 +- kernel/memremap.c | 10 ++--- mm/Kconfig | 5 +-- mm/gup.c | 2 +- mm/hmm.c | 2 - 15 files changed, 50 insertions(+), 52 deletions(-)
Signature · 1 line
2.21.0.dirty
PATCH v3 1/4 mm/memremap: Rename and consolidate SECTION_SIZE
Trying to activate ZONE_DEVICE for arm64 reveals that memremap's internal helpers for sparsemem sections conflict with and arm64's definitions for hugepages, which inherit the name of "sections" from earlier versions of the ARM architecture.
Disambiguate memremap (and now HMM too) by propagating sparsemem's PA_ prefix, to clarify that these values are in terms of addresses rather than PFNs (and because it's a heck of a lot easier than changing all the arch code). SECTION_MASK is unused, so it can just go.
[anshuman: Consolidated mm/hmm.c instance and updated the commit message]
--- include/linux/mmzone.h | 1 + kernel/memremap.c | 10 ++++------ mm/hmm.c | 2 -- 3 files changed, 5 insertions(+), 8 deletions(-)
include/linux/mmzone.h +1 -0
@@ -1134,6 +1134,7 @@ static inline unsigned long early_pfn_to_nid(unsigned long pfn)* PFN_SECTION_SHIFT pfn to/from section number */ #define PA_SECTION_SHIFT (SECTION_SIZE_BITS) +#define PA_SECTION_SIZE (1UL << PA_SECTION_SHIFT)#define PFN_SECTION_SHIFT (SECTION_SIZE_BITS - PAGE_SHIFT)#define NR_MEM_SECTIONS (1UL << SECTIONS_SHIFT)
kernel/memremap.c +4 -6
@@ -14,8 +14,6 @@#include <linux/hmm.h> static DEFINE_XARRAY (pgmap_array); -#define SECTION_MASK ~((1UL << PA_SECTION_SHIFT) - 1) -#define SECTION_SIZE (1UL << PA_SECTION_SHIFT)#if IS_ENABLED (CONFIG_DEVICE_PRIVATE ) vm_fault_t device_private_entry_fault(struct vm_area_struct *vma,
@@ -97,8 +95,8 @@ static void devm_memremap_pages_release(void *data)put_page (pfn_to_page (pfn));/* pages are dead and unused, undo the arch mapping */ - align_start = res->start & ~(SECTION_SIZE -1 ); - align_size =ALIGN (res->start +resource_size (res),SECTION_SIZE ) + align_start = res->start & ~(PA_SECTION_SIZE -1 ); + align_size =ALIGN (res->start +resource_size (res),PA_SECTION_SIZE ) - align_start; nid =page_to_nid (pfn_to_page (align_start >>PAGE_SHIFT ));
@@ -159,8 +157,8 @@ void *devm_memremap_pages(struct device *dev, struct dev_pagemap *pgmap)if (!pgmap->ref || !pgmap->kill)return ERR_PTR (-EINVAL ); - align_start = res->start & ~(SECTION_SIZE -1 ); - align_size =ALIGN (res->start +resource_size (res),SECTION_SIZE ) + align_start = res->start & ~(PA_SECTION_SIZE -1 ); + align_size =ALIGN (res->start +resource_size (res),PA_SECTION_SIZE ) - align_start; align_end = align_start + align_size -1 ;
mm/hmm.c +0 -2
@@ -34,8 +34,6 @@#include <linux/mmu_notifier.h> #include <linux/memory_hotplug.h> -#define PA_SECTION_SIZE (1UL << PA_SECTION_SHIFT) -#if IS_ENABLED (CONFIG_HMM_MIRROR )static const struct mmu_notifier_ops hmm_mmu_notifier_ops;
Signature · 1 line
2.21.0.dirty
PATCH v3 2/4 mm: clean up is_device_*_page() definitions
In reply to Robin MurphyRefactor is_device_{public,private}_page() with is_pci_p2pdma_page() to make them all consistent in depending on their respective config options even when CONFIG_DEV_PAGEMAP_OPS is enabled for other reasons. This allows a little more compile-time optimisation as well as the conceptual and cosmetic cleanup.
--- include/linux/mm.h | 43 +++++++++++++------------------------------ 1 file changed, 13 insertions(+), 30 deletions(-)
include/linux/mm.h +13 -30
@@ -942,32 +942,6 @@ static inline bool put_devmap_managed_page(struct page *page) }return false; } - -static inline bool is_device_private_page (const struct page *page) -{ -return is_zone_device_page (page) && - page->pgmap->type ==MEMORY_DEVICE_PRIVATE ; -} - -static inline bool is_device_public_page (const struct page *page) -{ -return is_zone_device_page (page) && - page->pgmap->type ==MEMORY_DEVICE_PUBLIC ; -} - -#ifdef CONFIG_PCI_P2PDMA -static inline bool is_pci_p2pdma_page (const struct page *page) -{ -return is_zone_device_page (page) && - page->pgmap->type ==MEMORY_DEVICE_PCI_P2PDMA ; -} -#else /* CONFIG_PCI_P2PDMA */ -static inline bool is_pci_p2pdma_page (const struct page *page) -{ -return false; -} -#endif /* CONFIG_PCI_P2PDMA */ -#else /* CONFIG_DEV_PAGEMAP_OPS */ static inline void dev_pagemap_get_ops (void ) {
@@ -981,22 +955,31 @@ static inline bool put_devmap_managed_page(struct page *page) {return false; } +#endif /* CONFIG_DEV_PAGEMAP_OPS */ static inline bool is_device_private_page (const struct page *page) { -return false; +return IS_ENABLED (CONFIG_DEV_PAGEMAP_OPS ) && +IS_ENABLED (CONFIG_DEVICE_PRIVATE ) && +is_zone_device_page (page) && + page->pgmap->type ==MEMORY_DEVICE_PRIVATE ; }static inline bool is_device_public_page (const struct page *page) { -return false; +return IS_ENABLED (CONFIG_DEV_PAGEMAP_OPS ) && +IS_ENABLED (CONFIG_DEVICE_PUBLIC ) && +is_zone_device_page (page) && + page->pgmap->type ==MEMORY_DEVICE_PUBLIC ; }static inline bool is_pci_p2pdma_page (const struct page *page) { -return false; +return IS_ENABLED (CONFIG_DEV_PAGEMAP_OPS ) && +IS_ENABLED (CONFIG_PCI_P2PDMA ) && +is_zone_device_page (page) && + page->pgmap->type ==MEMORY_DEVICE_PCI_P2PDMA ; } -#endif /* CONFIG_DEV_PAGEMAP_OPS */ /* 127: arbitrary random number, small enough to assemble well */ #define page_ref_zero_or_close_to_overflow (page) \
Signature · 1 line
2.21.0.dirty
PATCH v3 3/4 mm: introduce ARCH_HAS_PTE_DEVMAP
In reply to Robin MurphyARCH_HAS_ZONE_DEVICE is somewhat meaningless in itself, and combined with the long-out-of-date comment can lead to the impression than an architecture may just enable it (since __add_pages() now "comprehends device memory" for itself) and expect things to work.
In practice, however, ZONE_DEVICE users have little chance of functioning correctly without __HAVE_ARCH_PTE_DEVMAP, so let's clean that up the same way as ARCH_HAS_PTE_SPECIAL and make it the proper dependency so the real situation is clearer.
--- arch/powerpc/Kconfig | 2 +- arch/powerpc/include/asm/book3s/64/pgtable.h | 1 - arch/x86/Kconfig | 2 +- arch/x86/include/asm/pgtable.h | 4 ++-- arch/x86/include/asm/pgtable_types.h | 1 - include/linux/mm.h | 4 ++-- include/linux/pfn_t.h | 4 ++-- mm/Kconfig | 5 ++--- mm/gup.c | 2 +- 9 files changed, 11 insertions(+), 14 deletions(-)
arch/powerpc/Kconfig +1 -1
@@ -128,6 +128,7 @@ config PPC select ARCH_HAS_MMIOWB if PPC64 select ARCH_HAS_PHYS_TO_DMA select ARCH_HAS_PMEM_API if PPC64 + select ARCH_HAS_PTE_DEVMAP if PPC_BOOK3S_64 select ARCH_HAS_PTE_SPECIAL select ARCH_HAS_MEMBARRIER_CALLBACKS select ARCH_HAS_SCALED_CPUTIME if VIRT_CPU_ACCOUNTING_NATIVE && PPC64
@@ -135,7 +136,6 @@ config PPC select ARCH_HAS_TICK_BROADCAST if GENERIC_CLOCKEVENTS_BROADCAST select ARCH_HAS_UACCESS_FLUSHCACHE if PPC64 select ARCH_HAS_UBSAN_SANITIZE_ALL - select ARCH_HAS_ZONE_DEVICE if PPC_BOOK3S_64 select ARCH_HAVE_NMI_SAFE_CMPXCHG select ARCH_KEEP_MEMBLOCK select ARCH_MIGHT_HAVE_PC_PARPORT
arch/powerpc/include/asm/book3s/64/pgtable.h +0 -1
@@ -90,7 +90,6 @@#define _PAGE_SOFT_DIRTY _RPAGE_SW3/* software: software dirty tracking */ #define _PAGE_SPECIAL _RPAGE_SW2/* software: special page */ #define _PAGE_DEVMAP _RPAGE_SW1/* software: ZONE_DEVICE page */ -#define __HAVE_ARCH_PTE_DEVMAP/* * Drivers request for cache inhibited pte mapping using _PAGE_NO_CACHE
arch/x86/Kconfig +1 -1
@@ -69,6 +69,7 @@ config X86 select ARCH_HAS_KCOV if X86_64 select ARCH_HAS_MEMBARRIER_SYNC_CORE select ARCH_HAS_PMEM_API if X86_64 + select ARCH_HAS_PTE_DEVMAP if X86_64 select ARCH_HAS_PTE_SPECIAL select ARCH_HAS_REFCOUNT select ARCH_HAS_UACCESS_FLUSHCACHE if X86_64
@@ -79,7 +80,6 @@ config X86 select ARCH_HAS_STRICT_MODULE_RWX select ARCH_HAS_SYNC_CORE_BEFORE_USERMODE select ARCH_HAS_UBSAN_SANITIZE_ALL - select ARCH_HAS_ZONE_DEVICE if X86_64 select ARCH_HAVE_NMI_SAFE_CMPXCHG select ARCH_MIGHT_HAVE_ACPI_PDC if ACPI select ARCH_MIGHT_HAVE_PC_PARPORT
arch/x86/include/asm/pgtable.h +2 -2
@@ -271,7 +271,7 @@ static inline int has_transparent_hugepage(void)return boot_cpu_has (X86_FEATURE_PSE ); } -#ifdef __HAVE_ARCH_PTE_DEVMAP +#ifdef CONFIG_ARCH_HAS_PTE_DEVMAP static inline int pmd_devmap (pmd_t pmd) {return !!(pmd_val (pmd) & _PAGE_DEVMAP);
@@ -732,7 +732,7 @@ static inline int pte_present(pte_t a)return pte_flags (a) & (_PAGE_PRESENT | _PAGE_PROTNONE); } -#ifdef __HAVE_ARCH_PTE_DEVMAP +#ifdef CONFIG_ARCH_HAS_PTE_DEVMAP static inline int pte_devmap (pte_t a) {return (pte_flags (a) & _PAGE_DEVMAP) == _PAGE_DEVMAP;
arch/x86/include/asm/pgtable_types.h +0 -1
@@ -103,7 +103,6 @@#if defined(CONFIG_X86_64 ) || defined(CONFIG_X86_PAE )#define _PAGE_NX (_AT(pteval_t, 1) << _PAGE_BIT_NX)#define _PAGE_DEVMAP (_AT(u64, 1) << _PAGE_BIT_DEVMAP) -#define __HAVE_ARCH_PTE_DEVMAP#else #define _PAGE_NX (_AT(pteval_t, 0))#define _PAGE_DEVMAP (_AT(pteval_t, 0))
include/linux/mm.h +2 -2
@@ -536,7 +536,7 @@ static inline void vma_set_anonymous(struct vm_area_struct *vma)struct mmu_gather ;struct inode ; -#if !defined(__HAVE_ARCH_PTE_DEVMAP) || !defined(CONFIG_TRANSPARENT_HUGEPAGE ) +#if !defined(CONFIG_ARCH_HAS_PTE_DEVMAP ) || !defined(CONFIG_TRANSPARENT_HUGEPAGE )static inline int pmd_devmap (pmd_t pmd) {return 0 ;
@@ -1754,7 +1754,7 @@ static inline void sync_mm_rss(struct mm_struct *mm) }#endif -#ifndef __HAVE_ARCH_PTE_DEVMAP +#ifndef CONFIG_ARCH_HAS_PTE_DEVMAP static inline int pte_devmap (pte_t pte) {return 0 ;
include/linux/pfn_t.h +2 -2
@@ -104,7 +104,7 @@ static inline pud_t pfn_t_pud(pfn_t pfn, pgprot_t pgprot)#endif #endif -#ifdef __HAVE_ARCH_PTE_DEVMAP +#ifdef CONFIG_ARCH_HAS_PTE_DEVMAP static inline bool pfn_t_devmap (pfn_t pfn) {const u64 flags =PFN_DEV |PFN_MAP ;
@@ -122,7 +122,7 @@ pmd_t pmd_mkdevmap(pmd_t pmd);defined (CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD) pud_t pud_mkdevmap(pud_t pud);#endif -#endif /* __HAVE_ARCH_PTE_DEVMAP */ +#endif /* CONFIG_ARCH_HAS_PTE_DEVMAP */ #ifdef CONFIG_ARCH_HAS_PTE_SPECIAL static inline bool pfn_t_special (pfn_t pfn)
mm/Kconfig +2 -3
@@ -647,8 +647,7 @@ config IDLE_PAGE_TRACKING See Documentation/admin-guide/mm/idle_page_tracking.rst for more details. -# arch_add_memory() comprehends device memory -config ARCH_HAS_ZONE_DEVICE +config ARCH_HAS_PTE_DEVMAP bool config ZONE_DEVICE
@@ -656,7 +655,7 @@ config ZONE_DEVICE depends on MEMORY_HOTPLUG depends on MEMORY_HOTREMOVE depends on SPARSEMEM_VMEMMAP - depends on ARCH_HAS_ZONE_DEVICE + depends on ARCH_HAS_PTE_DEVMAP select XARRAY_MULTI help
mm/gup.c +1 -1
@@ -1799,7 +1799,7 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end, }#endif /* CONFIG_ARCH_HAS_PTE_SPECIAL */ -#if defined(__HAVE_ARCH_PTE_DEVMAP) && defined(CONFIG_TRANSPARENT_HUGEPAGE ) +#if defined(CONFIG_ARCH_HAS_PTE_DEVMAP ) && defined(CONFIG_TRANSPARENT_HUGEPAGE )static int __gup_device_huge (unsigned long pfn,unsigned long addr,unsigned long end,struct page **pages,int *nr) {
Signature · 1 line
2.21.0.dirty
PATCH v3 4/4 arm64: mm: Implement pte_devmap support
In reply to Robin MurphyIn order for things like get_user_pages() to work on ZONE_DEVICE memory, we need a software PTE bit to identify device-backed PFNs. Hook this up along with the relevant helpers to join in with ARCH_HAS_PTE_DEVMAP.
--- arch/arm64/Kconfig | 1 + arch/arm64/include/asm/pgtable-prot.h | 1 + arch/arm64/include/asm/pgtable.h | 19 +++++++++++++++++++ 3 files changed, 21 insertions(+)
arch/arm64/Kconfig +1 -0
@@ -23,6 +23,7 @@ config ARM64 select ARCH_HAS_KCOV select ARCH_HAS_KEEPINITRD select ARCH_HAS_MEMBARRIER_SYNC_CORE + select ARCH_HAS_PTE_DEVMAP select ARCH_HAS_PTE_SPECIAL select ARCH_HAS_SETUP_DMA_OPS select ARCH_HAS_SET_MEMORY
arch/arm64/include/asm/pgtable-prot.h +1 -0
@@ -28,6 +28,7 @@#define PTE_WRITE (PTE_DBM)/* same as DBM (51) */ #define PTE_DIRTY (_AT(pteval_t, 1) << 55)#define PTE_SPECIAL (_AT(pteval_t, 1) << 56) +#define PTE_DEVMAP (_AT(pteval_t, 1) << 57)#define PTE_PROT_NONE (_AT(pteval_t, 1) << 58)/* only when !PTE_VALID */ #ifndef __ASSEMBLY__
arch/arm64/include/asm/pgtable.h +19 -0
@@ -90,6 +90,7 @@ extern unsigned long empty_zero_page[PAGE_SIZE / sizeof(unsigned long)];#define pte_write (pte) (!!(pte_val(pte) & PTE_WRITE))#define pte_user_exec (pte) (!(pte_val(pte) & PTE_UXN))#define pte_cont (pte) (!!(pte_val(pte) & PTE_CONT)) +#define pte_devmap (pte) (!!(pte_val(pte) & PTE_DEVMAP))#define pte_cont_addr_end (addr, end) \ ({ unsigned long __boundary = ((addr) + CONT_PTE_SIZE) & CONT_PTE_MASK; \
@@ -217,6 +218,11 @@ static inline pmd_t pmd_mkcont(pmd_t pmd)return __pmd (pmd_val (pmd) |PMD_SECT_CONT ); } +static inline pte_t pte_mkdevmap (pte_t pte) +{ +return set_pte_bit (pte,__pgprot (PTE_DEVMAP )); +} +static inline void set_pte (pte_t *ptep,pte_t pte) {WRITE_ONCE (*ptep, pte);
@@ -381,6 +387,9 @@ static inline int pmd_protnone(pmd_t pmd)#define pmd_mkhuge (pmd) (__pmd(pmd_val(pmd) & ~PMD_TABLE_BIT)) +#define pmd_devmap (pmd) pte_devmap(pmd_pte(pmd)) +#define pmd_mkdevmap (pmd) pte_pmd(pte_mkdevmap(pmd_pte(pmd))) +#define __pmd_to_phys (pmd) __pte_to_phys(pmd_pte(pmd))#define __phys_to_pmd_val (phys) __phys_to_pte_val(phys)#define pmd_pfn (pmd) ((__pmd_to_phys(pmd) & PMD_MASK) >> PAGE_SHIFT)
@@ -537,6 +546,11 @@ static inline phys_addr_t pud_page_paddr(pud_t pud)return __pud_to_phys (pud); } +static inline int pud_devmap (pud_t pud) +{ +return 0 ; +} +/* Find an entry in the second-level page table. */ #define pmd_index (addr) (((addr) >> PMD_SHIFT) & (PTRS_PER_PMD - 1))
@@ -624,6 +638,11 @@ static inline phys_addr_t pgd_page_paddr(pgd_t pgd)#define pgd_ERROR (pgd) __pgd_error(__FILE__, __LINE__, pgd_val(pgd)) +static inline int pgd_devmap (pgd_t pgd) +{ +return 0 ; +} +/* to find an entry in a page-table-directory */ #define pgd_index (addr) (((addr) >> PGDIR_SHIFT) & (PTRS_PER_PGD - 1))
Signature · 1 line
2.21.0.dirty
Re: [PATCH v3 4/4] arm64: mm: Implement pte_devmap support
Quoted 47 lines · Robin Murphy
arch/arm64/include/asm/pgtable.h +14
-0@@ -90,6 +90,7 @@ extern unsigned long empty_zero_page[PAGE_SIZE / sizeof(unsigned long)];#define pte_write (pte) (!!(pte_val(pte) & PTE_WRITE))#define pte_user_exec (pte) (!(pte_val(pte) & PTE_UXN))#define pte_cont (pte) (!!(pte_val(pte) & PTE_CONT)) +#define pte_devmap (pte) (!!(pte_val(pte) & PTE_DEVMAP))#define pte_cont_addr_end (addr, end) \ ({ unsigned long __boundary = ((addr) + CONT_PTE_SIZE) & CONT_PTE_MASK; \@@ -217,6 +218,11 @@ static inline pmd_t pmd_mkcont(pmd_t pmd)return __pmd (pmd_val (pmd) |PMD_SECT_CONT ); } +static inline pte_t pte_mkdevmap (pte_t pte) +{ +return set_pte_bit (pte,__pgprot (PTE_DEVMAP )); +} +static inline void set_pte (pte_t *ptep,pte_t pte) {WRITE_ONCE (*ptep, pte);@@ -381,6 +387,9 @@ static inline int pmd_protnone(pmd_t pmd)#define pmd_mkhuge (pmd) (__pmd(pmd_val(pmd) & ~PMD_TABLE_BIT)) +#define pmd_devmap (pmd) pte_devmap(pmd_pte(pmd)) +#define pmd_mkdevmap (pmd) pte_pmd(pte_mkdevmap(pmd_pte(pmd))) +#define __pmd_to_phys (pmd) __pte_to_phys(pmd_pte(pmd))#define __phys_to_pmd_val (phys) __phys_to_pte_val(phys)#define pmd_pfn (pmd) ((__pmd_to_phys(pmd) & PMD_MASK) >> PAGE_SHIFT)@@ -537,6 +546,11 @@ static inline phys_addr_t pud_page_paddr(pud_t pud)return __pud_to_phys (pud); } +static inline int pud_devmap (pud_t pud) +{ +return 0 ; +} +/* Find an entry in the second-level page table. */ #define pmd_index (addr) (((addr) >> PMD_SHIFT) & (PTRS_PER_PMD - 1))
arch/arm64/include/asm/pgtable.h static inline phys_addr_t pgd_page_paddr(pgd_t pgd) · line 644
Quoted 7 lines
arch/arm64/include/asm/pgtable.h +4
-0@@ -624,6 +638,11 @@ static inline phys_addr_t pgd_page_paddr(pgd_t pgd)#define pgd_ERROR (pgd) __pgd_error(__FILE__, __LINE__, pgd_val(pgd)) +static inline int pgd_devmap (pgd_t pgd) +{ +return 0 ; +}
I think you need to guard this and pXd_devmap() with CONFIG_TRANSPARENT_HUGEPAGE, otherwise you'll conflict with the dummy definitions in mm.h and the build will fail.
Will
Re: [PATCH v3 4/4] arm64: mm: Implement pte_devmap support
Quoted 51 lines · Will Deacon
Quoted 50 lines · Robin Murphy
arch/arm64/include/asm/pgtable.h +14
-0@@ -90,6 +90,7 @@ extern unsigned long empty_zero_page[PAGE_SIZE / sizeof(unsigned long)];#define pte_write (pte) (!!(pte_val(pte) & PTE_WRITE))#define pte_user_exec (pte) (!(pte_val(pte) & PTE_UXN))#define pte_cont (pte) (!!(pte_val(pte) & PTE_CONT)) +#define pte_devmap (pte) (!!(pte_val(pte) & PTE_DEVMAP))#define pte_cont_addr_end (addr, end) \ ({ unsigned long __boundary = ((addr) + CONT_PTE_SIZE) & CONT_PTE_MASK; \@@ -217,6 +218,11 @@ static inline pmd_t pmd_mkcont(pmd_t pmd)return __pmd (pmd_val (pmd) |PMD_SECT_CONT ); } +static inline pte_t pte_mkdevmap (pte_t pte) +{ +return set_pte_bit (pte,__pgprot (PTE_DEVMAP )); +} +static inline void set_pte (pte_t *ptep,pte_t pte) {WRITE_ONCE (*ptep, pte);@@ -381,6 +387,9 @@ static inline int pmd_protnone(pmd_t pmd)#define pmd_mkhuge (pmd) (__pmd(pmd_val(pmd) & ~PMD_TABLE_BIT)) +#define pmd_devmap (pmd) pte_devmap(pmd_pte(pmd)) +#define pmd_mkdevmap (pmd) pte_pmd(pte_mkdevmap(pmd_pte(pmd))) +#define __pmd_to_phys (pmd) __pte_to_phys(pmd_pte(pmd))#define __phys_to_pmd_val (phys) __phys_to_pte_val(phys)#define pmd_pfn (pmd) ((__pmd_to_phys(pmd) & PMD_MASK) >> PAGE_SHIFT)@@ -537,6 +546,11 @@ static inline phys_addr_t pud_page_paddr(pud_t pud)return __pud_to_phys (pud); } +static inline int pud_devmap (pud_t pud) +{ +return 0 ; +} +/* Find an entry in the second-level page table. */ #define pmd_index (addr) (((addr) >> PMD_SHIFT) & (PTRS_PER_PMD - 1))@@ -624,6 +638,11 @@ static inline phys_addr_t pgd_page_paddr(pgd_t pgd)#define pgd_ERROR (pgd) __pgd_error(__FILE__, __LINE__, pgd_val(pgd))
Quoted 8 lines
arch/arm64/include/asm/pgtable.h static inline phys_addr_t pgd_page_paddr(pgd_t pgd) · line 644
Quoted 4 lines
arch/arm64/include/asm/pgtable.h +4
-0+static inline int pgd_devmap (pgd_t pgd) +{ +return 0 ; +}I think you need to guard this and
pXd_devmap()withCONFIG_TRANSPARENT_HUGEPAGE, otherwise you'll conflict with the dummy definitions inmm.hand the build will fail.
Ah, right you are - I got as far as catching similar issues with CONFIG_PGTABLE_LEVELS, but apparently I failed to spot the !THP guards in x86 and powerpc. Let me give this one a tweak and test a wider range of configs...
Robin.
Re: [PATCH v3 4/4] arm64: mm: Implement pte_devmap support
In reply to Will DeaconQuoted 51 lines · Will Deacon
Quoted 50 lines · Robin Murphy
arch/arm64/include/asm/pgtable.h +14
-0@@ -90,6 +90,7 @@ extern unsigned long empty_zero_page[PAGE_SIZE / sizeof(unsigned long)];#define pte_write (pte) (!!(pte_val(pte) & PTE_WRITE))#define pte_user_exec (pte) (!(pte_val(pte) & PTE_UXN))#define pte_cont (pte) (!!(pte_val(pte) & PTE_CONT)) +#define pte_devmap (pte) (!!(pte_val(pte) & PTE_DEVMAP))#define pte_cont_addr_end (addr, end) \ ({ unsigned long __boundary = ((addr) + CONT_PTE_SIZE) & CONT_PTE_MASK; \@@ -217,6 +218,11 @@ static inline pmd_t pmd_mkcont(pmd_t pmd)return __pmd (pmd_val (pmd) |PMD_SECT_CONT ); } +static inline pte_t pte_mkdevmap (pte_t pte) +{ +return set_pte_bit (pte,__pgprot (PTE_DEVMAP )); +} +static inline void set_pte (pte_t *ptep,pte_t pte) {WRITE_ONCE (*ptep, pte);@@ -381,6 +387,9 @@ static inline int pmd_protnone(pmd_t pmd)#define pmd_mkhuge (pmd) (__pmd(pmd_val(pmd) & ~PMD_TABLE_BIT)) +#define pmd_devmap (pmd) pte_devmap(pmd_pte(pmd)) +#define pmd_mkdevmap (pmd) pte_pmd(pte_mkdevmap(pmd_pte(pmd))) +#define __pmd_to_phys (pmd) __pte_to_phys(pmd_pte(pmd))#define __phys_to_pmd_val (phys) __phys_to_pte_val(phys)#define pmd_pfn (pmd) ((__pmd_to_phys(pmd) & PMD_MASK) >> PAGE_SHIFT)@@ -537,6 +546,11 @@ static inline phys_addr_t pud_page_paddr(pud_t pud)return __pud_to_phys (pud); } +static inline int pud_devmap (pud_t pud) +{ +return 0 ; +} +/* Find an entry in the second-level page table. */ #define pmd_index (addr) (((addr) >> PMD_SHIFT) & (PTRS_PER_PMD - 1))@@ -624,6 +638,11 @@ static inline phys_addr_t pgd_page_paddr(pgd_t pgd)#define pgd_ERROR (pgd) __pgd_error(__FILE__, __LINE__, pgd_val(pgd))
Quoted 8 lines
arch/arm64/include/asm/pgtable.h static inline phys_addr_t pgd_page_paddr(pgd_t pgd) · line 644
Quoted 4 lines
arch/arm64/include/asm/pgtable.h +4
-0+static inline int pgd_devmap (pgd_t pgd) +{ +return 0 ; +}I think you need to guard this and
pXd_devmap()withCONFIG_TRANSPARENT_HUGEPAGE, otherwise you'll conflict with the dummy definitions inmm.hand the build will fail.
Just curious why pgd_devmap() also needs to be wrapped in TRANSPARENT_HUGEPAGE config (or use this dummy otherwise). IIUC in case of DAX mappings there can never be a huge mapping at PGD level. It only supports PMD or PUD based ones.
PATCH v3 4/4 arm64: mm: Implement pte_devmap support
In reply to Robin MurphyIn order for things like get_user_pages() to work on ZONE_DEVICE memory, we need a software PTE bit to identify device-backed PFNs. Hook this up along with the relevant helpers to join in with ARCH_HAS_PTE_DEVMAP.
---
Fix to build correctly under all combinations of CONFIG_PGTABLE_LEVELS and CONFIG_TRANSPARENT_HUGEPAGE.
arch/arm64/Kconfig | 1 + arch/arm64/include/asm/pgtable-prot.h | 1 + arch/arm64/include/asm/pgtable.h | 21 +++++++++++++++++++++ 3 files changed, 23 insertions(+)
arch/arm64/Kconfig +1 -0
@@ -23,6 +23,7 @@ config ARM64 select ARCH_HAS_KCOV select ARCH_HAS_KEEPINITRD select ARCH_HAS_MEMBARRIER_SYNC_CORE + select ARCH_HAS_PTE_DEVMAP select ARCH_HAS_PTE_SPECIAL select ARCH_HAS_SETUP_DMA_OPS select ARCH_HAS_SET_MEMORY
arch/arm64/include/asm/pgtable-prot.h +1 -0
@@ -28,6 +28,7 @@#define PTE_WRITE (PTE_DBM)/* same as DBM (51) */ #define PTE_DIRTY (_AT(pteval_t, 1) << 55)#define PTE_SPECIAL (_AT(pteval_t, 1) << 56) +#define PTE_DEVMAP (_AT(pteval_t, 1) << 57)#define PTE_PROT_NONE (_AT(pteval_t, 1) << 58)/* only when !PTE_VALID */ #ifndef __ASSEMBLY__
arch/arm64/include/asm/pgtable.h +21 -0
@@ -90,6 +90,7 @@ extern unsigned long empty_zero_page[PAGE_SIZE / sizeof(unsigned long)];#define pte_write (pte) (!!(pte_val(pte) & PTE_WRITE))#define pte_user_exec (pte) (!(pte_val(pte) & PTE_UXN))#define pte_cont (pte) (!!(pte_val(pte) & PTE_CONT)) +#define pte_devmap (pte) (!!(pte_val(pte) & PTE_DEVMAP))#define pte_cont_addr_end (addr, end) \ ({ unsigned long __boundary = ((addr) + CONT_PTE_SIZE) & CONT_PTE_MASK; \
@@ -217,6 +218,11 @@ static inline pmd_t pmd_mkcont(pmd_t pmd)return __pmd (pmd_val (pmd) |PMD_SECT_CONT ); } +static inline pte_t pte_mkdevmap (pte_t pte) +{ +return set_pte_bit (pte,__pgprot (PTE_DEVMAP )); +} +static inline void set_pte (pte_t *ptep,pte_t pte) {WRITE_ONCE (*ptep, pte);
@@ -381,6 +387,11 @@ static inline int pmd_protnone(pmd_t pmd)#define pmd_mkhuge (pmd) (__pmd(pmd_val(pmd) & ~PMD_TABLE_BIT)) +#ifdef CONFIG_TRANSPARENT_HUGEPAGE +#define pmd_devmap (pmd) pte_devmap(pmd_pte(pmd)) +#endif +#define pmd_mkdevmap (pmd) pte_pmd(pte_mkdevmap(pmd_pte(pmd))) +#define __pmd_to_phys (pmd) __pte_to_phys(pmd_pte(pmd))#define __phys_to_pmd_val (phys) __phys_to_pte_val(phys)#define pmd_pfn (pmd) ((__pmd_to_phys(pmd) & PMD_MASK) >> PAGE_SHIFT)
@@ -666,6 +677,16 @@ static inline int pmdp_set_access_flags(struct vm_area_struct *vma, {return ptep_set_access_flags (vma, address, (pte_t *)pmdp,pmd_pte (entry), dirty); } + +static inline int pud_devmap (pud_t pud) +{ +return 0 ; +} + +static inline int pgd_devmap (pgd_t pgd) +{ +return 0 ; +}#endif /*
Signature · 1 line
2.21.0.dirty
Re: [PATCH v3.1 4/4] arm64: mm: Implement pte_devmap support
Quoted 7 lines · Robin Murphy
In order for things like
get_user_pages()to work onZONE_DEVICEmemory, we need a software PTE bit to identify device-backed PFNs. Hook this up along with the relevant helpers to join in withARCH_HAS_PTE_DEVMAP.Signed-off-by Robin Murphy <[email protected]>---
Quoted 7 lines
Fix to build correctly under all combinations of
CONFIG_PGTABLE_LEVELSandCONFIG_TRANSPARENT_HUGEPAGE.arch/arm64/Kconfig | 1 + arch/arm64/include/asm/pgtable-prot.h | 1 + arch/arm64/include/asm/pgtable.h | 21 +++++++++++++++++++++ 3 files changed, 23 insertions(+)
Andrew -- please can you update the previous version of this patch, which I think you picked up?
Thanks,
Will
Robin, Andrew:
I have a series for the hmm tree, which touches the section size bits, and remove device public memory support.
It might be best if we include this series in the hmm tree as well to avoid conflicts. Is it ok to include the rebase version of at least the cleanup part (which looks like it is not required for the actual arm64 support) in the hmm tree to avoid conflicts?
Mark Rutland
reviewerQuoted 1 line · Christoph Hellwig
Robin, Andrew:
As a heads-up, Robin is currently on holiday, so this is all down to Andrew's preference.
Quoted 7 lines
I have a series for the hmm tree, which touches the section size bits, and remove device public memory support.
It might be best if we include this series in the hmm tree as well to avoid conflicts. Is it ok to include the rebase version of at least the cleanup part (which looks like it is not required for the actual arm64 support) in the hmm tree to avoid conflicts?
Per the cover letter, the arm64 patch has a build dependency on the others, so that might require a stable brnach for the common prefix.
Thanks,
Mark.
Quoted 9 lines · Mark Rutland
Quoted 1 line · Christoph Hellwig
Robin, Andrew:
As a heads-up, Robin is currently on holiday, so this is all down to Andrew's preference.
Quoted 3 lines
I have a series for the hmm tree, which touches the section size bits, and remove device public memory support.
Quoted 7 lines
Quoted 4 lines
It might be best if we include this series in the hmm tree as well to avoid conflicts. Is it ok to include the rebase version of at least the cleanup part (which looks like it is not required for the actual arm64 support) in the hmm tree to avoid conflicts?
Per the cover letter, the arm64 patch has a build dependency on the others, so that might require a stable brnach for the common prefix.
I guess we'll just have to live with the merge errors then, as the mm tree is a patch series and thus can't easily use a stable base tree. That is unlike Andrew wants to pull in the hmm tree as a prep patch for the series.
Quoted 15 lines · Christoph Hellwig
Quoted 14 lines · Mark Rutland
Quoted 1 line · Christoph Hellwig
Robin, Andrew:
As a heads-up, Robin is currently on holiday, so this is all down to Andrew's preference.
Quoted 7 lines
I have a series for the hmm tree, which touches the section size bits, and remove device public memory support.
It might be best if we include this series in the hmm tree as well to avoid conflicts. Is it ok to include the rebase version of at least the cleanup part (which looks like it is not required for the actual arm64 support) in the hmm tree to avoid conflicts?
Quoted 7 lines
Quoted 2 lines
Per the cover letter, the arm64 patch has a build dependency on the others, so that might require a stable brnach for the common prefix.
I guess we'll just have to live with the merge errors then, as the mm tree is a patch series and thus can't easily use a stable base tree. That is unlike Andrew wants to pull in the hmm tree as a prep patch for the series.
It looks like the first three patches apply cleanly to hmm.git ..
So what we can do is base this 4 patch series off rc6 and pull the first 3 into hmm and the full 4 into arm.git. We use this workflow often with rdma and netdev.
Let me know and I can help orchestate this.
Jason
Andrew Morton
maintainerQuoted 24 lines · On Wed, 26 Jun 2019 15:45:47 +0000 Jason Gunthorpe <[email protected]> wrote:
Quoted 15 lines · Christoph Hellwig
Quoted 14 lines · Mark Rutland
Quoted 1 line · Christoph Hellwig
Robin, Andrew:
As a heads-up, Robin is currently on holiday, so this is all down to Andrew's preference.
Quoted 7 lines
I have a series for the hmm tree, which touches the section size bits, and remove device public memory support.
It might be best if we include this series in the hmm tree as well to avoid conflicts. Is it ok to include the rebase version of at least the cleanup part (which looks like it is not required for the actual arm64 support) in the hmm tree to avoid conflicts?
Quoted 7 lines
Quoted 2 lines
Per the cover letter, the arm64 patch has a build dependency on the others, so that might require a stable brnach for the common prefix.
I guess we'll just have to live with the merge errors then, as the mm tree is a patch series and thus can't easily use a stable base tree. That is unlike Andrew wants to pull in the hmm tree as a prep patch for the series.
Quoted 7 lines
It looks like the first three patches apply cleanly to hmm.git ..
So what we can do is base this 4 patch series off rc6 and pull the first 3 into hmm and the full 4 into arm.git. We use this workflow often with rdma and netdev.
Let me know and I can help orchestate this.
Well. Whatever works. In this situation I'd stage the patches after linux-next and would merge them up after the prereq patches have been merged into mainline. Easy.
Andrew Morton
maintainerQuoted 5 lines · On Wed, 26 Jun 2019 20:35:51 -0700 Andrew Morton <[email protected]>…
Quoted 1 line
Let me know and I can help orchestate this.
Well. Whatever works. In this situation I'd stage the patches after linux-next and would merge them up after the prereq patches have been merged into mainline. Easy.
All right, what the hell just happened? A bunch of new material has just been introduced into linux-next. I've partially unpicked the resulting mess, haven't dared trying to compile it yet. To get this far I'll need to drop two patch series and one individual patch:
mm-clean-up-is_device__page-definitions.patch
mm-introduce-arch_has_pte_devmap.patch
arm64-mm-implement-pte_devmap-support.patch
arm64-mm-implement-pte_devmap-support-fix.patch
mm-sparsemem-introduce-struct-mem_section_usage.patch mm-sparsemem-introduce-a-section_is_early-flag.patch mm-sparsemem-add-helpers-track-active-portions-of-a-section-at-boot.patch mm-hotplug-prepare-shrink_zone-pgdat_span-for-sub-section-removal.patch mm-sparsemem-convert-kmalloc_section_memmap-to-populate_section_memmap.patch mm-hotplug-kill-is_dev_zone-usage-in-__remove_pages.patch mm-kill-is_dev_zone-helper.patch mm-sparsemem-prepare-for-sub-section-ranges.patch mm-sparsemem-support-sub-section-hotplug.patch mm-document-zone_device-memory-model-implications.patch mm-document-zone_device-memory-model-implications-fix.patch mm-devm_memremap_pages-enable-sub-section-remap.patch libnvdimm-pfn-fix-fsdax-mode-namespace-info-block-zero-fields.patch libnvdimm-pfn-stop-padding-pmem-namespaces-to-section-alignment.patch
mm-sparsemem-cleanup-section-number-data-types.patch mm-sparsemem-cleanup-section-number-data-types-fix.patch
I thought you were just going to move material out of -mm and into hmm.git. Didn't begin to suspect that new and quite disruptive material would be introduced late in -rc7!!
Quoted 9 lines · Andrew Morton
Quoted 5 lines · On Wed, 26 Jun 2019 20:35:51 -0700 Andrew Morton <[email protected]>…
Quoted 1 line
Let me know and I can help orchestate this.
Well. Whatever works. In this situation I'd stage the patches after linux-next and would merge them up after the prereq patches have been merged into mainline. Easy.
All right, what the hell just happened?
Christoph's patch series for the devmap & hmm rework finally made it into linux-next, sorry, it took quite a few iterations on the list to get all the reviews and tests, and figure out how to resolve some other conflicting things. So it just made it this week.
Recall, this is the patch series I asked you about routing a few weeks ago, as it really exceeded the small area that hmm.git was supposed to cover. I think we are both caught off guard how big the conflict is!
Quoted 9 lines
A bunch of new material has just been introduced into linux-next. I've partially unpicked the resulting mess, haven't dared trying to compile it yet. To get this far I'll need to drop two patch series and one individual patch:
mm-clean-up-is_device__page-definitions.patch
mm-introduce-arch_has_pte_devmap.patch
arm64-mm-implement-pte_devmap-support.patch
arm64-mm-implement-pte_devmap-support-fix.patch
This one we discussed, and I thought we agreed would go to your 'stage after linux-next' flow (see above). I think the conflict was minor here.
Quoted 6 lines
mm-sparsemem-introduce-struct-mem_section_usage.patch mm-sparsemem-introduce-a-section_is_early-flag.patch mm-sparsemem-add-helpers-track-active-portions-of-a-section-at-boot.patch mm-hotplug-prepare-shrink_zone-pgdat_span-for-sub-section-removal.patch mm-sparsemem-convert-kmalloc_section_memmap-to-populate_section_memmap.patch mm-hotplug-kill-is_dev_zone-usage-in-__remove_pages.patch
Quoted 8 lines
mm-kill-is_dev_zone-helper.patch mm-sparsemem-prepare-for-sub-section-ranges.patch mm-sparsemem-support-sub-section-hotplug.patch mm-document-zone_device-memory-model-implications.patch mm-document-zone_device-memory-model-implications-fix.patch mm-devm_memremap_pages-enable-sub-section-remap.patch libnvdimm-pfn-fix-fsdax-mode-namespace-info-block-zero-fields.patch libnvdimm-pfn-stop-padding-pmem-namespaces-to-section-alignment.patch
Dan pointed to this while reviewing CH's series and said the conflicts would be manageable, but they are certainly larger than I expected!
This series is the one that seems to be the really big trouble. I already checked all the other stuff that Stephen resolved, and it looks OK and managable. Just this one conflict with kernel/memremap.c is beyond me.
What approach do you want to take to go forward? Here are some thoughts:
CH has said he is away for the long weekend, so the path that involves the fewest people is if Dan respins the above on linux-next and it goes later with the arm patches above, assuming defering it for now has no other adverse effects on -mm.
Pushing CH's series to -mm would need a respin on top of Dan's series above and would need to carry along the whole hmm.git (about 44 patches). Signs are that this could be managed with the code currently in the GPU trees.
If we give up on CH's series the hmm.git will not have conflicts, however we just kick the can to the next merge window where we will be back to having to co-ordinate amd/nouveau/rdma git trees and -mm's patch workflow - and I think we will be worse off as we will have totally given up on a git based work flow for this. :(
Quoted 2 lines
mm-sparsemem-cleanup-section-number-data-types.patch mm-sparsemem-cleanup-section-number-data-types-fix.patch
Stephen used a minor conflict resolution for this one, I checked it carefully and it looked OK.
Quoted 2 lines
I thought you were just going to move material out of -mm and into hmm.git.
Dan brought up a patch from Ira conflicting with CH's work and we did handle that by moving a single patch, as well I moved several hmm specific patches early in the cycle.
Quoted 2 lines
Didn't begin to suspect that new and quite disruptive material would be introduced late in -rc7!!
Unfortunately a non-rebasing tree like hmm.git should only get patches into linux-next once they are fully reviewed and done on the list. I did not attempt to run separately patches 'under review' into linux-next as you do.
Actually I didn't even know this would benefit your workflow, rebasing patches on top of linux-next is not part of the git based workflow I'm using :(
AFAIK Dan and CH were both tracking conflicts with linux-next, so I'd like to hear from Dan what he thinks about his series, maybe the rebase is simple & safe for him? Dan and CH were working pretty closely on CH's series.
Jason
Andrew Morton
maintainerQuoted 5 lines · On Thu, 4 Jul 2019 19:59:38 +0000 Jason Gunthorpe <[email protected]> wrote:
Quoted 4 lines · Andrew Morton
Quoted 2 lines · On Wed, 26 Jun 2019 20:35:51 -0700 Andrew Morton <[email protected]>…
Quoted 1 line
Let me know and I can help orchestate this.
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
Well. Whatever works. In this situation I'd stage the patches after linux-next and would merge them up after the prereq patches have been merged into mainline. Easy.
All right, what the hell just happened?
Christoph's patch series for the devmap & hmm rework finally made it into linux-next
We're talking about "dev_pagemap related cleanups v4", yes?
I note that linux-next contains "mm: remove the HMM config option" which was present in Christoph's v3 series but wasn't present in v4. Perhaps something has gone wrong here.
Quoted 7 lines
sorry, it took quite a few iterations on the list to get all the reviews and tests, and figure out how to resolve some other conflicting things. So it just made it this week.
Recall, this is the patch series I asked you about routing a few weeks ago, as it really exceeded the small area that hmm.git was supposed to cover. I think we are both caught off guard how big the conflict is!
I guess I was distracted - I should have taken a look to see how mergable it all was.
It's a large patchset and it appears to be mainly (entirely?) code cleanups. I don't think such material would be appropriate for a late -rc7 merge even if it didn't conflict with lots of other higher priority pending functional changes and fixes!
Quoted 13 lines · Andrew Morton
Quoted 11 lines · On Thu, 4 Jul 2019 19:59:38 +0000 Jason Gunthorpe <[email protected]> wrote:
Quoted 9 lines · Andrew Morton
Quoted 5 lines · On Wed, 26 Jun 2019 20:35:51 -0700 Andrew Morton <[email protected]>…
Quoted 1 line
Let me know and I can help orchestate this.
Well. Whatever works. In this situation I'd stage the patches after linux-next and would merge them up after the prereq patches have been merged into mainline. Easy.
All right, what the hell just happened?
Quoted 8 lines
Quoted 2 lines
Christoph's patch series for the devmap & hmm rework finally made it into linux-next
We're talking about "
dev_pagemaprelated cleanups v4", yes?I note that linux-next contains "mm: remove the HMM config option" which was present in Christoph's v3 series but wasn't present in v4. Perhaps something has gone wrong here.
When CH sent v4 to the list it was corrupted, v3 is the one to reference for content.
Here is the mailing thread discussing this:
https://lore.kernel.org/linux-mm/[email protected]/
Quoted 10 lines
Quoted 7 lines
sorry, it took quite a few iterations on the list to get all the reviews and tests, and figure out how to resolve some other conflicting things. So it just made it this week.
Recall, this is the patch series I asked you about routing a few weeks ago, as it really exceeded the small area that hmm.git was supposed to cover. I think we are both caught off guard how big the conflict is!
I guess I was distracted - I should have taken a look to see how mergable it all was.
Okay, fair enough. I also could have also done more checks myself with linux-next
Quoted 4 lines
It's a large patchset and it appears to be mainly (entirely?) code cleanups. I don't think such material would be appropriate for a late -rc7 merge even if it didn't conflict with lots of other higher priority pending functional changes and fixes!
I see your other email you resolved the conflicts - so please let me know if you want to proceed with dropping CH's series or not, I'll make a special effort to get that change into tomorrows linux-next if you want (it is already 6pm here)
Regards,
Jason
Quoted 9 lines · Jason Gunthorpe
Quoted 4 lines
It's a large patchset and it appears to be mainly (entirely?) code cleanups. I don't think such material would be appropriate for a late -rc7 merge even if it didn't conflict with lots of other higher priority pending functional changes and fixes!
I see your other email you resolved the conflicts - so please let me know if you want to proceed with dropping CH's series or not, I'll make a special effort to get that change into tomorrows linux-next if you want (it is already 6pm here)
I spent some time this morning looking at the various conflicts, and I think Dan is right, they are mangable. In the sense we can forward a merge resolution to Linus and it is not completely crazy. Most hunks are the usual mechanical sort of conflicts.
Like Stephen, only two small conflict hunks in the memremap.c give me any pause, and I'm confident with CH and Dan's help it can be resolved robustly. If Linus doesn't like it then we fall back to dropping CH's series.
So, here is a fourth idea..
We remove hmm.git entirely from your workflow (ie you revert commit "cc5dfd59e375f Merge branch 'hmm-devmem-cleanup.4' into rdma.git hmm" in your local version of linux-next) and I will send hmm.git to Linus after Dan's patches and others are merged by you to Linus. With Dan and CH's help I will forward the reviewed conflict resolution.
This will not disturb the -mm patch workflow at all, and you can put everything back the way it was on July 3.
What do you think about this?
Jason
Quoted 10 lines · Jason Gunthorpe
Quoted 9 lines · Andrew Morton
Quoted 5 lines · On Wed, 26 Jun 2019 20:35:51 -0700 Andrew Morton <[email protected]>…
Quoted 1 line
Let me know and I can help orchestate this.
Well. Whatever works. In this situation I'd stage the patches after linux-next and would merge them up after the prereq patches have been merged into mainline. Easy.
All right, what the hell just happened?
Aw crap, and I had this series chalked up as done...
Quoted 14 lines
Christoph's patch series for the devmap & hmm rework finally made it into linux-next, sorry, it took quite a few iterations on the list to get all the reviews and tests, and figure out how to resolve some other conflicting things. So it just made it this week.
Recall, this is the patch series I asked you about routing a few weeks ago, as it really exceeded the small area that hmm.git was supposed to cover. I think we are both caught off guard how big the conflict is!
Quoted 4 lines
A bunch of new material has just been introduced into linux-next. I've partially unpicked the resulting mess, haven't dared trying to compile it yet. To get this far I'll need to drop two patch series and one individual patch:
Quoted 8 lines
Quoted 4 lines
mm-clean-up-is_device__page-definitions.patch
mm-introduce-arch_has_pte_devmap.patch
arm64-mm-implement-pte_devmap-support.patch
arm64-mm-implement-pte_devmap-support-fix.patchThis one we discussed, and I thought we agreed would go to your 'stage after linux-next' flow (see above). I think the conflict was minor here.
I can rebase and resend tomorrow if there's an agreement on what exactly to base it on - I'd really like to get this ticked off for 5.3 if at all possible.
Thanks,
Robin.
Quoted 73 lines
Quoted 6 lines
mm-sparsemem-introduce-struct-mem_section_usage.patch mm-sparsemem-introduce-a-section_is_early-flag.patch mm-sparsemem-add-helpers-track-active-portions-of-a-section-at-boot.patch mm-hotplug-prepare-shrink_zone-pgdat_span-for-sub-section-removal.patch mm-sparsemem-convert-kmalloc_section_memmap-to-populate_section_memmap.patch mm-hotplug-kill-is_dev_zone-usage-in-__remove_pages.patch
Quoted 8 lines
mm-kill-is_dev_zone-helper.patch mm-sparsemem-prepare-for-sub-section-ranges.patch mm-sparsemem-support-sub-section-hotplug.patch mm-document-zone_device-memory-model-implications.patch mm-document-zone_device-memory-model-implications-fix.patch mm-devm_memremap_pages-enable-sub-section-remap.patch libnvdimm-pfn-fix-fsdax-mode-namespace-info-block-zero-fields.patch libnvdimm-pfn-stop-padding-pmem-namespaces-to-section-alignment.patch
Dan pointed to this while reviewing CH's series and said the conflicts would be manageable, but they are certainly larger than I expected!
This series is the one that seems to be the really big trouble. I already checked all the other stuff that Stephen resolved, and it looks OK and managable. Just this one conflict with
kernel/memremap.cis beyond me.What approach do you want to take to go forward? Here are some thoughts:
CH has said he is away for the long weekend, so the path that involves the fewest people is if Dan respins the above on linux-next and it goes later with the arm patches above, assuming defering it for now has no other adverse effects on -mm.
Pushing CH's series to -mm would need a respin on top of Dan's series above and would need to carry along the whole hmm.git (about 44 patches). Signs are that this could be managed with the code currently in the GPU trees.
If we give up on CH's series the hmm.git will not have conflicts, however we just kick the can to the next merge window where we will be back to having to co-ordinate amd/nouveau/rdma git trees and -mm's patch workflow - and I think we will be worse off as we will have totally given up on a git based work flow for this. :(
Quoted 2 lines
mm-sparsemem-cleanup-section-number-data-types.patch mm-sparsemem-cleanup-section-number-data-types-fix.patch
Stephen used a minor conflict resolution for this one, I checked it carefully and it looked OK.
Quoted 2 lines
I thought you were just going to move material out of -mm and into hmm.git.
Dan brought up a patch from Ira conflicting with CH's work and we did handle that by moving a single patch, as well I moved several hmm specific patches early in the cycle.
Quoted 2 lines
Didn't begin to suspect that new and quite disruptive material would be introduced late in -rc7!!
Unfortunately a non-rebasing tree like hmm.git should only get patches into linux-next once they are fully reviewed and done on the list. I did not attempt to run separately patches 'under review' into linux-next as you do.
Actually I didn't even know this would benefit your workflow, rebasing patches on top of linux-next is not part of the git based workflow I'm using :(
AFAIK Dan and CH were both tracking conflicts with linux-next, so I'd like to hear from Dan what he thinks about his series, maybe the rebase is simple & safe for him? Dan and CH were working pretty closely on CH's series.
Jason
Andrew Morton
maintainerQuoted 5 lines · On Thu, 4 Jul 2019 21:54:36 +0100 Robin Murphy <[email protected]> wrote:
Quoted 5 lines
Quoted 4 lines
mm-clean-up-is_device__page-definitions.patch
mm-introduce-arch_has_pte_devmap.patch
arm64-mm-implement-pte_devmap-support.patch
arm64-mm-implement-pte_devmap-support-fix.patch
Quoted 7 lines
Quoted 3 lines
This one we discussed, and I thought we agreed would go to your 'stage after linux-next' flow (see above). I think the conflict was minor here.
I can rebase and resend tomorrow if there's an agreement on what exactly to base it on - I'd really like to get this ticked off for 5.3 if at all possible.
I took another look. Yes, it looks like the repairs were simple.
Let me now try to compile all this...
Quoted 11 lines · Andrew Morton
Quoted 9 lines · On Thu, 4 Jul 2019 21:54:36 +0100 Robin Murphy <[email protected]> wrote:
Quoted 8 lines
Quoted 4 lines
mm-clean-up-is_device__page-definitions.patch
mm-introduce-arch_has_pte_devmap.patch
arm64-mm-implement-pte_devmap-support.patch
arm64-mm-implement-pte_devmap-support-fix.patchThis one we discussed, and I thought we agreed would go to your 'stage after linux-next' flow (see above). I think the conflict was minor here.
Quoted 7 lines
Quoted 3 lines
I can rebase and resend tomorrow if there's an agreement on what exactly to base it on - I'd really like to get this ticked off for 5.3 if at all possible.
I took another look. Yes, it looks like the repairs were simple.
Let me now try to compile all this...
Thanks, the revised patches look OK to me, and I've confirmed that today's -next builds and boots for arm64.
Cheers,
Robin.
Quoted 70 lines · Jason Gunthorpe
Quoted 9 lines · Andrew Morton
Quoted 5 lines · On Wed, 26 Jun 2019 20:35:51 -0700 Andrew Morton <[email protected]>…
Quoted 1 line
Let me know and I can help orchestate this.
Well. Whatever works. In this situation I'd stage the patches after linux-next and would merge them up after the prereq patches have been merged into mainline. Easy.
All right, what the hell just happened?
Christoph's patch series for the devmap & hmm rework finally made it into linux-next, sorry, it took quite a few iterations on the list to get all the reviews and tests, and figure out how to resolve some other conflicting things. So it just made it this week.
Recall, this is the patch series I asked you about routing a few weeks ago, as it really exceeded the small area that hmm.git was supposed to cover. I think we are both caught off guard how big the conflict is!
Quoted 9 lines
A bunch of new material has just been introduced into linux-next. I've partially unpicked the resulting mess, haven't dared trying to compile it yet. To get this far I'll need to drop two patch series and one individual patch:
mm-clean-up-is_device__page-definitions.patch
mm-introduce-arch_has_pte_devmap.patch
arm64-mm-implement-pte_devmap-support.patch
arm64-mm-implement-pte_devmap-support-fix.patchThis one we discussed, and I thought we agreed would go to your 'stage after linux-next' flow (see above). I think the conflict was minor here.
Quoted 6 lines
mm-sparsemem-introduce-struct-mem_section_usage.patch mm-sparsemem-introduce-a-section_is_early-flag.patch mm-sparsemem-add-helpers-track-active-portions-of-a-section-at-boot.patch mm-hotplug-prepare-shrink_zone-pgdat_span-for-sub-section-removal.patch mm-sparsemem-convert-kmalloc_section_memmap-to-populate_section_memmap.patch mm-hotplug-kill-is_dev_zone-usage-in-__remove_pages.patch
Quoted 8 lines
mm-kill-is_dev_zone-helper.patch mm-sparsemem-prepare-for-sub-section-ranges.patch mm-sparsemem-support-sub-section-hotplug.patch mm-document-zone_device-memory-model-implications.patch mm-document-zone_device-memory-model-implications-fix.patch mm-devm_memremap_pages-enable-sub-section-remap.patch libnvdimm-pfn-fix-fsdax-mode-namespace-info-block-zero-fields.patch libnvdimm-pfn-stop-padding-pmem-namespaces-to-section-alignment.patch
Dan pointed to this while reviewing CH's series and said the conflicts would be manageable, but they are certainly larger than I expected!
This series is the one that seems to be the really big trouble. I already checked all the other stuff that Stephen resolved, and it looks OK and managable. Just this one conflict with
kernel/memremap.cis beyond me.What approach do you want to take to go forward? Here are some thoughts:
CH has said he is away for the long weekend, so the path that involves the fewest people is if Dan respins the above on linux-next and it goes later with the arm patches above, assuming defering it for now has no other adverse effects on -mm.
Pushing CH's series to -mm would need a respin on top of Dan's series above and would need to carry along the whole hmm.git (about 44 patches). Signs are that this could be managed with the code currently in the GPU trees.
Quoted 5 lines
If we give up on CH's series the hmm.git will not have conflicts, however we just kick the can to the next merge window where we will be back to having to co-ordinate amd/nouveau/rdma git trees and -mm's patch workflow - and I think we will be worse off as we will have totally given up on a git based work flow for this. :(
I think the problem would be resolved going forward post-v5.3 since we won't have two tress managing kernel/memremap.c. This cycle however there is a backlog of kernel/memremap.c changes in -mm.
Quoted 9 lines · Dan Williams
Quoted 5 lines
If we give up on CH's series the hmm.git will not have conflicts, however we just kick the can to the next merge window where we will be back to having to co-ordinate amd/nouveau/rdma git trees and -mm's patch workflow - and I think we will be worse off as we will have totally given up on a git based work flow for this. :(
I think the problem would be resolved going forward post-v5.3 since we won't have two tress managing
kernel/memremap.c. This cycle however there is a backlog ofkernel/memremap.cchanges in -mm.
IHMO there is always something :(
CH's series had something like 5 different collisions already, and I think we did a good job of with everything but your subsection patches.
Jason