Skip to contentSkip to conversation

mm/vmalloc: cleanup after hugepage series

PATCH In review Nicholas Piggin 4 participants 15 messages Jan 26, 2021 – Jan 27, 2021

Subsystems (9)
lore
Share & tools
lore

Nicholas Piggin

author reviewer
#0 lore raw

cover mm/vmalloc: cleanup after hugepage series

Christoph pointed out some overdue cleanups required after the huge page series, and I had some other comment and warning changes.

Thanks,
Nick

Nicholas Piggin (5):
mm/vmalloc: remove map_kernel_range kernel/dma: remove unnecessary unmap_kernel_range powerpc/xive: remove unnecessary unmap_kernel_range mm/vmalloc: remove unmap_kernel_range mm/vmalloc: improve allocation failure error messages

 Documentation/core-api/cachetlb.rst |   4 +-
 arch/arm64/mm/init.c                |   2 +-
 arch/powerpc/kernel/isa-bridge.c    |   4 +-
 arch/powerpc/kernel/pci_64.c        |   2 +-
 arch/powerpc/mm/ioremap.c           |   2 +-
 arch/powerpc/sysdev/xive/common.c   |   4 -
 drivers/pci/pci.c                   |   2 +-
 include/linux/vmalloc.h             |  19 +---
 kernel/dma/remap.c                  |   1 -
 mm/internal.h                       |   7 ++
 mm/percpu-vm.c                      |   7 +-
 mm/vmalloc.c                        | 164 ++++++++++++++--------------
 12 files changed, 104 insertions(+), 114 deletions(-)
Signature · 1 line
2.23.0
author reviewer
#1 lore raw

PATCH 1/5 mm/vmalloc: remove map_kernel_range

This is a shim around vmap_pages_range, get rid of it.

Move the main API comment from the _noflush variant to the normal variant, and make _noflush internal to mm/.

---
 Documentation/core-api/cachetlb.rst |  2 +-
 include/linux/vmalloc.h             | 11 -----
 mm/internal.h                       |  6 +++
 mm/percpu-vm.c                      |  5 ++-
 mm/vmalloc.c                        | 65 +++++++++++++----------------
 5 files changed, 38 insertions(+), 51 deletions(-)
Documentation/core-api/cachetlb.rst +1 -1
index a1582cc79f0f..756f7bcf8191 100644
@@ -213,7 +213,7 @@ Here are the routines, one by one:
 	there will be no entries in the cache for the kernel address
 	space for virtual addresses in the range 'start' to 'end-1'.
 
-	The first of these two routines is invoked after map_kernel_range()
+	The first of these two routines is invoked after vmap_range()
 	has installed the page table entries.  The second is invoked
 	before unmap_kernel_range() deletes the page table entries.
 
include/linux/vmalloc.h +0 -11
index 93270adf5db5..913c9d4f5e03 100644
@@ -217,10 +217,6 @@ static inline bool is_vm_area_hugepages(const void *addr)
 int vmap_range(unsigned long addr, unsigned long end,
 			phys_addr_t phys_addr, pgprot_t prot,
 			unsigned int max_page_shift);
-extern int map_kernel_range_noflush(unsigned long start, unsigned long size,
-				    pgprot_t prot, struct page **pages);
-int map_kernel_range(unsigned long start, unsigned long size, pgprot_t prot,
-		struct page **pages);
 extern void unmap_kernel_range_noflush(unsigned long addr, unsigned long size);
 extern void unmap_kernel_range(unsigned long addr, unsigned long size);
 static inline void set_vm_flush_reset_perms(void *addr)
@@ -232,13 +228,6 @@ static inline void set_vm_flush_reset_perms(void *addr)
 }
 
 #else
-static inline int
-map_kernel_range_noflush(unsigned long start, unsigned long size,
-			pgprot_t prot, struct page **pages)
-{
-	return size >> PAGE_SHIFT;
-}
-#define map_kernel_range map_kernel_range_noflush
 static inline void
 unmap_kernel_range_noflush(unsigned long addr, unsigned long size)
 {
mm/internal.h +6 -0
index 25d2b2439f19..3ce246945882 100644
@@ -623,4 +623,10 @@ struct migration_target_control {
 	gfp_t gfp_mask;
 };
 
+/*
+ * mm/vmalloc.c
+ */
+int vmap_pages_range_noflush(unsigned long addr, unsigned long end,
+                pgprot_t prot, struct page **pages, unsigned int page_shift);
+
 #endif	/* __MM_INTERNAL_H */
mm/percpu-vm.c +3 -2
index e46f7a6917f9..88a53eb68a94 100644
@@ -8,6 +8,7 @@
  * Chunks are mapped into vmalloc areas and populated page by page.
  * This is the default chunk allocator.
  */
+#include "internal.h"
 
 static struct page *pcpu_chunk_page(struct pcpu_chunk *chunk,
 				    unsigned int cpu, int page_idx)
@@ -192,8 +193,8 @@ static void pcpu_post_unmap_tlb_flush(struct pcpu_chunk *chunk,
 static int __pcpu_map_pages(unsigned long addr, struct page **pages,
 			    int nr_pages)
 {
-	return map_kernel_range_noflush(addr, nr_pages << PAGE_SHIFT,
-					PAGE_KERNEL, pages);
+	return vmap_pages_range_noflush(addr, addr + (nr_pages << PAGE_SHIFT),
+					PAGE_KERNEL, pages, PAGE_SHIFT);
 }
 
 /**
mm/vmalloc.c +28 -37

107 lines not on this page · raw message

Signature · 1 line
2.23.0
author reviewer
#3 lore raw

PATCH 2/5 kernel/dma: remove unnecessary unmap_kernel_range

In reply to Nicholas Piggin

vunmap will remove ptes.

Cc Christoph Hellwig <[email protected]>
Cc Marek Szyprowski <[email protected]>
Cc Robin Murphy <[email protected]>
---
 kernel/dma/remap.c | 1 -
 1 file changed, 1 deletion(-)
kernel/dma/remap.c +0 -1
index 905c3fa005f1..b4526668072e 100644
@@ -66,6 +66,5 @@ void dma_common_free_remap(void *cpu_addr, size_t size)
 		return;
 	}
 
-	unmap_kernel_range((unsigned long)cpu_addr, PAGE_ALIGN(size));
 	vunmap(cpu_addr);
 }
Signature · 1 line
2.23.0
author reviewer
#8 lore raw

PATCH 3/5 powerpc/xive: remove unnecessary unmap_kernel_range

In reply to Nicholas Piggin

iounmap will remove ptes.

---
 arch/powerpc/sysdev/xive/common.c | 4 ----
 1 file changed, 4 deletions(-)
arch/powerpc/sysdev/xive/common.c +0 -4
index 595310e056f4..d6c2069cc828 100644
@@ -959,16 +959,12 @@ EXPORT_SYMBOL_GPL(is_xive_irq);
 void xive_cleanup_irq_data(struct xive_irq_data *xd)
 {
 	if (xd->eoi_mmio) {
-		unmap_kernel_range((unsigned long)xd->eoi_mmio,
-				   1u << xd->esb_shift);
 		iounmap(xd->eoi_mmio);
 		if (xd->eoi_mmio == xd->trig_mmio)
 			xd->trig_mmio = NULL;
 		xd->eoi_mmio = NULL;
 	}
 	if (xd->trig_mmio) {
-		unmap_kernel_range((unsigned long)xd->trig_mmio,
-				   1u << xd->esb_shift);
 		iounmap(xd->trig_mmio);
 		xd->trig_mmio = NULL;
 	}
Signature · 1 line
2.23.0
author reviewer
#11 lore raw

PATCH 4/5 mm/vmalloc: remove unmap_kernel_range

In reply to Nicholas Piggin

This is a shim around vunmap_range, get rid of it.

Move the main API comment from the _noflush variant to the normal variant, and make _noflush internal to mm/.

---
 Documentation/core-api/cachetlb.rst |  2 +-
 arch/arm64/mm/init.c                |  2 +-
 arch/powerpc/kernel/isa-bridge.c    |  4 +-
 arch/powerpc/kernel/pci_64.c        |  2 +-
 arch/powerpc/mm/ioremap.c           |  2 +-
 drivers/pci/pci.c                   |  2 +-
 include/linux/vmalloc.h             |  8 +---
 mm/internal.h                       |  1 +
 mm/percpu-vm.c                      |  2 +-
 mm/vmalloc.c                        | 59 ++++++++++++++---------------
 10 files changed, 38 insertions(+), 46 deletions(-)
Documentation/core-api/cachetlb.rst +1 -1
index 756f7bcf8191..fe4290e26729 100644
@@ -215,7 +215,7 @@ Here are the routines, one by one:
 
 	The first of these two routines is invoked after vmap_range()
 	has installed the page table entries.  The second is invoked
-	before unmap_kernel_range() deletes the page table entries.
+	before vunmap_range() deletes the page table entries.
 
 There exists another whole class of cpu cache issues which currently
 require a whole different set of interfaces to handle properly.
arch/arm64/mm/init.c +1 -1
index 709d98fea90c..7fe0a5074205 100644
@@ -498,7 +498,7 @@ void free_initmem(void)
 	 * prevents the region from being reused for kernel modules, which
 	 * is not supported by kallsyms.
 	 */
-	unmap_kernel_range((u64)__init_begin, (u64)(__init_end - __init_begin));
+	vunmap_range((u64)__init_begin, (u64)__init_end);
 }
 
 void dump_mem_limit(void)
arch/powerpc/kernel/isa-bridge.c +2 -2
index 2257d24e6a26..39c625737c09 100644
@@ -48,7 +48,7 @@ static void remap_isa_base(phys_addr_t pa, unsigned long size)
 	if (slab_is_available()) {
 		if (ioremap_page_range(ISA_IO_BASE, ISA_IO_BASE + size, pa,
 				pgprot_noncached(PAGE_KERNEL)))
-			unmap_kernel_range(ISA_IO_BASE, size);
+			vunmap_range(ISA_IO_BASE, ISA_IO_BASE + size);
 	} else {
 		early_ioremap_range(ISA_IO_BASE, pa, size,
 				pgprot_noncached(PAGE_KERNEL));
@@ -311,7 +311,7 @@ static void isa_bridge_remove(void)
 	isa_bridge_pcidev = NULL;
 
 	/* Unmap the ISA area */
-	unmap_kernel_range(ISA_IO_BASE, 0x10000);
+	vunmap_range(ISA_IO_BASE, ISA_IO_BASE + 0x10000);
 }
 
 /**
arch/powerpc/kernel/pci_64.c +1 -1
index 9312e6eda7ff..3fb7e572abed 100644
@@ -140,7 +140,7 @@ void __iomem *ioremap_phb(phys_addr_t paddr, unsigned long size)
 	addr = (unsigned long)area->addr;
 	if (ioremap_page_range(addr, addr + size, paddr,
 			pgprot_noncached(PAGE_KERNEL))) {
-		unmap_kernel_range(addr, size);
+		vunmap_range(addr, addr + size);
 		return NULL;
 	}
 
arch/powerpc/mm/ioremap.c +1 -1
index b1a0aebe8c48..57342154d2b0 100644
@@ -93,7 +93,7 @@ void __iomem *do_ioremap(phys_addr_t pa, phys_addr_t offset, unsigned long size,
 	if (!ret)
 		return (void __iomem *)area->addr + offset;
 
-	unmap_kernel_range(va, size);
+	vunmap_range(va, va + size);
 	free_vm_area(area);
 
 	return NULL;
drivers/pci/pci.c +1 -1
index b9fecc25d213..d1e5ee09b381 100644
@@ -4107,7 +4107,7 @@ void pci_unmap_iospace(struct resource *res)
 #if defined(PCI_IOBASE) && defined(CONFIG_MMU)
 	unsigned long vaddr = (unsigned long)PCI_IOBASE + res->start;
 
-	unmap_kernel_range(vaddr, resource_size(res));
+	vunmap_range(vaddr, vaddr + resource_size(res));
 #endif
 }
 EXPORT_SYMBOL(pci_unmap_iospace);
include/linux/vmalloc.h +1 -7
index 913c9d4f5e03..b569a13c9960 100644
@@ -217,8 +217,7 @@ static inline bool is_vm_area_hugepages(const void *addr)
 int vmap_range(unsigned long addr, unsigned long end,
 			phys_addr_t phys_addr, pgprot_t prot,
 			unsigned int max_page_shift);
-extern void unmap_kernel_range_noflush(unsigned long addr, unsigned long size);
-extern void unmap_kernel_range(unsigned long addr, unsigned long size);
+void vunmap_range(unsigned long addr, unsigned long end);
 static inline void set_vm_flush_reset_perms(void *addr)
 {
 	struct vm_struct *vm = find_vm_area(addr);
@@ -228,11 +227,6 @@ static inline void set_vm_flush_reset_perms(void *addr)
 }
 
 #else
-static inline void
-unmap_kernel_range_noflush(unsigned long addr, unsigned long size)
-{
-}
-#define unmap_kernel_range unmap_kernel_range_noflush
 static inline void set_vm_flush_reset_perms(void *addr)
 {
 }
mm/internal.h +1 -0
index 3ce246945882..3d4449f79582 100644
@@ -628,5 +628,6 @@ struct migration_target_control {
  */
 int vmap_pages_range_noflush(unsigned long addr, unsigned long end,
                 pgprot_t prot, struct page **pages, unsigned int page_shift);
+void vunmap_range_noflush(unsigned long start, unsigned long end);
 
 #endif	/* __MM_INTERNAL_H */
mm/percpu-vm.c +1 -1
index 88a53eb68a94..8d3844bc0c7c 100644
@@ -134,7 +134,7 @@ static void pcpu_pre_unmap_flush(struct pcpu_chunk *chunk,
 
 static void __pcpu_unmap_pages(unsigned long addr, int nr_pages)
 {
-	unmap_kernel_range_noflush(addr, nr_pages << PAGE_SHIFT);
+	vunmap_range_noflush(addr, addr + (nr_pages << PAGE_SHIFT));
 }
 
 /**
mm/vmalloc.c +28 -31

93 lines not on this page · raw message

Signature · 1 line
2.23.0
author reviewer
#13 lore raw

PATCH 5/5 mm/vmalloc: improve allocation failure error messages

In reply to Nicholas Piggin

There are several reasons why a vmalloc can fail, virtual space exhausted, page array allocation failure, page allocation failure, and kernel page table allocation failure.

Add distinct warning messages for the main causes of failure, with some added information like page order or allocation size where applicable.

---
 mm/vmalloc.c | 40 ++++++++++++++++++++++++++++------------
 1 file changed, 28 insertions(+), 12 deletions(-)
mm/vmalloc.c +28 -12
index 5ff190590fe4..4facf582a3be 100644
@@ -2790,6 +2790,10 @@ static void *__vmalloc_area_node(struct vm_struct *area, gfp_t gfp_mask,
 
 	if (!pages) {
 		free_vm_area(area);
+		warn_alloc(gfp_mask, NULL,
+			   "vmalloc size %lu allocation failure: "
+			   "page array size %lu allocation failed",
+			   area->nr_pages * PAGE_SIZE, array_size);
 		return NULL;
 	}
 
@@ -2813,6 +2817,10 @@ static void *__vmalloc_area_node(struct vm_struct *area, gfp_t gfp_mask,
 			/* Successfully allocated i pages, free them in __vfree() */
 			area->nr_pages = i;
 			atomic_long_add(area->nr_pages, &nr_vmalloc_pages);
+			warn_alloc(gfp_mask, NULL,
+				   "vmalloc size %lu allocation failure: "
+				   "page order %u allocation failed",
+				   area->nr_pages * PAGE_SIZE, page_order);
 			goto fail;
 		}
 
@@ -2824,15 +2832,17 @@ static void *__vmalloc_area_node(struct vm_struct *area, gfp_t gfp_mask,
 	}
 	atomic_long_add(area->nr_pages, &nr_vmalloc_pages);
 
-	if (vmap_pages_range(addr, addr + size, prot, pages, page_shift) < 0)
+	if (vmap_pages_range(addr, addr + size, prot, pages, page_shift) < 0) {
+		warn_alloc(gfp_mask, NULL,
+			   "vmalloc size %lu allocation failure: "
+			   "failed to map pages",
+			   area->nr_pages * PAGE_SIZE);
 		goto fail;
+	}
 
 	return area->addr;
 
 fail:
-	warn_alloc(gfp_mask, NULL,
-			  "vmalloc: allocation failure, allocated %ld of %ld bytes",
-			  (area->nr_pages*PAGE_SIZE), size);
 	__vfree(area->addr);
 	return NULL;
 }
@@ -2866,8 +2876,15 @@ void *__vmalloc_node_range(unsigned long size, unsigned long align,
 	unsigned long real_align = align;
 	unsigned int shift = PAGE_SHIFT;
 
-	if (!size || (size >> PAGE_SHIFT) > totalram_pages())
-		goto fail;
+	if (WARN_ON_ONCE(!size))
+		return NULL;
+
+	if ((size >> PAGE_SHIFT) > totalram_pages()) {
+		warn_alloc(gfp_mask, NULL,
+			   "vmalloc size %lu allocation failure: "
+			   "exceeds total pages", real_size);
+		return NULL;
+	}
 
 	if (vmap_allow_huge && !(vm_flags & VM_NO_HUGE_VMAP) &&
 			arch_vmap_pmd_supported(prot)) {
@@ -2894,8 +2911,12 @@ void *__vmalloc_node_range(unsigned long size, unsigned long align,
 	size = PAGE_ALIGN(size);
 	area = __get_vm_area_node(size, align, VM_ALLOC | VM_UNINITIALIZED |
 				vm_flags, start, end, node, gfp_mask, caller);
-	if (!area)
+	if (!area) {
+		warn_alloc(gfp_mask, NULL,
+			   "vmalloc size %lu allocation failure: "
+			   "vm_struct allocation failed", real_size);
 		goto fail;
+	}
 
 	addr = __vmalloc_area_node(area, gfp_mask, prot, shift, node);
 	if (!addr)
@@ -2920,11 +2941,6 @@ void *__vmalloc_node_range(unsigned long size, unsigned long align,
 		goto again;
 	}
 
-	if (!area) {
-		/* Warn for area allocation, page allocations already warn */
-		warn_alloc(gfp_mask, NULL,
-			  "vmalloc: allocation failure: %lu bytes", real_size);
-	}
 	return NULL;
 }
 
Signature · 1 line
2.23.0
Reviewed-by on v1 5/5
#14 lore raw

Re: [PATCH 5/5] mm/vmalloc: improve allocation failure error messages

In reply to Nicholas Piggin
Quoted 9 lines · Nicholas Piggin

There are several reasons why a vmalloc can fail, virtual space exhausted, page array allocation failure, page allocation failure, and kernel page table allocation failure.

Add distinct warning messages for the main causes of failure, with some added information like page order or allocation size where applicable.

Looks good,

Acked-by on v1 3/5
#10 lore raw

Re: [PATCH 3/5] powerpc/xive: remove unnecessary unmap_kernel_range

In reply to Nicholas Piggin
Quoted 5 lines · Nicholas Piggin

iounmap will remove ptes.

Looks good.

Thanks,

C.

Quoted 26 lines
---
 arch/powerpc/sysdev/xive/common.c | 4 ----
 1 file changed, 4 deletions(-)
arch/powerpc/sysdev/xive/common.c +0 -4
index 595310e056f4..d6c2069cc828 100644
@@ -959,16 +959,12 @@ EXPORT_SYMBOL_GPL(is_xive_irq);
 void xive_cleanup_irq_data(struct xive_irq_data *xd)
 {
 	if (xd->eoi_mmio) {
-		unmap_kernel_range((unsigned long)xd->eoi_mmio,
-				   1u << xd->esb_shift);
 		iounmap(xd->eoi_mmio);
 		if (xd->eoi_mmio == xd->trig_mmio)
 			xd->trig_mmio = NULL;
 		xd->eoi_mmio = NULL;
 	}
 	if (xd->trig_mmio) {
-		unmap_kernel_range((unsigned long)xd->trig_mmio,
-				   1u << xd->esb_shift);
 		iounmap(xd->trig_mmio);
 		xd->trig_mmio = NULL;
 	}
#5 lore raw

Re: [PATCH 2/5] kernel/dma: remove unnecessary unmap_kernel_range

In reply to Nicholas Piggin
Quoted 1 line · Nicholas Piggin

vunmap will remove ptes.

Should there be some ASSERT after the vunmap to make sure that is the case?

Quoted 28 lines
Cc Christoph Hellwig <[email protected]>
Cc Marek Szyprowski <[email protected]>
Cc Robin Murphy <[email protected]>
---
 kernel/dma/remap.c | 1 -
 1 file changed, 1 deletion(-)
kernel/dma/remap.c +0 -1
index 905c3fa005f1..b4526668072e 100644
@@ -66,6 +66,5 @@ void dma_common_free_remap(void *cpu_addr, size_t size)
 		return;
 	}
 
-	unmap_kernel_range((unsigned long)cpu_addr, PAGE_ALIGN(size));
 	vunmap(cpu_addr);
 }
Signature · 6 lines
2.23.0

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
#6 lore raw

Re: [PATCH 2/5] kernel/dma: remove unnecessary unmap_kernel_range

Quoted 5 lines · Konrad Rzeszutek Wilk
Quoted 1 line · Nicholas Piggin

vunmap will remove ptes.

Should there be some ASSERT after the vunmap to make sure that is the case?

Not really. removing the PTEs is the whole point of vunmap. Everything else is just house keeping.

author reviewer
#7 lore raw

Re: [PATCH 2/5] kernel/dma: remove unnecessary unmap_kernel_range

Excerpts from Christoph Hellwig's message of January 27, 2021 5:10 pm:

Quoted 9 lines · Konrad Rzeszutek Wilk
Quoted 5 lines · Nicholas Piggin
Quoted 1 line

vunmap will remove ptes.

Should there be some ASSERT after the vunmap to make sure that is the case?

Not really. removing the PTEs is the whole point of vunmap. Everything else is just house keeping.

Agree. I did double check this and wrote a quick test to check ptes were there before the vunmap and cleared after, just to make sure I didn't make a silly mistake with the patch. But in general drivers should be able to trust code behind the API call will do the right thing. Such assertions should go in the vunmap() implementation as appropriate.

Thanks,
Nick