MIPS: mm: align hugetlb mappings in arch_get_unmapped_area()

__update_tlb() probes for the 8K pair at the address it is given and, for a huge pmd, writes the huge entry with tlbwi at the probed index or with tlbwr when the probe misses. That works only if the caller passes the faulting address, so that the probe finds the 4K entry the refill handler loaded for the faulting page and the huge entry replaces it.

update_mmu_cache_pmd() is not called that way. do_set_pmd() has always passed the huge-aligned address, and since commit ebcfc63d6bca ("mm:
abstract THP allocation") the anonymous THP fault path does too (map_anon_folio_pmd()). The probe then misses the stale 4K entry, which sits at the faulting page somewhere else in the 2 MB range, tlbwr adds a huge entry next to it, and the TLB holds two entries matching the faulting address. Octeon raises "Machine Check exception - caused by multiple matching entries in the TLB" on the next refill of that page; a process on a CN63XX board died this way within a second of start, on the first anonymous THP of its bss.

Skip the write when the probe misses. The refill and TLBL/TLBS handlers probe the faulting address themselves and rewrite the stale entry with the huge one (they also set the software young bit), so the entry is installed on the next access at the cost of one exception.

Fixes fd062c847a8c ("MIPS: TLB support for hugetlbfs.")

Cc [email protected]

Assisted-by Claude:claude-opus-5

Signed-off-by Orgad Shaneh <[email protected]>

---

arch/mips/mm/tlb-r4k.c +14 -4

@@ -332,6 +332,19 @@ void __update_tlb(struct vm_area_struct * vma, unsigned long address, pte_t pte)
 	/* this could be a huge page  */
 	if (pmd_leaf(*pmdp)) {
 		unsigned long lo;
+
+		/*
+		 * The probe above only covers the 8K pair at @address, and
+		 * a huge mapping is installed with the huge-aligned address
+		 * while the refill that started the fault left a 4K entry
+		 * for the faulting page elsewhere in the range. Writing the
+		 * huge entry to a random index would leave two entries
+		 * matching the faulting address; leave it to the refill and
+		 * TLBL/TLBS handlers, which probe the faulting address.
+		 */
+		if (idx < 0)
+			goto out;
+
 		write_c0_pagemask(PM_HUGE_MASK);
 		ptep = (pte_t *)pmdp;
 		lo = pte_to_entrylo(pte_val(*ptep));

@@ -339,10 +352,7 @@ void __update_tlb(struct vm_area_struct * vma, unsigned long address, pte_t pte)
 		write_c0_entrylo1(lo + (HPAGE_SIZE >> 7));
 
 		mtc0_tlbw_hazard();
-		if (idx < 0)
-			tlb_write_random();
-		else
-			tlb_write_indexed();
+		tlb_write_indexed();
 		tlbw_use_hazard();
 		write_c0_pagemask(PM_DEFAULT_MASK);
 	} else

arch/mips/mm/tlb-r4k.c +0 -0

 	htw_start();
 	flush_micro_tlb_vm(vma);
 

Signature · 1 line

2.47.0

Since hugetlb mappings were made to go through the architecture's arch_get_unmapped_area{,_topdown}(), every architecture that implements those has to align hugetlb files itself. The generic implementation and loongarch do it with huge_page_mask_align(); MIPS was left out.

A non-MAP_FIXED mmap() of a hugetlbfs file therefore returns an address that is only SHMLBA aligned, and the kernel then installs 2 MB PMDs for a VMA that starts in the middle of a PMD. The consequences on an Octeon (CN63XX) board running a process that links libhugetlbfs with HUGETLB_ELFMAP=R and HUGETLB_MORECORE=yes, all within a minute of
start: the neighbouring 4 KB page table is clobbered, the TLB ends up with overlapping entries ("Caught Machine Check exception - caused by multiple matching entries in the TLB"), process exit trips BUG_ON(start & ~huge_page_mask(h)) in __unmap_hugepage_range(), and freed pages leak into unrelated kernel structures (oopses in the irq maple tree, in pte_offset_map, ...).

Do what loongarch does in commit 3109d5ff484b ("LoongArch: Set hugetlb mmap base address aligned with pmd size"): when the file is a hugetlb file, use its page mask as the search alignment instead of the cache colour mask.

Fixes 7bd3f1e1a9ae ("mm: make hugetlb mappings go through mm_get_unmapped_area_vmflags")

Cc [email protected] # 6.13+

Assisted-by Claude:claude-opus-5

Signed-off-by Orgad Shaneh <[email protected]>

---

arch/mips/mm/mmap.c +5 -1

@@ -9,6 +9,7 @@
 #include <linux/compiler.h>
 #include <linux/elf-randomize.h>
 #include <linux/errno.h>
+#include <linux/hugetlb.h>
 #include <linux/mm.h>
 #include <linux/mman.h>
 #include <linux/export.h>

@@ -72,8 +73,11 @@ static unsigned long arch_get_unmapped_area_common(struct file *filp,
 	}
 
 	info.length = len;
-	info.align_mask = do_color_align ? (PAGE_MASK & shm_align_mask) : 0;
 	info.align_offset = pgoff << PAGE_SHIFT;
+	if (filp && is_file_hugepages(filp))
+		info.align_mask = huge_page_mask_align(filp);
+	else
+		info.align_mask = do_color_align ? (PAGE_MASK & shm_align_mask) : 0;
 
 	if (dir == DOWN) {
 		info.flags = VM_UNMAPPED_AREA_TOPDOWN;

Signature · 1 line

2.47.0