Skip to contentSkip to conversation

thp: transparent hugepages on s390

RFC PATCH In review Gerald Schaefer 5 participants 16 messages Aug 29, 2012 – Sep 5, 2012

Subsystems (7)
lore
Share & tools
lore
#0 lore raw

v2 cover thp: transparent hugepages on s390

This patch series adds support for transparent hugepages on System z. Small changes to common code are necessary with regard to a different pgtable_t, tlb flushing and kvm behaviour on s390, see patches 1 to 3.

Changes to v2:
[PATCH 1/7] Fix build problem on non-s390 architecture. [PATCH 2/7] Fix build problem on non-s390 architecture. [PATCH 7/7] Make use of rrbm instruction if available.

Gerald Schaefer (7):
thp: remove assumptions on pgtable_t type
thp: introduce pmdp_invalidate()
thp: make MADV_HUGEPAGE check for mm->def_flags thp, s390: thp splitting backend for s390 thp, s390: thp pagetable pre-allocation for s390 thp, s390: disable thp for kvm host on s390 thp, s390: architecture backend for thp on s390

 arch/s390/include/asm/hugetlb.h |  18 +---
 arch/s390/include/asm/pgtable.h | 208 ++++++++++++++++++++++++++++++++++++++++
 arch/s390/include/asm/setup.h   |   3 +
 arch/s390/include/asm/tlb.h     |   1 +
 arch/s390/kernel/early.c        |   2 +
 arch/s390/mm/gup.c              |  11 ++-
 arch/s390/mm/pgtable.c          | 108 +++++++++++++++++++++
 include/asm-generic/pgtable.h   |  13 +++
 include/linux/huge_mm.h         |   1 -
 mm/Kconfig                      |   2 +-
 mm/huge_memory.c                |  53 +++-------
 mm/pgtable-generic.c            |  50 ++++++++++
 12 files changed, 408 insertions(+), 62 deletions(-)
Signature · 1 line
1.7.11.5
#1 lore raw

RFC PATCH v2 1/7 thp: remove assumptions on pgtable_t type

The thp page table pre-allocation code currently assumes that pgtable_t is of type "struct page *". This may not be true for all architectures, so this patch removes that assumption by replacing the functions prepare_pmd_huge_pte() and get_pmd_huge_pte() with two new functions that can be defined architecture-specific.

It also removes two VM_BUG_ON checks for page_count() and page_mapcount() operating on a pgtable_t. Apart from the VM_BUG_ON removal, there will be no functional change introduced by this patch.

---
 include/asm-generic/pgtable.h |  8 ++++++++
 include/linux/huge_mm.h       |  1 -
 mm/huge_memory.c              | 46 ++++++-------------------------------------
 mm/pgtable-generic.c          | 39 ++++++++++++++++++++++++++++++++++++
 4 files changed, 53 insertions(+), 41 deletions(-)
include/asm-generic/pgtable.h +8 -0
index ff4947b..f756f60 100644
@@ -162,6 +162,14 @@ extern void pmdp_splitting_flush(struct vm_area_struct *vma,
 				 unsigned long address, pmd_t *pmdp);
 #endif
 
+#ifndef __HAVE_ARCH_PGTABLE_DEPOSIT
+extern void pgtable_deposit(struct mm_struct *mm, pgtable_t pgtable);
+#endif
+
+#ifndef __HAVE_ARCH_PGTABLE_WITHDRAW
+extern pgtable_t pgtable_withdraw(struct mm_struct *mm);
+#endif
+
 #ifndef __HAVE_ARCH_PTE_SAME
 static inline int pte_same(pte_t pte_a, pte_t pte_b)
 {
include/linux/huge_mm.h +0 -1
index 4c59b11..6ab47af 100644
@@ -11,7 +11,6 @@ extern int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm,
 extern int do_huge_pmd_wp_page(struct mm_struct *mm, struct vm_area_struct *vma,
 			       unsigned long address, pmd_t *pmd,
 			       pmd_t orig_pmd);
-extern pgtable_t get_pmd_huge_pte(struct mm_struct *mm);
 extern struct page *follow_trans_huge_pmd(struct mm_struct *mm,
 					  unsigned long addr,
 					  pmd_t *pmd,
mm/huge_memory.c +6 -40

100 lines not on this page · raw message

mm/pgtable-generic.c +39 -0
index 74c0dda..308f1fb 100644
@@ -120,3 +120,42 @@ void pmdp_splitting_flush(struct vm_area_struct *vma, unsigned long address,
 }
 #endif /* CONFIG_TRANSPARENT_HUGEPAGE */
 #endif
+
+#ifndef __HAVE_ARCH_PGTABLE_DEPOSIT
+#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+void pgtable_deposit(struct mm_struct *mm, pgtable_t pgtable)
+{
+	assert_spin_locked(&mm->page_table_lock);
+
+	/* FIFO */
+	if (!mm->pmd_huge_pte)
+		INIT_LIST_HEAD(&pgtable->lru);
+	else
+		list_add(&pgtable->lru, &mm->pmd_huge_pte->lru);
+	mm->pmd_huge_pte = pgtable;
+}
+#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
+#endif
+
+#ifndef __HAVE_ARCH_PGTABLE_WITHDRAW
+#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+/* no "address" argument so destroys page coloring of some arch */
+pgtable_t pgtable_withdraw(struct mm_struct *mm)
+{
+	pgtable_t pgtable;
+
+	assert_spin_locked(&mm->page_table_lock);
+
+	/* FIFO */
+	pgtable = mm->pmd_huge_pte;
+	if (list_empty(&pgtable->lru))
+		mm->pmd_huge_pte = NULL;
+	else {
+		mm->pmd_huge_pte = list_entry(pgtable->lru.next,
+					      struct page, lru);
+		list_del(&pgtable->lru);
+	}
+	return pgtable;
+}
+#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
+#endif
Signature · 1 line
1.7.11.5
#2 lore raw

Re: [RFC v2 PATCH 1/7] thp: remove assumptions on pgtable_t type

Quoted 9 lines · Gerald Schaefer

The thp page table pre-allocation code currently assumes that pgtable_t is of type "struct page *". This may not be true for all architectures, so this patch removes that assumption by replacing the functions prepare_pmd_huge_pte() and get_pmd_huge_pte() with two new functions that can be defined architecture-specific.

It also removes two VM_BUG_ON checks for page_count() and page_mapcount() operating on a pgtable_t. Apart from the VM_BUG_ON removal, there will be no functional change introduced by this patch.

Why is that VM_BUG_ON not needed any more ? What is that changed which break that requirement ?

-aneesh

#3 lore raw

Re: [RFC v2 PATCH 1/7] thp: remove assumptions on pgtable_t type

Quoted 8 lines · On Fri, 31 Aug 2012 10:59:38 +0530 "Aneesh Kumar K.V"…
Quoted 6 lines · Gerald Schaefer

The thp page table pre-allocation code currently assumes that pgtable_t is of type "struct page *". This may not be true for all architectures, so this patch removes that assumption by replacing the functions prepare_pmd_huge_pte() and get_pmd_huge_pte() with two new functions that can be defined architecture-specific.

Quoted 6 lines
Quoted 3 lines

It also removes two VM_BUG_ON checks for page_count() and page_mapcount() operating on a pgtable_t. Apart from the VM_BUG_ON removal, there will be no functional change introduced by this patch.

Why is that VM_BUG_ON not needed any more ? What is that changed which break that requirement ?

Because pgtable_t for s390 is not a page and there simply is no page_count or page_mapcount.

Signature · 4 lines
blue skies,
   Martin.

"Reality continues to ruin my life." - Calvin.
#4 lore raw

Re: [RFC v2 PATCH 1/7] thp: remove assumptions on pgtable_t type

In reply to Gerald Schaefer

Hi Gerald,

include/asm-generic/pgtable.h extern void pmdp_splitting_flush(struct vm_area_struct *vma, · line 167

Quoted 3 lines · Gerald Schaefer
include/asm-generic/pgtable.h +3 -0
+#ifndef __HAVE_ARCH_PGTABLE_DEPOSIT
+extern void pgtable_deposit(struct mm_struct *mm, pgtable_t pgtable);
+#endif

One minor nitpick on the naming of the two functions: considering that those are global exports, that they can only be used by THP and they mangle internally with the mm->pmd_huge_pte field (also only defined if TRANSPARENT_HUGEPAGE=y), a more specific THP naming may be clearer.

If the current prepare_pmd_huge_pte/get_pmd_huge_pte wasn't good other options could be pgtable_trans_huge_deposit/withdraw.

#5 lore raw

Re: [RFC v2 PATCH 1/7] thp: remove assumptions on pgtable_t type

Quoted 12 lines · On Wed, 5 Sep 2012 00:56:33 +0200 Andrea Arcangeli <[email protected]> wrote:

Hi Gerald,

include/asm-generic/pgtable.h extern void pmdp_splitting_flush(struct vm_area_struct *vma, · line 167

Quoted 3 lines · Gerald Schaefer
include/asm-generic/pgtable.h +3 -0
+#ifndef __HAVE_ARCH_PGTABLE_DEPOSIT
+extern void pgtable_deposit(struct mm_struct *mm, pgtable_t pgtable);
+#endif

One minor nitpick on the naming of the two functions: considering that those are global exports, that they can only be used by THP and they mangle internally with the mm->pmd_huge_pte field (also only defined if TRANSPARENT_HUGEPAGE=y), a more specific THP naming may be clearer.

Quoted 3 lines

If the current prepare_pmd_huge_pte/get_pmd_huge_pte wasn't good other options could be pgtable_trans_huge_deposit/withdraw.

Thanks for your feedback. I'll send a new patch series with those functions renamed, as it has impact on 3 of the 7 patches, and also including the Kconfig change using HAVE_ARCH_TRANSPARENT_HUGEPAGE.

#6 lore raw

RFC PATCH v2 2/7 thp: introduce pmdp_invalidate()

In reply to Gerald Schaefer

On s390, a valid page table entry must not be changed while it is attached to any CPU. So instead of pmd_mknotpresent() and set_pmd_at(), an IDTE operation would be necessary there. This patch introduces the pmdp_invalidate() function, to allow architecture-specific implementations.

---
 include/asm-generic/pgtable.h |  5 +++++
 mm/huge_memory.c              |  3 +--
 mm/pgtable-generic.c          | 11 +++++++++++
 3 files changed, 17 insertions(+), 2 deletions(-)
include/asm-generic/pgtable.h +5 -0
index f756f60..47519ef 100644
@@ -170,6 +170,11 @@ extern void pgtable_deposit(struct mm_struct *mm, pgtable_t pgtable);
 extern pgtable_t pgtable_withdraw(struct mm_struct *mm);
 #endif
 
+#ifndef __HAVE_ARCH_PMDP_INVALIDATE
+extern void pmdp_invalidate(struct vm_area_struct *vma, unsigned long address,
+			    pmd_t *pmdp);
+#endif
+
 #ifndef __HAVE_ARCH_PTE_SAME
 static inline int pte_same(pte_t pte_a, pte_t pte_b)
 {
mm/huge_memory.c +1 -2
index 6805328..8bac516 100644
@@ -1374,8 +1374,7 @@ static int __split_huge_page_map(struct page *page,
 		 * SMP TLB and finally we write the non-huge version
 		 * of the pmd entry with pmd_populate.
 		 */
-		set_pmd_at(mm, address, pmd, pmd_mknotpresent(*pmd));
-		flush_tlb_range(vma, address, address + HPAGE_PMD_SIZE);
+		pmdp_invalidate(vma, address, pmd);
 		pmd_populate(mm, pmd, pgtable);
 		ret = 1;
 	}
mm/pgtable-generic.c +11 -0
index 308f1fb..ca01b4b 100644
@@ -159,3 +159,14 @@ pgtable_t pgtable_withdraw(struct mm_struct *mm)
 }
 #endif /* CONFIG_TRANSPARENT_HUGEPAGE */
 #endif
+
+#ifndef __HAVE_ARCH_PMDP_INVALIDATE
+#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+void pmdp_invalidate(struct vm_area_struct *vma, unsigned long address,
+		     pmd_t *pmdp)
+{
+	set_pmd_at(vma->vm_mm, address, pmdp, pmd_mknotpresent(*pmdp));
+	flush_tlb_range(vma, address, address + HPAGE_PMD_SIZE);
+}
+#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
+#endif
Signature · 1 line
1.7.11.5
#7 lore raw

RFC PATCH v2 3/7 thp: make MADV_HUGEPAGE check for mm->def_flags

In reply to Gerald Schaefer

This adds a check to hugepage_madvise(), to refuse MADV_HUGEPAGE if VM_NOHUGEPAGE is set in mm->def_flags. On s390, the VM_NOHUGEPAGE flag will be set in mm->def_flags for kvm processes, to prevent any future thp mappings. In order to also prevent MADV_HUGEPAGE on such an mm, hugepage_madvise() should check mm->def_flags.

---
 mm/huge_memory.c | 4 ++++
 1 file changed, 4 insertions(+)
mm/huge_memory.c +4 -0
index 8bac516..f39cb03 100644
@@ -1464,6 +1464,8 @@ out:
 int hugepage_madvise(struct vm_area_struct *vma,
 		     unsigned long *vm_flags, int advice)
 {
+	struct mm_struct *mm = vma->vm_mm;
+
 	switch (advice) {
 	case MADV_HUGEPAGE:
 		/*
@@ -1471,6 +1473,8 @@ int hugepage_madvise(struct vm_area_struct *vma,
 		 */
 		if (*vm_flags & (VM_HUGEPAGE | VM_NO_THP))
 			return -EINVAL;
+		if (mm->def_flags & VM_NOHUGEPAGE)
+			return -EINVAL;
 		*vm_flags &= ~VM_NOHUGEPAGE;
 		*vm_flags |= VM_HUGEPAGE;
 		/*
Signature · 1 line
1.7.11.5
#8 lore raw

RFC PATCH v2 4/7 thp, s390: thp splitting backend for s390

In reply to Gerald Schaefer

This patch is part of the architecture backend for thp on s390. It provides the functions related to thp splitting, including serialization against gup. Unlike other archs, pmdp_splitting_flush() cannot use a tlb flushing operation to serialize against gup on s390, because that wouldn't be stopped by the disabled IRQs. So instead, smp_call_function() is called with an empty function, which will have the expected effect.

---
 arch/s390/include/asm/pgtable.h | 13 +++++++++++++
 arch/s390/mm/gup.c              | 11 ++++++++++-
 arch/s390/mm/pgtable.c          | 18 ++++++++++++++++++
 3 files changed, 41 insertions(+), 1 deletion(-)
arch/s390/include/asm/pgtable.h +13 -0
index 6bd7d74..c3f0775 100644
@@ -347,6 +347,8 @@ extern struct page *vmemmap;
 
 #define _SEGMENT_ENTRY_LARGE	0x400	/* STE-format control, large page   */
 #define _SEGMENT_ENTRY_CO	0x100	/* change-recording override   */
+#define _SEGMENT_ENTRY_SPLIT_BIT 0	/* THP splitting bit number */
+#define _SEGMENT_ENTRY_SPLIT	(1UL << _SEGMENT_ENTRY_SPLIT_BIT)
 
 /* Page status table bits for virtualization */
 #define RCP_ACC_BITS	0xf000000000000000UL
@@ -506,6 +508,10 @@ static inline int pmd_bad(pmd_t pmd)
 	return (pmd_val(pmd) & mask) != _SEGMENT_ENTRY;
 }
 
+#define __HAVE_ARCH_PMDP_SPLITTING_FLUSH
+extern void pmdp_splitting_flush(struct vm_area_struct *vma,
+				 unsigned long addr, pmd_t *pmdp);
+
 static inline int pte_none(pte_t pte)
 {
 	return (pte_val(pte) & _PAGE_INVALID) && !(pte_val(pte) & _PAGE_SWT);
@@ -1159,6 +1165,13 @@ static inline pmd_t *pmd_offset(pud_t *pud, unsigned long address)
 #define pte_offset_map(pmd, address) pte_offset_kernel(pmd, address)
 #define pte_unmap(pte) do { } while (0)
 
+#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+static inline int pmd_trans_splitting(pmd_t pmd)
+{
+	return pmd_val(pmd) & _SEGMENT_ENTRY_SPLIT;
+}
+#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
+
 /*
  * 31 bit swap entry format:
  * A page-table entry has some bits we have to treat in a special way.
arch/s390/mm/gup.c +10 -1
index 65cb06e..d34fd6e 100644
@@ -115,7 +115,16 @@ static inline int gup_pmd_range(pud_t *pudp, pud_t pud, unsigned long addr,
 		pmd = *pmdp;
 		barrier();
 		next = pmd_addr_end(addr, end);
-		if (pmd_none(pmd))
+		/*
+		 * The pmd_trans_splitting() check below explains why
+		 * pmdp_splitting_flush() has to serialize with
+		 * smp_call_function() against our disabled IRQs, to stop
+		 * this gup-fast code from running while we set the
+		 * splitting bit in the pmd. Returning zero will take
+		 * the slow path that will call wait_split_huge_page()
+		 * if the pmd is still in splitting state.
+		 */
+		if (pmd_none(pmd) || pmd_trans_splitting(pmd))
 			return 0;
 		if (unlikely(pmd_huge(pmd))) {
 			if (!gup_huge_pmd(pmdp, pmd, addr, next,
arch/s390/mm/pgtable.c +18 -0
index 18df31d..5a73522 100644
@@ -866,3 +866,21 @@ bool kernel_page_present(struct page *page)
 	return cc == 0;
 }
 #endif /* CONFIG_HIBERNATION && CONFIG_DEBUG_PAGEALLOC */
+
+#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+static void pmdp_splitting_flush_sync(void *arg)
+{
+	/* Simply deliver the interrupt */
+}
+
+void pmdp_splitting_flush(struct vm_area_struct *vma, unsigned long address,
+			  pmd_t *pmdp)
+{
+	VM_BUG_ON(address & ~HPAGE_PMD_MASK);
+	if (!test_and_set_bit(_SEGMENT_ENTRY_SPLIT_BIT,
+			      (unsigned long *) pmdp)) {
+		/* need to serialize against gup-fast (IRQ disabled) */
+		smp_call_function(pmdp_splitting_flush_sync, NULL, 1);
+	}
+}
+#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
Signature · 1 line
1.7.11.5
#9 lore raw

RFC PATCH v2 5/7 thp, s390: thp pagetable pre-allocation for s390

In reply to Gerald Schaefer

This patch is part of the architecture backend for thp on s390. It provides the pagetable pre-allocation functions pgtable_deposit() and pgtable_withdraw(). Unlike other archs, s390 has no struct page * as pgtable_t, but rather a pointer to the page table. So instead of saving the pagetable pre-allocation list info inside the struct page, it is being saved within the pagetable itself.

---
 arch/s390/include/asm/pgtable.h |  6 ++++++
 arch/s390/mm/pgtable.c          | 38 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 44 insertions(+)
arch/s390/include/asm/pgtable.h +6 -0
index c3f0775..353590c 100644
@@ -1166,6 +1166,12 @@ static inline pmd_t *pmd_offset(pud_t *pud, unsigned long address)
 #define pte_unmap(pte) do { } while (0)
 
 #ifdef CONFIG_TRANSPARENT_HUGEPAGE
+#define __HAVE_ARCH_PGTABLE_DEPOSIT
+extern void pgtable_deposit(struct mm_struct *mm, pgtable_t pgtable);
+
+#define __HAVE_ARCH_PGTABLE_WITHDRAW
+extern pgtable_t pgtable_withdraw(struct mm_struct *mm);
+
 static inline int pmd_trans_splitting(pmd_t pmd)
 {
 	return pmd_val(pmd) & _SEGMENT_ENTRY_SPLIT;
arch/s390/mm/pgtable.c +38 -0
index 5a73522..3168fbd 100644
@@ -883,4 +883,42 @@ void pmdp_splitting_flush(struct vm_area_struct *vma, unsigned long address,
 		smp_call_function(pmdp_splitting_flush_sync, NULL, 1);
 	}
 }
+
+void pgtable_deposit(struct mm_struct *mm, pgtable_t pgtable)
+{
+	struct list_head *lh = (struct list_head *) pgtable;
+
+	assert_spin_locked(&mm->page_table_lock);
+
+	/* FIFO */
+	if (!mm->pmd_huge_pte)
+		INIT_LIST_HEAD(lh);
+	else
+		list_add(lh, (struct list_head *) mm->pmd_huge_pte);
+	mm->pmd_huge_pte = pgtable;
+}
+
+pgtable_t pgtable_withdraw(struct mm_struct *mm)
+{
+	struct list_head *lh;
+	pgtable_t pgtable;
+	pte_t *ptep;
+
+	assert_spin_locked(&mm->page_table_lock);
+
+	/* FIFO */
+	pgtable = mm->pmd_huge_pte;
+	lh = (struct list_head *) pgtable;
+	if (list_empty(lh))
+		mm->pmd_huge_pte = NULL;
+	else {
+		mm->pmd_huge_pte = (pgtable_t) lh->next;
+		list_del(lh);
+	}
+	ptep = (pte_t *) pgtable;
+	pte_val(*ptep) = _PAGE_TYPE_EMPTY;
+	ptep++;
+	pte_val(*ptep) = _PAGE_TYPE_EMPTY;
+	return pgtable;
+}
 #endif /* CONFIG_TRANSPARENT_HUGEPAGE */
Signature · 1 line
1.7.11.5
#10 lore raw

RFC PATCH v2 6/7 thp, s390: disable thp for kvm host on s390

In reply to Gerald Schaefer

This patch is part of the architecture backend for thp on s390. It disables thp for kvm hosts, because there is no kvm host hugepage support so far. Existing thp mappings are split by follow_page() with FOLL_SPLIT, and future thp mappings are prevented by setting VM_NOHUGEPAGE in mm->def_flags.

---
 arch/s390/mm/pgtable.c | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)
arch/s390/mm/pgtable.c +30 -0
index 3168fbd..7846072 100644
@@ -787,6 +787,30 @@ void tlb_remove_table(struct mmu_gather *tlb, void *table)
 		tlb_table_flush(tlb);
 }
 
+#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+void thp_split_vma(struct vm_area_struct *vma)
+{
+	unsigned long addr;
+	struct page *page;
+
+	for (addr = vma->vm_start; addr < vma->vm_end; addr += PAGE_SIZE) {
+		page = follow_page(vma, addr, FOLL_SPLIT);
+	}
+}
+
+void thp_split_mm(struct mm_struct *mm)
+{
+	struct vm_area_struct *vma = mm->mmap;
+
+	while (vma != NULL) {
+		thp_split_vma(vma);
+		vma->vm_flags &= ~VM_HUGEPAGE;
+		vma->vm_flags |= VM_NOHUGEPAGE;
+		vma = vma->vm_next;
+	}
+}
+#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
+
 /*
  * switch on pgstes for its userspace process (for kvm)
  */
@@ -824,6 +848,12 @@ int s390_enable_sie(void)
 	if (!mm)
 		return -ENOMEM;
 
+#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+	/* split thp mappings and disable thp for future mappings */
+	thp_split_mm(mm);
+	mm->def_flags |= VM_NOHUGEPAGE;
+#endif
+
 	/* Now lets check again if something happened */
 	task_lock(tsk);
 	if (!tsk->mm || atomic_read(&tsk->mm->mm_users) > 1 ||
Signature · 1 line
1.7.11.5
#11 lore raw

RFC PATCH v2 7/7 thp, s390: architecture backend for thp on s390

In reply to Gerald Schaefer

This implements the architecture backend for transparent hugepages on s390.

---
 arch/s390/include/asm/hugetlb.h |  18 +---
 arch/s390/include/asm/pgtable.h | 189 ++++++++++++++++++++++++++++++++++++++++
 arch/s390/include/asm/setup.h   |   3 +
 arch/s390/include/asm/tlb.h     |   1 +
 arch/s390/kernel/early.c        |   2 +
 arch/s390/mm/pgtable.c          |  22 +++++
 mm/Kconfig                      |   2 +-
 7 files changed, 219 insertions(+), 18 deletions(-)
arch/s390/include/asm/hugetlb.h +1 -17
index 799ed0f..a840662 100644
@@ -87,23 +87,6 @@ static inline void __pmd_csp(pmd_t *pmdp)
 		"	csp %1,%3"
 		: "=m" (*pmdp)
 		: "d" (reg2), "d" (reg3), "d" (reg4), "m" (*pmdp) : "cc");
-	pmd_val(*pmdp) = _SEGMENT_ENTRY_INV | _SEGMENT_ENTRY;
-}
-
-static inline void __pmd_idte(unsigned long address, pmd_t *pmdp)
-{
-	unsigned long sto = (unsigned long) pmdp -
-				pmd_index(address) * sizeof(pmd_t);
-
-	if (!(pmd_val(*pmdp) & _SEGMENT_ENTRY_INV)) {
-		asm volatile(
-			"	.insn	rrf,0xb98e0000,%2,%3,0,0"
-			: "=m" (*pmdp)
-			: "m" (*pmdp), "a" (sto),
-			  "a" ((address & HPAGE_MASK))
-		);
-	}
-	pmd_val(*pmdp) = _SEGMENT_ENTRY_INV | _SEGMENT_ENTRY;
 }
 
 static inline void huge_ptep_invalidate(struct mm_struct *mm,
@@ -115,6 +98,7 @@ static inline void huge_ptep_invalidate(struct mm_struct *mm,
 		__pmd_idte(address, pmdp);
 	else
 		__pmd_csp(pmdp);
+	pmd_val(*pmdp) = _SEGMENT_ENTRY_INV | _SEGMENT_ENTRY;
 }
 
 #define huge_ptep_set_access_flags(__vma, __addr, __ptep, __entry, __dirty) \
arch/s390/include/asm/pgtable.h +189 -0

213 lines not on this page · raw message

arch/s390/include/asm/setup.h +3 -0
index e6859d1..13b545f 100644
@@ -80,6 +80,7 @@ extern unsigned int addressing_mode;
 #define MACHINE_FLAG_LPAR	(1UL << 12)
 #define MACHINE_FLAG_SPP	(1UL << 13)
 #define MACHINE_FLAG_TOPOLOGY	(1UL << 14)
+#define MACHINE_FLAG_RRBM	(1UL << 16)
 
 #define MACHINE_IS_VM		(S390_lowcore.machine_flags & MACHINE_FLAG_VM)
 #define MACHINE_IS_KVM		(S390_lowcore.machine_flags & MACHINE_FLAG_KVM)
@@ -98,6 +99,7 @@ extern unsigned int addressing_mode;
 #define MACHINE_HAS_PFMF	(0)
 #define MACHINE_HAS_SPP		(0)
 #define MACHINE_HAS_TOPOLOGY	(0)
+#define MACHINE_HAS_RRBM	(0)
 #else /* CONFIG_64BIT */
 #define MACHINE_HAS_IEEE	(1)
 #define MACHINE_HAS_CSP		(1)
@@ -109,6 +111,7 @@ extern unsigned int addressing_mode;
 #define MACHINE_HAS_PFMF	(S390_lowcore.machine_flags & MACHINE_FLAG_PFMF)
 #define MACHINE_HAS_SPP		(S390_lowcore.machine_flags & MACHINE_FLAG_SPP)
 #define MACHINE_HAS_TOPOLOGY	(S390_lowcore.machine_flags & MACHINE_FLAG_TOPOLOGY)
+#define MACHINE_HAS_RRBM	(S390_lowcore.machine_flags & MACHINE_FLAG_RRBM)
 #endif /* CONFIG_64BIT */
 
 #define ZFCPDUMP_HSA_SIZE	(32UL<<20)
arch/s390/include/asm/tlb.h +1 -0
index 06e5acb..b75d7d6 100644
@@ -137,6 +137,7 @@ static inline void pud_free_tlb(struct mmu_gather *tlb, pud_t *pud,
 #define tlb_start_vma(tlb, vma)			do { } while (0)
 #define tlb_end_vma(tlb, vma)			do { } while (0)
 #define tlb_remove_tlb_entry(tlb, ptep, addr)	do { } while (0)
+#define tlb_remove_pmd_tlb_entry(tlb, pmdp, addr)	do { } while (0)
 #define tlb_migrate_finish(mm)			do { } while (0)
 
 #endif /* _S390_TLB_H */
arch/s390/kernel/early.c +2 -0
index 83c3271..fb03cce 100644
@@ -372,6 +372,8 @@ static __init void detect_machine_facilities(void)
 		S390_lowcore.machine_flags |= MACHINE_FLAG_MVCOS;
 	if (test_facility(40))
 		S390_lowcore.machine_flags |= MACHINE_FLAG_SPP;
+	if (test_facility(66))
+		S390_lowcore.machine_flags |= MACHINE_FLAG_RRBM;
 #endif
 }
 
arch/s390/mm/pgtable.c +22 -0
index 7846072..216254b 100644
@@ -898,6 +898,28 @@ bool kernel_page_present(struct page *page)
 #endif /* CONFIG_HIBERNATION && CONFIG_DEBUG_PAGEALLOC */
 
 #ifdef CONFIG_TRANSPARENT_HUGEPAGE
+int pmdp_clear_flush_young(struct vm_area_struct *vma, unsigned long address,
+			   pmd_t *pmdp)
+{
+	VM_BUG_ON(address & ~HPAGE_PMD_MASK);
+	/* No need to flush TLB
+	 * On s390 reference bits are in storage key and never in TLB */
+	return pmdp_test_and_clear_young(vma, address, pmdp);
+}
+
+int pmdp_set_access_flags(struct vm_area_struct *vma,
+			  unsigned long address, pmd_t *pmdp,
+			  pmd_t entry, int dirty)
+{
+	VM_BUG_ON(address & ~HPAGE_PMD_MASK);
+
+	if (pmd_same(*pmdp, entry))
+		return 0;
+	pmdp_invalidate(vma, address, pmdp);
+	set_pmd_at(vma->vm_mm, address, pmdp, entry);
+	return 1;
+}
+
 static void pmdp_splitting_flush_sync(void *arg)
 {
 	/* Simply deliver the interrupt */
mm/Kconfig +1 -1
index d5c8019..7d51edb 100644
@@ -318,7 +318,7 @@ config NOMMU_INITIAL_TRIM_EXCESS
 
 config TRANSPARENT_HUGEPAGE
 	bool "Transparent Hugepage Support"
-	depends on X86 && MMU
+	depends on (X86 || (S390 && 64BIT)) && MMU
 	select COMPACTION
 	help
 	  Transparent Hugepages allows the kernel to use huge pages and
Signature · 1 line
1.7.11.5
maintainer
#12 lore raw
In reply to Gerald Schaefer
Quoted 3 lines · On Wed, 29 Aug 2012 17:32:57 +0200 Gerald Schaefer…

This patch series adds support for transparent hugepages on System z. Small changes to common code are necessary with regard to a different pgtable_t, tlb flushing and kvm behaviour on s390, see patches 1 to 3.

"RFC" always worries me. I read it as "Really Flakey Code" ;) Is it still appropriate to this patchset?

I grabbed them all. Patches 1-3 look sane to me and I cheerfully didn't read the s390 changes at all. Hopefully Andrea will be able to review at least patches 1-3 for us.

If that all goes well, how do we play this? I'd prefer to merge 1-3 myself, as they do interact with ongoing MM development. I can also merge 4-7 if appropriate s390 maintainer acks are seen. Or I can drop them and the s390 parts can be merged via the s390 tree at a later date?

#13 lore raw
Quoted 9 lines · On Thu, 30 Aug 2012 12:54:44 -0700 Andrew Morton <[email protected]>…
Quoted 3 lines · On Wed, 29 Aug 2012 17:32:57 +0200 Gerald Schaefer…

This patch series adds support for transparent hugepages on System z. Small changes to common code are necessary with regard to a different pgtable_t, tlb flushing and kvm behaviour on s390, see patches 1 to 3.

"RFC" always worries me. I read it as "Really Flakey Code" ;) Is it still appropriate to this patchset?

The code quality is IMHO already good. We do change common mm code though and we want to get some feedback on that.

Quoted 9 lines

I grabbed them all. Patches 1-3 look sane to me and I cheerfully didn't read the s390 changes at all. Hopefully Andrea will be able to review at least patches 1-3 for us.

If that all goes well, how do we play this? I'd prefer to merge 1-3 myself, as they do interact with ongoing MM development. I can also merge 4-7 if appropriate s390 maintainer acks are seen. Or I can drop them and the s390 parts can be merged via the s390 tree at a later date?

I would really appreciate if Andrea could have a look at the code. I've read the patches and I am fine with them but it is very easy to miss some important bit.

As far as upstreaming is concerned: I can deal with the pure s390 parts via the s390 tree if that helps you. If you prefer the carry all of them, that is fine with me as well.

Signature · 4 lines
blue skies,
   Martin.

"Reality continues to ruin my life." - Calvin.
maintainer
#14 lore raw
Quoted 11 lines · On Fri, 31 Aug 2012 09:07:57 +0200 Martin Schwidefsky <[email protected]>…
Quoted 9 lines

I grabbed them all. Patches 1-3 look sane to me and I cheerfully didn't read the s390 changes at all. Hopefully Andrea will be able to review at least patches 1-3 for us.

If that all goes well, how do we play this? I'd prefer to merge 1-3 myself, as they do interact with ongoing MM development. I can also merge 4-7 if appropriate s390 maintainer acks are seen. Or I can drop them and the s390 parts can be merged via the s390 tree at a later date?

I would really appreciate if Andrea could have a look at the code.

Yes please ;)

Quoted 7 lines

I've
read the patches and I am fine with them but it is very easy to miss some important bit.

As far as upstreaming is concerned: I can deal with the pure s390 parts via the s390 tree if that helps you. If you prefer the carry all of them, that is fine with me as well.

It would be easiest/simplest if I were to merge it all. But that means that the s390 developers wouldn't have tested it much, unless they're testing linux-next?

I guess that's not the end of the world - you'll have a couple of months after -rc1 to find and fix any problems.

#15 lore raw

Hi Andrew and Martin,

Quoted 13 lines · Andrew Morton
Quoted 10 lines · On Fri, 31 Aug 2012 09:07:57 +0200 Martin Schwidefsky <[email protected]>…
Quoted 9 lines

I grabbed them all. Patches 1-3 look sane to me and I cheerfully didn't read the s390 changes at all. Hopefully Andrea will be able to review at least patches 1-3 for us.

If that all goes well, how do we play this? I'd prefer to merge 1-3 myself, as they do interact with ongoing MM development. I can also merge 4-7 if appropriate s390 maintainer acks are seen. Or I can drop them and the s390 parts can be merged via the s390 tree at a later date?

Quoted 3 lines
Quoted 1 line

I would really appreciate if Andrea could have a look at the code.

Yes please ;)

:) Sorry for the delay! I was at the Plumbers until last Friday and I'm catching up with the emails.

I finished reviewing them and they look perfectly fine. In fact it was a pleasure to read them. Patch 1 and 2 are noops in x86 context, patch 3 is fine as well.

The s390 solution to send a dummy IPIs to wait gup-fast to exit the critical section after setting the splitting bit looks fine as well.

The only additional code cleanup suggestion I can give is to also add a HAVE_ARCH_TRANSPARENT_HUGEPAGE on the same lines of:

http://git.kernel.org/?p=linux/kernel/git/andrea/aa.git;a=commitdiff;h=36c2d096cce61fcfc1726ea1c1fd6dd65ddf8b08;hp=b2a86603629e2401d85f9ed1acbcf6c0043b2b23

as patch 8. In order to convert the below:

+ depends on (X86 || (S390 && 64BIT)) && MMU

to:

depends on HAVE_ARCH_TRANSPARENT_HUGEPAGE

The MMU check looks already superflous, even more so after adding the HAVE_ARCH_TRANSPARENT_HUGEPAGE (if something it'll be up to the arch code to add a dependency on it before defining HAVE_ARCH_).

Thanks,
Andrea