MIPS: Add hugeTLBfs support.
cover MIPS: Add hugeTLBfs support.
This patch set adds hugetlbfs support to the MIPS architecture.
Huge pages are implemented by placing the PTE for the huge page (flagged by the _PAGE_HUGE) in the PMD. This gives a huge page size of 2M (PTRS_PER_PTE * PAGE_SIZE) when the PAGE_SIZE is 4K. Each huge page occupies a complete TLB entry with the first half of the page covered by entrylo0 and the second half by entrylo1. Since there is only a single PTE for the huge page in the PMD, the value of entrylo1 is derived from entrylo0 by adding a constant of half a huge page. We do this for several reasons:
1) The valid values of PageMask are such that this combination is allowed.
2) Since the PageMask applies to both entrylo0 *and* entrylo1 we cannot leave one of them empty because the current vma allocation mechanisms don't handle excluding areas that would fall in any gaps caused by an empty entrylo from allocation to normal sized pages.
When hugetlbfs is enabled, there is an approximately two instruction overhead in the normal path of the TLB refill handler. This is needed to check the _PAGE_HUGE bit of each PMD entry. With hugetlbfs disabled, there is no change.
Currently the patch only works for 64-bit kernels because the value of PTRS_PER_PTE in 32-bit kernels is such that it is impossible to have a valid PageMask. It is thought that by adjusting the page allocation scheme, 32-bit kernels could be supported in the future.
All of the patches except the forth touch only the arch/mips tree. The forth patch is the generic Kconfig change to allow MIPS to turn on hugetlbfs.
I will reply with the five patches.
David Daney (5):
MIPS: Add support files for hugeTLBfs.
MIPS: Add hugeTLBfs page defines.
MIPS: TLB support for hugeTLBfs.
Enable hugetlbfs for more systems.
MIPS: Add SYS_SUPPORTS_HUGETLBFS Kconfig variable and enable it for
some systems.
arch/mips/Kconfig | 11 +++ arch/mips/include/asm/hugetlb.h | 114 +++++++++++++++++++++++ arch/mips/include/asm/mipsregs.h | 16 ++++ arch/mips/include/asm/page.h | 5 + arch/mips/include/asm/pgtable-bits.h | 1 + arch/mips/include/asm/pgtable.h | 10 ++ arch/mips/mm/Makefile | 1 + arch/mips/mm/hugetlbpage.c | 101 +++++++++++++++++++++ arch/mips/mm/tlb-r4k.c | 43 +++++++--- arch/mips/mm/tlbex.c | 165 +++++++++++++++++++++++++++++++++- fs/Kconfig | 2 +- 11 files changed, 456 insertions(+), 13 deletions(-) create mode 100644 arch/mips/include/asm/hugetlb.h create mode 100644 arch/mips/mm/hugetlbpage.c
PATCH 1/5 MIPS: Add support files for hugeTLBfs.
--- arch/mips/include/asm/hugetlb.h | 114 +++++++++++++++++++++++++++++++++++++++ arch/mips/mm/Makefile | 1 + arch/mips/mm/hugetlbpage.c | 101 ++++++++++++++++++++++++++++++++++ 3 files changed, 216 insertions(+), 0 deletions(-) create mode 100644 arch/mips/include/asm/hugetlb.h create mode 100644 arch/mips/mm/hugetlbpage.c
arch/mips/include/asm/hugetlb.h new file +114 -0
@@ -0,0 +1,114 @@ +/* +* This file is subject to the terms and conditions of the GNU General Public +* License. See the file "COPYING" in the main directory of this archive +* for more details. +* +* Copyright (C) 2008, 2009 Cavium Networks, Inc. +*/ + +#ifndef __ASM_HUGETLB_H +#define __ASM_HUGETLB_H + +#include <asm/page.h> + + +static inline int is_hugepage_only_range (struct mm_struct *mm, +unsigned long addr, +unsigned long len) +{ +return 0 ; +} + +static inline int prepare_hugepage_range (struct file *file, +unsigned long addr, +unsigned long len) +{ +unsigned long task_size =STACK_TOP ; +struct hstate *h =hstate_file (file); + +if (len & ~huge_page_mask (h)) +return -EINVAL ; +if (addr & ~huge_page_mask (h)) +return -EINVAL ; +if (len > task_size) +return -ENOMEM ; +if (task_size - len < addr) +return -EINVAL ; +return 0 ; +} + +static inline void hugetlb_prefault_arch_hook (struct mm_struct *mm) +{ +} + +static inline void hugetlb_free_pgd_range (struct mmu_gather *tlb, +unsigned long addr, +unsigned long end, +unsigned long floor, +unsigned long ceiling) +{ +free_pgd_range (tlb, addr, end, floor, ceiling); +} + +static inline void set_huge_pte_at (struct mm_struct *mm,unsigned long addr, +pte_t *ptep,pte_t pte) +{ +set_pte_at (mm, addr, ptep, pte); +} + +static inline pte_t huge_ptep_get_and_clear (struct mm_struct *mm, +unsigned long addr,pte_t *ptep) +{ +pte_t clear; +pte_t pte = *ptep; + +pte_val (clear) = (unsigned long )invalid_pte_table; +set_pte_at (mm, addr, ptep, clear); +return pte; +} + +static inline void huge_ptep_clear_flush (struct vm_area_struct *vma, +unsigned long addr,pte_t *ptep) +{ +} + +static inline int huge_pte_none (pte_t pte) +{ +unsigned long val =pte_val (pte) & ~_PAGE_GLOBAL; +return !val || (val == (unsigned long )invalid_pte_table); +} + +static inline pte_t huge_pte_wrprotect (pte_t pte) +{ +return pte_wrprotect (pte); +} + +static inline void huge_ptep_set_wrprotect (struct mm_struct *mm, +unsigned long addr,pte_t *ptep) +{ +ptep_set_wrprotect (mm, addr, ptep); +} + +static inline int huge_ptep_set_access_flags (struct vm_area_struct *vma, +unsigned long addr, +pte_t *ptep,pte_t pte, +int dirty) +{ +return ptep_set_access_flags (vma, addr, ptep, pte, dirty); +} + +static inline pte_t huge_ptep_get (pte_t *ptep) +{ +return *ptep; +} + +static inline int arch_prepare_hugepage (struct page *page) +{ +return 0 ; +} + +static inline void arch_release_hugepage (struct page *page) +{ +} + +#endif /* __ASM_HUGETLB_H */
arch/mips/mm/Makefile +1 -0
@@ -8,6 +8,7 @@ obj-y += cache.o dma-default.o extable.o fault.o \ obj-$(CONFIG_32BIT) += ioremap.o pgtable-32.o obj-$(CONFIG_64BIT) += pgtable-64.o obj-$(CONFIG_HIGHMEM) += highmem.o +obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o obj-$(CONFIG_CPU_LOONGSON2) += c-r4k.o cex-gen.o tlb-r4k.o obj-$(CONFIG_CPU_MIPS32) += c-r4k.o cex-gen.o tlb-r4k.o
arch/mips/mm/hugetlbpage.c new file +101 -0
101 lines not on this page · raw message
Signature · 1 line
1.6.0.6
PATCH 2/5 MIPS: Add hugeTLBfs page defines.
In reply to David Daney--- arch/mips/include/asm/mipsregs.h | 16 ++++++++++++++++ arch/mips/include/asm/page.h | 5 +++++ arch/mips/include/asm/pgtable-bits.h | 1 + arch/mips/include/asm/pgtable.h | 10 ++++++++++ 4 files changed, 32 insertions(+), 0 deletions(-)
arch/mips/include/asm/mipsregs.h +16 -0
@@ -220,6 +220,22 @@#error Bad page size configuration!#endif +/* +* Default huge tlb size for a given kernel configuration +*/ +#ifdef CONFIG_PAGE_SIZE_4KB +#define PM_HUGE_MASK PM_1M +#elif defined(CONFIG_PAGE_SIZE_8KB ) +#define PM_HUGE_MASK PM_4M +#elif defined(CONFIG_PAGE_SIZE_16KB ) +#define PM_HUGE_MASK PM_16M +#elif defined(CONFIG_PAGE_SIZE_32KB ) +#define PM_HUGE_MASK PM_64M +#elif defined(CONFIG_PAGE_SIZE_64KB ) +#define PM_HUGE_MASK PM_256M +#elif defined(CONFIG_HUGETLB_PAGE ) +#error Bad page size configuration for hugetlbfs! +#endif /* * Values used for computation of new tlb entries
arch/mips/include/asm/page.h +5 -0
@@ -32,6 +32,11 @@#define PAGE_SIZE (1UL << PAGE_SHIFT)#define PAGE_MASK (~((1 << PAGE_SHIFT) - 1)) +#define HPAGE_SHIFT (PAGE_SHIFT + PAGE_SHIFT - 3) +#define HPAGE_SIZE ((1UL) << HPAGE_SHIFT) +#define HPAGE_MASK (~(HPAGE_SIZE - 1)) +#define HUGETLB_PAGE_ORDER (HPAGE_SHIFT - PAGE_SHIFT) +#ifndef __ASSEMBLY__#include <linux/pfn.h>
arch/mips/include/asm/pgtable-bits.h +1 -0
@@ -72,6 +72,7 @@#else #define _PAGE_R4KBUG (1<<5)/* workaround for r4k bug */ +#define _PAGE_HUGE (1<<5)/* huge tlb page */ #define _PAGE_GLOBAL (1<<6)#define _PAGE_VALID (1<<7)#define _PAGE_SILENT_READ (1<<7)/* synonym */
arch/mips/include/asm/pgtable.h +10 -0
@@ -292,6 +292,16 @@ static inline pte_t pte_mkyoung(pte_t pte)pte_val (pte) |= _PAGE_SILENT_READ; return pte; } + +#ifdef _PAGE_HUGE +static inline int pte_huge (pte_t pte) {return pte_val (pte) & _PAGE_HUGE; } + +static inline pte_t pte_mkhuge (pte_t pte) +{ +pte_val (pte) |= _PAGE_HUGE; +return pte; +} +#endif /* _PAGE_HUGE */ #endif static inline int pte_special (pte_t pte) {return 0 ; }static inline pte_t pte_mkspecial (pte_t pte) {return pte; }
Signature · 1 line
1.6.0.6
PATCH 3/5 MIPS: TLB support for hugeTLBfs.
In reply to David DaneyThe TLB handlers need to check for huge pages and give them special handling. Huge pages consist of two contiguous sub-pages of physical memory.
* Loading entrylo0 and entrylo1 need to be handled specially.
* The page mask must be set for huge pages and then restored after writing the TLB entries.
* The PTE for huge pages resides in the PMD, we halt traversal of the tables there.
--- arch/mips/mm/tlb-r4k.c | 43 +++++++++--- arch/mips/mm/tlbex.c | 165 +++++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 196 insertions(+), 12 deletions(-)
arch/mips/mm/tlb-r4k.c +32 -11
@@ -11,6 +11,7 @@#include <linux/init.h> #include <linux/sched.h> #include <linux/mm.h> +#include <linux/hugetlb.h> #include <asm/cpu.h> #include <asm/bootinfo.h>
@@ -297,21 +298,41 @@ void __update_tlb(struct vm_area_struct * vma, unsigned long address, pte_t pte) pudp =pud_offset (pgdp, address); pmdp =pmd_offset (pudp, address); idx =read_c0_index (); - ptep =pte_offset_map (pmdp, address); +#ifdef CONFIG_HUGETLB_PAGE +/* this could be a huge page */ +if (pmd_huge (*pmdp)) { +unsigned long lo; +write_c0_pagemask (PM_HUGE_MASK ); + ptep = (pte_t *)pmdp; + lo =pte_val (*ptep) >>6 ; +write_c0_entrylo0 (lo); +write_c0_entrylo1 (lo + (HPAGE_SIZE >>7 )); + +mtc0_tlbw_hazard (); +if (idx <0 ) +tlb_write_random (); +else +tlb_write_indexed (); +write_c0_pagemask (PM_DEFAULT_MASK ); + } else +#endif + { + ptep =pte_offset_map (pmdp, address);#if defined(CONFIG_64BIT_PHYS_ADDR ) && defined(CONFIG_CPU_MIPS32 ) -write_c0_entrylo0 (ptep->pte_high); - ptep++; -write_c0_entrylo1 (ptep->pte_high); +write_c0_entrylo0 (ptep->pte_high); + ptep++; +write_c0_entrylo1 (ptep->pte_high);#else -write_c0_entrylo0 (pte_val (*ptep++) >>6 ); -write_c0_entrylo1 (pte_val (*ptep) >>6 ); +write_c0_entrylo0 (pte_val (*ptep++) >>6 ); +write_c0_entrylo1 (pte_val (*ptep) >>6 );#endif -mtc0_tlbw_hazard (); -if (idx <0 ) -tlb_write_random (); -else -tlb_write_indexed (); +mtc0_tlbw_hazard (); +if (idx <0 ) +tlb_write_random (); +else +tlb_write_indexed (); + }tlbw_use_hazard ();FLUSH_ITLB_VM (vma);EXIT_CRITICAL (flags);
arch/mips/mm/tlbex.c +164 -1
237 lines not on this page · raw message
Signature · 1 line
1.6.0.6
PATCH 4/5 Enable hugetlbfs for more systems.
In reply to David DaneyAs part of adding hugetlbfs support for MIPS, I am adding a new kconfig variable 'SYS_SUPPORTS_HUGETLBFS'. Since some mips cpu varients don't yet support it, we can enable selection of HUGETLBFS on a system by system basis from the arch/mips/Kconfig.
--- fs/Kconfig | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
fs/Kconfig +1 -1
@@ -124,7 +124,7 @@ config TMPFS_POSIX_ACL config HUGETLBFS bool "HugeTLB file system support" depends on X86 || IA64 || PPC64 || SPARC64 || (SUPERH && MMU) || \ - (S390 && 64BIT) || BROKEN + (S390 && 64BIT) || SYS_SUPPORTS_HUGETLBFS || BROKEN help hugetlbfs is a filesystem backing for HugeTLB pages, based on ramfs. For architectures that support it, say Y here and read
Signature · 1 line
1.6.0.6
Re: [PATCH 4/5] Enable hugetlbfs for more systems.
Quoted 4 lines · David Daney
From: David Daney <[email protected]>
Date: Wed, 27 May 2009 17:47:45 -0700
To: [email protected], [email protected]
Cc: [email protected], David Daney <[email protected]>
Holomorphy.com is unreachable since like a year. Use [email protected].
Quoted 18 lines
Subject: [PATCH 4/5] Enable hugetlbfs for more systems.
As part of adding hugetlbfs support for MIPS, I am adding a new kconfig variable '
SYS_SUPPORTS_HUGETLBFS'. Since some mips cpu varients don't yet support it, we can enable selection of HUGETLBFS on a system by system basis from thearch/mips/Kconfig.CC William Irwin <[email protected]>Signed-off-by David Daney <[email protected]>--- fs/Kconfig | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)fs/Kconfig +0
-0
fs/Kconfig config TMPFS_POSIX_ACL · line 127
Quoted 5 lines
fs/Kconfig +1
-1@@ -124,7 +124,7 @@ config TMPFS_POSIX_ACL config HUGETLBFS bool "HugeTLB file system support" depends on X86 || IA64 || PPC64 || SPARC64 || (SUPERH && MMU) || \ - (S390 && 64BIT) || BROKEN + (S390 && 64BIT) || SYS_SUPPORTS_HUGETLBFS || BROKEN
I'm ok with this patch but it's really crying for some follup patch to convert all the other architectures to use SYS_SUPPORTS_HUGETLBFS.
Ralf
PATCH 5/5 MIPS: Add SYS_SUPPORTS_HUGETLBFS Kconfig variable and enable it for some systems.
In reply to David DaneyAdd new kconfig variables SYS_SUPPORTS_HUGETLBFS and CPU_SUPPORTS_HUGEPAGES. They are enabled for systems that are known to support huge pages.
--- arch/mips/Kconfig | 11 +++++++++++ 1 files changed, 11 insertions(+), 0 deletions(-)
arch/mips/Kconfig +11 -0
@@ -852,6 +852,11 @@ config SYS_SUPPORTS_BIG_ENDIAN config SYS_SUPPORTS_LITTLE_ENDIAN bool +config SYS_SUPPORTS_HUGETLBFS + bool + depends on CPU_SUPPORTS_HUGEPAGES && 64BIT + default y + config IRQ_CPU bool
@@ -1056,6 +1061,7 @@ config CPU_MIPS64_R1 select CPU_SUPPORTS_32BIT_KERNEL select CPU_SUPPORTS_64BIT_KERNEL select CPU_SUPPORTS_HIGHMEM + select CPU_SUPPORTS_HUGEPAGES help Choose this option to build a kernel for release 1 or later of the MIPS64 architecture. Many modern embedded systems with a 64-bit
@@ -1075,6 +1081,7 @@ config CPU_MIPS64_R2 select CPU_SUPPORTS_32BIT_KERNEL select CPU_SUPPORTS_64BIT_KERNEL select CPU_SUPPORTS_HIGHMEM + select CPU_SUPPORTS_HUGEPAGES help Choose this option to build a kernel for release 2 or later of the MIPS64 architecture. Many modern embedded systems with a 64-bit
@@ -1161,6 +1168,7 @@ config CPU_R5500 select CPU_HAS_LLSC select CPU_SUPPORTS_32BIT_KERNEL select CPU_SUPPORTS_64BIT_KERNEL + select CPU_SUPPORTS_HUGEPAGES help NEC VR5500 and VR5500A series processors implement 64-bit MIPS IV instruction set.
@@ -1246,6 +1254,7 @@ config CPU_CAVIUM_OCTEON select WEAK_ORDERING select WEAK_REORDERING_BEYOND_LLSC select CPU_SUPPORTS_HIGHMEM + select CPU_SUPPORTS_HUGEPAGES help The Cavium Octeon processor is a highly integrated chip containing many ethernet hardware widgets for networking tasks. The processor
@@ -1365,6 +1374,8 @@ config CPU_SUPPORTS_32BIT_KERNEL bool config CPU_SUPPORTS_64BIT_KERNEL bool +config CPU_SUPPORTS_HUGEPAGES + bool # # Set to y for ptrace access to watch registers.
Signature · 1 line
1.6.0.6
Re: [PATCH 5/5] MIPS: Add SYS_SUPPORTS_HUGETLBFS Kconfig variable and enable it for some systems.
Quoted 10 lines · David Daney
From: David Daney <[email protected]>
Date: Wed, 27 May 2009 17:47:46 -0700
To: [email protected], [email protected]
Cc: [email protected], David Daney <[email protected]>
Subject: [PATCH 5/5] MIPS: AddSYS_SUPPORTS_HUGETLBFSKconfigvariable and
enable it for some systems.Add new kconfig variables
SYS_SUPPORTS_HUGETLBFSandCPU_SUPPORTS_HUGEPAGES. They are enabled for systems that are known to support huge pages.
Which should be all processors except R2000 / R3000, R6000 and R8000, no?
Ralf