Implement use of HW assistance on TLB table walk on 8xx
v2 cover Implement use of HW assistance on TLB table walk on 8xx
The purpose of this serie is to implement hardware assistance for TLB table walk on the 8xx.
First part is to make L1 entries and L2 entries independant.
For that, we need to alter ioremap functions in order to handle GUARD attribute at the PGD/PMD level.
Last part is to reuse PTE fragment implemented on PPC64 in order to not waste 16k Pages for page tables as only 4k are used.
Tested successfully on 8xx.
Successfull compilation on following defconfigs:ppc64_defconfigppc64e_defconfigpseries_defconfigpmac32_defconfiglinkstation_defconfigcorenet32_smp_defconfigppc40x_defconfigstorcenter_defconfigppc44x_defconfig
Changes in v2:
- Removed the 3 first patchs which have been applied already
- Fixed compilation errors reported by Michael
- Squashed the commonalisation of ioremap functions into a single patch
- Fixed the use of pte_fragment
- Added a patch optimising perf counting of TLB misses and instructions
Christophe Leroy (14):
Revert "powerpc/8xx: Use L1 entry APG to handle _PAGE_ACCESSED for
CONFIG_SWAP"
powerpc: move io mapping functions into ioremap.c
powerpc: make ioremap_bot common to PPC32 and PPC64
powerpc: common ioremap functions.
powerpc: use _ALIGN_DOWN macro for VMALLOC_BASE powerpc/nohash32: allow setting GUARDED attribute in the PMD directly powerpc/8xx: set GUARDED attribute in the PMD directly powerpc/8xx: Remove PTE_ATOMIC_UPDATES
powerpc/mm: Use hardware assistance in TLB handlers on the 8xx powerpc/8xx: reunify TLB handler routines
powerpc/8xx: Free up SPRN_SPRG_SCRATCH2
powerpc/mm: Make pte_fragment_alloc() common to PPC32 and PPC64 powerpc/mm: Use pte_fragment_alloc() on 8xx
powerpc/8xx: Move SW perf counters in first 32kb of memory
arch/powerpc/include/asm/book3s/32/pgtable.h | 29 +- arch/powerpc/include/asm/book3s/64/pgtable.h | 2 + arch/powerpc/include/asm/highmem.h | 11 - arch/powerpc/include/asm/hugetlb.h | 4 +- arch/powerpc/include/asm/machdep.h | 2 +- arch/powerpc/include/asm/mmu-8xx.h | 38 +-- arch/powerpc/include/asm/mmu_context.h | 28 ++ arch/powerpc/include/asm/nohash/32/pgalloc.h | 56 +++- arch/powerpc/include/asm/nohash/32/pgtable.h | 77 +++-- arch/powerpc/include/asm/nohash/32/pte-8xx.h | 6 +- arch/powerpc/include/asm/nohash/pgtable.h | 4 + arch/powerpc/include/asm/page.h | 2 +- arch/powerpc/include/asm/pgtable-types.h | 4 + arch/powerpc/kernel/head_8xx.S | 409 +++++++++++---------------- arch/powerpc/mm/8xx_mmu.c | 12 +- arch/powerpc/mm/Makefile | 2 +- arch/powerpc/mm/dma-noncoherent.c | 2 +- arch/powerpc/mm/dump_linuxpagetables.c | 32 ++- arch/powerpc/mm/hugetlbpage.c | 12 + arch/powerpc/mm/init_32.c | 6 +- arch/powerpc/mm/ioremap.c | 254 +++++++++++++++++ arch/powerpc/mm/mem.c | 16 +- arch/powerpc/mm/mmu_context_book3s64.c | 28 -- arch/powerpc/mm/mmu_context_nohash.c | 4 + arch/powerpc/mm/pgtable.c | 75 +++++ arch/powerpc/mm/pgtable_32.c | 167 +++-------- arch/powerpc/mm/pgtable_64.c | 244 ---------------- arch/powerpc/platforms/Kconfig.cputype | 9 + 28 files changed, 790 insertions(+), 745 deletions(-) create mode 100644 arch/powerpc/mm/ioremap.c
Signature · 1 line
2.13.3
PATCH v2 1/14 Revert "powerpc/8xx: Use L1 entry APG to handle _PAGE_ACCESSED for CONFIG_SWAP"
This reverts commit 4f94b2c7462d9720b2afa7e8e8d4c19446bb31ce.
That commit was buggy, as it used rlwinm instead of rlwimi. Instead of fixing that bug, we revert the previous commit in order to reduce the dependency between L1 entries and L2 entries
--- arch/powerpc/include/asm/mmu-8xx.h | 34 +++++----------------------- arch/powerpc/kernel/head_8xx.S | 45 +++++++++++++++++++++++--------------- arch/powerpc/mm/8xx_mmu.c | 2 +- 3 files changed, 34 insertions(+), 47 deletions(-)
arch/powerpc/include/asm/mmu-8xx.h +6 -28
@@ -34,20 +34,12 @@* respectively NA for All or X for Supervisor and no access for User. * Then we use the APG to say whether accesses are according to Page rules or * "all Supervisor" rules (Access to all) -* We also use the 2nd APG bit for _PAGE_ACCESSED when having SWAP: -* When that bit is not set access is done iaw "all user" -* which means no access iaw page rules. -* Therefore, we define 4 APG groups. lsb is _PMD_USER, 2nd is _PAGE_ACCESSED -* 0x => No access => 11 (all accesses performed as user iaw page definition) -* 10 => No user => 01 (all accesses performed according to page definition) -* 11 => User => 00 (all accesses performed as supervisor iaw page definition) +* Therefore, we define 2 APG groups. lsb is _PMD_USER +* 0 => No user => 01 (all accesses performed according to page definition) +* 1 => User => 00 (all accesses performed as supervisor iaw page definition) * We define all 16 groups so that all other bits of APG can take any value */ -#ifdef CONFIG_SWAP -#define MI_APG_INIT 0xf4f4f4f4 -#else #define MI_APG_INIT 0x44444444 -#endif /* The effective page number register. When read, contains the information * about the last instruction TLB miss. When MI_RPN is written, bits in
@@ -115,20 +107,12 @@* Supervisor and no access for user and NA for ALL. * Then we use the APG to say whether accesses are according to Page rules or * "all Supervisor" rules (Access to all) -* We also use the 2nd APG bit for _PAGE_ACCESSED when having SWAP: -* When that bit is not set access is done iaw "all user" -* which means no access iaw page rules. -* Therefore, we define 4 APG groups. lsb is _PMD_USER, 2nd is _PAGE_ACCESSED -* 0x => No access => 11 (all accesses performed as user iaw page definition) -* 10 => No user => 01 (all accesses performed according to page definition) -* 11 => User => 00 (all accesses performed as supervisor iaw page definition) +* Therefore, we define 2 APG groups. lsb is _PMD_USER +* 0 => No user => 01 (all accesses performed according to page definition) +* 1 => User => 00 (all accesses performed as supervisor iaw page definition) * We define all 16 groups so that all other bits of APG can take any value */ -#ifdef CONFIG_SWAP -#define MD_APG_INIT 0xf4f4f4f4 -#else #define MD_APG_INIT 0x44444444 -#endif /* The effective page number register. When read, contains the information * about the last instruction TLB miss. When MD_RPN is written, bits in
@@ -180,12 +164,6 @@*/ #define SPRN_M_TW 799 -/* APGs */ -#define M_APG0 0x00000000 -#define M_APG1 0x00000020 -#define M_APG2 0x00000040 -#define M_APG3 0x00000060 -#ifdef CONFIG_PPC_MM_SLICES #include <asm/nohash/32/slice.h> #define SLICE_ARRAY_SIZE (1 << (32 - SLICE_LOW_SHIFT - 1))
arch/powerpc/kernel/head_8xx.S +27 -18
92 lines not on this page · raw message
arch/powerpc/mm/8xx_mmu.c +1 -1
@@ -79,7 +79,7 @@ void __init MMU_init_hw(void)for (; i <32 && mem >=LARGE_PAGE_SIZE_8M ; i++) {mtspr (SPRN_MD_CTR , ctr | (i <<8 ));mtspr (SPRN_MD_EPN , (unsigned long )__va (addr) |MD_EVALID ); -mtspr (SPRN_MD_TWC ,MD_PS8MEG |MD_SVALID |M_APG2 ); +mtspr (SPRN_MD_TWC ,MD_PS8MEG |MD_SVALID );mtspr (SPRN_MD_RPN , addr | flags | _PAGE_PRESENT); addr +=LARGE_PAGE_SIZE_8M ; mem -=LARGE_PAGE_SIZE_8M ;
Signature · 1 line
2.13.3
PATCH v2 2/14 powerpc: move io mapping functions into ioremap.c
In reply to Christophe LeroyThis patch is the first of a serie that intends to make io mappings common to PPC32 and PPC64.
It moves ioremap/unmap fonctions into a new file called ioremap.c with no other modification to the functions.
For the time being, the PPC32 and PPC64 parts get enclosed into #ifdef. Following patches will aim at making those functions as common as possible between PPC32 and PPC64.
This patch also moves EXPORT_SYMBOL at the end of each function
--- arch/powerpc/mm/Makefile | 2 +- arch/powerpc/mm/ioremap.c | 352 +++++++++++++++++++++++++++++++++++++++++++ arch/powerpc/mm/pgtable_32.c | 139 ----------------- arch/powerpc/mm/pgtable_64.c | 177 ---------------------- 4 files changed, 353 insertions(+), 317 deletions(-) create mode 100644 arch/powerpc/mm/ioremap.c
arch/powerpc/mm/Makefile +1 -1
@@ -9,7 +9,7 @@ ccflags-$(CONFIG_PPC64) := $(NO_MINIMAL_TOC) obj-y := fault.o mem.o pgtable.o mmap.o \ init_$(BITS).o pgtable_$(BITS).o \ - init-common.o mmu_context.o drmem.o + init-common.o mmu_context.o drmem.o ioremap.o obj-$(CONFIG_PPC_MMU_NOHASH) += mmu_context_nohash.o tlb_nohash.o \ tlb_nohash_low.o obj-$(CONFIG_PPC_BOOK3E) += tlb_low_$(BITS)e.o
arch/powerpc/mm/ioremap.c new file +352 -0
352 lines not on this page · raw message
arch/powerpc/mm/pgtable_32.c +0 -139
151 lines not on this page · raw message
arch/powerpc/mm/pgtable_64.c +0 -177
185 lines not on this page · raw message
Signature · 1 line
2.13.3
PATCH v2 3/14 powerpc: make ioremap_bot common to PPC32 and PPC64
In reply to Christophe LeroyToday, early ioremap maps from IOREMAP_BASE down to up on PPC64 and from IOREMAP_TOP up to down on PPC32
This patchs modifies PPC32 behaviour to get same behaviour as PPC64
--- arch/powerpc/include/asm/book3s/32/pgtable.h | 29 ++++++++++++++++++------ arch/powerpc/include/asm/highmem.h | 11 ---------- arch/powerpc/include/asm/nohash/32/pgtable.h | 33 ++++++++++++++++++---------- arch/powerpc/mm/dma-noncoherent.c | 2 +- arch/powerpc/mm/dump_linuxpagetables.c | 6 ++--- arch/powerpc/mm/init_32.c | 6 ++++- arch/powerpc/mm/ioremap.c | 21 +++++++++--------- arch/powerpc/mm/mem.c | 7 +++--- 8 files changed, 66 insertions(+), 49 deletions(-)
arch/powerpc/include/asm/book3s/32/pgtable.h +22 -7
@@ -50,20 +50,32 @@* virtual space that goes below PKMAP and FIXMAP */ #ifdef CONFIG_HIGHMEM +#ifdef CONFIG_PPC_4K_PAGES +#define PKMAP_ORDER PTE_SHIFT +#else +#define PKMAP_ORDER 9 +#endif +#define LAST_PKMAP (1 << PKMAP_ORDER) +#ifndef CONFIG_PPC_4K_PAGES +#define PKMAP_BASE (FIXADDR_START - PAGE_SIZE*(LAST_PKMAP + 1)) +#else +#define PKMAP_BASE ((FIXADDR_START - PAGE_SIZE*(LAST_PKMAP + 1)) & PMD_MASK) +#endif #define KVIRT_TOP PKMAP_BASE#else #define KVIRT_TOP (0xfe000000UL)/* for now, could be FIXMAP_BASE ? */ #endif +#define IOREMAP_BASE VMALLOC_BASE/* -* ioremap_bot starts at that address. Early ioremaps move down from there, -* until mem_init() at which point this becomes the top of the vmalloc +* ioremap_bot starts at IOREMAP_BASE. Early ioremaps move up from there, +* until mem_init() at which point this becomes the bottom of the vmalloc * and ioremap space */ #ifdef CONFIG_NOT_COHERENT_CACHE -#define IOREMAP_TOP ((KVIRT_TOP - CONFIG_CONSISTENT_SIZE) & PAGE_MASK) +#define IOREMAP_END ((KVIRT_TOP - CONFIG_CONSISTENT_SIZE) & PAGE_MASK)#else -#define IOREMAP_TOP KVIRT_TOP +#define IOREMAP_END KVIRT_TOP#endif /*
@@ -85,11 +97,12 @@*/ #define VMALLOC_OFFSET (0x1000000)/* 16M */ #ifdef PPC_PIN_SIZE -#define VMALLOC_START (((_ALIGN((long)high_memory, PPC_PIN_SIZE) + VMALLOC_OFFSET) & ~(VMALLOC_OFFSET-1))) +#define VMALLOC_BASE (((_ALIGN((long)high_memory, PPC_PIN_SIZE) + VMALLOC_OFFSET) & ~(VMALLOC_OFFSET-1)))#else -#define VMALLOC_START ((((long)high_memory + VMALLOC_OFFSET) & ~(VMALLOC_OFFSET-1))) +#define VMALLOC_BASE ((((long)high_memory + VMALLOC_OFFSET) & ~(VMALLOC_OFFSET-1)))#endif -#define VMALLOC_END ioremap_bot +#define VMALLOC_START ioremap_bot +#define VMALLOC_END IOREMAP_END#ifndef __ASSEMBLY__#include <linux/sched.h>
@@ -298,6 +311,8 @@ static inline void __ptep_set_access_flags(struct mm_struct *mm,int map_kernel_page (unsigned long va,phys_addr_t pa,int flags); +#include <asm/fixmap.h> +/* Generic accessors to PTE bits */ static inline int pte_write (pte_t pte) {return !!(pte_val (pte) & _PAGE_RW);}static inline int pte_read (pte_t pte) {return 1 ; }
arch/powerpc/include/asm/highmem.h +0 -11
@@ -44,17 +44,6 @@ extern pte_t *pkmap_page_table;* and PKMAP can be placed in a single pte table. We use 512 pages for PKMAP * in case of 16K/64K/256K page sizes. */ -#ifdef CONFIG_PPC_4K_PAGES -#define PKMAP_ORDER PTE_SHIFT -#else -#define PKMAP_ORDER 9 -#endif -#define LAST_PKMAP (1 << PKMAP_ORDER) -#ifndef CONFIG_PPC_4K_PAGES -#define PKMAP_BASE (FIXADDR_START - PAGE_SIZE*(LAST_PKMAP + 1)) -#else -#define PKMAP_BASE ((FIXADDR_START - PAGE_SIZE*(LAST_PKMAP + 1)) & PMD_MASK) -#endif #define LAST_PKMAP_MASK (LAST_PKMAP-1)#define PKMAP_NR (virt) ((virt-PKMAP_BASE) >> PAGE_SHIFT)#define PKMAP_ADDR (nr) (PKMAP_BASE + ((nr) << PAGE_SHIFT))
arch/powerpc/include/asm/nohash/32/pgtable.h +21 -12
@@ -69,6 +69,17 @@ extern int icache_44x_need_flush;* virtual space that goes below PKMAP and FIXMAP */ #ifdef CONFIG_HIGHMEM +#ifdef CONFIG_PPC_4K_PAGES +#define PKMAP_ORDER PTE_SHIFT +#else +#define PKMAP_ORDER 9 +#endif +#define LAST_PKMAP (1 << PKMAP_ORDER) +#ifndef CONFIG_PPC_4K_PAGES +#define PKMAP_BASE (FIXADDR_START - PAGE_SIZE*(LAST_PKMAP + 1)) +#else +#define PKMAP_BASE ((FIXADDR_START - PAGE_SIZE*(LAST_PKMAP + 1)) & PMD_MASK) +#endif #define KVIRT_TOP PKMAP_BASE#else #define KVIRT_TOP (0xfe000000UL)/* for now, could be FIXMAP_BASE ? */
@@ -80,10 +91,11 @@ extern int icache_44x_need_flush;* and ioremap space */ #ifdef CONFIG_NOT_COHERENT_CACHE -#define IOREMAP_TOP ((KVIRT_TOP - CONFIG_CONSISTENT_SIZE) & PAGE_MASK) +#define IOREMAP_END ((KVIRT_TOP - CONFIG_CONSISTENT_SIZE) & PAGE_MASK)#else -#define IOREMAP_TOP KVIRT_TOP +#define IOREMAP_END KVIRT_TOP#endif +#define IOREMAP_BASE VMALLOC_BASE/* * Just any arbitrary offset to the start of the vmalloc VM area: the
@@ -94,21 +106,16 @@ extern int icache_44x_need_flush;* area for the same reason. ;) * * We no longer map larger than phys RAM with the BATs so we don't have -* to worry about the VMALLOC_OFFSET causing problems. We do have to worry -* about clashes between our early calls to ioremap() that start growing down -* from IOREMAP_TOP being run into the VM area allocations (growing upwards -* from VMALLOC_START). For this reason we have ioremap_bot to check when -* we actually run into our mappings setup in the early boot with the VM -* system. This really does become a problem for machines with good amounts -* of RAM. -- Cort +* to worry about the VMALLOC_OFFSET causing problems. */ #define VMALLOC_OFFSET (0x1000000)/* 16M */ #ifdef PPC_PIN_SIZE -#define VMALLOC_START (((_ALIGN((long)high_memory, PPC_PIN_SIZE) + VMALLOC_OFFSET) & ~(VMALLOC_OFFSET-1))) +#define VMALLOC_BASE (((_ALIGN((long)high_memory, PPC_PIN_SIZE) + VMALLOC_OFFSET) & ~(VMALLOC_OFFSET-1)))#else -#define VMALLOC_START ((((long)high_memory + VMALLOC_OFFSET) & ~(VMALLOC_OFFSET-1))) +#define VMALLOC_BASE ((((long)high_memory + VMALLOC_OFFSET) & ~(VMALLOC_OFFSET-1)))#endif -#define VMALLOC_END ioremap_bot +#define VMALLOC_START ioremap_bot +#define VMALLOC_END IOREMAP_END/* * Bits in a linux-style PTE. These match the bits in the
@@ -325,6 +332,8 @@ static inline int pte_young(pte_t pte)int map_kernel_page (unsigned long va,phys_addr_t pa,int flags); +#include <asm/fixmap.h> +#endif /* !__ASSEMBLY__ */ #endif /* __ASM_POWERPC_NOHASH_32_PGTABLE_H */
arch/powerpc/mm/dma-noncoherent.c +1 -1
@@ -43,7 +43,7 @@* can be further configured for specific applications under * the "Advanced Setup" menu. -Matt */ -#define CONSISTENT_BASE (IOREMAP_TOP) +#define CONSISTENT_BASE (IOREMAP_END)#define CONSISTENT_END (CONSISTENT_BASE + CONFIG_CONSISTENT_SIZE)#define CONSISTENT_OFFSET (x) (((unsigned long)(x) - CONSISTENT_BASE) >> PAGE_SHIFT)
arch/powerpc/mm/dump_linuxpagetables.c +3 -3
@@ -452,11 +452,11 @@ static void populate_markers(void) address_markers[i++].start_address =VMEMMAP_BASE ;#endif #else /* !CONFIG_PPC64 */ + address_markers[i++].start_address =IOREMAP_BASE ; address_markers[i++].start_address = ioremap_bot; - address_markers[i++].start_address =IOREMAP_TOP ;#ifdef CONFIG_NOT_COHERENT_CACHE - address_markers[i++].start_address =IOREMAP_TOP ; - address_markers[i++].start_address =IOREMAP_TOP + + address_markers[i++].start_address =IOREMAP_END ; + address_markers[i++].start_address =IOREMAP_END +CONFIG_CONSISTENT_SIZE ;#endif #ifdef CONFIG_HIGHMEM
arch/powerpc/mm/init_32.c +5 -1
@@ -172,7 +172,11 @@ void __init MMU_init(void)mapin_ram ();/* Initialize early top-down ioremap allocator */ - ioremap_bot =IOREMAP_TOP ; +if (IS_ENABLED (CONFIG_HIGHMEM )) + high_memory = (void *)__va (lowmem_end_addr); +else + high_memory = (void *)__va (memblock_end_of_DRAM ()); + ioremap_bot =IOREMAP_BASE ;if (ppc_md.progress) ppc_md.progress ("MMU:exit" ,0x211 );
arch/powerpc/mm/ioremap.c +10 -11
@@ -28,11 +28,15 @@#include "mmu_decl.h" -#ifdef CONFIG_PPC32 - +#if defined(CONFIG_PPC_BOOK3S_64 ) || defined(CONFIG_PPC32 )unsigned long ioremap_bot; +#else +unsigned long ioremap_bot =IOREMAP_BASE ; +#endif EXPORT_SYMBOL (ioremap_bot);/* aka VMALLOC_END */ +#ifdef CONFIG_PPC32 +void __iomem *ioremap (phys_addr_t addr,unsigned long size) {
@@ -90,7 +94,7 @@ __ioremap_caller(phys_addr_t addr, unsigned long size, unsigned long flags,/* * Choose an address to map it to. * Once the vmalloc system is running, we use it. -* Before then, we use space going down from IOREMAP_TOP +* Before then, we use space going up from IOREMAP_BASE * (ioremap_bot records where we're up to). */ p = addr &PAGE_MASK ;
@@ -135,7 +139,8 @@ __ioremap_caller(phys_addr_t addr, unsigned long size, unsigned long flags, area->phys_addr = p; v = (unsigned long ) area->addr; }else { - v = (ioremap_bot -= size); + v = ioremap_bot; + ioremap_bot += size; }/*
@@ -164,19 +169,13 @@ void iounmap(volatile void __iomem *addr)if (v_block_mapped ((unsigned long )addr))return ; -if (addr > high_memory && (unsigned long ) addr < ioremap_bot) +if ((unsigned long ) addr >= ioremap_bot) vunmap((void *) (PAGE_MASK & (unsigned long)addr)); }EXPORT_SYMBOL (iounmap);#else -#ifdef CONFIG_PPC_BOOK3S_64 -unsigned long ioremap_bot; -#else /* !CONFIG_PPC_BOOK3S_64 */ -unsigned long ioremap_bot =IOREMAP_BASE ; -#endif -/** * __ioremap_at - Low level function to establish the page tables * for an IO mapping
arch/powerpc/mm/mem.c +4 -3
@@ -345,8 +345,9 @@ void __init mem_init(void)#ifdef CONFIG_SWIOTLB swiotlb_init (0 );#endif - +#ifdef CONFIG_PPC64 high_memory = (void *)__va (max_low_pfn *PAGE_SIZE ); +#endif set_max_mapnr (max_pfn);free_all_bootmem ();
@@ -383,10 +384,10 @@ void __init mem_init(void)#endif /* CONFIG_HIGHMEM */ #ifdef CONFIG_NOT_COHERENT_CACHE pr_info (" * 0x%08lx..0x%08lx : consistent mem\n" , -IOREMAP_TOP ,IOREMAP_TOP +CONFIG_CONSISTENT_SIZE ); +IOREMAP_END ,IOREMAP_END +CONFIG_CONSISTENT_SIZE );#endif /* CONFIG_NOT_COHERENT_CACHE */ pr_info (" * 0x%08lx..0x%08lx : early ioremap\n" , - ioremap_bot,IOREMAP_TOP ); +IOREMAP_BASE , ioremap_bot);pr_info (" * 0x%08lx..0x%08lx : vmalloc & ioremap\n" ,VMALLOC_START ,VMALLOC_END );#endif /* CONFIG_PPC32 */
Signature · 1 line
2.13.3
PATCH v2 4/14 powerpc: common ioremap functions.
In reply to Christophe Leroy__ioremap(), ioremap(), ioremap_wc() et ioremap_prot() are very similar between PPC32 and PPC64, they can easily be made common.
_PAGE_WRITE equals to _PAGE_RW on PPC32_PAGE_RO and _PAGE_HWWRITE are 0 on PPC64
iounmap() can also be made common by renaming the PPC32 iounmap() as __iounmap() then __iounmap() can also be made common to PPC32 and PPC64.
This patch also makes __ioremap_caller() common to PPC32 and PPC64
--- arch/powerpc/include/asm/book3s/64/pgtable.h | 2 + arch/powerpc/include/asm/machdep.h | 2 +- arch/powerpc/mm/ioremap.c | 190 ++++++--------------------- 3 files changed, 46 insertions(+), 148 deletions(-)
arch/powerpc/include/asm/book3s/64/pgtable.h +2 -0
@@ -17,6 +17,8 @@#define _PAGE_NA 0#define _PAGE_RO 0#define _PAGE_USER 0 +#define _PAGE_HWWRITE 0 +#define _PAGE_COHERENT 0#define _PAGE_EXEC 0x00001/* execute permission */ #define _PAGE_WRITE 0x00002/* write access allowed */
arch/powerpc/include/asm/machdep.h +1 -1
@@ -33,11 +33,11 @@ struct pci_host_bridge;struct machdep_calls {char *name; -#ifdef CONFIG_PPC64 void __iomem * (*ioremap)(phys_addr_t addr,unsigned long size,unsigned long flags,void *caller);void (*iounmap)(volatile void __iomem *token); +#ifdef CONFIG_PPC64 #ifdef CONFIG_PM void (*iommu_save)(void );void (*iommu_restore)(void );
arch/powerpc/mm/ioremap.c +43 -147
230 lines not on this page · raw message
Signature · 1 line
2.13.3
PATCH v2 5/14 powerpc: use _ALIGN_DOWN macro for VMALLOC_BASE
In reply to Christophe LeroyUse _ALIGN_DOWN macro instead of open coding in define of VMALLOC_BASE
--- arch/powerpc/include/asm/nohash/32/pgtable.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
arch/powerpc/include/asm/nohash/32/pgtable.h +2 -2
@@ -110,9 +110,9 @@ extern int icache_44x_need_flush;*/ #define VMALLOC_OFFSET (0x1000000)/* 16M */ #ifdef PPC_PIN_SIZE -#define VMALLOC_BASE (((_ALIGN((long)high_memory, PPC_PIN_SIZE) + VMALLOC_OFFSET) & ~(VMALLOC_OFFSET-1))) +#define VMALLOC_BASE _ALIGN_DOWN(_ALIGN((long)high_memory, PPC_PIN_SIZE) + VMALLOC_OFFSET, VMALLOC_OFFSET)#else -#define VMALLOC_BASE ((((long)high_memory + VMALLOC_OFFSET) & ~(VMALLOC_OFFSET-1))) +#define VMALLOC_BASE _ALIGN_DOWN((long)high_memory + VMALLOC_OFFSET, VMALLOC_OFFSET)#endif #define VMALLOC_START ioremap_bot#define VMALLOC_END IOREMAP_END
Signature · 1 line
2.13.3
PATCH v2 6/14 powerpc/nohash32: allow setting GUARDED attribute in the PMD directly
In reply to Christophe LeroyOn the 8xx, the GUARDED attribute of the pages is managed in the L1 entry, therefore to avoid having to copy it into L1 entry at each TLB miss, we have to set it in the PMD
In order to allow this, this patch splits the VM alloc space in two parts, one for VM alloc and non Guarded IO, and one for Guarded IO.
--- arch/powerpc/include/asm/nohash/32/pgalloc.h | 12 ++++++++++++ arch/powerpc/include/asm/nohash/32/pgtable.h | 18 ++++++++++++++++-- arch/powerpc/mm/dump_linuxpagetables.c | 26 ++++++++++++++++++++++++-- arch/powerpc/mm/ioremap.c | 13 ++++++++++--- arch/powerpc/mm/mem.c | 9 +++++++++ arch/powerpc/mm/pgtable_32.c | 28 +++++++++++++++++++++++++++- arch/powerpc/platforms/Kconfig.cputype | 2 ++ 7 files changed, 100 insertions(+), 8 deletions(-)
arch/powerpc/include/asm/nohash/32/pgalloc.h +12 -0
@@ -58,6 +58,14 @@ static inline void pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmdp, *pmdp =__pmd (__pa (pte) | _PMD_PRESENT); } +#ifdef CONFIG_PPC_GUARDED_PAGE_IN_PMD +static inline void pmd_populate_kernel_g (struct mm_struct *mm,pmd_t *pmdp, +pte_t *pte) +{ + *pmdp =__pmd (__pa (pte) | _PMD_PRESENT | _PMD_GUARDED); +} +#endif +static inline void pmd_populate (struct mm_struct *mm,pmd_t *pmdp,pgtable_t pte_page) {
@@ -83,6 +91,10 @@ static inline void pmd_populate(struct mm_struct *mm, pmd_t *pmdp,#define pmd_pgtable (pmd) pmd_page(pmd)#endif +#define pte_alloc_kernel_g (pmd, address) \ + ((unlikely(pmd_none(*(pmd))) && __pte_alloc_kernel_g(pmd, address))? \ + NULL: pte_offset_kernel(pmd, address)) +extern pte_t *pte_alloc_one_kernel (struct mm_struct *mm,unsigned long addr);extern pgtable_t pte_alloc_one (struct mm_struct *mm,unsigned long addr);
arch/powerpc/include/asm/nohash/32/pgtable.h +16 -2
@@ -80,9 +80,14 @@ extern int icache_44x_need_flush;#else #define PKMAP_BASE ((FIXADDR_START - PAGE_SIZE*(LAST_PKMAP + 1)) & PMD_MASK)#endif -#define KVIRT_TOP PKMAP_BASE +#define _KVIRT_TOP PKMAP_BASE#else -#define KVIRT_TOP (0xfe000000UL)/* for now, could be FIXMAP_BASE ? */ +#define _KVIRT_TOP (0xfe000000UL)/* for now, could be FIXMAP_BASE ? */ +#endif +#ifdef CONFIG_PPC_GUARDED_PAGE_IN_PMD +#define KVIRT_TOP _ALIGN_DOWN(_KVIRT_TOP, PGDIR_SIZE) +#else +#define KVIRT_TOP _KVIRT_TOP#endif /*
@@ -95,7 +100,11 @@ extern int icache_44x_need_flush;#else #define IOREMAP_END KVIRT_TOP#endif +#ifdef CONFIG_PPC_GUARDED_PAGE_IN_PMD +#define IOREMAP_BASE _ALIGN_UP(VMALLOC_BASE + (IOREMAP_END - VMALLOC_BASE) / 2, PGDIR_SIZE) +#else #define IOREMAP_BASE VMALLOC_BASE +#endif /* * Just any arbitrary offset to the start of the vmalloc VM area: the
@@ -114,8 +123,13 @@ extern int icache_44x_need_flush;#else #define VMALLOC_BASE _ALIGN_DOWN((long)high_memory + VMALLOC_OFFSET, VMALLOC_OFFSET)#endif +#ifdef CONFIG_PPC_GUARDED_PAGE_IN_PMD +#define VMALLOC_START VMALLOC_BASE +#define VMALLOC_END IOREMAP_BASE +#else #define VMALLOC_START ioremap_bot#define VMALLOC_END IOREMAP_END +#endif /* * Bits in a linux-style PTE. These match the bits in the
arch/powerpc/mm/dump_linuxpagetables.c +24 -2
@@ -74,9 +74,9 @@ struct addr_marker {static struct addr_marker address_markers[] = { {0 ,"Start of kernel VM" }, +#ifdef CONFIG_PPC64 {0 ,"vmalloc() Area" }, {0 ,"vmalloc() End" }, -#ifdef CONFIG_PPC64 {0 ,"isa I/O start" }, {0 ,"isa I/O end" }, { 0, "phb I/O start" },
@@ -85,8 +85,19 @@ static struct addr_marker address_markers[] = { { 0, "I/O remap end" }, { 0, "vmemmap start" }, #else +#ifdef CONFIG_PPC_GUARDED_PAGE_IN_PMD + { 0, "vmalloc() Area" }, + { 0, "vmalloc() End" }, { 0, "Early I/O remap start" }, { 0, "Early I/O remap end" }, + { 0, "I/O remap start" }, + { 0, "I/O remap end" }, +#else + { 0, "Early I/O remap start" }, + { 0, "Early I/O remap end" }, + { 0, "vmalloc() I/O remap start" }, + { 0, "vmalloc() I/O remap end" }, +#endif #ifdef CONFIG_NOT_COHERENT_CACHE { 0, "Consistent mem start" }, { 0, "Consistent mem end" },
@@ -437,9 +448,9 @@ static void populate_markers(void)int i =0 ; address_markers[i++].start_address =PAGE_OFFSET ; +#ifdef CONFIG_PPC64 address_markers[i++].start_address =VMALLOC_START ; address_markers[i++].start_address =VMALLOC_END ; -#ifdef CONFIG_PPC64 address_markers[i++].start_address =ISA_IO_BASE ; address_markers[i++].start_address =ISA_IO_END ; address_markers[i++].start_address =PHB_IO_BASE ;
@@ -452,8 +463,19 @@ static void populate_markers(void) address_markers[i++].start_address =VMEMMAP_BASE ;#endif #else /* !CONFIG_PPC64 */ +#ifdef CONFIG_PPC_GUARDED_PAGE_IN_PMD + address_markers[i++].start_address =VMALLOC_START ; + address_markers[i++].start_address =VMALLOC_END ; address_markers[i++].start_address =IOREMAP_BASE ; address_markers[i++].start_address = ioremap_bot; + address_markers[i++].start_address = ioremap_bot; + address_markers[i++].start_address =IOREMAP_END ; +#else + address_markers[i++].start_address =IOREMAP_BASE ; + address_markers[i++].start_address = ioremap_bot; + address_markers[i++].start_address = ioremap_bot; + address_markers[i++].start_address =IOREMAP_END ; +#endif #ifdef CONFIG_NOT_COHERENT_CACHE address_markers[i++].start_address =IOREMAP_END ; address_markers[i++].start_address = IOREMAP_END +
arch/powerpc/mm/ioremap.c +10 -3
@@ -134,9 +134,16 @@ void __iomem * __ioremap_caller(phys_addr_t addr, unsigned long size,if (slab_is_available ()) {struct vm_struct *area; - area =__get_vm_area_caller (size,VM_IOREMAP , - ioremap_bot,IOREMAP_END , - caller); +#ifdef CONFIG_PPC_GUARDED_PAGE_IN_PMD +if (!(flags & _PAGE_GUARDED)) + area =__get_vm_area_caller (size,VM_IOREMAP , +VMALLOC_START ,VMALLOC_END , + caller); + else +#endif + area =__get_vm_area_caller (size,VM_IOREMAP , + ioremap_bot,IOREMAP_END , + caller);if (area ==NULL )return NULL ;
arch/powerpc/mm/mem.c +9 -0
@@ -386,10 +386,19 @@ void __init mem_init(void)pr_info (" * 0x%08lx..0x%08lx : consistent mem\n" ,IOREMAP_END ,IOREMAP_END +CONFIG_CONSISTENT_SIZE );#endif /* CONFIG_NOT_COHERENT_CACHE */ +#ifdef CONFIG_PPC_GUARDED_PAGE_IN_PMD +pr_info (" * 0x%08lx..0x%08lx : ioremap\n" , + ioremap_bot,IOREMAP_END );pr_info (" * 0x%08lx..0x%08lx : early ioremap\n" ,IOREMAP_BASE , ioremap_bot); +pr_info (" * 0x%08lx..0x%08lx : vmalloc\n" , +VMALLOC_START ,VMALLOC_END ); +#else pr_info (" * 0x%08lx..0x%08lx : vmalloc & ioremap\n" ,VMALLOC_START ,VMALLOC_END ); +pr_info (" * 0x%08lx..0x%08lx : early ioremap\n" , +IOREMAP_BASE , ioremap_bot); +#endif #endif /* CONFIG_PPC32 */ }
arch/powerpc/mm/pgtable_32.c +27 -1
@@ -70,6 +70,27 @@ pgtable_t pte_alloc_one(struct mm_struct *mm, unsigned long address)return ptepage; } +#ifdef CONFIG_PPC_GUARDED_PAGE_IN_PMD +int __pte_alloc_kernel_g (pmd_t *pmd,unsigned long address) +{ +pte_t *new =pte_alloc_one_kernel (&init_mm, address); +if (!new) +return -ENOMEM ; + +smp_wmb ();/* See comment in __pte_alloc */ + +spin_lock (&init_mm.page_table_lock); +if (likely (pmd_none (*pmd))) {/* Has another populated it ? */ +pmd_populate_kernel_g (&init_mm, pmd, new); + new =NULL ; + } +spin_unlock (&init_mm.page_table_lock); +if (new) +pte_free_kernel (&init_mm, new); +return 0 ; +} +#endif +int map_kernel_page (unsigned long va,phys_addr_t pa,int flags) {pmd_t *pd;
@@ -79,7 +100,12 @@ int map_kernel_page(unsigned long va, phys_addr_t pa, int flags)/* Use upper 10 bits of VA to index the first level map */ pd =pmd_offset (pud_offset (pgd_offset_k (va), va), va);/* Use middle 10 bits of VA to index the second-level map */ - pg =pte_alloc_kernel (pd, va); +#ifdef CONFIG_PPC_GUARDED_PAGE_IN_PMD +if (flags & _PAGE_GUARDED) + pg =pte_alloc_kernel_g (pd, va); + else +#endif + pg =pte_alloc_kernel (pd, va);if (pg !=0 ) { err =0 ;/* The PTE should never be already set nor present in the
arch/powerpc/platforms/Kconfig.cputype +2 -0
@@ -319,6 +319,8 @@ config ARCH_ENABLE_HUGEPAGE_MIGRATION def_bool y depends on PPC_BOOK3S_64 && HUGETLB_PAGE && MIGRATION +config PPC_GUARDED_PAGE_IN_PMD + bool config PPC_MMU_NOHASH def_bool y
Signature · 1 line
2.13.3
PATCH v2 7/14 powerpc/8xx: set GUARDED attribute in the PMD directly
In reply to Christophe LeroyOn the 8xx, the GUARDED attribute of the pages is managed in the L1 entry, therefore to avoid having to copy it into L1 entry at each TLB miss, we set it in the PMD.
--- arch/powerpc/include/asm/nohash/32/pte-8xx.h | 3 ++- arch/powerpc/kernel/head_8xx.S | 18 +++++++----------- arch/powerpc/platforms/Kconfig.cputype | 1 + 3 files changed, 10 insertions(+), 12 deletions(-)
arch/powerpc/include/asm/nohash/32/pte-8xx.h +2 -1
@@ -47,10 +47,11 @@#define _PAGE_RO 0x0600/* Supervisor RO, User no access */ #define _PMD_PRESENT 0x0001 -#define _PMD_BAD 0x0fd0 +#define _PMD_BAD 0x0fc0#define _PMD_PAGE_MASK 0x000c#define _PMD_PAGE_8M 0x000c#define _PMD_PAGE_512K 0x0004 +#define _PMD_GUARDED 0x0010#define _PMD_USER 0x0020/* APG 1 */ /* Until my rework is finished, 8xx still needs atomic PTE updates */
arch/powerpc/kernel/head_8xx.S +7 -11
@@ -345,6 +345,10 @@ _ENTRY(ITLBMiss_cmp) rlwinm r10, r10, 32 - (PAGE_SHIFT - 2), 32 - PAGE_SHIFT, 29 #ifdef CONFIG_HUGETLB_PAGE mtcr r11 +#endif + /* Load the MI_TWC with the attributes for this "segment." */ + mtspr SPRN_MI_TWC, r11 /* Set segment attributes */ +#ifdef CONFIG_HUGETLB_PAGE bt- 28, 10f /* bit 28 = Large page (8M) */ bt- 29, 20f /* bit 29 = Large page (8M or 512k) */ #endif
@@ -354,8 +358,6 @@ _ENTRY(ITLBMiss_cmp) #if defined(ITLB_MISS_KERNEL) || defined(CONFIG_HUGETLB_PAGE) mtcr r12 #endif - /* Load the MI_TWC with the attributes for this "segment." */ - mtspr SPRN_MI_TWC, r11 /* Set segment attributes */ #ifdef CONFIG_SWAP rlwinm r11, r10, 32-5, _PAGE_PRESENT
@@ -457,6 +459,9 @@ _ENTRY(DTLBMiss_jmp) rlwinm r10, r10, 32 - (PAGE_SHIFT - 2), 32 - PAGE_SHIFT, 29 #ifdef CONFIG_HUGETLB_PAGE mtcr r11 +#endif + mtspr SPRN_MD_TWC, r11 +#ifdef CONFIG_HUGETLB_PAGE bt- 28, 10f /* bit 28 = Large page (8M) */ bt- 29, 20f /* bit 29 = Large page (8M or 512k) */ #endif
@@ -465,15 +470,6 @@ _ENTRY(DTLBMiss_jmp) 4: mtcr r12 - /* Insert the Guarded flag into the TWC from the Linux PTE. - * It is bit 27 of both the Linux PTE and the TWC (at least - * I got that right :-). It will be better when we can put - * this into the Linux pgd/pmd and load it in the operation - * above. - */ - rlwimi r11, r10, 0, _PAGE_GUARDED - mtspr SPRN_MD_TWC, r11 - /* Both _PAGE_ACCESSED and _PAGE_PRESENT has to be set. * We also need to know if the insn is a load/store, so: * Clear _PAGE_PRESENT and load that which will
arch/powerpc/platforms/Kconfig.cputype +1 -0
@@ -34,6 +34,7 @@ config PPC_8xx bool "Freescale 8xx" select FSL_SOC select SYS_SUPPORTS_HUGETLBFS + select PPC_GUARDED_PAGE_IN_PMD config 40x bool "AMCC 40x"
Signature · 1 line
2.13.3
PATCH v2 8/14 powerpc/8xx: Remove PTE_ATOMIC_UPDATES
In reply to Christophe Leroycommit 1bc54c03117b9 ("powerpc: rework 4xx PTE access and TLB miss") introduced non atomic PTE updates and started the work of removing PTE updates in TLB miss handlers, but kept PTE_ATOMIC_UPDATES for the 8xx with the following comment:
/* Until my rework is finished, 8xx still needs atomic PTE updates */
commit fe11dc3f9628e ("powerpc/8xx: Update TLB asm so it behaves as linux mm expects") removed all PTE updates done in TLB miss handlers
Therefore, atomic PTE updates are not needed anymore for the 8xx
--- arch/powerpc/include/asm/nohash/32/pte-8xx.h | 3 --- 1 file changed, 3 deletions(-)
arch/powerpc/include/asm/nohash/32/pte-8xx.h +0 -3
@@ -54,9 +54,6 @@#define _PMD_GUARDED 0x0010#define _PMD_USER 0x0020/* APG 1 */ -/* Until my rework is finished, 8xx still needs atomic PTE updates */ -#define PTE_ATOMIC_UPDATES 1 -#ifdef CONFIG_PPC_16K_PAGES #define _PAGE_PSIZE _PAGE_HUGE#endif
Signature · 1 line
2.13.3
PATCH v2 9/14 powerpc/mm: Use hardware assistance in TLB handlers on the 8xx
In reply to Christophe LeroyToday, on the 8xx the TLB handlers do SW tablewalk by doing all the calculation in ASM, in order to match with the Linux page table structure.
The 8xx offers hardware assistance which allows significant size reduction of the TLB handlers, hence also reduces the time spent in the handlers.
However, using this HW assistance implies some constraints on the page table structure:
- Regardless of the main page size used (4k or 16k), the level 1 table (PGD) contains 1024 entries and each PGD entry covers a 4Mbytes area which is managed by a level 2 table (PTE) containing also 1024 entries each describing a 4k page.
- 16k pages require 4 identifical entries in the L2 table
- 512k pages PTE have to be spread every 128 bytes in the L2 table
- 8M pages PTE are at the address pointed by the L1 entry and each 8M page require 2 identical entries in the PGD.
In order to use hardware assistance, this patch does the following modifications:
- Make PGD size independant of the main page size
- In 16k pages mode, redefine pte_t as a struct with 4 elements, and populate those 4 elements in __set_pte_at() and pte_update()
- Modify the TLB handlers to use HW assistance
- Adapt the size of the hugepage tables.
--- arch/powerpc/include/asm/hugetlb.h | 4 +- arch/powerpc/include/asm/nohash/32/pgtable.h | 16 +- arch/powerpc/include/asm/nohash/pgtable.h | 4 + arch/powerpc/include/asm/pgtable-types.h | 4 + arch/powerpc/kernel/head_8xx.S | 225 +++++++++------------------ arch/powerpc/mm/8xx_mmu.c | 10 +- arch/powerpc/mm/hugetlbpage.c | 12 ++ 7 files changed, 111 insertions(+), 164 deletions(-)
arch/powerpc/include/asm/hugetlb.h +3 -1
@@ -77,7 +77,9 @@ static inline pte_t *hugepte_offset(hugepd_t hpd, unsigned long addr,unsigned long idx =0 ;pte_t *dir =hugepd_page (hpd); -#ifndef CONFIG_PPC_FSL_BOOK3E +#ifdef CONFIG_PPC_8xx + idx = (addr & ((1UL << pdshift) -1 )) >>PAGE_SHIFT ; +#elif !defined(CONFIG_PPC_FSL_BOOK3E ) idx = (addr & ((1UL << pdshift) -1 )) >>hugepd_shift (hpd);#endif
arch/powerpc/include/asm/nohash/32/pgtable.h +15 -1
@@ -18,7 +18,11 @@ extern int icache_44x_need_flush;#endif /* __ASSEMBLY__ */ +#if defined(CONFIG_PPC_8xx) && defined(CONFIG_PPC_16K_PAGES ) +#define PTE_INDEX_SIZE (PTE_SHIFT - 2) +#else #define PTE_INDEX_SIZE PTE_SHIFT +#endif #define PMD_INDEX_SIZE 0#define PUD_INDEX_SIZE 0#define PGD_INDEX_SIZE (32 - PGDIR_SHIFT)
@@ -47,7 +51,11 @@ extern int icache_44x_need_flush;* -Matt */ /* PGDIR_SHIFT determines what a top-level page table entry can map */ +#ifdef CONFIG_PPC_8xx +#define PGDIR_SHIFT 22 +#else #define PGDIR_SHIFT (PAGE_SHIFT + PTE_INDEX_SIZE) +#endif #define PGDIR_SIZE (1UL << PGDIR_SHIFT)#define PGDIR_MASK (~(PGDIR_SIZE-1))
@@ -201,7 +209,13 @@ static inline unsigned long pte_update(pte_t *p, : "cc" );#else /* PTE_ATOMIC_UPDATES */ unsigned long old =pte_val (*p); - *p =__pte ((old & ~clr) | set); +unsigned long new = (old & ~clr) | set; + +#if defined(CONFIG_PPC_8xx) && defined(CONFIG_PPC_16K_PAGES ) + p->pte = p->pte1 = p->pte2 = p->pte3 = new; +#else + *p =__pte (new); +#endif #endif /* !PTE_ATOMIC_UPDATES */ #ifdef CONFIG_44x
arch/powerpc/include/asm/nohash/pgtable.h +4 -0
@@ -164,7 +164,11 @@ static inline void __set_pte_at(struct mm_struct *mm, unsigned long addr,/* Anything else just stores the PTE normally. That covers all 64-bit * cases, and 32-bit non-hash with 32-bit PTEs. */ +#if defined(CONFIG_PPC_8xx) && defined(CONFIG_PPC_16K_PAGES ) + ptep->pte = ptep->pte1 = ptep->pte2 = ptep->pte3 =pte_val (pte); +#else *ptep = pte; +#endif /* * With hardware tablewalk, a sync is needed to ensure that
arch/powerpc/include/asm/pgtable-types.h +4 -0
@@ -3,7 +3,11 @@#define _ASM_POWERPC_PGTABLE_TYPES_H/* PTE level */ +#if defined(CONFIG_PPC_8xx) && defined(CONFIG_PPC_16K_PAGES ) +typedef struct {pte_basic_t pte, pte1, pte2, pte3; }pte_t ; +#else typedef struct {pte_basic_t pte; }pte_t ; +#endif #define __pte (x) ((pte_t) { (x) })static inline pte_basic_t pte_val (pte_t x) {
arch/powerpc/kernel/head_8xx.S +71 -154
409 lines not on this page · raw message
arch/powerpc/mm/8xx_mmu.c +2 -8
@@ -173,8 +173,6 @@ void __init setup_initial_memory_limit(phys_addr_t first_memblock_base,*/ void set_context (unsigned long id,pgd_t *pgd) { -s16 offset = (s16)(__pa (swapper_pg_dir)); -#ifdef CONFIG_BDI_SWITCH pgd_t **ptr = *(pgd_t ***)(KERNELBASE +0xf0 );
@@ -184,12 +182,8 @@ void set_context(unsigned long id, pgd_t *pgd) *(ptr +1 ) = pgd;#endif -/* Register M_TW will contain base address of level 1 table minus the -* lower part of the kernel PGDIR base address, so that all accesses to -* level 1 table are done relative to lower part of kernel PGDIR base -* address. -*/ -mtspr (SPRN_M_TW ,__pa (pgd) - offset); +/* Register M_TWB will contain base address of level 1 table */ +mtspr (SPRN_M_TWB ,__pa (pgd));/* Update context */ mtspr (SPRN_M_CASID , id -1 );
arch/powerpc/mm/hugetlbpage.c +12 -0
@@ -63,7 +63,11 @@ static int __hugepte_alloc(struct mm_struct *mm, hugepd_t *hpdp, cachep = hugepte_cache; num_hugepd =1 << (pshift - pdshift); }else { +#ifdef CONFIG_PPC_8xx + cachep =PGT_CACHE (PTE_SHIFT ); +#else cachep =PGT_CACHE (pdshift - pshift); +#endif num_hugepd =1 ; }
@@ -328,7 +332,11 @@ static void free_hugepd_range(struct mmu_gather *tlb, hugepd_t *hpdp, int pdshifif (shift >= pdshift)hugepd_free (tlb, hugepte); else +#ifdef CONFIG_PPC_8xx +pgtable_free_tlb (tlb, hugepte,PTE_SHIFT ); +#else pgtable_free_tlb (tlb, hugepte, pdshift - shift); +#endif } static void hugetlb_free_pmd_range(struct mmu_gather *tlb, pud_t *pud,
@@ -696,7 +704,11 @@ static int __init hugetlbpage_init(void)* use pgt cache for hugepd. */ if (pdshift > shift) +#ifdef CONFIG_PPC_8xx +pgtable_cache_add (PTE_SHIFT ,NULL ); +#else pgtable_cache_add (pdshift - shift,NULL ); +#endif #if defined(CONFIG_PPC_FSL_BOOK3E ) || defined(CONFIG_PPC_8xx) else if (!hugepte_cache) {/*
Signature · 1 line
2.13.3
PATCH v2 10/14 powerpc/8xx: reunify TLB handler routines
In reply to Christophe LeroyEach handler must not exceed 64 instructions to fit into the main exception area.
Following the significant size reduction of TLB handler routines, the side handlers can be brought back close to the main part.
In the worst case:
Main part of ITLB handler is 45 insn, side part is 9 insn ==> total 54 Main part of DTLB handler is 37 insn, side part is 23 insn ==> total 60
--- arch/powerpc/kernel/head_8xx.S | 108 ++++++++++++++++++++--------------------- 1 file changed, 52 insertions(+), 56 deletions(-)
arch/powerpc/kernel/head_8xx.S +52 -56
@@ -388,6 +388,23 @@ _ENTRY(itlb_miss_perf) mfspr r11, SPRN_SPRG_SCRATCH1 rfi +#ifndef CONFIG_PIN_TLB_TEXT +ITLBMissLinear: + mtcr r11 + /* Set 8M byte page and mark it valid */ + li r11, MI_PS8MEG | MI_SVALID + mtspr SPRN_MI_TWC, r11 + rlwinm r10, r10, 20, 0x0f800000 /* 8xx supports max 256Mb RAM */ + ori r10, r10, 0xf0 | MI_SPS16K | _PAGE_PRIVILEGED | _PAGE_DIRTY | \ + _PAGE_PRESENT + mtspr SPRN_MI_RPN, r10 /* Update TLB entry */ + +_ENTRY(itlb_miss_exit_2) + mfspr r10, SPRN_SPRG_SCRATCH0 + mfspr r11, SPRN_SPRG_SCRATCH1 + rfi +#endif + . = 0x1200 DataStoreTLBMiss: mtspr SPRN_SPRG_SCRATCH0, r10
@@ -463,6 +480,41 @@ _ENTRY(dtlb_miss_perf) mfspr r11, SPRN_SPRG_SCRATCH1 rfi +DTLBMissIMMR: + mtcr r11 + /* Set 512k byte guarded page and mark it valid */ + li r10, MD_PS512K | MD_GUARDED | MD_SVALID + mtspr SPRN_MD_TWC, r10 + mfspr r10, SPRN_IMMR /* Get current IMMR */ + rlwinm r10, r10, 0, 0xfff80000 /* Get 512 kbytes boundary */ + ori r10, r10, 0xf0 | MD_SPS16K | _PAGE_PRIVILEGED | _PAGE_DIRTY | \ + _PAGE_PRESENT | _PAGE_NO_CACHE + mtspr SPRN_MD_RPN, r10 /* Update TLB entry */ + + li r11, RPN_PATTERN + mtspr SPRN_DAR, r11 /* Tag DAR */ +_ENTRY(dtlb_miss_exit_2) + mfspr r10, SPRN_SPRG_SCRATCH0 + mfspr r11, SPRN_SPRG_SCRATCH1 + rfi + +DTLBMissLinear: + mtcr r11 + /* Set 8M byte page and mark it valid */ + li r11, MD_PS8MEG | MD_SVALID + mtspr SPRN_MD_TWC, r11 + rlwinm r10, r10, 20, 0x0f800000 /* 8xx supports max 256Mb RAM */ + ori r10, r10, 0xf0 | MD_SPS16K | _PAGE_PRIVILEGED | _PAGE_DIRTY | \ + _PAGE_PRESENT + mtspr SPRN_MD_RPN, r10 /* Update TLB entry */ + + li r11, RPN_PATTERN + mtspr SPRN_DAR, r11 /* Tag DAR */ +_ENTRY(dtlb_miss_exit_3) + mfspr r10, SPRN_SPRG_SCRATCH0 + mfspr r11, SPRN_SPRG_SCRATCH1 + rfi + /* This is an instruction TLB error on the MPC8xx. This could be due * to many reasons, such as executing guarded memory or illegal instruction * addresses. There is nothing to do but handle a big time error fault.
@@ -565,62 +617,6 @@ InstructionBreakpoint: . = 0x2000 -/* - * Bottom part of DataStoreTLBMiss handlers for IMMR area and linear RAM. - * not enough space in the DataStoreTLBMiss area. - */ -DTLBMissIMMR: - mtcr r11 - /* Set 512k byte guarded page and mark it valid */ - li r10, MD_PS512K | MD_GUARDED | MD_SVALID - mtspr SPRN_MD_TWC, r10 - mfspr r10, SPRN_IMMR /* Get current IMMR */ - rlwinm r10, r10, 0, 0xfff80000 /* Get 512 kbytes boundary */ - ori r10, r10, 0xf0 | MD_SPS16K | _PAGE_PRIVILEGED | _PAGE_DIRTY | \ - _PAGE_PRESENT | _PAGE_NO_CACHE - mtspr SPRN_MD_RPN, r10 /* Update TLB entry */ - - li r11, RPN_PATTERN - mtspr SPRN_DAR, r11 /* Tag DAR */ -_ENTRY(dtlb_miss_exit_2) - mfspr r10, SPRN_SPRG_SCRATCH0 - mfspr r11, SPRN_SPRG_SCRATCH1 - rfi - -DTLBMissLinear: - mtcr r11 - /* Set 8M byte page and mark it valid */ - li r11, MD_PS8MEG | MD_SVALID - mtspr SPRN_MD_TWC, r11 - rlwinm r10, r10, 20, 0x0f800000 /* 8xx supports max 256Mb RAM */ - ori r10, r10, 0xf0 | MD_SPS16K | _PAGE_PRIVILEGED | _PAGE_DIRTY | \ - _PAGE_PRESENT - mtspr SPRN_MD_RPN, r10 /* Update TLB entry */ - - li r11, RPN_PATTERN - mtspr SPRN_DAR, r11 /* Tag DAR */ -_ENTRY(dtlb_miss_exit_3) - mfspr r10, SPRN_SPRG_SCRATCH0 - mfspr r11, SPRN_SPRG_SCRATCH1 - rfi - -#ifndef CONFIG_PIN_TLB_TEXT -ITLBMissLinear: - mtcr r11 - /* Set 8M byte page and mark it valid */ - li r11, MI_PS8MEG | MI_SVALID - mtspr SPRN_MI_TWC, r11 - rlwinm r10, r10, 20, 0x0f800000 /* 8xx supports max 256Mb RAM */ - ori r10, r10, 0xf0 | MI_SPS16K | _PAGE_PRIVILEGED | _PAGE_DIRTY | \ - _PAGE_PRESENT - mtspr SPRN_MI_RPN, r10 /* Update TLB entry */ - -_ENTRY(itlb_miss_exit_2) - mfspr r10, SPRN_SPRG_SCRATCH0 - mfspr r11, SPRN_SPRG_SCRATCH1 - rfi -#endif - /* This is the procedure to calculate the data EA for buggy dcbx,dcbi instructions * by decoding the registers used by the dcbx instruction and adding them. * DAR is set to the calculated address.
Signature · 1 line
2.13.3
PATCH v2 11/14 powerpc/8xx: Free up SPRN_SPRG_SCRATCH2
In reply to Christophe LeroyWe can now use SPRN_M_TW in the DAR Fixup code, freeing SPRN_SPRG_SCRATCH2
Then SPRN_SPRG_SCRATCH2 may be used for something else in the future.
--- arch/powerpc/kernel/head_8xx.S | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
arch/powerpc/kernel/head_8xx.S +4 -4
@@ -624,7 +624,7 @@ InstructionBreakpoint: /* define if you don't want to use self modifying code */ #define NO_SELF_MODIFYING_CODE FixupDAR:/* Entry point for dcbx workaround. */ - mtspr SPRN_SPRG_SCRATCH2, r10 + mtspr SPRN_M_TW, r10 /* fetch instruction from memory. */ mfspr r10, SPRN_SRR0 mtspr SPRN_MD_EPN, r10
@@ -668,7 +668,7 @@ _ENTRY(FixupDAR_cmp) beq+ 142f cmpwi cr0, r10, 1964 /* Is icbi? */ beq+ 142f -141: mfspr r10,SPRN_SPRG_SCRATCH2 +141: mfspr r10,SPRN_M_TW b DARFixed /* Nope, go back to normal TLB processing */ 200:
@@ -703,7 +703,7 @@ modified_instr: bne+ 143f subf r10,r0,r10 /* r10=r10-r0, only if reg RA is r0 */ 143: mtdar r10 /* store faulting EA in DAR */ - mfspr r10,SPRN_SPRG_SCRATCH2 + mfspr r10,SPRN_M_TW b DARFixed /* Go back to normal TLB handling */ #else mfctr r10
@@ -757,7 +757,7 @@ modified_instr: mfdar r11 mtctr r11 /* restore ctr reg from DAR */ mtdar r10 /* save fault EA to DAR */ - mfspr r10,SPRN_SPRG_SCRATCH2 + mfspr r10,SPRN_M_TW b DARFixed /* Go back to normal TLB handling */ /* special handling for r10,r11 since these are modified already */
Signature · 1 line
2.13.3
PATCH v2 12/14 powerpc/mm: Make pte_fragment_alloc() common to PPC32 and PPC64
In reply to Christophe LeroyIn order to allow the 8xx to handle pte_fragments, this patch makes it common to PPC32 and PPC64 by moving the related code to common files and by defining a new config item called CONFIG_NEED_PTE_FRAG
--- arch/powerpc/include/asm/mmu_context.h | 28 ++++++++++++++ arch/powerpc/include/asm/page.h | 2 +- arch/powerpc/mm/mmu_context_book3s64.c | 28 -------------- arch/powerpc/mm/pgtable.c | 67 ++++++++++++++++++++++++++++++++++ arch/powerpc/mm/pgtable_64.c | 67 ---------------------------------- arch/powerpc/platforms/Kconfig.cputype | 5 +++ 6 files changed, 101 insertions(+), 96 deletions(-)
arch/powerpc/include/asm/mmu_context.h +28 -0
@@ -262,5 +262,33 @@ static inline u64 pte_to_hpte_pkey_bits(u64 pteflags)#endif /* CONFIG_PPC_MEM_KEYS */ +#ifdef CONFIG_NEED_PTE_FRAG +static inline void destroy_pagetable_page (struct mm_struct *mm) +{ +int count; +void *pte_frag; +struct page *page; + + pte_frag = mm->context.pte_frag; +if (!pte_frag) +return ; + + page =virt_to_page (pte_frag); +/* drop all the pending references */ + count = ((unsigned long )pte_frag & ~PAGE_MASK ) >>PTE_FRAG_SIZE_SHIFT ; +/* We allow PTE_FRAG_NR fragments from a PTE page */ +if (page_ref_sub_and_test (page,PTE_FRAG_NR - count)) { +pgtable_page_dtor (page); +free_unref_page (page); + } +} + +#else +static inline void destroy_pagetable_page (struct mm_struct *mm) +{ +return ; +} +#endif +#endif /* __KERNEL__ */ #endif /* __ASM_POWERPC_MMU_CONTEXT_H */
arch/powerpc/include/asm/page.h +1 -1
@@ -344,7 +344,7 @@ struct vm_area_struct;*/ typedef pte_t *pgtable_t ;#else -#if defined(CONFIG_PPC_64K_PAGES ) && defined(CONFIG_PPC64 ) +#if defined(CONFIG_NEED_PTE_FRAG )typedef pte_t *pgtable_t ;#else typedef struct page *pgtable_t ;
arch/powerpc/mm/mmu_context_book3s64.c +0 -28
@@ -192,34 +192,6 @@ static void destroy_contexts(mm_context_t *ctx)spin_unlock (&mmu_context_lock); } -#ifdef CONFIG_PPC_64K_PAGES -static void destroy_pagetable_page (struct mm_struct *mm) -{ -int count; -void *pte_frag; -struct page *page; - - pte_frag = mm->context.pte_frag; -if (!pte_frag) -return ; - - page =virt_to_page (pte_frag); -/* drop all the pending references */ - count = ((unsigned long )pte_frag & ~PAGE_MASK ) >>PTE_FRAG_SIZE_SHIFT ; -/* We allow PTE_FRAG_NR fragments from a PTE page */ -if (page_ref_sub_and_test (page,PTE_FRAG_NR - count)) { -pgtable_page_dtor (page); -free_unref_page (page); - } -} - -#else -static inline void destroy_pagetable_page (struct mm_struct *mm) -{ -return ; -} -#endif -void destroy_context (struct mm_struct *mm) {#ifdef CONFIG_SPAPR_TCE_IOMMU
arch/powerpc/mm/pgtable.c +67 -0
70 lines not on this page · raw message
arch/powerpc/mm/pgtable_64.c +0 -67
73 lines not on this page · raw message
arch/powerpc/platforms/Kconfig.cputype +5 -0
@@ -337,6 +337,11 @@ config PPC_MM_SLICES default y if PPC_8xx && HUGETLB_PAGE default n +config NEED_PTE_FRAG + bool + default y if PPC_BOOK3S_64 && PPC_64K_PAGES + default n + config PPC_HAVE_PMU_SUPPORT bool
Signature · 1 line
2.13.3
PATCH v2 13/14 powerpc/mm: Use pte_fragment_alloc() on 8xx
In reply to Christophe LeroyIn 16k page size mode, the 8xx need only 4k for a page table.
This patch makes use of the pte_fragment functions in order to avoid wasting memory space
--- arch/powerpc/include/asm/mmu-8xx.h | 4 +++ arch/powerpc/include/asm/nohash/32/pgalloc.h | 44 +++++++++++++++++++++++++++- arch/powerpc/include/asm/nohash/32/pgtable.h | 10 ++++++- arch/powerpc/mm/mmu_context_nohash.c | 4 +++ arch/powerpc/mm/pgtable.c | 10 ++++++- arch/powerpc/mm/pgtable_32.c | 12 ++++++++ arch/powerpc/platforms/Kconfig.cputype | 1 + 7 files changed, 82 insertions(+), 3 deletions(-)
arch/powerpc/include/asm/mmu-8xx.h +4 -0
@@ -190,6 +190,10 @@ typedef struct {struct slice_mask mask_8m;# endif #endif +#ifdef CONFIG_NEED_PTE_FRAG +/* for 4K PTE fragment support */ +void *pte_frag; +#endif } mm_context_t;#define PHYS_IMMR_BASE (mfspr(SPRN_IMMR) & 0xfff80000)
arch/powerpc/include/asm/nohash/32/pgalloc.h +43 -1
78 lines not on this page · raw message
arch/powerpc/include/asm/nohash/32/pgtable.h +9 -1
@@ -20,6 +20,9 @@ extern int icache_44x_need_flush;#if defined(CONFIG_PPC_8xx) && defined(CONFIG_PPC_16K_PAGES )#define PTE_INDEX_SIZE (PTE_SHIFT - 2) +#define PTE_FRAG_NR 4 +#define PTE_FRAG_SIZE_SHIFT 12 +#define PTE_FRAG_SIZE (1UL << PTE_FRAG_SIZE_SHIFT)#else #define PTE_INDEX_SIZE PTE_SHIFT#endif
@@ -319,7 +322,7 @@ static inline int pte_young(pte_t pte)*/ #ifndef CONFIG_BOOKE #define pmd_page_vaddr (pmd) \ - ((unsigned long) __va(pmd_val(pmd) & PAGE_MASK)) + ((unsigned long) __va(pmd_val(pmd) & ~(PTE_TABLE_SIZE - 1)))#define pmd_page (pmd) \ pfn_to_page(pmd_val(pmd) >> PAGE_SHIFT)#else
@@ -342,9 +345,14 @@ static inline int pte_young(pte_t pte)#define pte_offset_kernel (dir, addr) \ (pmd_bad(*(dir)) ? NULL : (pte_t *)pmd_page_vaddr(*(dir)) + \ pte_index(addr)) +#ifdef CONFIG_NEED_PTE_FRAG +#define pte_offset_map (dir, addr) pte_offset_kernel(dir, addr) +#define pte_unmap (pte) do {} while (0) +#else #define pte_offset_map (dir, addr) \ ((pte_t *) kmap_atomic(pmd_page(*(dir))) + pte_index(addr))#define pte_unmap (pte) kunmap_atomic(pte) +#endif /* * Encode and decode a swap entry.
arch/powerpc/mm/mmu_context_nohash.c +4 -0
@@ -390,6 +390,9 @@ int init_new_context(struct task_struct *t, struct mm_struct *mm)#endif mm->context.id =MMU_NO_CONTEXT ; mm->context.active =0 ; +#ifdef CONFIG_NEED_PTE_FRAG + mm->context.pte_frag =NULL ; +#endif return 0 ; }
@@ -418,6 +421,7 @@ void destroy_context(struct mm_struct *mm) nr_free_contexts++; }raw_spin_unlock_irqrestore (&context_lock, flags); +destroy_pagetable_page (mm); }#ifdef CONFIG_SMP
arch/powerpc/mm/pgtable.c +9 -1
@@ -23,6 +23,7 @@#include <linux/kernel.h> #include <linux/gfp.h> +#include <linux/memblock.h> #include <linux/mm.h> #include <linux/percpu.h> #include <linux/hardirq.h>
@@ -320,10 +321,17 @@ static pte_t *__alloc_for_cache(struct mm_struct *mm, int kernel)return (pte_t *)ret; } -pte_t *pte_fragment_alloc (struct mm_struct *mm,unsigned long vmaddr,int kernel) +__ref pte_t *pte_fragment_alloc (struct mm_struct *mm,unsigned long vmaddr,int kernel) {pte_t *pte; +if (kernel && !slab_is_available ()) { + pte =__va (memblock_alloc (PTE_FRAG_SIZE ,PTE_FRAG_SIZE )); +if (pte) +memset (pte,0 ,PTE_FRAG_SIZE ); + +return pte; + } pte =get_from_cache (mm);if (pte)return pte;
arch/powerpc/mm/pgtable_32.c +12 -0
@@ -40,6 +40,17 @@extern char etext[], _stext[], _sinittext[], _einittext[]; +#ifdef CONFIG_NEED_PTE_FRAG +void pte_fragment_free (unsigned long *table,int kernel) +{ +struct page *page =virt_to_page (table); +if (put_page_testzero (page)) { +if (!kernel) +pgtable_page_dtor (page); +free_unref_page (page); + } +} +#else __ref pte_t *pte_alloc_one_kernel (struct mm_struct *mm,unsigned long address) {pte_t *pte;
@@ -69,6 +80,7 @@ pgtable_t pte_alloc_one(struct mm_struct *mm, unsigned long address) }return ptepage; } +#endif #ifdef CONFIG_PPC_GUARDED_PAGE_IN_PMD int __pte_alloc_kernel_g (pmd_t *pmd,unsigned long address)
arch/powerpc/platforms/Kconfig.cputype +1 -0
@@ -340,6 +340,7 @@ config PPC_MM_SLICES config NEED_PTE_FRAG bool default y if PPC_BOOK3S_64 && PPC_64K_PAGES + default y if PPC_8xx && PPC_16K_PAGES default n config PPC_HAVE_PMU_SUPPORT
Signature · 1 line
2.13.3
PATCH v2 14/14 powerpc/8xx: Move SW perf counters in first 32kb of memory
In reply to Christophe LeroyIn order to simplify time critical exceptions handling 8xx specific SW perf counters, this patch moves the counters into the begining of memory. This is possible because .text is readable and the counters are never modified outside of the handlers.
By doing this, we avoid having to set a second register with the upper part of the address of the counters.
--- arch/powerpc/kernel/head_8xx.S | 75 +++++++++++++++++++----------------------- 1 file changed, 34 insertions(+), 41 deletions(-)
arch/powerpc/kernel/head_8xx.S +34 -41
@@ -106,6 +106,23 @@ turn_on_mmu: mtspr SPRN_SRR0,r0 rfi /* enables MMU */ + +#ifdef CONFIG_PERF_EVENTS + .align 4 + + .globl itlb_miss_counter +itlb_miss_counter: + .space 4 + + .globl dtlb_miss_counter +dtlb_miss_counter: + .space 4 + + .globl instruction_counter +instruction_counter: + .space 4 +#endif + /* * Exception entry code. This code runs with address translation * turned off, i.e. using physical addresses.
@@ -368,25 +385,20 @@ _ENTRY(ITLBMiss_cmp) mtspr SPRN_MI_RPN, r10 /* Update TLB entry */ /* Restore registers */ -_ENTRY(itlb_miss_exit_1) - mfspr r10, SPRN_SPRG_SCRATCH0 #if defined(ITLB_MISS_KERNEL) || defined(CONFIG_SWAP) mfspr r11, SPRN_SPRG_SCRATCH1 #endif +_ENTRY(itlb_miss_exit_1) + mfspr r10, SPRN_SPRG_SCRATCH0 rfi #ifdef CONFIG_PERF_EVENTS _ENTRY(itlb_miss_perf) -#if !defined(ITLB_MISS_KERNEL) && !defined(CONFIG_SWAP) - mtspr SPRN_SPRG_SCRATCH1, r11 -#endif - lis r10, (itlb_miss_counter - PAGE_OFFSET)@ha - lwz r11, (itlb_miss_counter - PAGE_OFFSET)@l(r10) - addi r11, r11, 1 - stw r11, (itlb_miss_counter - PAGE_OFFSET)@l(r10) -#endif + lwz r10, (itlb_miss_counter - PAGE_OFFSET)@l(0) + addi r10, r10, 1 + stw r10, (itlb_miss_counter - PAGE_OFFSET)@l(0) mfspr r10, SPRN_SPRG_SCRATCH0 - mfspr r11, SPRN_SPRG_SCRATCH1 rfi +#endif #ifndef CONFIG_PIN_TLB_TEXT ITLBMissLinear:
@@ -399,9 +411,9 @@ ITLBMissLinear: _PAGE_PRESENT mtspr SPRN_MI_RPN, r10 /* Update TLB entry */ + mfspr r11, SPRN_SPRG_SCRATCH1 _ENTRY(itlb_miss_exit_2) mfspr r10, SPRN_SPRG_SCRATCH0 - mfspr r11, SPRN_SPRG_SCRATCH1 rfi #endif
@@ -465,20 +477,18 @@ _ENTRY(DTLBMiss_jmp) /* Restore registers */ mtspr SPRN_DAR, r11 /* Tag DAR */ + mfspr r11, SPRN_SPRG_SCRATCH1 _ENTRY(dtlb_miss_exit_1) mfspr r10, SPRN_SPRG_SCRATCH0 - mfspr r11, SPRN_SPRG_SCRATCH1 rfi #ifdef CONFIG_PERF_EVENTS _ENTRY(dtlb_miss_perf) - lis r10, (dtlb_miss_counter - PAGE_OFFSET)@ha - lwz r11, (dtlb_miss_counter - PAGE_OFFSET)@l(r10) - addi r11, r11, 1 - stw r11, (dtlb_miss_counter - PAGE_OFFSET)@l(r10) -#endif + lwz r10, (dtlb_miss_counter - PAGE_OFFSET)@l(0) + addi r10, r10, 1 + stw r10, (dtlb_miss_counter - PAGE_OFFSET)@l(0) mfspr r10, SPRN_SPRG_SCRATCH0 - mfspr r11, SPRN_SPRG_SCRATCH1 rfi +#endif DTLBMissIMMR: mtcr r11
@@ -493,9 +503,9 @@ DTLBMissIMMR: li r11, RPN_PATTERN mtspr SPRN_DAR, r11 /* Tag DAR */ + mfspr r11, SPRN_SPRG_SCRATCH1 _ENTRY(dtlb_miss_exit_2) mfspr r10, SPRN_SPRG_SCRATCH0 - mfspr r11, SPRN_SPRG_SCRATCH1 rfi DTLBMissLinear:
@@ -510,9 +520,9 @@ DTLBMissLinear: li r11, RPN_PATTERN mtspr SPRN_DAR, r11 /* Tag DAR */ + mfspr r11, SPRN_SPRG_SCRATCH1 _ENTRY(dtlb_miss_exit_3) mfspr r10, SPRN_SPRG_SCRATCH0 - mfspr r11, SPRN_SPRG_SCRATCH1 rfi /* This is an instruction TLB error on the MPC8xx. This could be due
@@ -598,16 +608,13 @@ DataBreakpoint: . = 0x1d00 InstructionBreakpoint: mtspr SPRN_SPRG_SCRATCH0, r10 - mtspr SPRN_SPRG_SCRATCH1, r11 - lis r10, (instruction_counter - PAGE_OFFSET)@ha - lwz r11, (instruction_counter - PAGE_OFFSET)@l(r10) - addi r11, r11, -1 - stw r11, (instruction_counter - PAGE_OFFSET)@l(r10) + lwz r10, (instruction_counter - PAGE_OFFSET)@l(0) + addi r10, r10, -1 + stw r10, (instruction_counter - PAGE_OFFSET)@l(0) lis r10, 0xffff ori r10, r10, 0x01 mtspr SPRN_COUNTA, r10 mfspr r10, SPRN_SPRG_SCRATCH0 - mfspr r11, SPRN_SPRG_SCRATCH1 rfi #else EXCEPTION(0x1d00, Trap_1d, unknown_exception, EXC_XFER_EE)
@@ -966,17 +973,3 @@ swapper_pg_dir: */ abatron_pteptrs: .space 8 - -#ifdef CONFIG_PERF_EVENTS - .globl itlb_miss_counter -itlb_miss_counter: - .space 4 - - .globl dtlb_miss_counter -dtlb_miss_counter: - .space 4 - - .globl instruction_counter -instruction_counter: - .space 4 -#endif
Signature · 1 line
2.13.3