mm: introduce VM_MIXEDMAP
Subsystems (3)
A message that is not in the archive (sent off-list or not yet received): [email protected]
PATCH 2/8 mm: introduce VM_MIXEDMAP
Introduce a new type of mapping, VM_MIXEDMAP. This is unlike VM_PFNMAP in that it can support COW mappings of arbitrary ranges including ranges without struct page (PFNMAP can only support COW in those cases where the un-COW-ed translations are mapped linearly in the virtual address).
VM_MIXEDMAP achieves this by refcounting all pfn_valid pages, and not refcounting !pfn_valid pages (which is not an option for VM_PFNMAP, because it needs to avoid refcounting pfn_valid pages eg. for /dev/mem mappings).
(Needs Jared's SOB)
--- include/linux/mm.h | 1 mm/memory.c | 79 ++++++++++++++++++++++++++++++++++++++--------------- 2 files changed, 59 insertions(+), 21 deletions(-)
include/linux/mm.h +1 -0
@@ -107,6 +107,7 @@ extern unsigned int kobjsize(const void#define VM_ALWAYSDUMP 0x04000000/* Always include in core dumps */ #define VM_CAN_NONLINEAR 0x08000000/* Has ->fault & does nonlinear pages */ +#define VM_MIXEDMAP 0x10000000/* Can contain "struct page" and pure PFN pages */ #ifndef VM_STACK_DEFAULT_FLAGS /* arch can override this */ #define VM_STACK_DEFAULT_FLAGS VM_DATA_DEFAULT_FLAGS
mm/memory.c +58 -21
117 lines not on this page · raw message
Signature · 5 lines
-- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 8 lines · On Thursday 14 February 2008 17:16, [email protected] wrote:
Introduce a new type of mapping,
VM_MIXEDMAP. This is unlikeVM_PFNMAPin that it can support COW mappings of arbitrary ranges including ranges without struct page (PFNMAP can only support COW in those cases where the un-COW-ed translations are mapped linearly in the virtual address).
VM_MIXEDMAPachieves this by refcounting allpfn_validpages, and not refcounting !pfn_valid pages (which is not an option forVM_PFNMAP, because it needs to avoid refcountingpfn_validpages eg. for/dev/memmappings).
BTW. sorry quilt mail outsmarted me here... I had just intended to send these privately to Badari, Dave, and the other powerpc guys. I wasn't intending this for you Linus, or linux-mm etc.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 1 line
(Needs Jared's SOB)
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 3/8 mm: introduce pte_special pte bit
In reply to a message not in the archives390 for one, cannot implement VM_MIXEDMAP with pfn_valid, due to their memory model (which is more dynamic than most). Instead, they had proposed to implement it with an additional path through vm_normal_page(), using a bit in the pte to determine whether or not the page should be refcounted:
vm_normal_page()
{
...
if (unlikely(vma->vm_flags & (VM_PFNMAP|VM_MIXEDMAP))) {
if (vma->vm_flags & VM_MIXEDMAP) {
#ifdef s390
if (!mixedmap_refcount_pte(pte))
return NULL;
#else
if (!pfn_valid(pfn))
return NULL;
#endif
goto out;
}
...
}This is fine, however if we are allowed to use a bit in the pte to determine refcountedness, we can use that to _completely_ replace all the vma based schemes. So instead of adding more cases to the already complex vma-based scheme, we can have a clearly seperate and simple pte-based scheme (and get slightly better code generation in the process):
vm_normal_page()
{
#ifdef s390 if (!mixedmap_refcount_pte(pte)) return NULL; return pte_page(pte); #else ...
#endif
}
And finally, we may rather make this concept usable by any architecture rather than making it s390 only, so implement a new type of pte state for this. Unfortunately the old vma based code must stay, because some architectures may not be able to spare pte bits. This makes vm_normal_page a little bit more ugly than we would like, but the 2 cases are clearly seperate.
So introduce a pte_special pte state, and use it in mm/memory.c. It is currently a noop for all architectures, so this doesn't actually result in any compiled code changes to mm/memory.o.
--- include/asm-alpha/pgtable.h | 2 include/asm-avr32/pgtable.h | 8 ++ include/asm-cris/pgtable.h | 2 include/asm-frv/pgtable.h | 2 include/asm-ia64/pgtable.h | 3 + include/asm-m32r/pgtable.h | 10 +++ include/asm-m68k/motorola_pgtable.h | 2 include/asm-m68k/sun3_pgtable.h | 2 include/asm-mips/pgtable.h | 2 include/asm-parisc/pgtable.h | 2 include/asm-powerpc/pgtable-ppc32.h | 3 + include/asm-powerpc/pgtable-ppc64.h | 3 + include/asm-ppc/pgtable.h | 3 + include/asm-s390/pgtable.h | 10 +++ include/asm-sh64/pgtable.h | 2 include/asm-sparc/pgtable.h | 7 ++ include/asm-sparc64/pgtable.h | 10 +++ include/asm-um/pgtable.h | 10 +++ include/asm-x86/pgtable_32.h | 2 include/asm-x86/pgtable_64.h | 2 include/asm-xtensa/pgtable.h | 4 + include/linux/mm.h | 3 - mm/memory.c | 98 +++++++++++++++++++----------------- 23 files changed, 147 insertions(+), 45 deletions(-)
include/asm-um/pgtable.h +10 -0
@@ -173,6 +173,11 @@ static inline int pte_newprot(pte_t pte)return (pte_present (pte) && (pte_get_bits (pte, _PAGE_NEWPROT))); } +static inline int pte_special (pte_t pte) +{ +return 0 ; +} +/* * ================================= * Flags setting section.
@@ -241,6 +246,11 @@ static inline pte_t pte_mknewpage(pte_treturn (pte); } +static inline pte_t pte_mkspecial (pte_t pte) +{ +return (pte); +} +static inline void set_pte (pte_t *pteptr,pte_t pteval) {pte_copy (*pteptr, pteval);
include/linux/mm.h +2 -1
@@ -712,7 +712,8 @@ struct zap_details {unsigned long truncate_count;/* Compare vm_truncate_count */ }; -struct page *vm_normal_page (struct vm_area_struct *,unsigned long ,pte_t ); +struct page *vm_normal_page (struct vm_area_struct *vma,unsigned long addr,pte_t pte); +unsigned long zap_page_range (struct vm_area_struct *vma,unsigned long address,unsigned long size,struct zap_details *); unsigned long unmap_vmas(struct mmu_gather **tlb,
mm/memory.c +54 -44
156 lines not on this page · raw message
include/asm-alpha/pgtable.h +2 -0
@@ -268,6 +268,7 @@ extern inline int pte_write(pte_t pte)extern inline int pte_dirty (pte_t pte) {return pte_val (pte) & _PAGE_DIRTY; }extern inline int pte_young (pte_t pte) {return pte_val (pte) & _PAGE_ACCESSED; }extern inline int pte_file (pte_t pte) {return pte_val (pte) & _PAGE_FILE; } +extern inline int pte_special (pte_t pte) {return 0 ; }extern inline pte_t pte_wrprotect (pte_t pte) {pte_val (pte) |= _PAGE_FOW;return pte; }extern inline pte_t pte_mkclean (pte_t pte) {pte_val (pte) &= ~(__DIRTY_BITS);return pte; }
@@ -275,6 +276,7 @@ extern inline pte_t pte_mkold(pte_t pte)extern inline pte_t pte_mkwrite (pte_t pte) {pte_val (pte) &= ~_PAGE_FOW;return pte; }extern inline pte_t pte_mkdirty (pte_t pte) {pte_val (pte) |= __DIRTY_BITS;return pte; }extern inline pte_t pte_mkyoung (pte_t pte) {pte_val (pte) |= __ACCESS_BITS;return pte; } +extern inline pte_t pte_mkspecial (pte_t pte) {return pte; }#define PAGE_DIR_OFFSET (tsk,address) pgd_offset((tsk),(address))
include/asm-avr32/pgtable.h +8 -0
@@ -211,6 +211,10 @@ static inline int pte_young(pte_t pte) {return pte_val (pte) & _PAGE_ACCESSED; } +static inline int pte_special (pte_t pte) +{ +return 0 ; +}/* * The following only work if pte_present() is not true.
@@ -251,6 +255,10 @@ static inline pte_t pte_mkyoung(pte_t ptset_pte (&pte,__pte (pte_val (pte) | _PAGE_ACCESSED));return pte; } +static inline pte_t pte_mkspecial (pte_t pte) +{ +return pte; +}#define pmd_none (x) (!pmd_val(x))#define pmd_present (x) (pmd_val(x) & _PAGE_PRESENT)
include/asm-cris/pgtable.h +2 -0
@@ -115,6 +115,7 @@ static inline int pte_write(pte_t pte)static inline int pte_dirty (pte_t pte) {return pte_val (pte) & _PAGE_MODIFIED; }static inline int pte_young (pte_t pte) {return pte_val (pte) & _PAGE_ACCESSED; }static inline int pte_file (pte_t pte) {return pte_val (pte) & _PAGE_FILE; } +static inline int pte_special (pte_t pte) {return 0 ; }static inline pte_t pte_wrprotect (pte_t pte) {
@@ -162,6 +163,7 @@ static inline pte_t pte_mkyoung(pte_t pt }return pte; } +static inline pte_t pte_mkspecial (pte_t pte) {return pte; }/* * Conversion functions: convert a page and protection to a page entry,
include/asm-frv/pgtable.h +2 -0
@@ -380,6 +380,7 @@ static inline pmd_t *pmd_offset(pud_t *dstatic inline int pte_dirty (pte_t pte) {return (pte).pte & _PAGE_DIRTY; }static inline int pte_young (pte_t pte) {return (pte).pte & _PAGE_ACCESSED; }static inline int pte_write (pte_t pte) {return !((pte).pte & _PAGE_WP); } +static inline int pte_special (pte_t pte) {return 0 ; }static inline pte_t pte_mkclean (pte_t pte) { (pte).pte &= ~_PAGE_DIRTY;return pte; }static inline pte_t pte_mkold (pte_t pte) { (pte).pte &= ~_PAGE_ACCESSED;return pte; }
@@ -387,6 +388,7 @@ static inline pte_t pte_wrprotect(pte_tstatic inline pte_t pte_mkdirty (pte_t pte) { (pte).pte |= _PAGE_DIRTY;return pte; }static inline pte_t pte_mkyoung (pte_t pte) { (pte).pte |= _PAGE_ACCESSED;return pte; }static inline pte_t pte_mkwrite (pte_t pte) { (pte).pte &= ~_PAGE_WP;return pte; } +static inline pte_t pte_mkspecial (pte_t pte) {return pte; }static inline int ptep_test_and_clear_young (struct vm_area_struct *vma,unsigned long addr,pte_t *ptep) {
include/asm-ia64/pgtable.h +3 -0
@@ -302,6 +302,8 @@ ia64_phys_addr_valid (unsigned long addr#define pte_dirty (pte) ((pte_val(pte) & _PAGE_D) != 0)#define pte_young (pte) ((pte_val(pte) & _PAGE_A) != 0)#define pte_file (pte) ((pte_val(pte) & _PAGE_FILE) != 0) +#define pte_special (pte) 0 +/* * Note: we convert AR_RWX to AR_RX and AR_RW to AR_R by clearing the 2nd bit in the * access rights:
@@ -313,6 +315,7 @@ ia64_phys_addr_valid (unsigned long addr#define pte_mkclean (pte) (__pte(pte_val(pte) & ~_PAGE_D))#define pte_mkdirty (pte) (__pte(pte_val(pte) | _PAGE_D))#define pte_mkhuge (pte) (__pte(pte_val(pte))) +#define pte_mkspecial (pte) (pte)/* * Because ia64's Icache and Dcache is not coherent (on a cpu), we need to
include/asm-m32r/pgtable.h +10 -0
@@ -214,6 +214,11 @@ static inline int pte_file(pte_t pte)return pte_val (pte) & _PAGE_FILE; } +static inline int pte_special (pte_t pte) +{ +return 0 ; +} +static inline pte_t pte_mkclean (pte_t pte) {pte_val (pte) &= ~_PAGE_DIRTY;
@@ -250,6 +255,11 @@ static inline pte_t pte_mkwrite(pte_t ptreturn pte; } +static inline pte_t pte_mkspecial (pte_t pte) +{ +return pte; +} +static inline int ptep_test_and_clear_young (struct vm_area_struct *vma,unsigned long addr,pte_t *ptep) {return test_and_clear_bit (_PAGE_BIT_ACCESSED, ptep);
include/asm-m68k/motorola_pgtable.h +2 -0
@@ -168,6 +168,7 @@ static inline int pte_write(pte_t pte)static inline int pte_dirty (pte_t pte) {return pte_val (pte) & _PAGE_DIRTY; }static inline int pte_young (pte_t pte) {return pte_val (pte) & _PAGE_ACCESSED; }static inline int pte_file (pte_t pte) {return pte_val (pte) & _PAGE_FILE; } +static inline int pte_special (pte_t pte) {return 0 ; }static inline pte_t pte_wrprotect (pte_t pte) {pte_val (pte) |= _PAGE_RONLY;return pte; }static inline pte_t pte_mkclean (pte_t pte) {pte_val (pte) &= ~_PAGE_DIRTY;return pte; }
@@ -185,6 +186,7 @@ static inline pte_t pte_mkcache(pte_t ptpte_val (pte) = (pte_val (pte) & _CACHEMASK040) | m68k_supervisor_cachemode;return pte; } +static inline pte_t pte_mkspecial (pte_t pte) {return pte; }#define PAGE_DIR_OFFSET (tsk,address) pgd_offset((tsk),(address))
include/asm-m68k/sun3_pgtable.h +2 -0
@@ -169,6 +169,7 @@ static inline int pte_write(pte_t pte)static inline int pte_dirty (pte_t pte) {return pte_val (pte) &SUN3_PAGE_MODIFIED ; }static inline int pte_young (pte_t pte) {return pte_val (pte) &SUN3_PAGE_ACCESSED ; }static inline int pte_file (pte_t pte) {return pte_val (pte) &SUN3_PAGE_ACCESSED ; } +static inline int pte_special (pte_t pte) {return 0 ; }static inline pte_t pte_wrprotect (pte_t pte) {pte_val (pte) &= ~SUN3_PAGE_WRITEABLE ;return pte; }static inline pte_t pte_mkclean (pte_t pte) {pte_val (pte) &= ~SUN3_PAGE_MODIFIED ;return pte; }
@@ -181,6 +182,7 @@ static inline pte_t pte_mknocache(pte_t//static inline pte_t pte_mkcache(pte_t pte) { pte_val(pte) &= SUN3_PAGE_NOCACHE; return pte; } // until then, use: static inline pte_t pte_mkcache (pte_t pte) {return pte; } +static inline pte_t pte_mkspecial (pte_t pte) {return pte; }extern pgd_t swapper_pg_dir[PTRS_PER_PGD ];extern pgd_t kernel_pg_dir[PTRS_PER_PGD ];
include/asm-mips/pgtable.h +2 -0
@@ -285,6 +285,8 @@ static inline pte_t pte_mkyoung(pte_t pt return pte; }#endif +static inline int pte_special (pte_t pte) {return 0 ; } +static inline pte_t pte_mkspecial (pte_t pte) {return pte; }/* * Macro to make mark a page protection value as "uncacheable". Note
include/asm-parisc/pgtable.h +2 -0
@@ -331,6 +331,7 @@ static inline int pte_dirty(pte_t pte)static inline int pte_young (pte_t pte) {return pte_val (pte) & _PAGE_ACCESSED; }static inline int pte_write (pte_t pte) {return pte_val (pte) & _PAGE_WRITE; }static inline int pte_file (pte_t pte) {return pte_val (pte) & _PAGE_FILE; } +static inline int pte_special (pte_t pte) {return 0 ; }static inline pte_t pte_mkclean (pte_t pte) {pte_val (pte) &= ~_PAGE_DIRTY;return pte; }static inline pte_t pte_mkold (pte_t pte) {pte_val (pte) &= ~_PAGE_ACCESSED;return pte; }
@@ -338,6 +339,7 @@ static inline pte_t pte_wrprotect(pte_tstatic inline pte_t pte_mkdirty (pte_t pte) {pte_val (pte) |= _PAGE_DIRTY;return pte; }static inline pte_t pte_mkyoung (pte_t pte) {pte_val (pte) |= _PAGE_ACCESSED;return pte; }static inline pte_t pte_mkwrite (pte_t pte) {pte_val (pte) |= _PAGE_WRITE;return pte; } +static inline pte_t pte_mkspecial (pte_t pte) {return pte; }/* * Conversion functions: convert a page and protection to a page entry,
include/asm-powerpc/pgtable-ppc32.h +3 -0
@@ -514,6 +514,7 @@ static inline int pte_write(pte_t pte)static inline int pte_dirty (pte_t pte) {return pte_val (pte) & _PAGE_DIRTY; }static inline int pte_young (pte_t pte) {return pte_val (pte) & _PAGE_ACCESSED; }static inline int pte_file (pte_t pte) {return pte_val (pte) & _PAGE_FILE; } +static inline int pte_special (pte_t pte) {return 0 ; }static inline void pte_uncache (pte_t pte) {pte_val (pte) |= _PAGE_NO_CACHE; }static inline void pte_cache (pte_t pte) {pte_val (pte) &= ~_PAGE_NO_CACHE; }
@@ -531,6 +532,8 @@ static inline pte_t pte_mkdirty(pte_t ptpte_val (pte) |= _PAGE_DIRTY;return pte; }static inline pte_t pte_mkyoung (pte_t pte) {pte_val (pte) |= _PAGE_ACCESSED;return pte; } +static inline pte_t pte_mkspecial (pte_t pte) { +return pte; }static inline pte_t pte_modify (pte_t pte,pgprot_t newprot) {
include/asm-powerpc/pgtable-ppc64.h +3 -0
@@ -239,6 +239,7 @@ static inline int pte_write(pte_t pte) {static inline int pte_dirty (pte_t pte) {return pte_val (pte) & _PAGE_DIRTY;}static inline int pte_young (pte_t pte) {return pte_val (pte) & _PAGE_ACCESSED;}static inline int pte_file (pte_t pte) {return pte_val (pte) & _PAGE_FILE;} +static inline int pte_special (pte_t pte) {return 0 ; }static inline void pte_uncache (pte_t pte) {pte_val (pte) |= _PAGE_NO_CACHE; }static inline void pte_cache (pte_t pte) {pte_val (pte) &= ~_PAGE_NO_CACHE; }
@@ -257,6 +258,8 @@ static inline pte_t pte_mkyoung(pte_t ptpte_val (pte) |= _PAGE_ACCESSED;return pte; }static inline pte_t pte_mkhuge (pte_t pte) {return pte; } +static inline pte_t pte_mkspecial (pte_t pte) { +return pte; }/* Atomic PTE updates */ static inline unsigned long pte_update(struct mm_struct *mm,
include/asm-ppc/pgtable.h +3 -0
@@ -491,6 +491,7 @@ static inline int pte_write(pte_t pte)static inline int pte_dirty (pte_t pte) {return pte_val (pte) & _PAGE_DIRTY; }static inline int pte_young (pte_t pte) {return pte_val (pte) & _PAGE_ACCESSED; }static inline int pte_file (pte_t pte) {return pte_val (pte) & _PAGE_FILE; } +static inline int pte_special (pte_t pte) {return 0 ; }static inline void pte_uncache (pte_t pte) {pte_val (pte) |= _PAGE_NO_CACHE; }static inline void pte_cache (pte_t pte) {pte_val (pte) &= ~_PAGE_NO_CACHE; }
@@ -508,6 +509,8 @@ static inline pte_t pte_mkdirty(pte_t ptpte_val (pte) |= _PAGE_DIRTY;return pte; }static inline pte_t pte_mkyoung (pte_t pte) {pte_val (pte) |= _PAGE_ACCESSED;return pte; } +static inline pte_t pte_mkspecial (pte_t pte) { +return pte; }static inline pte_t pte_modify (pte_t pte,pgprot_t newprot) {
include/asm-s390/pgtable.h +10 -0
@@ -510,6 +510,11 @@ static inline int pte_file(pte_t pte)return (pte_val (pte) & mask) == _PAGE_TYPE_FILE; } +static inline int pte_special (pte_t pte) +{ +return 0 ; +} +#define __HAVE_ARCH_PTE_SAME#define pte_same (a,b) (pte_val(a) == pte_val(b))
@@ -663,6 +668,11 @@ static inline pte_t pte_mkyoung(pte_t ptreturn pte; } +static inline pte_t pte_mkspecial (pte_t pte) +{ +return pte; +} +#define __HAVE_ARCH_PTEP_TEST_AND_CLEAR_YOUNGstatic inline int ptep_test_and_clear_young (struct vm_area_struct *vma,unsigned long addr,pte_t *ptep)
include/asm-sparc/pgtable.h +7 -0
@@ -219,6 +219,11 @@ static inline int pte_file(pte_t pte)return pte_val (pte) &BTFIXUP_HALF (pte_filei); } +static inline int pte_special (pte_t pte) +{ +return 0 ; +} +/* */ BTFIXUPDEF_HALF (pte_wrprotecti)
@@ -251,6 +256,8 @@ BTFIXUPDEF_CALL_CONST(pte_t, pte_mkyoung#define pte_mkdirty (pte) BTFIXUP_CALL(pte_mkdirty)(pte)#define pte_mkyoung (pte) BTFIXUP_CALL(pte_mkyoung)(pte) +#define pte_mkspecial(pte_t pte) (pte) +#define pfn_pte (pfn, prot) mk_pte(pfn_to_page(pfn), prot) BTFIXUPDEF_CALL(unsigned long, pte_pfn, pte_t)
include/asm-sparc64/pgtable.h +10 -0
@@ -506,6 +506,11 @@ static inline pte_t pte_mkyoung(pte_t ptreturn __pte (pte_val (pte) | mask); } +static inline pte_t pte_mkspecial (pte_t pte) +{ +return pte; +} +static inline unsigned long pte_young (pte_t pte) {unsigned long mask;
@@ -608,6 +613,11 @@ static inline unsigned long pte_present(return val; } +static inline int pte_special (pte_t pte) +{ +return 0 ; +} +#define pmd_set (pmdp, ptep) \ (pmd_val(*(pmdp)) = (__pa((unsigned long) (ptep)) >> 11UL))#define pud_set (pudp, pmdp) \
include/asm-xtensa/pgtable.h +4 -0
@@ -212,6 +212,8 @@ static inline int pte_write(pte_t pte) {static inline int pte_dirty (pte_t pte) {return pte_val (pte) & _PAGE_DIRTY; }static inline int pte_young (pte_t pte) {return pte_val (pte) & _PAGE_ACCESSED; }static inline int pte_file (pte_t pte) {return pte_val (pte) & _PAGE_FILE; } +static inline int pte_special (pte_t pte) {return 0 ; } +static inline pte_t pte_wrprotect (pte_t pte) {pte_val (pte) &= ~(_PAGE_WRITABLE | _PAGE_HW_WRITE);return pte; }static inline pte_t pte_mkclean (pte_t pte)
@@ -224,6 +226,8 @@ static inline pte_t pte_mkyoung(pte_t pt {pte_val (pte) |= _PAGE_ACCESSED;return pte; }static inline pte_t pte_mkwrite (pte_t pte) {pte_val (pte) |= _PAGE_WRITABLE;return pte; } +static inline pte_t pte_mkspecial (pte_t pte) + {return pte; }/* * Conversion functions: convert a page and protection to a page entry,
Signature · 5 lines
-- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [patch 3/8] mm: introduce pte_special pte bit
Quoted 1 line · On Thu, 2008-02-14 at 17:17 +1100, [email protected] wrote:
plain text document attachment (mm-normal-pte-bit.patch)
updated version.
Thanks,
Badari
s390 for one, cannot implement VM_MIXEDMAP with pfn_valid, due to their memory model (which is more dynamic than most). Instead, they had proposed to implement it with an additional path through vm_normal_page(), using a bit in the pte to determine whether or not the page should be refcounted:
vm_normal_page()
{
...
if (unlikely(vma->vm_flags & (VM_PFNMAP|VM_MIXEDMAP))) {
if (vma->vm_flags & VM_MIXEDMAP) {
#ifdef s390
if (!mixedmap_refcount_pte(pte))
return NULL;
#else
if (!pfn_valid(pfn))
return NULL;
#endif
goto out;
}
...
}This is fine, however if we are allowed to use a bit in the pte to determine refcountedness, we can use that to _completely_ replace all the vma based schemes. So instead of adding more cases to the already complex vma-based scheme, we can have a clearly seperate and simple pte-based scheme (and get slightly better code generation in the process):
vm_normal_page()
{
#ifdef s390 if (!mixedmap_refcount_pte(pte)) return NULL; return pte_page(pte); #else ...
#endif
}
And finally, we may rather make this concept usable by any architecture rather than making it s390 only, so implement a new type of pte state for this. Unfortunately the old vma based code must stay, because some architectures may not be able to spare pte bits. This makes vm_normal_page a little bit more ugly than we would like, but the 2 cases are clearly seperate.
So introduce a pte_special pte state, and use it in mm/memory.c. It is currently a noop for all architectures, so this doesn't actually result in any compiled code changes to mm/memory.o.
--- include/asm-alpha/pgtable.h | 2 include/asm-avr32/pgtable.h | 8 ++ include/asm-cris/pgtable.h | 2 include/asm-frv/pgtable.h | 2 include/asm-ia64/pgtable.h | 3 + include/asm-m32r/pgtable.h | 10 +++ include/asm-m68k/motorola_pgtable.h | 2 include/asm-m68k/sun3_pgtable.h | 2 include/asm-mips/pgtable.h | 2 include/asm-parisc/pgtable.h | 2 include/asm-powerpc/pgtable-ppc32.h | 3 + include/asm-powerpc/pgtable-ppc64.h | 3 + include/asm-ppc/pgtable.h | 3 + include/asm-s390/pgtable.h | 10 +++ include/asm-sparc/pgtable.h | 7 ++ include/asm-sparc64/pgtable.h | 10 +++ include/asm-um/pgtable.h | 10 +++ include/asm-x86/pgtable.h | 2 include/asm-xtensa/pgtable.h | 4 + include/linux/mm.h | 3 - mm/memory.c | 98 +++++++++++++++++++----------------- 21 files changed, 143 insertions(+), 45 deletions(-)
include/asm-um/pgtable.h +10 -0
@@ -173,6 +173,11 @@ static inline int pte_newprot(pte_t pte)return (pte_present (pte) && (pte_get_bits (pte, _PAGE_NEWPROT))); } +static inline int pte_special (pte_t pte) +{ +return 0 ; +} +/* * ================================= * Flags setting section.
@@ -241,6 +246,11 @@ static inline pte_t pte_mknewpage(pte_treturn (pte); } +static inline pte_t pte_mkspecial (pte_t pte) +{ +return (pte); +} +static inline void set_pte (pte_t *pteptr,pte_t pteval) {pte_copy (*pteptr, pteval);
include/linux/mm.h +2 -1
@@ -712,7 +712,8 @@ struct zap_details {unsigned long truncate_count;/* Compare vm_truncate_count */ }; -struct page *vm_normal_page (struct vm_area_struct *,unsigned long ,pte_t ); +struct page *vm_normal_page (struct vm_area_struct *vma,unsigned long addr,pte_t pte); +unsigned long zap_page_range (struct vm_area_struct *vma,unsigned long address,unsigned long size,struct zap_details *); unsigned long unmap_vmas(struct mmu_gather **tlb,
mm/memory.c +54 -44
156 lines not on this page · raw message
include/asm-alpha/pgtable.h +2 -0
@@ -268,6 +268,7 @@ extern inline int pte_write(pte_t pte)extern inline int pte_dirty (pte_t pte) {return pte_val (pte) & _PAGE_DIRTY; }extern inline int pte_young (pte_t pte) {return pte_val (pte) & _PAGE_ACCESSED; }extern inline int pte_file (pte_t pte) {return pte_val (pte) & _PAGE_FILE; } +extern inline int pte_special (pte_t pte) {return 0 ; }extern inline pte_t pte_wrprotect (pte_t pte) {pte_val (pte) |= _PAGE_FOW;return pte; }extern inline pte_t pte_mkclean (pte_t pte) {pte_val (pte) &= ~(__DIRTY_BITS);return pte; }
@@ -275,6 +276,7 @@ extern inline pte_t pte_mkold(pte_t pte)extern inline pte_t pte_mkwrite (pte_t pte) {pte_val (pte) &= ~_PAGE_FOW;return pte; }extern inline pte_t pte_mkdirty (pte_t pte) {pte_val (pte) |= __DIRTY_BITS;return pte; }extern inline pte_t pte_mkyoung (pte_t pte) {pte_val (pte) |= __ACCESS_BITS;return pte; } +extern inline pte_t pte_mkspecial (pte_t pte) {return pte; }#define PAGE_DIR_OFFSET (tsk,address) pgd_offset((tsk),(address))
include/asm-avr32/pgtable.h +8 -0
@@ -211,6 +211,10 @@ static inline int pte_young(pte_t pte) {return pte_val (pte) & _PAGE_ACCESSED; } +static inline int pte_special (pte_t pte) +{ +return 0 ; +}/* * The following only work if pte_present() is not true.
@@ -251,6 +255,10 @@ static inline pte_t pte_mkyoung(pte_t ptset_pte (&pte,__pte (pte_val (pte) | _PAGE_ACCESSED));return pte; } +static inline pte_t pte_mkspecial (pte_t pte) +{ +return pte; +}#define pmd_none (x) (!pmd_val(x))#define pmd_present (x) (pmd_val(x) & _PAGE_PRESENT)
include/asm-cris/pgtable.h +2 -0
@@ -115,6 +115,7 @@ static inline int pte_write(pte_t pte)static inline int pte_dirty (pte_t pte) {return pte_val (pte) & _PAGE_MODIFIED; }static inline int pte_young (pte_t pte) {return pte_val (pte) & _PAGE_ACCESSED; }static inline int pte_file (pte_t pte) {return pte_val (pte) & _PAGE_FILE; } +static inline int pte_special (pte_t pte) {return 0 ; }static inline pte_t pte_wrprotect (pte_t pte) {
@@ -162,6 +163,7 @@ static inline pte_t pte_mkyoung(pte_t pt }return pte; } +static inline pte_t pte_mkspecial (pte_t pte) {return pte; }/* * Conversion functions: convert a page and protection to a page entry,
include/asm-frv/pgtable.h +2 -0
@@ -380,6 +380,7 @@ static inline pmd_t *pmd_offset(pud_t *dstatic inline int pte_dirty (pte_t pte) {return (pte).pte & _PAGE_DIRTY; }static inline int pte_young (pte_t pte) {return (pte).pte & _PAGE_ACCESSED; }static inline int pte_write (pte_t pte) {return !((pte).pte & _PAGE_WP); } +static inline int pte_special (pte_t pte) {return 0 ; }static inline pte_t pte_mkclean (pte_t pte) { (pte).pte &= ~_PAGE_DIRTY;return pte; }static inline pte_t pte_mkold (pte_t pte) { (pte).pte &= ~_PAGE_ACCESSED;return pte; }
@@ -387,6 +388,7 @@ static inline pte_t pte_wrprotect(pte_tstatic inline pte_t pte_mkdirty (pte_t pte) { (pte).pte |= _PAGE_DIRTY;return pte; }static inline pte_t pte_mkyoung (pte_t pte) { (pte).pte |= _PAGE_ACCESSED;return pte; }static inline pte_t pte_mkwrite (pte_t pte) { (pte).pte &= ~_PAGE_WP;return pte; } +static inline pte_t pte_mkspecial (pte_t pte) {return pte; }static inline int ptep_test_and_clear_young (struct vm_area_struct *vma,unsigned long addr,pte_t *ptep) {
include/asm-ia64/pgtable.h +3 -0
@@ -302,6 +302,8 @@ ia64_phys_addr_valid (unsigned long addr#define pte_dirty (pte) ((pte_val(pte) & _PAGE_D) != 0)#define pte_young (pte) ((pte_val(pte) & _PAGE_A) != 0)#define pte_file (pte) ((pte_val(pte) & _PAGE_FILE) != 0) +#define pte_special (pte) 0 +/* * Note: we convert AR_RWX to AR_RX and AR_RW to AR_R by clearing the 2nd bit in the * access rights:
@@ -313,6 +315,7 @@ ia64_phys_addr_valid (unsigned long addr#define pte_mkclean (pte) (__pte(pte_val(pte) & ~_PAGE_D))#define pte_mkdirty (pte) (__pte(pte_val(pte) | _PAGE_D))#define pte_mkhuge (pte) (__pte(pte_val(pte))) +#define pte_mkspecial (pte) (pte)/* * Because ia64's Icache and Dcache is not coherent (on a cpu), we need to
include/asm-m32r/pgtable.h +10 -0
@@ -214,6 +214,11 @@ static inline int pte_file(pte_t pte)return pte_val (pte) & _PAGE_FILE; } +static inline int pte_special (pte_t pte) +{ +return 0 ; +} +static inline pte_t pte_mkclean (pte_t pte) {pte_val (pte) &= ~_PAGE_DIRTY;
@@ -250,6 +255,11 @@ static inline pte_t pte_mkwrite(pte_t ptreturn pte; } +static inline pte_t pte_mkspecial (pte_t pte) +{ +return pte; +} +static inline int ptep_test_and_clear_young (struct vm_area_struct *vma,unsigned long addr,pte_t *ptep) {return test_and_clear_bit (_PAGE_BIT_ACCESSED, ptep);
include/asm-m68k/motorola_pgtable.h +2 -0
@@ -168,6 +168,7 @@ static inline int pte_write(pte_t pte)static inline int pte_dirty (pte_t pte) {return pte_val (pte) & _PAGE_DIRTY; }static inline int pte_young (pte_t pte) {return pte_val (pte) & _PAGE_ACCESSED; }static inline int pte_file (pte_t pte) {return pte_val (pte) & _PAGE_FILE; } +static inline int pte_special (pte_t pte) {return 0 ; }static inline pte_t pte_wrprotect (pte_t pte) {pte_val (pte) |= _PAGE_RONLY;return pte; }static inline pte_t pte_mkclean (pte_t pte) {pte_val (pte) &= ~_PAGE_DIRTY;return pte; }
@@ -185,6 +186,7 @@ static inline pte_t pte_mkcache(pte_t ptpte_val (pte) = (pte_val (pte) & _CACHEMASK040) | m68k_supervisor_cachemode;return pte; } +static inline pte_t pte_mkspecial (pte_t pte) {return pte; }#define PAGE_DIR_OFFSET (tsk,address) pgd_offset((tsk),(address))
include/asm-m68k/sun3_pgtable.h +2 -0
@@ -169,6 +169,7 @@ static inline int pte_write(pte_t pte)static inline int pte_dirty (pte_t pte) {return pte_val (pte) &SUN3_PAGE_MODIFIED ; }static inline int pte_young (pte_t pte) {return pte_val (pte) &SUN3_PAGE_ACCESSED ; }static inline int pte_file (pte_t pte) {return pte_val (pte) &SUN3_PAGE_ACCESSED ; } +static inline int pte_special (pte_t pte) {return 0 ; }static inline pte_t pte_wrprotect (pte_t pte) {pte_val (pte) &= ~SUN3_PAGE_WRITEABLE ;return pte; }static inline pte_t pte_mkclean (pte_t pte) {pte_val (pte) &= ~SUN3_PAGE_MODIFIED ;return pte; }
@@ -181,6 +182,7 @@ static inline pte_t pte_mknocache(pte_t//static inline pte_t pte_mkcache(pte_t pte) { pte_val(pte) &= SUN3_PAGE_NOCACHE; return pte; } // until then, use: static inline pte_t pte_mkcache (pte_t pte) {return pte; } +static inline pte_t pte_mkspecial (pte_t pte) {return pte; }extern pgd_t swapper_pg_dir[PTRS_PER_PGD ];extern pgd_t kernel_pg_dir[PTRS_PER_PGD ];
include/asm-mips/pgtable.h +2 -0
@@ -285,6 +285,8 @@ static inline pte_t pte_mkyoung(pte_t pt return pte; }#endif +static inline int pte_special (pte_t pte) {return 0 ; } +static inline pte_t pte_mkspecial (pte_t pte) {return pte; }/* * Macro to make mark a page protection value as "uncacheable". Note
include/asm-parisc/pgtable.h +2 -0
@@ -331,6 +331,7 @@ static inline int pte_dirty(pte_t pte)static inline int pte_young (pte_t pte) {return pte_val (pte) & _PAGE_ACCESSED; }static inline int pte_write (pte_t pte) {return pte_val (pte) & _PAGE_WRITE; }static inline int pte_file (pte_t pte) {return pte_val (pte) & _PAGE_FILE; } +static inline int pte_special (pte_t pte) {return 0 ; }static inline pte_t pte_mkclean (pte_t pte) {pte_val (pte) &= ~_PAGE_DIRTY;return pte; }static inline pte_t pte_mkold (pte_t pte) {pte_val (pte) &= ~_PAGE_ACCESSED;return pte; }
@@ -338,6 +339,7 @@ static inline pte_t pte_wrprotect(pte_tstatic inline pte_t pte_mkdirty (pte_t pte) {pte_val (pte) |= _PAGE_DIRTY;return pte; }static inline pte_t pte_mkyoung (pte_t pte) {pte_val (pte) |= _PAGE_ACCESSED;return pte; }static inline pte_t pte_mkwrite (pte_t pte) {pte_val (pte) |= _PAGE_WRITE;return pte; } +static inline pte_t pte_mkspecial (pte_t pte) {return pte; }/* * Conversion functions: convert a page and protection to a page entry,
include/asm-powerpc/pgtable-ppc32.h +3 -0
@@ -514,6 +514,7 @@ static inline int pte_write(pte_t pte)static inline int pte_dirty (pte_t pte) {return pte_val (pte) & _PAGE_DIRTY; }static inline int pte_young (pte_t pte) {return pte_val (pte) & _PAGE_ACCESSED; }static inline int pte_file (pte_t pte) {return pte_val (pte) & _PAGE_FILE; } +static inline int pte_special (pte_t pte) {return 0 ; }static inline void pte_uncache (pte_t pte) {pte_val (pte) |= _PAGE_NO_CACHE; }static inline void pte_cache (pte_t pte) {pte_val (pte) &= ~_PAGE_NO_CACHE; }
@@ -531,6 +532,8 @@ static inline pte_t pte_mkdirty(pte_t ptpte_val (pte) |= _PAGE_DIRTY;return pte; }static inline pte_t pte_mkyoung (pte_t pte) {pte_val (pte) |= _PAGE_ACCESSED;return pte; } +static inline pte_t pte_mkspecial (pte_t pte) { +return pte; }static inline pte_t pte_modify (pte_t pte,pgprot_t newprot) {
include/asm-powerpc/pgtable-ppc64.h +3 -0
@@ -239,6 +239,7 @@ static inline int pte_write(pte_t pte) {static inline int pte_dirty (pte_t pte) {return pte_val (pte) & _PAGE_DIRTY;}static inline int pte_young (pte_t pte) {return pte_val (pte) & _PAGE_ACCESSED;}static inline int pte_file (pte_t pte) {return pte_val (pte) & _PAGE_FILE;} +static inline int pte_special (pte_t pte) {return 0 ; }static inline void pte_uncache (pte_t pte) {pte_val (pte) |= _PAGE_NO_CACHE; }static inline void pte_cache (pte_t pte) {pte_val (pte) &= ~_PAGE_NO_CACHE; }
@@ -257,6 +258,8 @@ static inline pte_t pte_mkyoung(pte_t ptpte_val (pte) |= _PAGE_ACCESSED;return pte; }static inline pte_t pte_mkhuge (pte_t pte) {return pte; } +static inline pte_t pte_mkspecial (pte_t pte) { +return pte; }/* Atomic PTE updates */ static inline unsigned long pte_update(struct mm_struct *mm,
include/asm-ppc/pgtable.h +3 -0
@@ -491,6 +491,7 @@ static inline int pte_write(pte_t pte)static inline int pte_dirty (pte_t pte) {return pte_val (pte) & _PAGE_DIRTY; }static inline int pte_young (pte_t pte) {return pte_val (pte) & _PAGE_ACCESSED; }static inline int pte_file (pte_t pte) {return pte_val (pte) & _PAGE_FILE; } +static inline int pte_special (pte_t pte) {return 0 ; }static inline void pte_uncache (pte_t pte) {pte_val (pte) |= _PAGE_NO_CACHE; }static inline void pte_cache (pte_t pte) {pte_val (pte) &= ~_PAGE_NO_CACHE; }
@@ -508,6 +509,8 @@ static inline pte_t pte_mkdirty(pte_t ptpte_val (pte) |= _PAGE_DIRTY;return pte; }static inline pte_t pte_mkyoung (pte_t pte) {pte_val (pte) |= _PAGE_ACCESSED;return pte; } +static inline pte_t pte_mkspecial (pte_t pte) { +return pte; }static inline pte_t pte_modify (pte_t pte,pgprot_t newprot) {
include/asm-s390/pgtable.h +10 -0
@@ -510,6 +510,11 @@ static inline int pte_file(pte_t pte)return (pte_val (pte) & mask) == _PAGE_TYPE_FILE; } +static inline int pte_special (pte_t pte) +{ +return 0 ; +} +#define __HAVE_ARCH_PTE_SAME#define pte_same (a,b) (pte_val(a) == pte_val(b))
@@ -663,6 +668,11 @@ static inline pte_t pte_mkyoung(pte_t ptreturn pte; } +static inline pte_t pte_mkspecial (pte_t pte) +{ +return pte; +} +#define __HAVE_ARCH_PTEP_TEST_AND_CLEAR_YOUNGstatic inline int ptep_test_and_clear_young (struct vm_area_struct *vma,unsigned long addr,pte_t *ptep)
include/asm-sparc/pgtable.h +7 -0
@@ -219,6 +219,11 @@ static inline int pte_file(pte_t pte)return pte_val (pte) &BTFIXUP_HALF (pte_filei); } +static inline int pte_special (pte_t pte) +{ +return 0 ; +} +/* */ BTFIXUPDEF_HALF (pte_wrprotecti)
@@ -251,6 +256,8 @@ BTFIXUPDEF_CALL_CONST(pte_t, pte_mkyoung#define pte_mkdirty (pte) BTFIXUP_CALL(pte_mkdirty)(pte)#define pte_mkyoung (pte) BTFIXUP_CALL(pte_mkyoung)(pte) +#define pte_mkspecial(pte_t pte) (pte) +#define pfn_pte (pfn, prot) mk_pte(pfn_to_page(pfn), prot) BTFIXUPDEF_CALL(unsigned long, pte_pfn, pte_t)
include/asm-sparc64/pgtable.h +10 -0
@@ -506,6 +506,11 @@ static inline pte_t pte_mkyoung(pte_t ptreturn __pte (pte_val (pte) | mask); } +static inline pte_t pte_mkspecial (pte_t pte) +{ +return pte; +} +static inline unsigned long pte_young (pte_t pte) {unsigned long mask;
@@ -608,6 +613,11 @@ static inline unsigned long pte_present(return val; } +static inline int pte_special (pte_t pte) +{ +return 0 ; +} +#define pmd_set (pmdp, ptep) \ (pmd_val(*(pmdp)) = (__pa((unsigned long) (ptep)) >> 11UL))#define pud_set (pudp, pmdp) \
include/asm-xtensa/pgtable.h +4 -0
@@ -212,6 +212,8 @@ static inline int pte_write(pte_t pte) {static inline int pte_dirty (pte_t pte) {return pte_val (pte) & _PAGE_DIRTY; }static inline int pte_young (pte_t pte) {return pte_val (pte) & _PAGE_ACCESSED; }static inline int pte_file (pte_t pte) {return pte_val (pte) & _PAGE_FILE; } +static inline int pte_special (pte_t pte) {return 0 ; } +static inline pte_t pte_wrprotect (pte_t pte) {pte_val (pte) &= ~(_PAGE_WRITABLE | _PAGE_HW_WRITE);return pte; }static inline pte_t pte_mkclean (pte_t pte)
@@ -224,6 +226,8 @@ static inline pte_t pte_mkyoung(pte_t pt {pte_val (pte) |= _PAGE_ACCESSED;return pte; }static inline pte_t pte_mkwrite (pte_t pte) {pte_val (pte) |= _PAGE_WRITABLE;return pte; } +static inline pte_t pte_mkspecial (pte_t pte) + {return pte; }/* * Conversion functions: convert a page and protection to a page entry,
include/asm-x86/pgtable.h +2 -0
@@ -149,6 +149,7 @@ static inline int pte_file(pte_t pte) {static inline int pte_huge (pte_t pte) {return pte_val (pte) & _PAGE_PSE; }static inline int pte_global (pte_t pte) {return pte_val (pte) & _PAGE_GLOBAL; }static inline int pte_exec (pte_t pte) {return !(pte_val (pte) & _PAGE_NX); } +static inline int pte_special (pte_t pte) {return 0 ; }static inline int pmd_large (pmd_t pte) { return (pmd_val(pte) & (_PAGE_PSE|_PAGE_PRESENT)) ==
@@ -166,6 +167,7 @@ static inline pte_t pte_mkhuge(pte_t ptestatic inline pte_t pte_clrhuge (pte_t pte) {return __pte (pte_val (pte) & ~(pteval_t )_PAGE_PSE); }static inline pte_t pte_mkglobal (pte_t pte) {return __pte (pte_val (pte) | _PAGE_GLOBAL); }static inline pte_t pte_clrglobal (pte_t pte) {return __pte (pte_val (pte) & ~(pteval_t )_PAGE_GLOBAL); } +static inline pte_t pte_mkspecial (pte_t pte) {return pte; }extern pteval_t __supported_pte_mask;
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>