x86: Full support of PAT
Subsystems (5)
Juergen Gross
maintainerv6 cover x86: Full support of PAT
The x86 architecture offers via the PAT (Page Attribute Table) a way to specify different caching modes in page table entries. The PAT MSR contains 8 entries each specifying one of 6 possible cache modes. A pte references one of those entries via 3 bits: _PAGE_PAT, _PAGE_PWT and _PAGE_PCD.
The Linux kernel currently supports only 4 different cache modes. The PAT MSR is set up in a way that the setting of _PAGE_PAT in a pte doesn't matter: the top 4 entries in the PAT MSR are the same as the 4 lower entries.
This results in the kernel not supporting e.g. write-through mode. Especially this cache mode would speed up drivers of video cards which now have to use uncached accesses.
OTOH some old processors (Pentium) don't support PAT correctly and the Xen hypervisor has been using a different PAT MSR configuration for some time now and can't change that as this setting is part of the ABI.
This patch set abstracts the cache mode from the pte and introduces tables to translate between cache mode and pte bits (the default cache mode "write back" is hard-wired to PAT entry 0). The tables are statically initialized with values being compatible to old processors and current usage. As soon as the PAT MSR is changed (or - in case of Xen - is read at boot time) the tables are changed accordingly. Requests of mappings with special cache modes are always possible now, in case they are not supported there will be a fallback to a compatible but slower mode.
Summing it up, this patch set adds the following features:
- capability to support WT and WP cache modes on processors with full PAT support
- processors with no or uncorrect PAT support are still working as today, even if WT or WP cache mode are selected by drivers for some pages
- reduction of Xen special handling regarding cache mode
Changes in V6:
- add new patch 10 (x86: Remove looking for setting of _PAGE_PAT_LARGE in pageattr.c) as suggested by Thomas Gleixner
- replaced SOB of Stefan Bader by "Based-on-patch-by:" as suggested by Borislav Petkov
Changes in V5:
- split up first patch as requested by Ingo Molnar and Thomas Gleixner
- add a helper function in pat_init_cache_modes() as requested by Ingo Molnar
Changes in V4:
- rebased to 3.18-rc2
Changes in V3:
- corrected two minor nits (UC_MINUS, again) detected by Toshi Kani
Changes in V2:
- simplified handling of PAT MSR write under Xen as suggested by David Vrabel
- removed resetting of pat_enabled under Xen
- two small corrections requested by Toshi Kani (UC_MINUS cache mode in vermilion driver, fix 32 bit kernel build failure)
- correct build error on non-x86 arch by moving definition of update_cache_mode_entry() to x86 specific header
Changes since RFC:
- renamed functions and variables as suggested by Toshi Kani
- corrected cache mode bits for WT and WP
- modified handling of PAT MSR write under Xen as suggested by Jan Beulich
Juergen Gross (18):
x86: Make page cache mode a real type
x86: Use new cache mode type in include/asm/fb.h
x86: Use new cache mode type in drivers/video/fbdev/gbefb.c
x86: Use new cache mode type in drivers/video/fbdev/vermilion
x86: Use new cache mode type in arch/x86/pci
x86: Use new cache mode type in arch/x86/mm/init_64.c
x86: Use new cache mode type in asm/pgtable.h
x86: Use new cache mode type in mm/iomap_32.c
x86: Use new cache mode type in track_pfn_remap() and
track_pfn_insert()
x86: Remove looking for setting of _PAGE_PAT_LARGE in pageattr.c
x86: Use new cache mode type in setting page attributes
x86: Use new cache mode type in mm/ioremap.c
x86: Use new cache mode type in memtype related functions
x86: Clean up pgtable_types.h
x86: Support PAT bit in pagetable dump for lower levels
x86: Respect PAT bit when copying pte values between large and normal
pages
x86: Enable PAT to use cache mode translation tables
xen: Support Xen pv-domains using PAT
arch/x86/include/asm/cacheflush.h | 38 ++++--- arch/x86/include/asm/fb.h | 6 +- arch/x86/include/asm/io.h | 2 +- arch/x86/include/asm/pat.h | 7 +- arch/x86/include/asm/pgtable.h | 19 ++-- arch/x86/include/asm/pgtable_types.h | 96 ++++++++++++---- arch/x86/mm/dump_pagetables.c | 24 ++-- arch/x86/mm/init.c | 37 +++++++ arch/x86/mm/init_64.c | 9 +- arch/x86/mm/iomap_32.c | 12 +- arch/x86/mm/ioremap.c | 63 ++++++----- arch/x86/mm/mm_internal.h | 2 + arch/x86/mm/pageattr.c | 84 ++++++++------ arch/x86/mm/pat.c | 176 +++++++++++++++++++----------- arch/x86/mm/pat_internal.h | 22 ++-- arch/x86/mm/pat_rbtree.c | 8 +- arch/x86/pci/i386.c | 4 +- arch/x86/xen/enlighten.c | 25 ++--- arch/x86/xen/mmu.c | 47 +------- arch/x86/xen/xen-ops.h | 1 - drivers/video/fbdev/gbefb.c | 3 +- drivers/video/fbdev/vermilion/vermilion.c | 6 +- 22 files changed, 412 insertions(+), 279 deletions(-)
Signature · 1 line
1.8.4.5
Juergen Gross
maintainerPATCH v6 1/18 x86: Make page cache mode a real type
At the moment there are a lot of places that handle setting or getting the page cache mode by treating the pgprot bits equal to the cache mode. This is only true because there are a lot of assumptions about the setup of the PAT MSR. Otherwise the cache type needs to get translated into pgprot bits and vice versa.
This patch tries to prepare for that by introducing a separate type for the cache mode and adding functions to translate between those and pgprot values.
To avoid too much performance penalty the translation between cache mode and pgprot values is done via tables which contain the relevant information. Write-back cache mode is hard-wired to be 0, all other modes are configurable via those tables. For large pages there are translation functions as the PAT bit is located at different positions in the ptes of 4k and large pages.
--- arch/x86/include/asm/pgtable_types.h | 73 +++++++++++++++++++++++++++++++++++- arch/x86/mm/init.c | 29 ++++++++++++++ 2 files changed, 101 insertions(+), 1 deletion(-)
arch/x86/include/asm/pgtable_types.h +72 -1
@@ -128,12 +128,34 @@ _PAGE_SOFT_DIRTY | _PAGE_NUMA)#define _HPAGE_CHG_MASK (_PAGE_CHG_MASK | _PAGE_PSE | _PAGE_NUMA) -#define _PAGE_CACHE_MASK (_PAGE_PCD | _PAGE_PWT)#define _PAGE_CACHE_WB (0)#define _PAGE_CACHE_WC (_PAGE_PWT)#define _PAGE_CACHE_UC_MINUS (_PAGE_PCD)#define _PAGE_CACHE_UC (_PAGE_PCD | _PAGE_PWT) +/* +* The cache modes defined here are used to translate between pure SW usage +* and the HW defined cache mode bits and/or PAT entries. +* +* The resulting bits for PWT, PCD and PAT should be chosen in a way +* to have the WB mode at index 0 (all bits clear). This is the default +* right now and likely would break too much if changed. +*/ +#ifndef __ASSEMBLY__ +enum page_cache_mode { + _PAGE_CACHE_MODE_WB =0 , + _PAGE_CACHE_MODE_WC =1 , + _PAGE_CACHE_MODE_UC_MINUS =2 , + _PAGE_CACHE_MODE_UC =3 , + _PAGE_CACHE_MODE_WT =4 , + _PAGE_CACHE_MODE_WP =5 , + _PAGE_CACHE_MODE_NUM =8 +}; +#endif + +#define _PAGE_CACHE_MASK (_PAGE_PAT | _PAGE_PCD | _PAGE_PWT) +#define _PAGE_NOCACHE (cachemode2protval(_PAGE_CACHE_MODE_UC)) +#define PAGE_NONE __pgprot(_PAGE_PROTNONE | _PAGE_ACCESSED)#define PAGE_SHARED __pgprot(_PAGE_PRESENT | _PAGE_RW | _PAGE_USER | \ _PAGE_ACCESSED | _PAGE_NX)
@@ -341,6 +363,55 @@ static inline pmdval_t pmdnuma_flags(pmd_t pmd)#define pgprot_val (x) ((x).pgprot)#define __pgprot (x) ((pgprot_t) { (x) } ) +extern uint16_t __cachemode2pte_tbl[_PAGE_CACHE_MODE_NUM]; +extern uint8_t __pte2cachemode_tbl[8 ]; + +#define __pte2cm_idx (cb) \ + ((((cb) >> (_PAGE_BIT_PAT - 2)) & 4) | \ + (((cb) >> (_PAGE_BIT_PCD - 1)) & 2) | \ + (((cb) >> _PAGE_BIT_PWT) & 1)) + +static inline unsigned long cachemode2protval (enum page_cache_mode pcm) +{ +if (likely (pcm ==0 )) +return 0 ; +return __cachemode2pte_tbl[pcm]; +} +static inline pgprot_t cachemode2pgprot (enum page_cache_mode pcm) +{ +return __pgprot (cachemode2protval (pcm)); +} +static inline enum page_cache_mode pgprot2cachemode (pgprot_t pgprot) +{ +unsigned long masked; + + masked =pgprot_val (pgprot) & _PAGE_CACHE_MASK; +if (likely (masked ==0 )) +return 0 ; +return __pte2cachemode_tbl[__pte2cm_idx (masked)]; +} +static inline pgprot_t pgprot_4k_2_large (pgprot_t pgprot) +{ +pgprot_t new; +unsigned long val; + + val =pgprot_val (pgprot); +pgprot_val (new) = (val & ~(_PAGE_PAT | _PAGE_PAT_LARGE)) | + ((val & _PAGE_PAT) << (_PAGE_BIT_PAT_LARGE - _PAGE_BIT_PAT)); +return new; +} +static inline pgprot_t pgprot_large_2_4k (pgprot_t pgprot) +{ +pgprot_t new; +unsigned long val; + + val =pgprot_val (pgprot); +pgprot_val (new) = (val & ~(_PAGE_PAT | _PAGE_PAT_LARGE)) | + ((val & _PAGE_PAT_LARGE) >> + (_PAGE_BIT_PAT_LARGE - _PAGE_BIT_PAT)); +return new; +} +typedef struct page *pgtable_t ;
arch/x86/mm/init.c +29 -0
@@ -27,6 +27,35 @@#include "mm_internal.h" +/* +* Tables translating between page_cache_type_t and pte encoding. +* Minimal supported modes are defined statically, modified if more supported +* cache modes are available. +* Index into __cachemode2pte_tbl is the cachemode. +* Index into __pte2cachemode_tbl are the caching attribute bits of the pte +* (_PAGE_PWT, _PAGE_PCD, _PAGE_PAT) at index bit positions 0, 1, 2. +*/ +uint16_t __cachemode2pte_tbl[_PAGE_CACHE_MODE_NUM] = { + [_PAGE_CACHE_MODE_WB] =0 , + [_PAGE_CACHE_MODE_WC] = _PAGE_PWT, + [_PAGE_CACHE_MODE_UC_MINUS] = _PAGE_PCD, + [_PAGE_CACHE_MODE_UC] = _PAGE_PCD | _PAGE_PWT, + [_PAGE_CACHE_MODE_WT] = _PAGE_PCD, + [_PAGE_CACHE_MODE_WP] = _PAGE_PCD, +}; +EXPORT_SYMBOL_GPL (__cachemode2pte_tbl); +uint8_t __pte2cachemode_tbl[8 ] = { + [__pte2cm_idx (0 )] = _PAGE_CACHE_MODE_WB, + [__pte2cm_idx (_PAGE_PWT)] = _PAGE_CACHE_MODE_WC, + [__pte2cm_idx (_PAGE_PCD)] = _PAGE_CACHE_MODE_UC_MINUS, + [__pte2cm_idx (_PAGE_PWT | _PAGE_PCD)] = _PAGE_CACHE_MODE_UC, + [__pte2cm_idx (_PAGE_PAT)] = _PAGE_CACHE_MODE_WB, + [__pte2cm_idx (_PAGE_PWT | _PAGE_PAT)] = _PAGE_CACHE_MODE_WC, + [__pte2cm_idx (_PAGE_PCD | _PAGE_PAT)] = _PAGE_CACHE_MODE_UC_MINUS, + [__pte2cm_idx (_PAGE_PWT | _PAGE_PCD | _PAGE_PAT)] = _PAGE_CACHE_MODE_UC, +}; +EXPORT_SYMBOL_GPL (__pte2cachemode_tbl); +static unsigned long __initdata pgt_buf_start;static unsigned long __initdata pgt_buf_end;static unsigned long __initdata pgt_buf_top;
Signature · 1 line
1.8.4.5
tip-bot for Juergen Gross
merge bot[tip:x86/mm] x86: Make page cache mode a real type
x86: Make page cache mode a real type
Commit-ID: 281d4078bec366d60990add9d91a952953bd0d72 Gitweb: http://git.kernel.org/tip/281d4078bec366d60990add9d91a952953bd0d72 Author: Juergen Gross <[email protected]> AuthorDate: Mon, 3 Nov 2014 14:01:47 +0100 Committer: Thomas Gleixner <[email protected]> CommitDate: Sun, 16 Nov 2014 11:04:24 +0100
x86: Make page cache mode a real type
At the moment there are a lot of places that handle setting or getting the page cache mode by treating the pgprot bits equal to the cache mode. This is only true because there are a lot of assumptions about the setup of the PAT MSR. Otherwise the cache type needs to get translated into pgprot bits and vice versa.
This patch tries to prepare for that by introducing a separate type for the cache mode and adding functions to translate between those and pgprot values.
To avoid too much performance penalty the translation between cache mode and pgprot values is done via tables which contain the relevant information. Write-back cache mode is hard-wired to be 0, all other modes are configurable via those tables. For large pages there are translation functions as the PAT bit is located at different positions in the ptes of 4k and large pages.
--- arch/x86/include/asm/pgtable_types.h | 73 +++++++++++++++++++++++++++++++++++- arch/x86/mm/init.c | 29 ++++++++++++++ 2 files changed, 101 insertions(+), 1 deletion(-)
arch/x86/include/asm/pgtable_types.h +72 -1
@@ -128,12 +128,34 @@ _PAGE_SOFT_DIRTY | _PAGE_NUMA)#define _HPAGE_CHG_MASK (_PAGE_CHG_MASK | _PAGE_PSE | _PAGE_NUMA) -#define _PAGE_CACHE_MASK (_PAGE_PCD | _PAGE_PWT)#define _PAGE_CACHE_WB (0)#define _PAGE_CACHE_WC (_PAGE_PWT)#define _PAGE_CACHE_UC_MINUS (_PAGE_PCD)#define _PAGE_CACHE_UC (_PAGE_PCD | _PAGE_PWT) +/* +* The cache modes defined here are used to translate between pure SW usage +* and the HW defined cache mode bits and/or PAT entries. +* +* The resulting bits for PWT, PCD and PAT should be chosen in a way +* to have the WB mode at index 0 (all bits clear). This is the default +* right now and likely would break too much if changed. +*/ +#ifndef __ASSEMBLY__ +enum page_cache_mode { + _PAGE_CACHE_MODE_WB =0 , + _PAGE_CACHE_MODE_WC =1 , + _PAGE_CACHE_MODE_UC_MINUS =2 , + _PAGE_CACHE_MODE_UC =3 , + _PAGE_CACHE_MODE_WT =4 , + _PAGE_CACHE_MODE_WP =5 , + _PAGE_CACHE_MODE_NUM =8 +}; +#endif + +#define _PAGE_CACHE_MASK (_PAGE_PAT | _PAGE_PCD | _PAGE_PWT) +#define _PAGE_NOCACHE (cachemode2protval(_PAGE_CACHE_MODE_UC)) +#define PAGE_NONE __pgprot(_PAGE_PROTNONE | _PAGE_ACCESSED)#define PAGE_SHARED __pgprot(_PAGE_PRESENT | _PAGE_RW | _PAGE_USER | \ _PAGE_ACCESSED | _PAGE_NX)
@@ -341,6 +363,55 @@ static inline pmdval_t pmdnuma_flags(pmd_t pmd)#define pgprot_val (x) ((x).pgprot)#define __pgprot (x) ((pgprot_t) { (x) } ) +extern uint16_t __cachemode2pte_tbl[_PAGE_CACHE_MODE_NUM]; +extern uint8_t __pte2cachemode_tbl[8 ]; + +#define __pte2cm_idx (cb) \ + ((((cb) >> (_PAGE_BIT_PAT - 2)) & 4) | \ + (((cb) >> (_PAGE_BIT_PCD - 1)) & 2) | \ + (((cb) >> _PAGE_BIT_PWT) & 1)) + +static inline unsigned long cachemode2protval (enum page_cache_mode pcm) +{ +if (likely (pcm ==0 )) +return 0 ; +return __cachemode2pte_tbl[pcm]; +} +static inline pgprot_t cachemode2pgprot (enum page_cache_mode pcm) +{ +return __pgprot (cachemode2protval (pcm)); +} +static inline enum page_cache_mode pgprot2cachemode (pgprot_t pgprot) +{ +unsigned long masked; + + masked =pgprot_val (pgprot) & _PAGE_CACHE_MASK; +if (likely (masked ==0 )) +return 0 ; +return __pte2cachemode_tbl[__pte2cm_idx (masked)]; +} +static inline pgprot_t pgprot_4k_2_large (pgprot_t pgprot) +{ +pgprot_t new; +unsigned long val; + + val =pgprot_val (pgprot); +pgprot_val (new) = (val & ~(_PAGE_PAT | _PAGE_PAT_LARGE)) | + ((val & _PAGE_PAT) << (_PAGE_BIT_PAT_LARGE - _PAGE_BIT_PAT)); +return new; +} +static inline pgprot_t pgprot_large_2_4k (pgprot_t pgprot) +{ +pgprot_t new; +unsigned long val; + + val =pgprot_val (pgprot); +pgprot_val (new) = (val & ~(_PAGE_PAT | _PAGE_PAT_LARGE)) | + ((val & _PAGE_PAT_LARGE) >> + (_PAGE_BIT_PAT_LARGE - _PAGE_BIT_PAT)); +return new; +} +typedef struct page *pgtable_t ;
arch/x86/mm/init.c +29 -0
@@ -27,6 +27,35 @@#include "mm_internal.h" +/* +* Tables translating between page_cache_type_t and pte encoding. +* Minimal supported modes are defined statically, modified if more supported +* cache modes are available. +* Index into __cachemode2pte_tbl is the cachemode. +* Index into __pte2cachemode_tbl are the caching attribute bits of the pte +* (_PAGE_PWT, _PAGE_PCD, _PAGE_PAT) at index bit positions 0, 1, 2. +*/ +uint16_t __cachemode2pte_tbl[_PAGE_CACHE_MODE_NUM] = { + [_PAGE_CACHE_MODE_WB] =0 , + [_PAGE_CACHE_MODE_WC] = _PAGE_PWT, + [_PAGE_CACHE_MODE_UC_MINUS] = _PAGE_PCD, + [_PAGE_CACHE_MODE_UC] = _PAGE_PCD | _PAGE_PWT, + [_PAGE_CACHE_MODE_WT] = _PAGE_PCD, + [_PAGE_CACHE_MODE_WP] = _PAGE_PCD, +}; +EXPORT_SYMBOL_GPL (__cachemode2pte_tbl); +uint8_t __pte2cachemode_tbl[8 ] = { + [__pte2cm_idx (0 )] = _PAGE_CACHE_MODE_WB, + [__pte2cm_idx (_PAGE_PWT)] = _PAGE_CACHE_MODE_WC, + [__pte2cm_idx (_PAGE_PCD)] = _PAGE_CACHE_MODE_UC_MINUS, + [__pte2cm_idx (_PAGE_PWT | _PAGE_PCD)] = _PAGE_CACHE_MODE_UC, + [__pte2cm_idx (_PAGE_PAT)] = _PAGE_CACHE_MODE_WB, + [__pte2cm_idx (_PAGE_PWT | _PAGE_PAT)] = _PAGE_CACHE_MODE_WC, + [__pte2cm_idx (_PAGE_PCD | _PAGE_PAT)] = _PAGE_CACHE_MODE_UC_MINUS, + [__pte2cm_idx (_PAGE_PWT | _PAGE_PCD | _PAGE_PAT)] = _PAGE_CACHE_MODE_UC, +}; +EXPORT_SYMBOL_GPL (__pte2cachemode_tbl); +static unsigned long __initdata pgt_buf_start;static unsigned long __initdata pgt_buf_end;static unsigned long __initdata pgt_buf_top;
Re: [PATCH V6 01/18] x86: Make page cache mode a real type
In reply to Juergen GrossQuoted 148 lines · Juergen Gross
At the moment there are a lot of places that handle setting or getting the page cache mode by treating the pgprot bits equal to the cache mode. This is only true because there are a lot of assumptions about the setup of the PAT MSR. Otherwise the cache type needs to get translated into pgprot bits and vice versa.
This patch tries to prepare for that by introducing a separate type for the cache mode and adding functions to translate between those and pgprot values.
To avoid too much performance penalty the translation between cache mode and pgprot values is done via tables which contain the relevant information. Write-back cache mode is hard-wired to be 0, all other modes are configurable via those tables. For large pages there are translation functions as the PAT bit is located at different positions in the ptes of 4k and large pages.
Based-on-patch-by Stefan Bader <[email protected]>Signed-off-by Juergen Gross <[email protected]>Reviewed-by Thomas Gleixner <[email protected]>--- arch/x86/include/asm/pgtable_types.h | 73 +++++++++++++++++++++++++++++++++++- arch/x86/mm/init.c | 29 ++++++++++++++ 2 files changed, 101 insertions(+), 1 deletion(-)arch/x86/include/asm/pgtable_types.h +72
-1@@ -128,12 +128,34 @@ _PAGE_SOFT_DIRTY | _PAGE_NUMA)#define _HPAGE_CHG_MASK (_PAGE_CHG_MASK | _PAGE_PSE | _PAGE_NUMA) -#define _PAGE_CACHE_MASK (_PAGE_PCD | _PAGE_PWT)#define _PAGE_CACHE_WB (0)#define _PAGE_CACHE_WC (_PAGE_PWT)#define _PAGE_CACHE_UC_MINUS (_PAGE_PCD)#define _PAGE_CACHE_UC (_PAGE_PCD | _PAGE_PWT) +/* +* The cache modes defined here are used to translate between pure SW usage +* and the HW defined cache mode bits and/or PAT entries. +* +* The resulting bits for PWT, PCD and PAT should be chosen in a way +* to have the WB mode at index 0 (all bits clear). This is the default +* right now and likely would break too much if changed. +*/ +#ifndef __ASSEMBLY__ +enum page_cache_mode { + _PAGE_CACHE_MODE_WB =0 , + _PAGE_CACHE_MODE_WC =1 , + _PAGE_CACHE_MODE_UC_MINUS =2 , + _PAGE_CACHE_MODE_UC =3 , + _PAGE_CACHE_MODE_WT =4 , + _PAGE_CACHE_MODE_WP =5 , + _PAGE_CACHE_MODE_NUM =8 +}; +#endif + +#define _PAGE_CACHE_MASK (_PAGE_PAT | _PAGE_PCD | _PAGE_PWT) +#define _PAGE_NOCACHE (cachemode2protval(_PAGE_CACHE_MODE_UC)) +#define PAGE_NONE __pgprot(_PAGE_PROTNONE | _PAGE_ACCESSED)#define PAGE_SHARED __pgprot(_PAGE_PRESENT | _PAGE_RW | _PAGE_USER | \ _PAGE_ACCESSED | _PAGE_NX)@@ -341,6 +363,55 @@ static inline pmdval_t pmdnuma_flags(pmd_t pmd)#define pgprot_val (x) ((x).pgprot)#define __pgprot (x) ((pgprot_t) { (x) } ) +extern uint16_t __cachemode2pte_tbl[_PAGE_CACHE_MODE_NUM]; +extern uint8_t __pte2cachemode_tbl[8 ]; + +#define __pte2cm_idx (cb) \ + ((((cb) >> (_PAGE_BIT_PAT - 2)) & 4) | \ + (((cb) >> (_PAGE_BIT_PCD - 1)) & 2) | \ + (((cb) >> _PAGE_BIT_PWT) & 1)) + +static inline unsigned long cachemode2protval (enum page_cache_mode pcm) +{ +if (likely (pcm ==0 )) +return 0 ; +return __cachemode2pte_tbl[pcm]; +} +static inline pgprot_t cachemode2pgprot (enum page_cache_mode pcm) +{ +return __pgprot (cachemode2protval (pcm)); +} +static inline enum page_cache_mode pgprot2cachemode (pgprot_t pgprot) +{ +unsigned long masked; + + masked =pgprot_val (pgprot) & _PAGE_CACHE_MASK; +if (likely (masked ==0 )) +return 0 ; +return __pte2cachemode_tbl[__pte2cm_idx (masked)]; +} +static inline pgprot_t pgprot_4k_2_large (pgprot_t pgprot) +{ +pgprot_t new; +unsigned long val; + + val =pgprot_val (pgprot); +pgprot_val (new) = (val & ~(_PAGE_PAT | _PAGE_PAT_LARGE)) | + ((val & _PAGE_PAT) << (_PAGE_BIT_PAT_LARGE - _PAGE_BIT_PAT)); +return new; +} +static inline pgprot_t pgprot_large_2_4k (pgprot_t pgprot) +{ +pgprot_t new; +unsigned long val; + + val =pgprot_val (pgprot); +pgprot_val (new) = (val & ~(_PAGE_PAT | _PAGE_PAT_LARGE)) | + ((val & _PAGE_PAT_LARGE) >> + (_PAGE_BIT_PAT_LARGE - _PAGE_BIT_PAT)); +return new; +} +typedef struct page *pgtable_t ;arch/x86/mm/init.c +19
-0@@ -27,6 +27,35 @@#include "mm_internal.h" +/* +* Tables translating between page_cache_type_t and pte encoding. +* Minimal supported modes are defined statically, modified if more supported +* cache modes are available. +* Index into __cachemode2pte_tbl is the cachemode. +* Index into __pte2cachemode_tbl are the caching attribute bits of the pte +* (_PAGE_PWT, _PAGE_PCD, _PAGE_PAT) at index bit positions 0, 1, 2. +*/ +uint16_t __cachemode2pte_tbl[_PAGE_CACHE_MODE_NUM] = { + [_PAGE_CACHE_MODE_WB] =0 , + [_PAGE_CACHE_MODE_WC] = _PAGE_PWT, + [_PAGE_CACHE_MODE_UC_MINUS] = _PAGE_PCD, + [_PAGE_CACHE_MODE_UC] = _PAGE_PCD | _PAGE_PWT, + [_PAGE_CACHE_MODE_WT] = _PAGE_PCD, + [_PAGE_CACHE_MODE_WP] = _PAGE_PCD, +}; +EXPORT_SYMBOL_GPL (__cachemode2pte_tbl); +uint8_t __pte2cachemode_tbl[8 ] = { + [__pte2cm_idx (0 )] = _PAGE_CACHE_MODE_WB,
arch/x86/mm/init.c line 58
Quoted 10 lines
arch/x86/mm/init.c +10
-0+ [__pte2cm_idx(_PAGE_PWT)] = _PAGE_CACHE_MODE_WC, + [__pte2cm_idx(_PAGE_PCD)] = _PAGE_CACHE_MODE_UC_MINUS, + [__pte2cm_idx(_PAGE_PWT | _PAGE_PCD)] = _PAGE_CACHE_MODE_UC, + [__pte2cm_idx(_PAGE_PAT)] = _PAGE_CACHE_MODE_WB, + [__pte2cm_idx(_PAGE_PWT | _PAGE_PAT)] = _PAGE_CACHE_MODE_WC, + [__pte2cm_idx(_PAGE_PCD | _PAGE_PAT)] = _PAGE_CACHE_MODE_UC_MINUS, + [__pte2cm_idx (_PAGE_PWT | _PAGE_PCD | _PAGE_PAT)] = _PAGE_CACHE_MODE_UC, +}; +EXPORT_SYMBOL_GPL (__pte2cachemode_tbl); +
I notice these two symbols are exported GPL-only. This breaks builds of several out-of-tree non-GPL modules such as the NVIDIA driver, and VMware modules, etc. What is the appropriate code path for proprietary modules to use when setting page cache mode flags? Alternatively, is it possible for these EXPORT_SYMBOL_GPLs to be changed to EXPORT_SYMBOL?
Quoted 11 lines
arch/x86/mm/init.c +0
-1static unsigned long __initdata pgt_buf_start;static unsigned long __initdata pgt_buf_end;static unsigned long __initdata pgt_buf_top; --1.8.4.5
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Juergen Gross
maintainerRe: [PATCH V6 01/18] x86: Make page cache mode a real type
Quoted 22 lines · Steven Noonan
Quoted 21 lines · Juergen Gross
At the moment there are a lot of places that handle setting or getting the page cache mode by treating the pgprot bits equal to the cache mode. This is only true because there are a lot of assumptions about the setup of the PAT MSR. Otherwise the cache type needs to get translated into pgprot bits and vice versa.
This patch tries to prepare for that by introducing a separate type for the cache mode and adding functions to translate between those and pgprot values.
To avoid too much performance penalty the translation between cache mode and pgprot values is done via tables which contain the relevant information. Write-back cache mode is hard-wired to be 0, all other modes are configurable via those tables. For large pages there are translation functions as the PAT bit is located at different positions in the ptes of 4k and large pages.
Based-on-patch-by Stefan Bader <[email protected]>Signed-off-by Juergen Gross <[email protected]>Reviewed-by Thomas Gleixner <[email protected]>---
...
Quoted 36 lines
Quoted 36 lines
arch/x86/mm/init.c +28
-0@@ -27,6 +27,35 @@#include "mm_internal.h" +/* +* Tables translating between page_cache_type_t and pte encoding. +* Minimal supported modes are defined statically, modified if more supported +* cache modes are available. +* Index into __cachemode2pte_tbl is the cachemode. +* Index into __pte2cachemode_tbl are the caching attribute bits of the pte +* (_PAGE_PWT, _PAGE_PCD, _PAGE_PAT) at index bit positions 0, 1, 2. +*/ +uint16_t __cachemode2pte_tbl[_PAGE_CACHE_MODE_NUM] = { + [_PAGE_CACHE_MODE_WB] =0 , + [_PAGE_CACHE_MODE_WC] = _PAGE_PWT, + [_PAGE_CACHE_MODE_UC_MINUS] = _PAGE_PCD, + [_PAGE_CACHE_MODE_UC] = _PAGE_PCD | _PAGE_PWT, + [_PAGE_CACHE_MODE_WT] = _PAGE_PCD, + [_PAGE_CACHE_MODE_WP] = _PAGE_PCD, +}; +EXPORT_SYMBOL_GPL (__cachemode2pte_tbl); +uint8_t __pte2cachemode_tbl[8 ] = { + [__pte2cm_idx (0 )] = _PAGE_CACHE_MODE_WB, + [__pte2cm_idx (_PAGE_PWT)] = _PAGE_CACHE_MODE_WC, + [__pte2cm_idx (_PAGE_PCD)] = _PAGE_CACHE_MODE_UC_MINUS, + [__pte2cm_idx (_PAGE_PWT | _PAGE_PCD)] = _PAGE_CACHE_MODE_UC, + [__pte2cm_idx (_PAGE_PAT)] = _PAGE_CACHE_MODE_WB, + [__pte2cm_idx (_PAGE_PWT | _PAGE_PAT)] = _PAGE_CACHE_MODE_WC, + [__pte2cm_idx (_PAGE_PCD | _PAGE_PAT)] = _PAGE_CACHE_MODE_UC_MINUS, + [__pte2cm_idx (_PAGE_PWT | _PAGE_PCD | _PAGE_PAT)] = _PAGE_CACHE_MODE_UC, +}; +EXPORT_SYMBOL_GPL (__pte2cachemode_tbl);
Quoted 8 lines
arch/x86/mm/init.c line 58
Quoted 1 line
arch/x86/mm/init.c +1
-0+I notice these two symbols are exported GPL-only. This breaks builds of several out-of-tree non-GPL modules such as the NVIDIA driver, and VMware modules, etc. What is the appropriate code path for proprietary modules to use when setting page cache mode flags? Alternatively, is it possible for these
EXPORT_SYMBOL_GPLsto be changed toEXPORT_SYMBOL?
I don't mind you sending a patch to change this. I won't object such a patch. OTOH this is more kind of a political question and I don't want to spend my time on arguing.
Juergen
Re: [PATCH V6 01/18] x86: Make page cache mode a real type
Quoted 11 lines · Juergen Gross
Quoted 6 lines · Steven Noonan
I notice these two symbols are exported GPL-only. This breaks builds of several out-of-tree non-GPL modules such as the NVIDIA driver, and VMware modules, etc. What is the appropriate code path for proprietary modules to use when setting page cache mode flags? Alternatively, is it possible for these
EXPORT_SYMBOL_GPLsto be changed toEXPORT_SYMBOL?I don't mind you sending a patch to change this. I won't object such a patch. OTOH this is more kind of a political question and I don't want to spend my time on arguing.
It's rather simple. If the new cache stuff replaces code which was available under EXPORT_SYMBOL, we either need a compat export or make the new exports non GPL.
Not that I like it, but that has been the policy so far. We only break out of tree stuff if the export is broken or security relevant, like we did with the init_mm export a few years ago.
Thanks,
tglx
Juergen Gross
maintainerRe: [PATCH V6 01/18] x86: Make page cache mode a real type
Quoted 13 lines · Thomas Gleixner
Quoted 11 lines · Juergen Gross
Quoted 6 lines · Steven Noonan
I notice these two symbols are exported GPL-only. This breaks builds of several out-of-tree non-GPL modules such as the NVIDIA driver, and VMware modules, etc. What is the appropriate code path for proprietary modules to use when setting page cache mode flags? Alternatively, is it possible for these
EXPORT_SYMBOL_GPLsto be changed toEXPORT_SYMBOL?I don't mind you sending a patch to change this. I won't object such a patch. OTOH this is more kind of a political question and I don't want to spend my time on arguing.
Quoted 7 lines
It's rather simple. If the new cache stuff replaces code which was available under
EXPORT_SYMBOL, we either need a compat export or make the new exports non GPL.Not that I like it, but that has been the policy so far. We only break out of tree stuff if the export is broken or security relevant, like we did with the
init_mmexport a few years ago.
Okay, in this case I'll write patch.
Thanks for clarifying this.
Juergen
Juergen Gross
maintainerPATCH v6 2/18 x86: Use new cache mode type in include/asm/fb.h
In reply to Juergen GrossInstead of directly using cache mode bits in the pte switch to usage of the new cache mode type.
--- arch/x86/include/asm/fb.h | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
arch/x86/include/asm/fb.h +5 -1
@@ -8,8 +8,12 @@static inline void fb_pgprotect (struct file *file,struct vm_area_struct *vma,unsigned long off) { +unsigned long prot; + + prot =pgprot_val (vma->vm_page_prot) & ~_PAGE_CACHE_MASK;if (boot_cpu_data.x86 >3 ) -pgprot_val (vma->vm_page_prot) |= _PAGE_PCD; +pgprot_val (vma->vm_page_prot) = + prot |cachemode2protval (_PAGE_CACHE_MODE_UC_MINUS); }extern int fb_is_primary_device (struct fb_info *info);
Signature · 1 line
1.8.4.5
tip-bot for Juergen Gross
merge bot[tip:x86/mm] x86: Use new cache mode type in include/asm/fb.h
x86: Use new cache mode type in include/asm/fb.h
Commit-ID: c27ce0af896b7cc1718995f2b3e66e2892d5081c Gitweb: http://git.kernel.org/tip/c27ce0af896b7cc1718995f2b3e66e2892d5081c Author: Juergen Gross <[email protected]> AuthorDate: Mon, 3 Nov 2014 14:01:48 +0100 Committer: Thomas Gleixner <[email protected]> CommitDate: Sun, 16 Nov 2014 11:04:24 +0100
x86: Use new cache mode type in include/asm/fb.h
Instead of directly using cache mode bits in the pte switch to usage of the new cache mode type.
--- arch/x86/include/asm/fb.h | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
arch/x86/include/asm/fb.h +5 -1
@@ -8,8 +8,12 @@static inline void fb_pgprotect (struct file *file,struct vm_area_struct *vma,unsigned long off) { +unsigned long prot; + + prot =pgprot_val (vma->vm_page_prot) & ~_PAGE_CACHE_MASK;if (boot_cpu_data.x86 >3 ) -pgprot_val (vma->vm_page_prot) |= _PAGE_PCD; +pgprot_val (vma->vm_page_prot) = + prot |cachemode2protval (_PAGE_CACHE_MODE_UC_MINUS); }extern int fb_is_primary_device (struct fb_info *info);
Juergen Gross
maintainerPATCH v6 3/18 x86: Use new cache mode type in drivers/video/fbdev/gbefb.c
In reply to Juergen GrossInstead of directly using the cache mode bits in the pte switch to using the cache mode type.
--- drivers/video/fbdev/gbefb.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/video/fbdev/gbefb.c +2 -1
@@ -54,7 +54,8 @@ struct gbefb_par {#endif #endif #ifdef CONFIG_X86 -#define pgprot_fb (_prot) ((_prot) | _PAGE_PCD) +#define pgprot_fb (_prot) (((_prot) & ~_PAGE_CACHE_MASK) | \ + cachemode2protval(_PAGE_CACHE_MODE_UC_MINUS))#endif /*
Signature · 1 line
1.8.4.5
Re: [PATCH V6 03/18] x86: Use new cache mode type in drivers/video/fbdev/gbefb.c
Quoted 16 lines · Juergen Gross
Instead of directly using the cache mode bits in the pte switch to using the cache mode type.
Based-on-patch-by Stefan Bader <[email protected]>Signed-off-by Juergen Gross <[email protected]>Reviewed-by Thomas Gleixner <[email protected]>--- drivers/video/fbdev/gbefb.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)drivers/video/fbdev/gbefb.c +0
-0
drivers/video/fbdev/gbefb.c struct gbefb_par { · line 61
Quoted 9 lines
drivers/video/fbdev/gbefb.c +2
-1@@ -54,7 +54,8 @@ struct gbefb_par {#endif #endif #ifdef CONFIG_X86 -#define pgprot_fb (_prot) ((_prot) | _PAGE_PCD) +#define pgprot_fb (_prot) (((_prot) & ~_PAGE_CACHE_MASK) | \ + cachemode2protval(_PAGE_CACHE_MODE_UC_MINUS))#endif /*
For this and vermilion fb:
Tomi
tip-bot for Juergen Gross
merge bot[tip:x86/mm] x86: Use new cache mode type in drivers/video/fbdev/ gbefb.c
In reply to Juergen Grossx86: Use new cache mode type in drivers/video/fbdev/gbefb.c
Commit-ID: 2d85ebf8e12e14694d6f9a4f34359c19f0738ace Gitweb: http://git.kernel.org/tip/2d85ebf8e12e14694d6f9a4f34359c19f0738ace Author: Juergen Gross <[email protected]> AuthorDate: Mon, 3 Nov 2014 14:01:49 +0100 Committer: Thomas Gleixner <[email protected]> CommitDate: Sun, 16 Nov 2014 11:04:25 +0100
x86: Use new cache mode type in drivers/video/fbdev/gbefb.c
Instead of directly using the cache mode bits in the pte switch to using the cache mode type.
--- drivers/video/fbdev/gbefb.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/video/fbdev/gbefb.c +2 -1
@@ -54,7 +54,8 @@ struct gbefb_par {#endif #endif #ifdef CONFIG_X86 -#define pgprot_fb (_prot) ((_prot) | _PAGE_PCD) +#define pgprot_fb (_prot) (((_prot) & ~_PAGE_CACHE_MASK) | \ + cachemode2protval(_PAGE_CACHE_MODE_UC_MINUS))#endif /*
Juergen Gross
maintainerPATCH v6 4/18 x86: Use new cache mode type in drivers/video/fbdev/vermilion
In reply to Juergen GrossInstead of directly using the cache mode bits in the pte switch to using the cache mode type.
--- drivers/video/fbdev/vermilion/vermilion.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
drivers/video/fbdev/vermilion/vermilion.c +4 -2
@@ -1003,13 +1003,15 @@ static int vmlfb_mmap(struct fb_info *info, struct vm_area_struct *vma) struct vml_info *vinfo = container_of(info, struct vml_info, info);unsigned long offset = vma->vm_pgoff <<PAGE_SHIFT ;int ret; +unsigned long prot; ret =vmlfb_vram_offset (vinfo, offset);if (ret)return -EINVAL ; -pgprot_val (vma->vm_page_prot) |= _PAGE_PCD; -pgprot_val (vma->vm_page_prot) &= ~_PAGE_PWT; + prot =pgprot_val (vma->vm_page_prot) & ~_PAGE_CACHE_MASK; +pgprot_val (vma->vm_page_prot) = + prot |cachemode2protval (_PAGE_CACHE_MODE_UC_MINUS);return vm_iomap_memory (vma, vinfo->vram_start, vinfo->vram_contig_size);
Signature · 1 line
1.8.4.5
tip-bot for Juergen Gross
merge bot[tip:x86/mm] x86: Use new cache mode type in drivers/video/fbdev/ vermilion
x86: Use new cache mode type in drivers/video/fbdev/vermilion
Commit-ID: 5006e45a6bc293b490638210d1a88ac391d2eb92 Gitweb: http://git.kernel.org/tip/5006e45a6bc293b490638210d1a88ac391d2eb92 Author: Juergen Gross <[email protected]> AuthorDate: Mon, 3 Nov 2014 14:01:50 +0100 Committer: Thomas Gleixner <[email protected]> CommitDate: Sun, 16 Nov 2014 11:04:25 +0100
x86: Use new cache mode type in drivers/video/fbdev/vermilion
Instead of directly using the cache mode bits in the pte switch to using the cache mode type.
--- drivers/video/fbdev/vermilion/vermilion.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
drivers/video/fbdev/vermilion/vermilion.c +4 -2
@@ -1003,13 +1003,15 @@ static int vmlfb_mmap(struct fb_info *info, struct vm_area_struct *vma) struct vml_info *vinfo = container_of(info, struct vml_info, info);unsigned long offset = vma->vm_pgoff <<PAGE_SHIFT ;int ret; +unsigned long prot; ret =vmlfb_vram_offset (vinfo, offset);if (ret)return -EINVAL ; -pgprot_val (vma->vm_page_prot) |= _PAGE_PCD; -pgprot_val (vma->vm_page_prot) &= ~_PAGE_PWT; + prot =pgprot_val (vma->vm_page_prot) & ~_PAGE_CACHE_MASK; +pgprot_val (vma->vm_page_prot) = + prot |cachemode2protval (_PAGE_CACHE_MODE_UC_MINUS);return vm_iomap_memory (vma, vinfo->vram_start, vinfo->vram_contig_size);
Juergen Gross
maintainerPATCH v6 5/18 x86: Use new cache mode type in arch/x86/pci
In reply to Juergen GrossInstead of directly using the cache mode bits in the pte switch to using the cache mode type.
--- arch/x86/pci/i386.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/pci/i386.c +2 -2
@@ -433,14 +433,14 @@ int pci_mmap_page_range(struct pci_dev *dev, struct vm_area_struct *vma,return -EINVAL ;if (pat_enabled && write_combine) - prot |= _PAGE_CACHE_WC; + prot |=cachemode2protval (_PAGE_CACHE_MODE_WC);else if (pat_enabled || boot_cpu_data.x86 >3 )/* * ioremap() and ioremap_nocache() defaults to UC MINUS for now. * To avoid attribute conflicts, request UC MINUS here * as well. */ - prot |= _PAGE_CACHE_UC_MINUS; + prot |=cachemode2protval (_PAGE_CACHE_MODE_UC_MINUS); vma->vm_page_prot =__pgprot (prot);
Signature · 1 line
1.8.4.5
tip-bot for Juergen Gross
merge bot[tip:x86/mm] x86: Use new cache mode type in arch/x86/pci
x86: Use new cache mode type in arch/x86/pci
Commit-ID: 1c64216be16404df7fab33c793890bb5076e8123 Gitweb: http://git.kernel.org/tip/1c64216be16404df7fab33c793890bb5076e8123 Author: Juergen Gross <[email protected]> AuthorDate: Mon, 3 Nov 2014 14:01:51 +0100 Committer: Thomas Gleixner <[email protected]> CommitDate: Sun, 16 Nov 2014 11:04:25 +0100
x86: Use new cache mode type in arch/x86/pci
Instead of directly using the cache mode bits in the pte switch to using the cache mode type.
--- arch/x86/pci/i386.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/pci/i386.c +2 -2
@@ -433,14 +433,14 @@ int pci_mmap_page_range(struct pci_dev *dev, struct vm_area_struct *vma,return -EINVAL ;if (pat_enabled && write_combine) - prot |= _PAGE_CACHE_WC; + prot |=cachemode2protval (_PAGE_CACHE_MODE_WC);else if (pat_enabled || boot_cpu_data.x86 >3 )/* * ioremap() and ioremap_nocache() defaults to UC MINUS for now. * To avoid attribute conflicts, request UC MINUS here * as well. */ - prot |= _PAGE_CACHE_UC_MINUS; + prot |=cachemode2protval (_PAGE_CACHE_MODE_UC_MINUS); vma->vm_page_prot =__pgprot (prot);
Juergen Gross
maintainerPATCH v6 6/18 x86: Use new cache mode type in arch/x86/mm/init_64.c
In reply to Juergen GrossInstead of directly using the cache mode bits in the pte switch to using the cache mode type.
--- arch/x86/mm/init_64.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
arch/x86/mm/init_64.c +6 -3
@@ -338,12 +338,15 @@ pte_t * __init populate_extra_pte(unsigned long vaddr)* Create large page table mappings for a range of physical addresses. */ static void __init __init_extra_mapping (unsigned long phys,unsigned long size, -pgprot_t prot) +enum page_cache_mode cache) {pgd_t *pgd;pud_t *pud;pmd_t *pmd; +pgprot_t prot; +pgprot_val (prot) =pgprot_val (PAGE_KERNEL_LARGE ) | +pgprot_val (pgprot_4k_2_large (cachemode2pgprot (cache)));BUG_ON ((phys & ~PMD_MASK ) || (size & ~PMD_MASK ));for (; size; phys +=PMD_SIZE , size -=PMD_SIZE ) { pgd =pgd_offset_k ((unsigned long )__va (phys));
@@ -366,12 +369,12 @@ static void __init __init_extra_mapping(unsigned long phys, unsigned long size,void __init init_extra_mapping_wb (unsigned long phys,unsigned long size) { -__init_extra_mapping (phys, size,PAGE_KERNEL_LARGE ); +__init_extra_mapping (phys, size, _PAGE_CACHE_MODE_WB); }void __init init_extra_mapping_uc (unsigned long phys,unsigned long size) { -__init_extra_mapping (phys, size,PAGE_KERNEL_LARGE_NOCACHE ); +__init_extra_mapping (phys, size, _PAGE_CACHE_MODE_UC); }/*
Signature · 1 line
1.8.4.5
tip-bot for Juergen Gross
merge bot[tip:x86/mm] x86: Use new cache mode type in arch/x86/mm/ init_64.c
x86: Use new cache mode type in arch/x86/mm/init_64.c
Commit-ID: 2df58b6d35306e8dab48923c9fbe9e1ad17537e2 Gitweb: http://git.kernel.org/tip/2df58b6d35306e8dab48923c9fbe9e1ad17537e2 Author: Juergen Gross <[email protected]> AuthorDate: Mon, 3 Nov 2014 14:01:52 +0100 Committer: Thomas Gleixner <[email protected]> CommitDate: Sun, 16 Nov 2014 11:04:25 +0100
x86: Use new cache mode type in arch/x86/mm/init_64.c
Instead of directly using the cache mode bits in the pte switch to using the cache mode type.
--- arch/x86/mm/init_64.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
arch/x86/mm/init_64.c +6 -3
@@ -337,12 +337,15 @@ pte_t * __init populate_extra_pte(unsigned long vaddr)* Create large page table mappings for a range of physical addresses. */ static void __init __init_extra_mapping (unsigned long phys,unsigned long size, -pgprot_t prot) +enum page_cache_mode cache) {pgd_t *pgd;pud_t *pud;pmd_t *pmd; +pgprot_t prot; +pgprot_val (prot) =pgprot_val (PAGE_KERNEL_LARGE ) | +pgprot_val (pgprot_4k_2_large (cachemode2pgprot (cache)));BUG_ON ((phys & ~PMD_MASK ) || (size & ~PMD_MASK ));for (; size; phys +=PMD_SIZE , size -=PMD_SIZE ) { pgd =pgd_offset_k ((unsigned long )__va (phys));
@@ -365,12 +368,12 @@ static void __init __init_extra_mapping(unsigned long phys, unsigned long size,void __init init_extra_mapping_wb (unsigned long phys,unsigned long size) { -__init_extra_mapping (phys, size,PAGE_KERNEL_LARGE ); +__init_extra_mapping (phys, size, _PAGE_CACHE_MODE_WB); }void __init init_extra_mapping_uc (unsigned long phys,unsigned long size) { -__init_extra_mapping (phys, size,PAGE_KERNEL_LARGE_NOCACHE ); +__init_extra_mapping (phys, size, _PAGE_CACHE_MODE_UC); }/*
Juergen Gross
maintainerPATCH v6 7/18 x86: Use new cache mode type in asm/pgtable.h
In reply to Juergen GrossInstead of directly using the cache mode bits in the pte switch to using the cache mode type. This requires changing some callers of is_new_memtype_allowed() to be changed as well.
--- arch/x86/include/asm/pgtable.h | 19 ++++++++++--------- arch/x86/mm/ioremap.c | 3 ++- arch/x86/mm/pat.c | 8 ++++++-- 3 files changed, 18 insertions(+), 12 deletions(-)
arch/x86/include/asm/pgtable.h +10 -9
@@ -9,9 +9,10 @@/* * Macro to mark a page protection value as UC- */ -#define pgprot_noncached (prot) \ - ((boot_cpu_data.x86 > 3) \ - ? (__pgprot(pgprot_val(prot) | _PAGE_CACHE_UC_MINUS)) \ +#define pgprot_noncached (prot) \ + ((boot_cpu_data.x86 > 3) \ + ? (__pgprot(pgprot_val(prot) | \ + cachemode2protval(_PAGE_CACHE_MODE_UC_MINUS))) \ : (prot))#ifndef __ASSEMBLY__
@@ -404,8 +405,8 @@ static inline pgprot_t pgprot_modify(pgprot_t oldprot, pgprot_t newprot)#define canon_pgprot (p) __pgprot(massage_pgprot(p))static inline int is_new_memtype_allowed (u64 paddr,unsigned long size, -unsigned long flags, -unsigned long new_flags) +enum page_cache_mode pcm, +enum page_cache_mode new_pcm) {/* * PAT type is always WB for untracked ranges, so no need to check.
@@ -419,10 +420,10 @@ static inline int is_new_memtype_allowed(u64 paddr, unsigned long size,* - request is uncached, return cannot be write-back * - request is write-combine, return cannot be write-back */ -if ((flags == _PAGE_CACHE_UC_MINUS && - new_flags == _PAGE_CACHE_WB) || - (flags == _PAGE_CACHE_WC && - new_flags == _PAGE_CACHE_WB)) { +if ((pcm == _PAGE_CACHE_MODE_UC_MINUS && + new_pcm == _PAGE_CACHE_MODE_WB) || + (pcm == _PAGE_CACHE_MODE_WC && + new_pcm == _PAGE_CACHE_MODE_WB)) {return 0 ; }
arch/x86/mm/ioremap.c +2 -1
@@ -142,7 +142,8 @@ static void __iomem *__ioremap_caller(resource_size_t phys_addr,if (prot_val != new_prot_val) {if (!is_new_memtype_allowed (phys_addr, size, - prot_val, new_prot_val)) { +pgprot2cachemode (__pgprot (prot_val)), +pgprot2cachemode (__pgprot (new_prot_val)))) { printk(KERN_ERR "ioremap error for 0x%llx-0x%llx, requested 0x%lx, got 0x%lx\n", (unsigned long long)phys_addr,
arch/x86/mm/pat.c +6 -2
@@ -455,7 +455,9 @@ int io_reserve_memtype(resource_size_t start, resource_size_t end,if (ret) goto out_err; -if (!is_new_memtype_allowed (start, size, req_type, new_type)) +if (!is_new_memtype_allowed (start, size, +pgprot2cachemode (__pgprot (req_type)), +pgprot2cachemode (__pgprot (new_type)))) goto out_free;if (kernel_map_sync_memtype (start, size, new_type) <0 )
@@ -630,7 +632,9 @@ static int reserve_pfn_range(u64 paddr, unsigned long size, pgprot_t *vma_prot,if (flags != want_flags) {if (strict_prot || - !is_new_memtype_allowed (paddr, size, want_flags, flags)) { + !is_new_memtype_allowed (paddr, size, +pgprot2cachemode (__pgprot (want_flags)), +pgprot2cachemode (__pgprot (flags)))) {free_memtype (paddr, paddr + size); printk(KERN_ERR "%s:%d map pfn expected mapping type %s" " for [mem %#010Lx-%#010Lx], got %s\n",
Signature · 1 line
1.8.4.5
tip-bot for Juergen Gross
merge bot[tip:x86/mm] x86: Use new cache mode type in asm/pgtable.h
x86: Use new cache mode type in asm/pgtable.h
Commit-ID: d85f33342a0f57acfbe078cdd0c4f590d5608bb7 Gitweb: http://git.kernel.org/tip/d85f33342a0f57acfbe078cdd0c4f590d5608bb7 Author: Juergen Gross <[email protected]> AuthorDate: Mon, 3 Nov 2014 14:01:53 +0100 Committer: Thomas Gleixner <[email protected]> CommitDate: Sun, 16 Nov 2014 11:04:25 +0100
x86: Use new cache mode type in asm/pgtable.h
Instead of directly using the cache mode bits in the pte switch to using the cache mode type. This requires changing some callers of is_new_memtype_allowed() to be changed as well.
--- arch/x86/include/asm/pgtable.h | 19 ++++++++++--------- arch/x86/mm/ioremap.c | 3 ++- arch/x86/mm/pat.c | 8 ++++++-- 3 files changed, 18 insertions(+), 12 deletions(-)
arch/x86/include/asm/pgtable.h +10 -9
@@ -9,9 +9,10 @@/* * Macro to mark a page protection value as UC- */ -#define pgprot_noncached (prot) \ - ((boot_cpu_data.x86 > 3) \ - ? (__pgprot(pgprot_val(prot) | _PAGE_CACHE_UC_MINUS)) \ +#define pgprot_noncached (prot) \ + ((boot_cpu_data.x86 > 3) \ + ? (__pgprot(pgprot_val(prot) | \ + cachemode2protval(_PAGE_CACHE_MODE_UC_MINUS))) \ : (prot))#ifndef __ASSEMBLY__
@@ -404,8 +405,8 @@ static inline pgprot_t pgprot_modify(pgprot_t oldprot, pgprot_t newprot)#define canon_pgprot (p) __pgprot(massage_pgprot(p))static inline int is_new_memtype_allowed (u64 paddr,unsigned long size, -unsigned long flags, -unsigned long new_flags) +enum page_cache_mode pcm, +enum page_cache_mode new_pcm) {/* * PAT type is always WB for untracked ranges, so no need to check.
@@ -419,10 +420,10 @@ static inline int is_new_memtype_allowed(u64 paddr, unsigned long size,* - request is uncached, return cannot be write-back * - request is write-combine, return cannot be write-back */ -if ((flags == _PAGE_CACHE_UC_MINUS && - new_flags == _PAGE_CACHE_WB) || - (flags == _PAGE_CACHE_WC && - new_flags == _PAGE_CACHE_WB)) { +if ((pcm == _PAGE_CACHE_MODE_UC_MINUS && + new_pcm == _PAGE_CACHE_MODE_WB) || + (pcm == _PAGE_CACHE_MODE_WC && + new_pcm == _PAGE_CACHE_MODE_WB)) {return 0 ; }
arch/x86/mm/ioremap.c +2 -1
@@ -142,7 +142,8 @@ static void __iomem *__ioremap_caller(resource_size_t phys_addr,if (prot_val != new_prot_val) {if (!is_new_memtype_allowed (phys_addr, size, - prot_val, new_prot_val)) { +pgprot2cachemode (__pgprot (prot_val)), +pgprot2cachemode (__pgprot (new_prot_val)))) { printk(KERN_ERR "ioremap error for 0x%llx-0x%llx, requested 0x%lx, got 0x%lx\n", (unsigned long long)phys_addr,
arch/x86/mm/pat.c +6 -2
@@ -455,7 +455,9 @@ int io_reserve_memtype(resource_size_t start, resource_size_t end,if (ret) goto out_err; -if (!is_new_memtype_allowed (start, size, req_type, new_type)) +if (!is_new_memtype_allowed (start, size, +pgprot2cachemode (__pgprot (req_type)), +pgprot2cachemode (__pgprot (new_type)))) goto out_free;if (kernel_map_sync_memtype (start, size, new_type) <0 )
@@ -630,7 +632,9 @@ static int reserve_pfn_range(u64 paddr, unsigned long size, pgprot_t *vma_prot,if (flags != want_flags) {if (strict_prot || - !is_new_memtype_allowed (paddr, size, want_flags, flags)) { + !is_new_memtype_allowed (paddr, size, +pgprot2cachemode (__pgprot (want_flags)), +pgprot2cachemode (__pgprot (flags)))) {free_memtype (paddr, paddr + size); printk(KERN_ERR "%s:%d map pfn expected mapping type %s" " for [mem %#010Lx-%#010Lx], got %s\n",
Juergen Gross
maintainerPATCH v6 8/18 x86: Use new cache mode type in mm/iomap_32.c
In reply to Juergen GrossInstead of directly using the cache mode bits in the pte switch to using the cache mode type. This requires to change io_reserve_memtype() as well.
--- arch/x86/include/asm/pat.h | 2 +- arch/x86/mm/iomap_32.c | 12 +++++++----- arch/x86/mm/pat.c | 18 ++++++++++-------- 3 files changed, 18 insertions(+), 14 deletions(-)
arch/x86/include/asm/pat.h +1 -1
@@ -20,7 +20,7 @@ extern int kernel_map_sync_memtype(u64 base, unsigned long size, unsigned long flag);int io_reserve_memtype (resource_size_t start,resource_size_t end, -unsigned long *type); +enum page_cache_mode *pcm);void io_free_memtype (resource_size_t start,resource_size_t end);
arch/x86/mm/iomap_32.c +7 -5
@@ -33,17 +33,17 @@ static int is_io_mapping_possible(resource_size_t base, unsigned long size)int iomap_create_wc (resource_size_t base,unsigned long size,pgprot_t *prot) { -unsigned long flag = _PAGE_CACHE_WC; +enum page_cache_mode pcm = _PAGE_CACHE_MODE_WC;int ret;if (!is_io_mapping_possible (base, size))return -EINVAL ; - ret =io_reserve_memtype (base, base + size, &flag); + ret =io_reserve_memtype (base, base + size, &pcm);if (ret)return ret; - *prot =__pgprot (__PAGE_KERNEL | flag); + *prot =__pgprot (__PAGE_KERNEL |cachemode2protval (pcm));return 0 ; }EXPORT_SYMBOL_GPL (iomap_create_wc);
@@ -82,8 +82,10 @@ iomap_atomic_prot_pfn(unsigned long pfn, pgprot_t prot)* MTRR is UC or WC. UC_MINUS gets the real intention, of the * user, which is "WC if the MTRR is WC, UC if you can't do that." */ -if (!pat_enabled &&pgprot_val (prot) ==pgprot_val (PAGE_KERNEL_WC )) - prot =PAGE_KERNEL_UC_MINUS ; +if (!pat_enabled &&pgprot_val (prot) == + (__PAGE_KERNEL |cachemode2protval (_PAGE_CACHE_MODE_WC))) + prot =__pgprot (__PAGE_KERNEL | +cachemode2protval (_PAGE_CACHE_MODE_UC_MINUS));return (void __force __iomem *)kmap_atomic_prot_pfn (pfn, prot); }
arch/x86/mm/pat.c +10 -8
@@ -442,25 +442,27 @@ static unsigned long lookup_memtype(u64 paddr)* On failure, returns non-zero */ int io_reserve_memtype (resource_size_t start,resource_size_t end, -unsigned long *type) +enum page_cache_mode *type) {resource_size_t size = end - start; -unsigned long req_type = *type; -unsigned long new_type; +enum page_cache_mode req_type = *type; +enum page_cache_mode new_type; +unsigned long new_prot;int ret;WARN_ON_ONCE (iomem_map_sanity_check (start, size)); - ret =reserve_memtype (start, end, req_type, &new_type); + ret =reserve_memtype (start, end,cachemode2protval (req_type), + &new_prot);if (ret) goto out_err; -if (!is_new_memtype_allowed (start, size, -pgprot2cachemode (__pgprot (req_type)), -pgprot2cachemode (__pgprot (new_type)))) + new_type =pgprot2cachemode (__pgprot (new_prot)); + +if (!is_new_memtype_allowed (start, size, req_type, new_type)) goto out_free; -if (kernel_map_sync_memtype (start, size, new_type) <0 ) +if (kernel_map_sync_memtype (start, size, new_prot) <0 ) goto out_free; *type = new_type;
Signature · 1 line
1.8.4.5
tip-bot for Juergen Gross
merge bot[tip:x86/mm] x86: Use new cache mode type in mm/iomap_32.c
x86: Use new cache mode type in mm/iomap_32.c
Commit-ID: 49a3b3cbdf1621678a39bd95a3e67c0f858539c7 Gitweb: http://git.kernel.org/tip/49a3b3cbdf1621678a39bd95a3e67c0f858539c7 Author: Juergen Gross <[email protected]> AuthorDate: Mon, 3 Nov 2014 14:01:54 +0100 Committer: Thomas Gleixner <[email protected]> CommitDate: Sun, 16 Nov 2014 11:04:25 +0100
x86: Use new cache mode type in mm/iomap_32.c
Instead of directly using the cache mode bits in the pte switch to using the cache mode type. This requires to change io_reserve_memtype() as well.
--- arch/x86/include/asm/pat.h | 2 +- arch/x86/mm/iomap_32.c | 12 +++++++----- arch/x86/mm/pat.c | 18 ++++++++++-------- 3 files changed, 18 insertions(+), 14 deletions(-)
arch/x86/include/asm/pat.h +1 -1
@@ -20,7 +20,7 @@ extern int kernel_map_sync_memtype(u64 base, unsigned long size, unsigned long flag);int io_reserve_memtype (resource_size_t start,resource_size_t end, -unsigned long *type); +enum page_cache_mode *pcm);void io_free_memtype (resource_size_t start,resource_size_t end);
arch/x86/mm/iomap_32.c +7 -5
@@ -33,17 +33,17 @@ static int is_io_mapping_possible(resource_size_t base, unsigned long size)int iomap_create_wc (resource_size_t base,unsigned long size,pgprot_t *prot) { -unsigned long flag = _PAGE_CACHE_WC; +enum page_cache_mode pcm = _PAGE_CACHE_MODE_WC;int ret;if (!is_io_mapping_possible (base, size))return -EINVAL ; - ret =io_reserve_memtype (base, base + size, &flag); + ret =io_reserve_memtype (base, base + size, &pcm);if (ret)return ret; - *prot =__pgprot (__PAGE_KERNEL | flag); + *prot =__pgprot (__PAGE_KERNEL |cachemode2protval (pcm));return 0 ; }EXPORT_SYMBOL_GPL (iomap_create_wc);
@@ -82,8 +82,10 @@ iomap_atomic_prot_pfn(unsigned long pfn, pgprot_t prot)* MTRR is UC or WC. UC_MINUS gets the real intention, of the * user, which is "WC if the MTRR is WC, UC if you can't do that." */ -if (!pat_enabled &&pgprot_val (prot) ==pgprot_val (PAGE_KERNEL_WC )) - prot =PAGE_KERNEL_UC_MINUS ; +if (!pat_enabled &&pgprot_val (prot) == + (__PAGE_KERNEL |cachemode2protval (_PAGE_CACHE_MODE_WC))) + prot =__pgprot (__PAGE_KERNEL | +cachemode2protval (_PAGE_CACHE_MODE_UC_MINUS));return (void __force __iomem *)kmap_atomic_prot_pfn (pfn, prot); }
arch/x86/mm/pat.c +10 -8
@@ -442,25 +442,27 @@ static unsigned long lookup_memtype(u64 paddr)* On failure, returns non-zero */ int io_reserve_memtype (resource_size_t start,resource_size_t end, -unsigned long *type) +enum page_cache_mode *type) {resource_size_t size = end - start; -unsigned long req_type = *type; -unsigned long new_type; +enum page_cache_mode req_type = *type; +enum page_cache_mode new_type; +unsigned long new_prot;int ret;WARN_ON_ONCE (iomem_map_sanity_check (start, size)); - ret =reserve_memtype (start, end, req_type, &new_type); + ret =reserve_memtype (start, end,cachemode2protval (req_type), + &new_prot);if (ret) goto out_err; -if (!is_new_memtype_allowed (start, size, -pgprot2cachemode (__pgprot (req_type)), -pgprot2cachemode (__pgprot (new_type)))) + new_type =pgprot2cachemode (__pgprot (new_prot)); + +if (!is_new_memtype_allowed (start, size, req_type, new_type)) goto out_free; -if (kernel_map_sync_memtype (start, size, new_type) <0 ) +if (kernel_map_sync_memtype (start, size, new_prot) <0 ) goto out_free; *type = new_type;
Juergen Gross
maintainerPATCH v6 9/18 x86: Use new cache mode type in track_pfn_remap() and track_pfn_insert()
In reply to Juergen GrossInstead of directly using the cache mode bits in the pte switch to using the cache mode type. As those are the main callers of lookup_memtype(), change this as well.
--- arch/x86/mm/pat.c | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-)
arch/x86/mm/pat.c +16 -16
@@ -394,12 +394,12 @@ int free_memtype(u64 start, u64 end)* * Only to be called when PAT is enabled * -* Returns _PAGE_CACHE_WB, _PAGE_CACHE_WC, _PAGE_CACHE_UC_MINUS or -* _PAGE_CACHE_UC +* Returns _PAGE_CACHE_MODE_WB, _PAGE_CACHE_MODE_WC, _PAGE_CACHE_MODE_UC_MINUS +* or _PAGE_CACHE_MODE_UC */ -static unsigned long lookup_memtype (u64 paddr) +static enum page_cache_mode lookup_memtype (u64 paddr) { -int rettype = _PAGE_CACHE_WB; +enum page_cache_mode rettype = _PAGE_CACHE_MODE_WB;struct memtype *entry;if (x86_platform.is_untracked_pat_range (paddr, paddr +PAGE_SIZE ))
@@ -408,13 +408,13 @@ static unsigned long lookup_memtype(u64 paddr)if (pat_pagerange_is_ram (paddr, paddr +PAGE_SIZE )) {struct page *page; page =pfn_to_page (paddr >>PAGE_SHIFT ); - rettype =get_page_memtype (page); + rettype =pgprot2cachemode (__pgprot (get_page_memtype (page)));/* * -1 from get_page_memtype() implies RAM page is in its * default state and not reserved, and hence of type WB */ if (rettype ==-1 ) - rettype = _PAGE_CACHE_WB; + rettype = _PAGE_CACHE_MODE_WB;return rettype; }
@@ -423,9 +423,9 @@ static unsigned long lookup_memtype(u64 paddr) entry =rbt_memtype_lookup (paddr);if (entry !=NULL ) - rettype = entry->type; + rettype =pgprot2cachemode (__pgprot (entry->type));else - rettype = _PAGE_CACHE_UC_MINUS; + rettype = _PAGE_CACHE_MODE_UC_MINUS;spin_unlock (&memtype_lock);return rettype;
@@ -613,7 +613,7 @@ static int reserve_pfn_range(u64 paddr, unsigned long size, pgprot_t *vma_prot,if (!pat_enabled)return 0 ; - flags =lookup_memtype (paddr); + flags =cachemode2protval (lookup_memtype (paddr));if (want_flags != flags) { printk(KERN_WARNING "%s:%d map pfn RAM range req %s for [mem %#010Lx-%#010Lx], got %s\n", current->comm, current->pid,
@@ -715,7 +715,7 @@ int track_pfn_remap(struct vm_area_struct *vma, pgprot_t *prot, unsigned long pfn, unsigned long addr, unsigned long size) { resource_size_t paddr = (resource_size_t)pfn << PAGE_SHIFT; -unsigned long flags; +enum page_cache_mode pcm;/* reserve the whole chunk starting from paddr */ if (addr == vma->vm_start && size == (vma->vm_end - vma->vm_start)) {
@@ -734,18 +734,18 @@ int track_pfn_remap(struct vm_area_struct *vma, pgprot_t *prot,* For anything smaller than the vma size we set prot based on the * lookup. */ - flags =lookup_memtype (paddr); + pcm =lookup_memtype (paddr);/* Check memtype for the remaining pages */ while (size >PAGE_SIZE ) { size -=PAGE_SIZE ; paddr +=PAGE_SIZE ; -if (flags !=lookup_memtype (paddr)) +if (pcm !=lookup_memtype (paddr))return -EINVAL ; } *prot =__pgprot ((pgprot_val (vma->vm_page_prot) & (~_PAGE_CACHE_MASK)) | - flags); +cachemode2protval (pcm));return 0 ; }
@@ -753,15 +753,15 @@ int track_pfn_remap(struct vm_area_struct *vma, pgprot_t *prot,int track_pfn_insert (struct vm_area_struct *vma,pgprot_t *prot,unsigned long pfn) { -unsigned long flags; +enum page_cache_mode pcm;if (!pat_enabled)return 0 ;/* Set prot based on lookup */ - flags =lookup_memtype ((resource_size_t )pfn <<PAGE_SHIFT ); + pcm =lookup_memtype ((resource_size_t )pfn <<PAGE_SHIFT ); *prot =__pgprot ((pgprot_val (vma->vm_page_prot) & (~_PAGE_CACHE_MASK)) | - flags); +cachemode2protval (pcm));return 0 ; }
Signature · 1 line
1.8.4.5
tip-bot for Juergen Gross
merge bot[tip:x86/mm] x86: Use new cache mode type in track_pfn_remap() and track_pfn_insert()
x86: Use new cache mode type in track_pfn_remap() and track_pfn_insert()
Commit-ID: 2a3746984c98b17b565e6a2c2bbaaaef757db1b4 Gitweb: http://git.kernel.org/tip/2a3746984c98b17b565e6a2c2bbaaaef757db1b4 Author: Juergen Gross <[email protected]> AuthorDate: Mon, 3 Nov 2014 14:01:55 +0100 Committer: Thomas Gleixner <[email protected]> CommitDate: Sun, 16 Nov 2014 11:04:25 +0100
x86: Use new cache mode type in track_pfn_remap() and track_pfn_insert()
Instead of directly using the cache mode bits in the pte switch to using the cache mode type. As those are the main callers of lookup_memtype(), change this as well.
--- arch/x86/mm/pat.c | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-)
arch/x86/mm/pat.c +16 -16
@@ -394,12 +394,12 @@ int free_memtype(u64 start, u64 end)* * Only to be called when PAT is enabled * -* Returns _PAGE_CACHE_WB, _PAGE_CACHE_WC, _PAGE_CACHE_UC_MINUS or -* _PAGE_CACHE_UC +* Returns _PAGE_CACHE_MODE_WB, _PAGE_CACHE_MODE_WC, _PAGE_CACHE_MODE_UC_MINUS +* or _PAGE_CACHE_MODE_UC */ -static unsigned long lookup_memtype (u64 paddr) +static enum page_cache_mode lookup_memtype (u64 paddr) { -int rettype = _PAGE_CACHE_WB; +enum page_cache_mode rettype = _PAGE_CACHE_MODE_WB;struct memtype *entry;if (x86_platform.is_untracked_pat_range (paddr, paddr +PAGE_SIZE ))
@@ -408,13 +408,13 @@ static unsigned long lookup_memtype(u64 paddr)if (pat_pagerange_is_ram (paddr, paddr +PAGE_SIZE )) {struct page *page; page =pfn_to_page (paddr >>PAGE_SHIFT ); - rettype =get_page_memtype (page); + rettype =pgprot2cachemode (__pgprot (get_page_memtype (page)));/* * -1 from get_page_memtype() implies RAM page is in its * default state and not reserved, and hence of type WB */ if (rettype ==-1 ) - rettype = _PAGE_CACHE_WB; + rettype = _PAGE_CACHE_MODE_WB;return rettype; }
@@ -423,9 +423,9 @@ static unsigned long lookup_memtype(u64 paddr) entry =rbt_memtype_lookup (paddr);if (entry !=NULL ) - rettype = entry->type; + rettype =pgprot2cachemode (__pgprot (entry->type));else - rettype = _PAGE_CACHE_UC_MINUS; + rettype = _PAGE_CACHE_MODE_UC_MINUS;spin_unlock (&memtype_lock);return rettype;
@@ -613,7 +613,7 @@ static int reserve_pfn_range(u64 paddr, unsigned long size, pgprot_t *vma_prot,if (!pat_enabled)return 0 ; - flags =lookup_memtype (paddr); + flags =cachemode2protval (lookup_memtype (paddr));if (want_flags != flags) { printk(KERN_WARNING "%s:%d map pfn RAM range req %s for [mem %#010Lx-%#010Lx], got %s\n", current->comm, current->pid,
@@ -715,7 +715,7 @@ int track_pfn_remap(struct vm_area_struct *vma, pgprot_t *prot, unsigned long pfn, unsigned long addr, unsigned long size) { resource_size_t paddr = (resource_size_t)pfn << PAGE_SHIFT; -unsigned long flags; +enum page_cache_mode pcm;/* reserve the whole chunk starting from paddr */ if (addr == vma->vm_start && size == (vma->vm_end - vma->vm_start)) {
@@ -734,18 +734,18 @@ int track_pfn_remap(struct vm_area_struct *vma, pgprot_t *prot,* For anything smaller than the vma size we set prot based on the * lookup. */ - flags =lookup_memtype (paddr); + pcm =lookup_memtype (paddr);/* Check memtype for the remaining pages */ while (size >PAGE_SIZE ) { size -=PAGE_SIZE ; paddr +=PAGE_SIZE ; -if (flags !=lookup_memtype (paddr)) +if (pcm !=lookup_memtype (paddr))return -EINVAL ; } *prot =__pgprot ((pgprot_val (vma->vm_page_prot) & (~_PAGE_CACHE_MASK)) | - flags); +cachemode2protval (pcm));return 0 ; }
@@ -753,15 +753,15 @@ int track_pfn_remap(struct vm_area_struct *vma, pgprot_t *prot,int track_pfn_insert (struct vm_area_struct *vma,pgprot_t *prot,unsigned long pfn) { -unsigned long flags; +enum page_cache_mode pcm;if (!pat_enabled)return 0 ;/* Set prot based on lookup */ - flags =lookup_memtype ((resource_size_t )pfn <<PAGE_SHIFT ); + pcm =lookup_memtype ((resource_size_t )pfn <<PAGE_SHIFT ); *prot =__pgprot ((pgprot_val (vma->vm_page_prot) & (~_PAGE_CACHE_MASK)) | - flags); +cachemode2protval (pcm));return 0 ; }
Juergen Gross
maintainerPATCH v6 10/18 x86: Remove looking for setting of _PAGE_PAT_LARGE in pageattr.c
In reply to Juergen GrossWhen modifying page attributes via change_page_attr_set_clr() don't test for setting _PAGE_PAT_LARGE, as this is
- never done
- PAT support for large pages is not included in the kernel up to now
--- arch/x86/mm/pageattr.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/mm/pageattr.c +1 -1
@@ -1307,7 +1307,7 @@ static int __change_page_attr_set_clr(struct cpa_data *cpa, int checkalias)static inline int cache_attr (pgprot_t attr) {return pgprot_val (attr) & - (_PAGE_PAT | _PAGE_PAT_LARGE | _PAGE_PWT | _PAGE_PCD); + (_PAGE_PAT | _PAGE_PWT | _PAGE_PCD); } static int change_page_attr_set_clr(unsigned long *addr, int numpages,
Signature · 1 line
1.8.4.5
Re: [PATCH V6 10/18] x86: Remove looking for setting of _PAGE_PAT_LARGE in pageattr.c
Quoted 6 lines · Juergen Gross
When modifying page attributes via
change_page_attr_set_clr()don't test for setting_PAGE_PAT_LARGE, as this is
- never done
- PAT support for large pages is not included in the kernel up to nowSigned-off-by Juergen Gross <[email protected]>
tip-bot for Juergen Gross
merge bot[tip:x86/mm] x86: Remove looking for setting of _PAGE_PAT_LARGE in pageattr.c
In reply to Juergen Grossx86: Remove looking for setting of _PAGE_PAT_LARGE in pageattr.c
Commit-ID: 102e19e1955d85f31475416b1ee22980c6462cf8 Gitweb: http://git.kernel.org/tip/102e19e1955d85f31475416b1ee22980c6462cf8 Author: Juergen Gross <[email protected]> AuthorDate: Mon, 3 Nov 2014 14:01:56 +0100 Committer: Thomas Gleixner <[email protected]> CommitDate: Sun, 16 Nov 2014 11:04:25 +0100
x86: Remove looking for setting of _PAGE_PAT_LARGE in pageattr.c
When modifying page attributes via change_page_attr_set_clr() don't test for setting _PAGE_PAT_LARGE, as this is
- never done
- PAT support for large pages is not included in the kernel up to now
--- arch/x86/mm/pageattr.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/mm/pageattr.c +1 -1
@@ -1307,7 +1307,7 @@ static int __change_page_attr_set_clr(struct cpa_data *cpa, int checkalias)static inline int cache_attr (pgprot_t attr) {return pgprot_val (attr) & - (_PAGE_PAT | _PAGE_PAT_LARGE | _PAGE_PWT | _PAGE_PCD); + (_PAGE_PAT | _PAGE_PWT | _PAGE_PCD); } static int change_page_attr_set_clr(unsigned long *addr, int numpages,
Juergen Gross
maintainerPATCH v6 11/18 x86: Use new cache mode type in setting page attributes
In reply to Juergen GrossInstead of directly using the cache mode bits in the pte switch to using the cache mode type in the functions for modifying page attributes.
--- arch/x86/mm/pageattr.c | 52 +++++++++++++++++++++++++++----------------------- 1 file changed, 28 insertions(+), 24 deletions(-)
arch/x86/mm/pageattr.c +28 -24
@@ -1304,12 +1304,6 @@ static int __change_page_attr_set_clr(struct cpa_data *cpa, int checkalias)return 0 ; } -static inline int cache_attr (pgprot_t attr) -{ -return pgprot_val (attr) & - (_PAGE_PAT | _PAGE_PWT | _PAGE_PCD); -} - static int change_page_attr_set_clr(unsigned long *addr, int numpages, pgprot_t mask_set, pgprot_t mask_clr, int force_split, int in_flag,
@@ -1390,7 +1384,7 @@ static int change_page_attr_set_clr(unsigned long *addr, int numpages,* No need to flush, when we did not set any of the caching * attributes: */ - cache =cache_attr (mask_set); + cache = !!pgprot2cachemode (mask_set);/* * On success we use CLFLUSH, when the CPU supports it to
@@ -1445,7 +1439,8 @@ int _set_memory_uc(unsigned long addr, int numpages)* for now UC MINUS. see comments in ioremap_nocache() */ return change_page_attr_set (&addr, numpages, -__pgprot (_PAGE_CACHE_UC_MINUS),0 ); +cachemode2pgprot (_PAGE_CACHE_MODE_UC_MINUS), +0 ); }int set_memory_uc (unsigned long addr,int numpages)
@@ -1474,7 +1469,7 @@ out_err:EXPORT_SYMBOL (set_memory_uc);static int _set_memory_array (unsigned long *addr,int addrinarray, -unsigned long new_type) +enum page_cache_mode new_type) {int i, j;int ret;
@@ -1484,17 +1479,19 @@ static int _set_memory_array(unsigned long *addr, int addrinarray,*/ for (i =0 ; i < addrinarray; i++) { ret =reserve_memtype (__pa (addr[i]),__pa (addr[i]) +PAGE_SIZE , - new_type,NULL ); +cachemode2protval (new_type),NULL );if (ret) goto out_free; } ret =change_page_attr_set (addr, addrinarray, -__pgprot (_PAGE_CACHE_UC_MINUS),1 ); +cachemode2pgprot (_PAGE_CACHE_MODE_UC_MINUS), +1 ); -if (!ret && new_type == _PAGE_CACHE_WC) +if (!ret && new_type == _PAGE_CACHE_MODE_WC) ret =change_page_attr_set_clr (addr, addrinarray, -__pgprot (_PAGE_CACHE_WC), +cachemode2pgprot ( + _PAGE_CACHE_MODE_WC),__pgprot (_PAGE_CACHE_MASK),0 ,CPA_ARRAY ,NULL );if (ret)
@@ -1511,13 +1508,13 @@ out_free:int set_memory_array_uc (unsigned long *addr,int addrinarray) { -return _set_memory_array (addr, addrinarray, _PAGE_CACHE_UC_MINUS); +return _set_memory_array (addr, addrinarray, _PAGE_CACHE_MODE_UC_MINUS); }EXPORT_SYMBOL (set_memory_array_uc);int set_memory_array_wc (unsigned long *addr,int addrinarray) { -return _set_memory_array (addr, addrinarray, _PAGE_CACHE_WC); +return _set_memory_array (addr, addrinarray, _PAGE_CACHE_MODE_WC); }EXPORT_SYMBOL (set_memory_array_wc);
@@ -1527,10 +1524,12 @@ int _set_memory_wc(unsigned long addr, int numpages)unsigned long addr_copy = addr; ret =change_page_attr_set (&addr, numpages, -__pgprot (_PAGE_CACHE_UC_MINUS),0 ); +cachemode2pgprot (_PAGE_CACHE_MODE_UC_MINUS), +0 );if (!ret) { ret =change_page_attr_set_clr (&addr_copy, numpages, -__pgprot (_PAGE_CACHE_WC), +cachemode2pgprot ( + _PAGE_CACHE_MODE_WC),__pgprot (_PAGE_CACHE_MASK),0 ,0 ,NULL ); }
@@ -1564,6 +1563,7 @@ EXPORT_SYMBOL(set_memory_wc);int _set_memory_wb (unsigned long addr,int numpages) { +/* WB cache mode is hard wired to all cache attribute bits being 0 */ return change_page_attr_clear (&addr, numpages,__pgprot (_PAGE_CACHE_MASK),0 ); }
@@ -1586,6 +1586,7 @@ int set_memory_array_wb(unsigned long *addr, int addrinarray)int i;int ret; +/* WB cache mode is hard wired to all cache attribute bits being 0 */ ret =change_page_attr_clear (addr, addrinarray,__pgprot (_PAGE_CACHE_MASK),1 );if (ret)
@@ -1648,7 +1649,7 @@ int set_pages_uc(struct page *page, int numpages)EXPORT_SYMBOL (set_pages_uc);static int _set_pages_array (struct page **pages,int addrinarray, -unsigned long new_type) +enum page_cache_mode new_type) {unsigned long start;unsigned long end;
@@ -1661,15 +1662,17 @@ static int _set_pages_array(struct page **pages, int addrinarray,continue ; start =page_to_pfn (pages[i]) <<PAGE_SHIFT ; end = start +PAGE_SIZE ; -if (reserve_memtype (start, end, new_type,NULL )) +if (reserve_memtype (start, end,cachemode2protval (new_type), +NULL )) goto err_out; } ret =cpa_set_pages_array (pages, addrinarray, -__pgprot (_PAGE_CACHE_UC_MINUS)); -if (!ret && new_type == _PAGE_CACHE_WC) +cachemode2pgprot (_PAGE_CACHE_MODE_UC_MINUS)); +if (!ret && new_type == _PAGE_CACHE_MODE_WC) ret =change_page_attr_set_clr (NULL , addrinarray, -__pgprot (_PAGE_CACHE_WC), +cachemode2pgprot ( + _PAGE_CACHE_MODE_WC),__pgprot (_PAGE_CACHE_MASK),0 ,CPA_PAGES_ARRAY , pages);if (ret)
@@ -1689,13 +1692,13 @@ err_out:int set_pages_array_uc (struct page **pages,int addrinarray) { -return _set_pages_array (pages, addrinarray, _PAGE_CACHE_UC_MINUS); +return _set_pages_array (pages, addrinarray, _PAGE_CACHE_MODE_UC_MINUS); }EXPORT_SYMBOL (set_pages_array_uc);int set_pages_array_wc (struct page **pages,int addrinarray) { -return _set_pages_array (pages, addrinarray, _PAGE_CACHE_WC); +return _set_pages_array (pages, addrinarray, _PAGE_CACHE_MODE_WC); }EXPORT_SYMBOL (set_pages_array_wc);
@@ -1714,6 +1717,7 @@ int set_pages_array_wb(struct page **pages, int addrinarray)unsigned long end;int i; +/* WB cache mode is hard wired to all cache attribute bits being 0 */ retval =cpa_clear_pages_array (pages, addrinarray,__pgprot (_PAGE_CACHE_MASK));if (retval)
Signature · 1 line
1.8.4.5
tip-bot for Juergen Gross
merge bot[tip:x86/mm] x86: Use new cache mode type in setting page attributes
x86: Use new cache mode type in setting page attributes
Commit-ID: c06814d8419a74528500f85faf5fc01f67f8e7e6 Gitweb: http://git.kernel.org/tip/c06814d8419a74528500f85faf5fc01f67f8e7e6 Author: Juergen Gross <[email protected]> AuthorDate: Mon, 3 Nov 2014 14:01:57 +0100 Committer: Thomas Gleixner <[email protected]> CommitDate: Sun, 16 Nov 2014 11:04:25 +0100
x86: Use new cache mode type in setting page attributes
Instead of directly using the cache mode bits in the pte switch to using the cache mode type in the functions for modifying page attributes.
--- arch/x86/mm/pageattr.c | 52 +++++++++++++++++++++++++++----------------------- 1 file changed, 28 insertions(+), 24 deletions(-)
arch/x86/mm/pageattr.c +28 -24
@@ -1304,12 +1304,6 @@ static int __change_page_attr_set_clr(struct cpa_data *cpa, int checkalias)return 0 ; } -static inline int cache_attr (pgprot_t attr) -{ -return pgprot_val (attr) & - (_PAGE_PAT | _PAGE_PWT | _PAGE_PCD); -} - static int change_page_attr_set_clr(unsigned long *addr, int numpages, pgprot_t mask_set, pgprot_t mask_clr, int force_split, int in_flag,
@@ -1390,7 +1384,7 @@ static int change_page_attr_set_clr(unsigned long *addr, int numpages,* No need to flush, when we did not set any of the caching * attributes: */ - cache =cache_attr (mask_set); + cache = !!pgprot2cachemode (mask_set);/* * On success we use CLFLUSH, when the CPU supports it to
@@ -1445,7 +1439,8 @@ int _set_memory_uc(unsigned long addr, int numpages)* for now UC MINUS. see comments in ioremap_nocache() */ return change_page_attr_set (&addr, numpages, -__pgprot (_PAGE_CACHE_UC_MINUS),0 ); +cachemode2pgprot (_PAGE_CACHE_MODE_UC_MINUS), +0 ); }int set_memory_uc (unsigned long addr,int numpages)
@@ -1474,7 +1469,7 @@ out_err:EXPORT_SYMBOL (set_memory_uc);static int _set_memory_array (unsigned long *addr,int addrinarray, -unsigned long new_type) +enum page_cache_mode new_type) {int i, j;int ret;
@@ -1484,17 +1479,19 @@ static int _set_memory_array(unsigned long *addr, int addrinarray,*/ for (i =0 ; i < addrinarray; i++) { ret =reserve_memtype (__pa (addr[i]),__pa (addr[i]) +PAGE_SIZE , - new_type,NULL ); +cachemode2protval (new_type),NULL );if (ret) goto out_free; } ret =change_page_attr_set (addr, addrinarray, -__pgprot (_PAGE_CACHE_UC_MINUS),1 ); +cachemode2pgprot (_PAGE_CACHE_MODE_UC_MINUS), +1 ); -if (!ret && new_type == _PAGE_CACHE_WC) +if (!ret && new_type == _PAGE_CACHE_MODE_WC) ret =change_page_attr_set_clr (addr, addrinarray, -__pgprot (_PAGE_CACHE_WC), +cachemode2pgprot ( + _PAGE_CACHE_MODE_WC),__pgprot (_PAGE_CACHE_MASK),0 ,CPA_ARRAY ,NULL );if (ret)
@@ -1511,13 +1508,13 @@ out_free:int set_memory_array_uc (unsigned long *addr,int addrinarray) { -return _set_memory_array (addr, addrinarray, _PAGE_CACHE_UC_MINUS); +return _set_memory_array (addr, addrinarray, _PAGE_CACHE_MODE_UC_MINUS); }EXPORT_SYMBOL (set_memory_array_uc);int set_memory_array_wc (unsigned long *addr,int addrinarray) { -return _set_memory_array (addr, addrinarray, _PAGE_CACHE_WC); +return _set_memory_array (addr, addrinarray, _PAGE_CACHE_MODE_WC); }EXPORT_SYMBOL (set_memory_array_wc);
@@ -1527,10 +1524,12 @@ int _set_memory_wc(unsigned long addr, int numpages)unsigned long addr_copy = addr; ret =change_page_attr_set (&addr, numpages, -__pgprot (_PAGE_CACHE_UC_MINUS),0 ); +cachemode2pgprot (_PAGE_CACHE_MODE_UC_MINUS), +0 );if (!ret) { ret =change_page_attr_set_clr (&addr_copy, numpages, -__pgprot (_PAGE_CACHE_WC), +cachemode2pgprot ( + _PAGE_CACHE_MODE_WC),__pgprot (_PAGE_CACHE_MASK),0 ,0 ,NULL ); }
@@ -1564,6 +1563,7 @@ EXPORT_SYMBOL(set_memory_wc);int _set_memory_wb (unsigned long addr,int numpages) { +/* WB cache mode is hard wired to all cache attribute bits being 0 */ return change_page_attr_clear (&addr, numpages,__pgprot (_PAGE_CACHE_MASK),0 ); }
@@ -1586,6 +1586,7 @@ int set_memory_array_wb(unsigned long *addr, int addrinarray)int i;int ret; +/* WB cache mode is hard wired to all cache attribute bits being 0 */ ret =change_page_attr_clear (addr, addrinarray,__pgprot (_PAGE_CACHE_MASK),1 );if (ret)
@@ -1648,7 +1649,7 @@ int set_pages_uc(struct page *page, int numpages)EXPORT_SYMBOL (set_pages_uc);static int _set_pages_array (struct page **pages,int addrinarray, -unsigned long new_type) +enum page_cache_mode new_type) {unsigned long start;unsigned long end;
@@ -1661,15 +1662,17 @@ static int _set_pages_array(struct page **pages, int addrinarray,continue ; start =page_to_pfn (pages[i]) <<PAGE_SHIFT ; end = start +PAGE_SIZE ; -if (reserve_memtype (start, end, new_type,NULL )) +if (reserve_memtype (start, end,cachemode2protval (new_type), +NULL )) goto err_out; } ret =cpa_set_pages_array (pages, addrinarray, -__pgprot (_PAGE_CACHE_UC_MINUS)); -if (!ret && new_type == _PAGE_CACHE_WC) +cachemode2pgprot (_PAGE_CACHE_MODE_UC_MINUS)); +if (!ret && new_type == _PAGE_CACHE_MODE_WC) ret =change_page_attr_set_clr (NULL , addrinarray, -__pgprot (_PAGE_CACHE_WC), +cachemode2pgprot ( + _PAGE_CACHE_MODE_WC),__pgprot (_PAGE_CACHE_MASK),0 ,CPA_PAGES_ARRAY , pages);if (ret)
@@ -1689,13 +1692,13 @@ err_out:int set_pages_array_uc (struct page **pages,int addrinarray) { -return _set_pages_array (pages, addrinarray, _PAGE_CACHE_UC_MINUS); +return _set_pages_array (pages, addrinarray, _PAGE_CACHE_MODE_UC_MINUS); }EXPORT_SYMBOL (set_pages_array_uc);int set_pages_array_wc (struct page **pages,int addrinarray) { -return _set_pages_array (pages, addrinarray, _PAGE_CACHE_WC); +return _set_pages_array (pages, addrinarray, _PAGE_CACHE_MODE_WC); }EXPORT_SYMBOL (set_pages_array_wc);
@@ -1714,6 +1717,7 @@ int set_pages_array_wb(struct page **pages, int addrinarray)unsigned long end;int i; +/* WB cache mode is hard wired to all cache attribute bits being 0 */ retval =cpa_clear_pages_array (pages, addrinarray,__pgprot (_PAGE_CACHE_MASK));if (retval)
Juergen Gross
maintainerPATCH v6 12/18 x86: Use new cache mode type in mm/ioremap.c
In reply to Juergen GrossInstead of directly using the cache mode bits in the pte switch to using the cache mode type.
--- arch/x86/include/asm/io.h | 2 +- arch/x86/include/asm/pat.h | 2 +- arch/x86/mm/ioremap.c | 65 +++++++++++++++++++++++++--------------------- arch/x86/mm/pat.c | 12 +++++---- 4 files changed, 44 insertions(+), 37 deletions(-)
arch/x86/include/asm/io.h +1 -1
@@ -314,7 +314,7 @@ extern void *xlate_dev_mem_ptr(unsigned long phys);extern void unxlate_dev_mem_ptr (unsigned long phys,void *addr);extern int ioremap_change_attr (unsigned long vaddr,unsigned long size, -unsigned long prot_val); +enum page_cache_mode pcm);extern void __iomem *ioremap_wc (resource_size_t offset,unsigned long size);extern bool is_early_ioremap_ptep (pte_t *ptep);
arch/x86/include/asm/pat.h +1 -1
@@ -17,7 +17,7 @@ extern int reserve_memtype(u64 start, u64 end,extern int free_memtype (u64 start,u64 end);extern int kernel_map_sync_memtype (u64 base,unsigned long size, -unsigned long flag); +enum page_cache_mode pcm);int io_reserve_memtype (resource_size_t start,resource_size_t end,enum page_cache_mode *pcm);
arch/x86/mm/ioremap.c +35 -30
149 lines not on this page · raw message
arch/x86/mm/pat.c +7 -5
@@ -462,7 +462,7 @@ int io_reserve_memtype(resource_size_t start, resource_size_t end,if (!is_new_memtype_allowed (start, size, req_type, new_type)) goto out_free; -if (kernel_map_sync_memtype (start, size, new_prot) <0 ) +if (kernel_map_sync_memtype (start, size, new_type) <0 ) goto out_free; *type = new_type;
@@ -560,7 +560,8 @@ int phys_mem_access_prot_allowed(struct file *file, unsigned long pfn,* Change the memory type for the physial address range in kernel identity * mapping space if that range is a part of identity map. */ -int kernel_map_sync_memtype (u64 base,unsigned long size,unsigned long flags) +int kernel_map_sync_memtype (u64 base,unsigned long size, +enum page_cache_mode pcm) {unsigned long id_sz;
@@ -578,11 +579,11 @@ int kernel_map_sync_memtype(u64 base, unsigned long size, unsigned long flags) __pa(high_memory) - base : size; -if (ioremap_change_attr ((unsigned long )__va (base), id_sz, flags) <0 ) { +if (ioremap_change_attr ((unsigned long )__va (base), id_sz, pcm) <0 ) {printk (KERN_INFO "%s:%d ioremap_change_attr failed %s " "for [mem %#010Lx-%#010Lx]\n" , current->comm, current->pid, -cattr_name (flags), +cattr_name (cachemode2protval (pcm)), base, (unsigned long long )(base + size-1 ));return -EINVAL ; }
@@ -656,7 +657,8 @@ static int reserve_pfn_range(u64 paddr, unsigned long size, pgprot_t *vma_prot, flags); } -if (kernel_map_sync_memtype (paddr, size, flags) <0 ) { +if (kernel_map_sync_memtype (paddr, size, +pgprot2cachemode (__pgprot (flags))) <0 ) {free_memtype (paddr, paddr + size);return -EINVAL ; }
Signature · 1 line
1.8.4.5
tip-bot for Juergen Gross
merge bot[tip:x86/mm] x86: Use new cache mode type in mm/ioremap.c
x86: Use new cache mode type in mm/ioremap.c
Commit-ID: b14097bd911c2554b0b5271b3a6b2d84044d1843 Gitweb: http://git.kernel.org/tip/b14097bd911c2554b0b5271b3a6b2d84044d1843 Author: Juergen Gross <[email protected]> AuthorDate: Mon, 3 Nov 2014 14:01:58 +0100 Committer: Thomas Gleixner <[email protected]> CommitDate: Sun, 16 Nov 2014 11:04:26 +0100
x86: Use new cache mode type in mm/ioremap.c
Instead of directly using the cache mode bits in the pte switch to using the cache mode type.
--- arch/x86/include/asm/io.h | 2 +- arch/x86/include/asm/pat.h | 2 +- arch/x86/mm/ioremap.c | 65 +++++++++++++++++++++++++--------------------- arch/x86/mm/pat.c | 12 +++++---- 4 files changed, 44 insertions(+), 37 deletions(-)
arch/x86/include/asm/io.h +1 -1
@@ -314,7 +314,7 @@ extern void *xlate_dev_mem_ptr(unsigned long phys);extern void unxlate_dev_mem_ptr (unsigned long phys,void *addr);extern int ioremap_change_attr (unsigned long vaddr,unsigned long size, -unsigned long prot_val); +enum page_cache_mode pcm);extern void __iomem *ioremap_wc (resource_size_t offset,unsigned long size);extern bool is_early_ioremap_ptep (pte_t *ptep);
arch/x86/include/asm/pat.h +1 -1
@@ -17,7 +17,7 @@ extern int reserve_memtype(u64 start, u64 end,extern int free_memtype (u64 start,u64 end);extern int kernel_map_sync_memtype (u64 base,unsigned long size, -unsigned long flag); +enum page_cache_mode pcm);int io_reserve_memtype (resource_size_t start,resource_size_t end,enum page_cache_mode *pcm);
arch/x86/mm/ioremap.c +35 -30
149 lines not on this page · raw message
arch/x86/mm/pat.c +7 -5
@@ -462,7 +462,7 @@ int io_reserve_memtype(resource_size_t start, resource_size_t end,if (!is_new_memtype_allowed (start, size, req_type, new_type)) goto out_free; -if (kernel_map_sync_memtype (start, size, new_prot) <0 ) +if (kernel_map_sync_memtype (start, size, new_type) <0 ) goto out_free; *type = new_type;
@@ -560,7 +560,8 @@ int phys_mem_access_prot_allowed(struct file *file, unsigned long pfn,* Change the memory type for the physial address range in kernel identity * mapping space if that range is a part of identity map. */ -int kernel_map_sync_memtype (u64 base,unsigned long size,unsigned long flags) +int kernel_map_sync_memtype (u64 base,unsigned long size, +enum page_cache_mode pcm) {unsigned long id_sz;
@@ -578,11 +579,11 @@ int kernel_map_sync_memtype(u64 base, unsigned long size, unsigned long flags) __pa(high_memory) - base : size; -if (ioremap_change_attr ((unsigned long )__va (base), id_sz, flags) <0 ) { +if (ioremap_change_attr ((unsigned long )__va (base), id_sz, pcm) <0 ) {printk (KERN_INFO "%s:%d ioremap_change_attr failed %s " "for [mem %#010Lx-%#010Lx]\n" , current->comm, current->pid, -cattr_name (flags), +cattr_name (cachemode2protval (pcm)), base, (unsigned long long )(base + size-1 ));return -EINVAL ; }
@@ -656,7 +657,8 @@ static int reserve_pfn_range(u64 paddr, unsigned long size, pgprot_t *vma_prot, flags); } -if (kernel_map_sync_memtype (paddr, size, flags) <0 ) { +if (kernel_map_sync_memtype (paddr, size, +pgprot2cachemode (__pgprot (flags))) <0 ) {free_memtype (paddr, paddr + size);return -EINVAL ; }
Juergen Gross
maintainerPATCH v6 13/18 x86: Use new cache mode type in memtype related functions
In reply to Juergen GrossInstead of directly using the cache mode bits in the pte switch to using the cache mode type.
--- arch/x86/include/asm/cacheflush.h | 38 ++++++++------ arch/x86/include/asm/pat.h | 2 +- arch/x86/mm/ioremap.c | 5 +- arch/x86/mm/pageattr.c | 9 ++-- arch/x86/mm/pat.c | 102 ++++++++++++++++++-------------------- arch/x86/mm/pat_internal.h | 22 ++++---- arch/x86/mm/pat_rbtree.c | 8 +-- 7 files changed, 96 insertions(+), 90 deletions(-)
arch/x86/include/asm/cacheflush.h +24 -14
@@ -9,10 +9,10 @@/* * X86 PAT uses page flags WC and Uncached together to keep track of * memory type of pages that have backing page struct. X86 PAT supports 3 -* different memory types, _PAGE_CACHE_WB, _PAGE_CACHE_WC and -* _PAGE_CACHE_UC_MINUS and fourth state where page's memory type has not +* different memory types, _PAGE_CACHE_MODE_WB, _PAGE_CACHE_MODE_WC and +* _PAGE_CACHE_MODE_UC_MINUS and fourth state where page's memory type has not * been changed from its default (value of -1 used to denote this). -* Note we do not support _PAGE_CACHE_UC here. +* Note we do not support _PAGE_CACHE_MODE_UC here. */ #define _PGMT_DEFAULT 0
@@ -22,36 +22,40 @@#define _PGMT_MASK (1UL << PG_uncached | 1UL << PG_arch_1)#define _PGMT_CLEAR_MASK (~_PGMT_MASK) -static inline unsigned long get_page_memtype (struct page *pg) +static inline enum page_cache_mode get_page_memtype (struct page *pg) {unsigned long pg_flags = pg->flags & _PGMT_MASK;if (pg_flags == _PGMT_DEFAULT)return -1 ;else if (pg_flags == _PGMT_WC) -return _PAGE_CACHE_WC; +return _PAGE_CACHE_MODE_WC;else if (pg_flags == _PGMT_UC_MINUS) -return _PAGE_CACHE_UC_MINUS; +return _PAGE_CACHE_MODE_UC_MINUS;else -return _PAGE_CACHE_WB; +return _PAGE_CACHE_MODE_WB; } -static inline void set_page_memtype (struct page *pg,unsigned long memtype) +static inline void set_page_memtype (struct page *pg, +enum page_cache_mode memtype) { -unsigned long memtype_flags = _PGMT_DEFAULT; +unsigned long memtype_flags;unsigned long old_flags;unsigned long new_flags;switch (memtype) { -case _PAGE_CACHE_WC: +case _PAGE_CACHE_MODE_WC: memtype_flags = _PGMT_WC;break ; -case _PAGE_CACHE_UC_MINUS: +case _PAGE_CACHE_MODE_UC_MINUS: memtype_flags = _PGMT_UC_MINUS;break ; -case _PAGE_CACHE_WB: +case _PAGE_CACHE_MODE_WB: memtype_flags = _PGMT_WB;break ; +default : + memtype_flags = _PGMT_DEFAULT; +break ; } do {
@@ -60,8 +64,14 @@ static inline void set_page_memtype(struct page *pg, unsigned long memtype) } while (cmpxchg(&pg->flags, old_flags, new_flags) != old_flags); }#else -static inline unsigned long get_page_memtype (struct page *pg) {return -1 ; } -static inline void set_page_memtype (struct page *pg,unsigned long memtype) { } +static inline enum page_cache_mode get_page_memtype (struct page *pg) +{ +return -1 ; +} +static inline void set_page_memtype (struct page *pg, +enum page_cache_mode memtype) +{ +}#endif /*
arch/x86/include/asm/pat.h +1 -1
@@ -13,7 +13,7 @@ static const int pat_enabled;extern void pat_init (void );extern int reserve_memtype (u64 start,u64 end, -unsigned long req_type,unsigned long *ret_type); +enum page_cache_mode req_pcm,enum page_cache_mode *ret_pcm);extern int free_memtype (u64 start,u64 end); extern int kernel_map_sync_memtype(u64 base, unsigned long size,
arch/x86/mm/ioremap.c +1 -4
@@ -83,7 +83,6 @@ static void __iomem *__ioremap_caller(resource_size_t phys_addr,const unsigned long unaligned_size = size;struct vm_struct *area;enum page_cache_mode new_pcm; -unsigned long new_prot_val;pgprot_t prot;int retval;void __iomem *ret_addr;
@@ -135,14 +134,12 @@ static void __iomem *__ioremap_caller(resource_size_t phys_addr, size =PAGE_ALIGN (last_addr+1 ) - phys_addr; retval =reserve_memtype (phys_addr, (u64 )phys_addr + size, -cachemode2protval (pcm), &new_prot_val); + pcm, &new_pcm);if (retval) {printk (KERN_ERR "ioremap reserve_memtype failed %d\n" , retval);return NULL ; } - new_pcm =pgprot2cachemode (__pgprot (new_prot_val)); -if (pcm != new_pcm) {if (!is_new_memtype_allowed (phys_addr, size, pcm, new_pcm)) { printk(KERN_ERR
arch/x86/mm/pageattr.c +4 -5
@@ -1451,7 +1451,7 @@ int set_memory_uc(unsigned long addr, int numpages)* for now UC MINUS. see comments in ioremap_nocache() */ ret =reserve_memtype (__pa (addr),__pa (addr) + numpages *PAGE_SIZE , - _PAGE_CACHE_UC_MINUS,NULL ); + _PAGE_CACHE_MODE_UC_MINUS,NULL );if (ret) goto out_err;
@@ -1479,7 +1479,7 @@ static int _set_memory_array(unsigned long *addr, int addrinarray,*/ for (i =0 ; i < addrinarray; i++) { ret =reserve_memtype (__pa (addr[i]),__pa (addr[i]) +PAGE_SIZE , -cachemode2protval (new_type),NULL ); + new_type,NULL );if (ret) goto out_free; }
@@ -1544,7 +1544,7 @@ int set_memory_wc(unsigned long addr, int numpages)return set_memory_uc (addr, numpages); ret =reserve_memtype (__pa (addr),__pa (addr) + numpages *PAGE_SIZE , - _PAGE_CACHE_WC,NULL ); + _PAGE_CACHE_MODE_WC,NULL );if (ret) goto out_err;
@@ -1662,8 +1662,7 @@ static int _set_pages_array(struct page **pages, int addrinarray,continue ; start =page_to_pfn (pages[i]) <<PAGE_SHIFT ; end = start +PAGE_SIZE ; -if (reserve_memtype (start, end,cachemode2protval (new_type), -NULL )) +if (reserve_memtype (start, end, new_type,NULL )) goto err_out; }
arch/x86/mm/pat.c +49 -53
267 lines not on this page · raw message
arch/x86/mm/pat_internal.h +12 -10
@@ -10,30 +10,32 @@ struct memtype {u64 start;u64 end;u64 subtree_max_end; -unsigned long type; +enum page_cache_mode type;struct rb_node rb; }; -static inline char *cattr_name (unsigned long flags) +static inline char *cattr_name (enum page_cache_mode pcm) { -switch (flags & _PAGE_CACHE_MASK) { -case _PAGE_CACHE_UC:return "uncached" ; -case _PAGE_CACHE_UC_MINUS:return "uncached-minus" ; -case _PAGE_CACHE_WB:return "write-back" ; -case _PAGE_CACHE_WC:return "write-combining" ; -default :return "broken" ; +switch (pcm) { +case _PAGE_CACHE_MODE_UC:return "uncached" ; +case _PAGE_CACHE_MODE_UC_MINUS:return "uncached-minus" ; +case _PAGE_CACHE_MODE_WB:return "write-back" ; +case _PAGE_CACHE_MODE_WC:return "write-combining" ; +case _PAGE_CACHE_MODE_WT:return "write-through" ; +case _PAGE_CACHE_MODE_WP:return "write-protected" ; +default :return "broken" ; } }#ifdef CONFIG_X86_PAT extern int rbt_memtype_check_insert (struct memtype *new, -unsigned long *new_type); +enum page_cache_mode *new_type);extern struct memtype *rbt_memtype_erase (u64 start,u64 end);extern struct memtype *rbt_memtype_lookup (u64 addr);extern int rbt_memtype_copy_nth_element (struct memtype *out,loff_t pos);#else static inline int rbt_memtype_check_insert (struct memtype *new, -unsigned long *new_type) +enum page_cache_mode *new_type) {return 0 ; }static inline struct memtype *rbt_memtype_erase (u64 start,u64 end) {return NULL ; }
arch/x86/mm/pat_rbtree.c +5 -3
@@ -122,11 +122,12 @@ static struct memtype *memtype_rb_exact_match(struct rb_root *root,static int memtype_rb_check_conflict (struct rb_root *root,u64 start,u64 end, -unsigned long reqtype,unsigned long *newtype) +enum page_cache_mode reqtype, +enum page_cache_mode *newtype) {struct rb_node *node;struct memtype *match; -int found_type = reqtype; +enum page_cache_mode found_type = reqtype; match =memtype_rb_lowest_match (&memtype_rbroot, start, end);if (match ==NULL )
@@ -187,7 +188,8 @@ static void memtype_rb_insert(struct rb_root *root, struct memtype *newdata)rb_insert_augmented (&newdata->rb, root, &memtype_rb_augment_cb); } -int rbt_memtype_check_insert (struct memtype *new,unsigned long *ret_type) +int rbt_memtype_check_insert (struct memtype *new, +enum page_cache_mode *ret_type) {int err =0 ;
Signature · 1 line
1.8.4.5
tip-bot for Juergen Gross
merge bot[tip:x86/mm] x86: Use new cache mode type in memtype related functions
x86: Use new cache mode type in memtype related functions
Commit-ID: e00c8cc93c1ac01ecd5049929a50fb47b62bb041 Gitweb: http://git.kernel.org/tip/e00c8cc93c1ac01ecd5049929a50fb47b62bb041 Author: Juergen Gross <[email protected]> AuthorDate: Mon, 3 Nov 2014 14:01:59 +0100 Committer: Thomas Gleixner <[email protected]> CommitDate: Sun, 16 Nov 2014 11:04:26 +0100
x86: Use new cache mode type in memtype related functions
Instead of directly using the cache mode bits in the pte switch to using the cache mode type.
--- arch/x86/include/asm/cacheflush.h | 38 ++++++++------ arch/x86/include/asm/pat.h | 2 +- arch/x86/mm/ioremap.c | 5 +- arch/x86/mm/pageattr.c | 9 ++-- arch/x86/mm/pat.c | 102 ++++++++++++++++++-------------------- arch/x86/mm/pat_internal.h | 22 ++++---- arch/x86/mm/pat_rbtree.c | 8 +-- 7 files changed, 96 insertions(+), 90 deletions(-)
arch/x86/include/asm/cacheflush.h +24 -14
@@ -9,10 +9,10 @@/* * X86 PAT uses page flags WC and Uncached together to keep track of * memory type of pages that have backing page struct. X86 PAT supports 3 -* different memory types, _PAGE_CACHE_WB, _PAGE_CACHE_WC and -* _PAGE_CACHE_UC_MINUS and fourth state where page's memory type has not +* different memory types, _PAGE_CACHE_MODE_WB, _PAGE_CACHE_MODE_WC and +* _PAGE_CACHE_MODE_UC_MINUS and fourth state where page's memory type has not * been changed from its default (value of -1 used to denote this). -* Note we do not support _PAGE_CACHE_UC here. +* Note we do not support _PAGE_CACHE_MODE_UC here. */ #define _PGMT_DEFAULT 0
@@ -22,36 +22,40 @@#define _PGMT_MASK (1UL << PG_uncached | 1UL << PG_arch_1)#define _PGMT_CLEAR_MASK (~_PGMT_MASK) -static inline unsigned long get_page_memtype (struct page *pg) +static inline enum page_cache_mode get_page_memtype (struct page *pg) {unsigned long pg_flags = pg->flags & _PGMT_MASK;if (pg_flags == _PGMT_DEFAULT)return -1 ;else if (pg_flags == _PGMT_WC) -return _PAGE_CACHE_WC; +return _PAGE_CACHE_MODE_WC;else if (pg_flags == _PGMT_UC_MINUS) -return _PAGE_CACHE_UC_MINUS; +return _PAGE_CACHE_MODE_UC_MINUS;else -return _PAGE_CACHE_WB; +return _PAGE_CACHE_MODE_WB; } -static inline void set_page_memtype (struct page *pg,unsigned long memtype) +static inline void set_page_memtype (struct page *pg, +enum page_cache_mode memtype) { -unsigned long memtype_flags = _PGMT_DEFAULT; +unsigned long memtype_flags;unsigned long old_flags;unsigned long new_flags;switch (memtype) { -case _PAGE_CACHE_WC: +case _PAGE_CACHE_MODE_WC: memtype_flags = _PGMT_WC;break ; -case _PAGE_CACHE_UC_MINUS: +case _PAGE_CACHE_MODE_UC_MINUS: memtype_flags = _PGMT_UC_MINUS;break ; -case _PAGE_CACHE_WB: +case _PAGE_CACHE_MODE_WB: memtype_flags = _PGMT_WB;break ; +default : + memtype_flags = _PGMT_DEFAULT; +break ; } do {
@@ -60,8 +64,14 @@ static inline void set_page_memtype(struct page *pg, unsigned long memtype) } while (cmpxchg(&pg->flags, old_flags, new_flags) != old_flags); }#else -static inline unsigned long get_page_memtype (struct page *pg) {return -1 ; } -static inline void set_page_memtype (struct page *pg,unsigned long memtype) { } +static inline enum page_cache_mode get_page_memtype (struct page *pg) +{ +return -1 ; +} +static inline void set_page_memtype (struct page *pg, +enum page_cache_mode memtype) +{ +}#endif /*
arch/x86/include/asm/pat.h +1 -1
@@ -13,7 +13,7 @@ static const int pat_enabled;extern void pat_init (void );extern int reserve_memtype (u64 start,u64 end, -unsigned long req_type,unsigned long *ret_type); +enum page_cache_mode req_pcm,enum page_cache_mode *ret_pcm);extern int free_memtype (u64 start,u64 end); extern int kernel_map_sync_memtype(u64 base, unsigned long size,
arch/x86/mm/ioremap.c +1 -4
@@ -83,7 +83,6 @@ static void __iomem *__ioremap_caller(resource_size_t phys_addr,const unsigned long unaligned_size = size;struct vm_struct *area;enum page_cache_mode new_pcm; -unsigned long new_prot_val;pgprot_t prot;int retval;void __iomem *ret_addr;
@@ -135,14 +134,12 @@ static void __iomem *__ioremap_caller(resource_size_t phys_addr, size =PAGE_ALIGN (last_addr+1 ) - phys_addr; retval =reserve_memtype (phys_addr, (u64 )phys_addr + size, -cachemode2protval (pcm), &new_prot_val); + pcm, &new_pcm);if (retval) {printk (KERN_ERR "ioremap reserve_memtype failed %d\n" , retval);return NULL ; } - new_pcm =pgprot2cachemode (__pgprot (new_prot_val)); -if (pcm != new_pcm) {if (!is_new_memtype_allowed (phys_addr, size, pcm, new_pcm)) { printk(KERN_ERR
arch/x86/mm/pageattr.c +4 -5
@@ -1451,7 +1451,7 @@ int set_memory_uc(unsigned long addr, int numpages)* for now UC MINUS. see comments in ioremap_nocache() */ ret =reserve_memtype (__pa (addr),__pa (addr) + numpages *PAGE_SIZE , - _PAGE_CACHE_UC_MINUS,NULL ); + _PAGE_CACHE_MODE_UC_MINUS,NULL );if (ret) goto out_err;
@@ -1479,7 +1479,7 @@ static int _set_memory_array(unsigned long *addr, int addrinarray,*/ for (i =0 ; i < addrinarray; i++) { ret =reserve_memtype (__pa (addr[i]),__pa (addr[i]) +PAGE_SIZE , -cachemode2protval (new_type),NULL ); + new_type,NULL );if (ret) goto out_free; }
@@ -1544,7 +1544,7 @@ int set_memory_wc(unsigned long addr, int numpages)return set_memory_uc (addr, numpages); ret =reserve_memtype (__pa (addr),__pa (addr) + numpages *PAGE_SIZE , - _PAGE_CACHE_WC,NULL ); + _PAGE_CACHE_MODE_WC,NULL );if (ret) goto out_err;
@@ -1662,8 +1662,7 @@ static int _set_pages_array(struct page **pages, int addrinarray,continue ; start =page_to_pfn (pages[i]) <<PAGE_SHIFT ; end = start +PAGE_SIZE ; -if (reserve_memtype (start, end,cachemode2protval (new_type), -NULL )) +if (reserve_memtype (start, end, new_type,NULL )) goto err_out; }
arch/x86/mm/pat.c +49 -53
267 lines not on this page · raw message
arch/x86/mm/pat_internal.h +12 -10
@@ -10,30 +10,32 @@ struct memtype {u64 start;u64 end;u64 subtree_max_end; -unsigned long type; +enum page_cache_mode type;struct rb_node rb; }; -static inline char *cattr_name (unsigned long flags) +static inline char *cattr_name (enum page_cache_mode pcm) { -switch (flags & _PAGE_CACHE_MASK) { -case _PAGE_CACHE_UC:return "uncached" ; -case _PAGE_CACHE_UC_MINUS:return "uncached-minus" ; -case _PAGE_CACHE_WB:return "write-back" ; -case _PAGE_CACHE_WC:return "write-combining" ; -default :return "broken" ; +switch (pcm) { +case _PAGE_CACHE_MODE_UC:return "uncached" ; +case _PAGE_CACHE_MODE_UC_MINUS:return "uncached-minus" ; +case _PAGE_CACHE_MODE_WB:return "write-back" ; +case _PAGE_CACHE_MODE_WC:return "write-combining" ; +case _PAGE_CACHE_MODE_WT:return "write-through" ; +case _PAGE_CACHE_MODE_WP:return "write-protected" ; +default :return "broken" ; } }#ifdef CONFIG_X86_PAT extern int rbt_memtype_check_insert (struct memtype *new, -unsigned long *new_type); +enum page_cache_mode *new_type);extern struct memtype *rbt_memtype_erase (u64 start,u64 end);extern struct memtype *rbt_memtype_lookup (u64 addr);extern int rbt_memtype_copy_nth_element (struct memtype *out,loff_t pos);#else static inline int rbt_memtype_check_insert (struct memtype *new, -unsigned long *new_type) +enum page_cache_mode *new_type) {return 0 ; }static inline struct memtype *rbt_memtype_erase (u64 start,u64 end) {return NULL ; }
arch/x86/mm/pat_rbtree.c +5 -3
@@ -122,11 +122,12 @@ static struct memtype *memtype_rb_exact_match(struct rb_root *root,static int memtype_rb_check_conflict (struct rb_root *root,u64 start,u64 end, -unsigned long reqtype,unsigned long *newtype) +enum page_cache_mode reqtype, +enum page_cache_mode *newtype) {struct rb_node *node;struct memtype *match; -int found_type = reqtype; +enum page_cache_mode found_type = reqtype; match =memtype_rb_lowest_match (&memtype_rbroot, start, end);if (match ==NULL )
@@ -187,7 +188,8 @@ static void memtype_rb_insert(struct rb_root *root, struct memtype *newdata)rb_insert_augmented (&newdata->rb, root, &memtype_rb_augment_cb); } -int rbt_memtype_check_insert (struct memtype *new,unsigned long *ret_type) +int rbt_memtype_check_insert (struct memtype *new, +enum page_cache_mode *ret_type) {int err =0 ;
Juergen Gross
maintainerPATCH v6 14/18 x86: Clean up pgtable_types.h
In reply to Juergen GrossRemove no longer used defines from pgtable_types.h as they are not used any longer.
Switch __PAGE_KERNEL_NOCACHE to use cache mode type instead of pte bits.
--- arch/x86/include/asm/pgtable_types.h | 21 +-------------------- 1 file changed, 1 insertion(+), 20 deletions(-)
arch/x86/include/asm/pgtable_types.h +1 -20
@@ -128,11 +128,6 @@ _PAGE_SOFT_DIRTY | _PAGE_NUMA)#define _HPAGE_CHG_MASK (_PAGE_CHG_MASK | _PAGE_PSE | _PAGE_NUMA) -#define _PAGE_CACHE_WB (0) -#define _PAGE_CACHE_WC (_PAGE_PWT) -#define _PAGE_CACHE_UC_MINUS (_PAGE_PCD) -#define _PAGE_CACHE_UC (_PAGE_PCD | _PAGE_PWT) -/* * The cache modes defined here are used to translate between pure SW usage * and the HW defined cache mode bits and/or PAT entries.
@@ -178,41 +173,27 @@ enum page_cache_mode {#define __PAGE_KERNEL_RO (__PAGE_KERNEL & ~_PAGE_RW)#define __PAGE_KERNEL_RX (__PAGE_KERNEL_EXEC & ~_PAGE_RW) -#define __PAGE_KERNEL_EXEC_NOCACHE (__PAGE_KERNEL_EXEC | _PAGE_PCD | _PAGE_PWT) -#define __PAGE_KERNEL_WC (__PAGE_KERNEL | _PAGE_CACHE_WC) -#define __PAGE_KERNEL_NOCACHE (__PAGE_KERNEL | _PAGE_PCD | _PAGE_PWT) -#define __PAGE_KERNEL_UC_MINUS (__PAGE_KERNEL | _PAGE_PCD) +#define __PAGE_KERNEL_NOCACHE (__PAGE_KERNEL | _PAGE_NOCACHE)#define __PAGE_KERNEL_VSYSCALL (__PAGE_KERNEL_RX | _PAGE_USER)#define __PAGE_KERNEL_VVAR (__PAGE_KERNEL_RO | _PAGE_USER) -#define __PAGE_KERNEL_VVAR_NOCACHE (__PAGE_KERNEL_VVAR | _PAGE_PCD | _PAGE_PWT)#define __PAGE_KERNEL_LARGE (__PAGE_KERNEL | _PAGE_PSE) -#define __PAGE_KERNEL_LARGE_NOCACHE (__PAGE_KERNEL | _PAGE_CACHE_UC | _PAGE_PSE)#define __PAGE_KERNEL_LARGE_EXEC (__PAGE_KERNEL_EXEC | _PAGE_PSE)#define __PAGE_KERNEL_IO (__PAGE_KERNEL)#define __PAGE_KERNEL_IO_NOCACHE (__PAGE_KERNEL_NOCACHE) -#define __PAGE_KERNEL_IO_UC_MINUS (__PAGE_KERNEL_UC_MINUS) -#define __PAGE_KERNEL_IO_WC (__PAGE_KERNEL_WC)#define PAGE_KERNEL __pgprot(__PAGE_KERNEL)#define PAGE_KERNEL_RO __pgprot(__PAGE_KERNEL_RO)#define PAGE_KERNEL_EXEC __pgprot(__PAGE_KERNEL_EXEC)#define PAGE_KERNEL_RX __pgprot(__PAGE_KERNEL_RX) -#define PAGE_KERNEL_WC __pgprot(__PAGE_KERNEL_WC)#define PAGE_KERNEL_NOCACHE __pgprot(__PAGE_KERNEL_NOCACHE) -#define PAGE_KERNEL_UC_MINUS __pgprot(__PAGE_KERNEL_UC_MINUS) -#define PAGE_KERNEL_EXEC_NOCACHE __pgprot(__PAGE_KERNEL_EXEC_NOCACHE)#define PAGE_KERNEL_LARGE __pgprot(__PAGE_KERNEL_LARGE) -#define PAGE_KERNEL_LARGE_NOCACHE __pgprot(__PAGE_KERNEL_LARGE_NOCACHE)#define PAGE_KERNEL_LARGE_EXEC __pgprot(__PAGE_KERNEL_LARGE_EXEC)#define PAGE_KERNEL_VSYSCALL __pgprot(__PAGE_KERNEL_VSYSCALL)#define PAGE_KERNEL_VVAR __pgprot(__PAGE_KERNEL_VVAR) -#define PAGE_KERNEL_VVAR_NOCACHE __pgprot(__PAGE_KERNEL_VVAR_NOCACHE)#define PAGE_KERNEL_IO __pgprot(__PAGE_KERNEL_IO)#define PAGE_KERNEL_IO_NOCACHE __pgprot(__PAGE_KERNEL_IO_NOCACHE) -#define PAGE_KERNEL_IO_UC_MINUS __pgprot(__PAGE_KERNEL_IO_UC_MINUS) -#define PAGE_KERNEL_IO_WC __pgprot(__PAGE_KERNEL_IO_WC)/* xwr */ #define __P000 PAGE_NONE
Signature · 1 line
1.8.4.5
tip-bot for Juergen Gross
merge bot[tip:x86/mm] x86: Clean up pgtable_types.h
x86: Clean up pgtable_types.h
Commit-ID: 87ad0b713b1034b6caf559976c35ce47f6d1d1e9 Gitweb: http://git.kernel.org/tip/87ad0b713b1034b6caf559976c35ce47f6d1d1e9 Author: Juergen Gross <[email protected]> AuthorDate: Mon, 3 Nov 2014 14:02:00 +0100 Committer: Thomas Gleixner <[email protected]> CommitDate: Sun, 16 Nov 2014 11:04:26 +0100
x86: Clean up pgtable_types.h
Remove no longer used defines from pgtable_types.h as they are not used any longer.
Switch __PAGE_KERNEL_NOCACHE to use cache mode type instead of pte bits.
--- arch/x86/include/asm/pgtable_types.h | 21 +-------------------- 1 file changed, 1 insertion(+), 20 deletions(-)
arch/x86/include/asm/pgtable_types.h +1 -20
@@ -128,11 +128,6 @@ _PAGE_SOFT_DIRTY | _PAGE_NUMA)#define _HPAGE_CHG_MASK (_PAGE_CHG_MASK | _PAGE_PSE | _PAGE_NUMA) -#define _PAGE_CACHE_WB (0) -#define _PAGE_CACHE_WC (_PAGE_PWT) -#define _PAGE_CACHE_UC_MINUS (_PAGE_PCD) -#define _PAGE_CACHE_UC (_PAGE_PCD | _PAGE_PWT) -/* * The cache modes defined here are used to translate between pure SW usage * and the HW defined cache mode bits and/or PAT entries.
@@ -178,41 +173,27 @@ enum page_cache_mode {#define __PAGE_KERNEL_RO (__PAGE_KERNEL & ~_PAGE_RW)#define __PAGE_KERNEL_RX (__PAGE_KERNEL_EXEC & ~_PAGE_RW) -#define __PAGE_KERNEL_EXEC_NOCACHE (__PAGE_KERNEL_EXEC | _PAGE_PCD | _PAGE_PWT) -#define __PAGE_KERNEL_WC (__PAGE_KERNEL | _PAGE_CACHE_WC) -#define __PAGE_KERNEL_NOCACHE (__PAGE_KERNEL | _PAGE_PCD | _PAGE_PWT) -#define __PAGE_KERNEL_UC_MINUS (__PAGE_KERNEL | _PAGE_PCD) +#define __PAGE_KERNEL_NOCACHE (__PAGE_KERNEL | _PAGE_NOCACHE)#define __PAGE_KERNEL_VSYSCALL (__PAGE_KERNEL_RX | _PAGE_USER)#define __PAGE_KERNEL_VVAR (__PAGE_KERNEL_RO | _PAGE_USER) -#define __PAGE_KERNEL_VVAR_NOCACHE (__PAGE_KERNEL_VVAR | _PAGE_PCD | _PAGE_PWT)#define __PAGE_KERNEL_LARGE (__PAGE_KERNEL | _PAGE_PSE) -#define __PAGE_KERNEL_LARGE_NOCACHE (__PAGE_KERNEL | _PAGE_CACHE_UC | _PAGE_PSE)#define __PAGE_KERNEL_LARGE_EXEC (__PAGE_KERNEL_EXEC | _PAGE_PSE)#define __PAGE_KERNEL_IO (__PAGE_KERNEL)#define __PAGE_KERNEL_IO_NOCACHE (__PAGE_KERNEL_NOCACHE) -#define __PAGE_KERNEL_IO_UC_MINUS (__PAGE_KERNEL_UC_MINUS) -#define __PAGE_KERNEL_IO_WC (__PAGE_KERNEL_WC)#define PAGE_KERNEL __pgprot(__PAGE_KERNEL)#define PAGE_KERNEL_RO __pgprot(__PAGE_KERNEL_RO)#define PAGE_KERNEL_EXEC __pgprot(__PAGE_KERNEL_EXEC)#define PAGE_KERNEL_RX __pgprot(__PAGE_KERNEL_RX) -#define PAGE_KERNEL_WC __pgprot(__PAGE_KERNEL_WC)#define PAGE_KERNEL_NOCACHE __pgprot(__PAGE_KERNEL_NOCACHE) -#define PAGE_KERNEL_UC_MINUS __pgprot(__PAGE_KERNEL_UC_MINUS) -#define PAGE_KERNEL_EXEC_NOCACHE __pgprot(__PAGE_KERNEL_EXEC_NOCACHE)#define PAGE_KERNEL_LARGE __pgprot(__PAGE_KERNEL_LARGE) -#define PAGE_KERNEL_LARGE_NOCACHE __pgprot(__PAGE_KERNEL_LARGE_NOCACHE)#define PAGE_KERNEL_LARGE_EXEC __pgprot(__PAGE_KERNEL_LARGE_EXEC)#define PAGE_KERNEL_VSYSCALL __pgprot(__PAGE_KERNEL_VSYSCALL)#define PAGE_KERNEL_VVAR __pgprot(__PAGE_KERNEL_VVAR) -#define PAGE_KERNEL_VVAR_NOCACHE __pgprot(__PAGE_KERNEL_VVAR_NOCACHE)#define PAGE_KERNEL_IO __pgprot(__PAGE_KERNEL_IO)#define PAGE_KERNEL_IO_NOCACHE __pgprot(__PAGE_KERNEL_IO_NOCACHE) -#define PAGE_KERNEL_IO_UC_MINUS __pgprot(__PAGE_KERNEL_IO_UC_MINUS) -#define PAGE_KERNEL_IO_WC __pgprot(__PAGE_KERNEL_IO_WC)/* xwr */ #define __P000 PAGE_NONE
Juergen Gross
maintainerPATCH v6 15/18 x86: Support PAT bit in pagetable dump for lower levels
In reply to Juergen GrossDumping page table protection bits is not correct for entries on levels 2 and 3 regarding the PAT bit, which is at a different position as on level 4.
--- arch/x86/mm/dump_pagetables.c | 24 +++++++++++------------- 1 file changed, 11 insertions(+), 13 deletions(-)
arch/x86/mm/dump_pagetables.c +11 -13
@@ -126,7 +126,7 @@ static void printk_prot(struct seq_file *m, pgprot_t prot, int level, bool dmsg)if (!pgprot_val (prot)) {/* Not present */ -pt_dump_cont_printf (m, dmsg," " ); +pt_dump_cont_printf (m, dmsg," " ); }else {if (pr & _PAGE_USER)pt_dump_cont_printf (m, dmsg,"USR " );
@@ -145,18 +145,16 @@ static void printk_prot(struct seq_file *m, pgprot_t prot, int level, bool dmsg)else pt_dump_cont_printf (m, dmsg," " ); -/* Bit 9 has a different meaning on level 3 vs 4 */ -if (level <=3 ) { -if (pr & _PAGE_PSE) -pt_dump_cont_printf (m, dmsg,"PSE " ); -else -pt_dump_cont_printf (m, dmsg," " ); - }else { -if (pr & _PAGE_PAT) -pt_dump_cont_printf (m, dmsg,"pat " ); -else -pt_dump_cont_printf (m, dmsg," " ); - } +/* Bit 7 has a different meaning on level 3 vs 4 */ +if (level <=3 && pr & _PAGE_PSE) +pt_dump_cont_printf (m, dmsg,"PSE " ); +else +pt_dump_cont_printf (m, dmsg," " ); +if ((level ==4 && pr & _PAGE_PAT) || + ((level ==3 || level ==2 ) && pr & _PAGE_PAT_LARGE)) +pt_dump_cont_printf (m, dmsg,"pat " ); +else +pt_dump_cont_printf (m, dmsg," " );if (pr & _PAGE_GLOBAL)pt_dump_cont_printf (m, dmsg,"GLB " );else
Signature · 1 line
1.8.4.5
tip-bot for Juergen Gross
merge bot[tip:x86/mm] x86: Support PAT bit in pagetable dump for lower levels
x86: Support PAT bit in pagetable dump for lower levels
Commit-ID: f439c429c320981943f8b64b2a4049d946cb492b Gitweb: http://git.kernel.org/tip/f439c429c320981943f8b64b2a4049d946cb492b Author: Juergen Gross <[email protected]> AuthorDate: Mon, 3 Nov 2014 14:02:01 +0100 Committer: Thomas Gleixner <[email protected]> CommitDate: Sun, 16 Nov 2014 11:04:26 +0100
x86: Support PAT bit in pagetable dump for lower levels
Dumping page table protection bits is not correct for entries on levels 2 and 3 regarding the PAT bit, which is at a different position as on level 4.
--- arch/x86/mm/dump_pagetables.c | 24 +++++++++++------------- 1 file changed, 11 insertions(+), 13 deletions(-)
arch/x86/mm/dump_pagetables.c +11 -13
@@ -126,7 +126,7 @@ static void printk_prot(struct seq_file *m, pgprot_t prot, int level, bool dmsg)if (!pgprot_val (prot)) {/* Not present */ -pt_dump_cont_printf (m, dmsg," " ); +pt_dump_cont_printf (m, dmsg," " ); }else {if (pr & _PAGE_USER)pt_dump_cont_printf (m, dmsg,"USR " );
@@ -145,18 +145,16 @@ static void printk_prot(struct seq_file *m, pgprot_t prot, int level, bool dmsg)else pt_dump_cont_printf (m, dmsg," " ); -/* Bit 9 has a different meaning on level 3 vs 4 */ -if (level <=3 ) { -if (pr & _PAGE_PSE) -pt_dump_cont_printf (m, dmsg,"PSE " ); -else -pt_dump_cont_printf (m, dmsg," " ); - }else { -if (pr & _PAGE_PAT) -pt_dump_cont_printf (m, dmsg,"pat " ); -else -pt_dump_cont_printf (m, dmsg," " ); - } +/* Bit 7 has a different meaning on level 3 vs 4 */ +if (level <=3 && pr & _PAGE_PSE) +pt_dump_cont_printf (m, dmsg,"PSE " ); +else +pt_dump_cont_printf (m, dmsg," " ); +if ((level ==4 && pr & _PAGE_PAT) || + ((level ==3 || level ==2 ) && pr & _PAGE_PAT_LARGE)) +pt_dump_cont_printf (m, dmsg,"pat " ); +else +pt_dump_cont_printf (m, dmsg," " );if (pr & _PAGE_GLOBAL)pt_dump_cont_printf (m, dmsg,"GLB " );else
Juergen Gross
maintainerPATCH v6 16/18 x86: Respect PAT bit when copying pte values between large and normal pages
In reply to Juergen GrossThe PAT bit in the ptes is not moved to the correct position when copying page protection attributes between entries of different sized pages. Translate the ptes according to their page size.
--- arch/x86/mm/pageattr.c | 33 +++++++++++++++++++++++---------- 1 file changed, 23 insertions(+), 10 deletions(-)
arch/x86/mm/pageattr.c +23 -10
@@ -485,14 +485,23 @@ try_preserve_large_page(pte_t *kpte, unsigned long address,/* * We are safe now. Check whether the new pgprot is the same: +* Convert protection attributes to 4k-format, as cpa->mask* are set +* up accordingly. */ old_pte = *kpte; - old_prot = req_prot =pte_pgprot (old_pte); + old_prot = req_prot =pgprot_large_2_4k (pte_pgprot (old_pte));pgprot_val (req_prot) &= ~pgprot_val (cpa->mask_clr);pgprot_val (req_prot) |=pgprot_val (cpa->mask_set);/* +* req_prot is in format of 4k pages. It must be converted to large +* page format: the caching mode includes the PAT bit located at +* different bit positions in the two formats. +*/ + req_prot =pgprot_4k_2_large (req_prot); + +/* * Set the PSE and GLOBAL flags only if the PRESENT flag is * set otherwise pmd_present/pmd_huge will return true even on * a non present pmd. The canon_pgprot will clear _PAGE_GLOBAL
@@ -585,13 +594,10 @@ __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address,paravirt_alloc_pte (&init_mm,page_to_pfn (base)); ref_prot =pte_pgprot (pte_clrhuge (*kpte)); -/* -* If we ever want to utilize the PAT bit, we need to -* update this function to make sure it's converted from -* bit 12 to bit 7 when we cross from the 2MB level to -* the 4K level: -*/ -WARN_ON_ONCE (pgprot_val (ref_prot) & _PAGE_PAT_LARGE); + +/* promote PAT bit to correct position */ +if (level ==PG_LEVEL_2M ) + ref_prot =pgprot_large_2_4k (ref_prot);#ifdef CONFIG_X86_64 if (level ==PG_LEVEL_1G ) {
@@ -879,6 +885,7 @@ static int populate_pmd(struct cpa_data *cpa, {unsigned int cur_pages =0 ;pmd_t *pmd; +pgprot_t pmd_pgprot;/* * Not on a 2M boundary?
@@ -910,6 +917,8 @@ static int populate_pmd(struct cpa_data *cpa,if (num_pages == cur_pages)return cur_pages; + pmd_pgprot =pgprot_4k_2_large (pgprot); +while (end - start >=PMD_SIZE ) {/*
@@ -921,7 +930,8 @@ static int populate_pmd(struct cpa_data *cpa, pmd =pmd_offset (pud, start); -set_pmd (pmd,__pmd (cpa->pfn | _PAGE_PSE |massage_pgprot (pgprot))); +set_pmd (pmd,__pmd (cpa->pfn | _PAGE_PSE | +massage_pgprot (pmd_pgprot))); start +=PMD_SIZE ; cpa->pfn +=PMD_SIZE ;
@@ -949,6 +959,7 @@ static int populate_pud(struct cpa_data *cpa, unsigned long start, pgd_t *pgd,pud_t *pud;unsigned long end;int cur_pages =0 ; +pgprot_t pud_pgprot; end = start + (cpa->numpages <<PAGE_SHIFT );
@@ -986,12 +997,14 @@ static int populate_pud(struct cpa_data *cpa, unsigned long start, pgd_t *pgd,return cur_pages; pud =pud_offset (pgd, start); + pud_pgprot =pgprot_4k_2_large (pgprot);/* * Map everything starting from the Gb boundary, possibly with 1G pages */ while (end - start >=PUD_SIZE ) { -set_pud (pud,__pud (cpa->pfn | _PAGE_PSE |massage_pgprot (pgprot))); +set_pud (pud,__pud (cpa->pfn | _PAGE_PSE | +massage_pgprot (pud_pgprot))); start +=PUD_SIZE ; cpa->pfn +=PUD_SIZE ;
Signature · 1 line
1.8.4.5
tip-bot for Juergen Gross
merge bot[tip:x86/mm] x86: Respect PAT bit when copying pte values between large and normal pages
x86: Respect PAT bit when copying pte values between large and normal pages
Commit-ID: f5b2831d654167d77da8afbef4d2584897b12d0c Gitweb: http://git.kernel.org/tip/f5b2831d654167d77da8afbef4d2584897b12d0c Author: Juergen Gross <[email protected]> AuthorDate: Mon, 3 Nov 2014 14:02:02 +0100 Committer: Thomas Gleixner <[email protected]> CommitDate: Sun, 16 Nov 2014 11:04:26 +0100
x86: Respect PAT bit when copying pte values between large and normal pages
The PAT bit in the ptes is not moved to the correct position when copying page protection attributes between entries of different sized pages. Translate the ptes according to their page size.
--- arch/x86/mm/pageattr.c | 33 +++++++++++++++++++++++---------- 1 file changed, 23 insertions(+), 10 deletions(-)
arch/x86/mm/pageattr.c +23 -10
@@ -485,14 +485,23 @@ try_preserve_large_page(pte_t *kpte, unsigned long address,/* * We are safe now. Check whether the new pgprot is the same: +* Convert protection attributes to 4k-format, as cpa->mask* are set +* up accordingly. */ old_pte = *kpte; - old_prot = req_prot =pte_pgprot (old_pte); + old_prot = req_prot =pgprot_large_2_4k (pte_pgprot (old_pte));pgprot_val (req_prot) &= ~pgprot_val (cpa->mask_clr);pgprot_val (req_prot) |=pgprot_val (cpa->mask_set);/* +* req_prot is in format of 4k pages. It must be converted to large +* page format: the caching mode includes the PAT bit located at +* different bit positions in the two formats. +*/ + req_prot =pgprot_4k_2_large (req_prot); + +/* * Set the PSE and GLOBAL flags only if the PRESENT flag is * set otherwise pmd_present/pmd_huge will return true even on * a non present pmd. The canon_pgprot will clear _PAGE_GLOBAL
@@ -585,13 +594,10 @@ __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address,paravirt_alloc_pte (&init_mm,page_to_pfn (base)); ref_prot =pte_pgprot (pte_clrhuge (*kpte)); -/* -* If we ever want to utilize the PAT bit, we need to -* update this function to make sure it's converted from -* bit 12 to bit 7 when we cross from the 2MB level to -* the 4K level: -*/ -WARN_ON_ONCE (pgprot_val (ref_prot) & _PAGE_PAT_LARGE); + +/* promote PAT bit to correct position */ +if (level ==PG_LEVEL_2M ) + ref_prot =pgprot_large_2_4k (ref_prot);#ifdef CONFIG_X86_64 if (level ==PG_LEVEL_1G ) {
@@ -879,6 +885,7 @@ static int populate_pmd(struct cpa_data *cpa, {unsigned int cur_pages =0 ;pmd_t *pmd; +pgprot_t pmd_pgprot;/* * Not on a 2M boundary?
@@ -910,6 +917,8 @@ static int populate_pmd(struct cpa_data *cpa,if (num_pages == cur_pages)return cur_pages; + pmd_pgprot =pgprot_4k_2_large (pgprot); +while (end - start >=PMD_SIZE ) {/*
@@ -921,7 +930,8 @@ static int populate_pmd(struct cpa_data *cpa, pmd =pmd_offset (pud, start); -set_pmd (pmd,__pmd (cpa->pfn | _PAGE_PSE |massage_pgprot (pgprot))); +set_pmd (pmd,__pmd (cpa->pfn | _PAGE_PSE | +massage_pgprot (pmd_pgprot))); start +=PMD_SIZE ; cpa->pfn +=PMD_SIZE ;
@@ -949,6 +959,7 @@ static int populate_pud(struct cpa_data *cpa, unsigned long start, pgd_t *pgd,pud_t *pud;unsigned long end;int cur_pages =0 ; +pgprot_t pud_pgprot; end = start + (cpa->numpages <<PAGE_SHIFT );
@@ -986,12 +997,14 @@ static int populate_pud(struct cpa_data *cpa, unsigned long start, pgd_t *pgd,return cur_pages; pud =pud_offset (pgd, start); + pud_pgprot =pgprot_4k_2_large (pgprot);/* * Map everything starting from the Gb boundary, possibly with 1G pages */ while (end - start >=PUD_SIZE ) { -set_pud (pud,__pud (cpa->pfn | _PAGE_PSE |massage_pgprot (pgprot))); +set_pud (pud,__pud (cpa->pfn | _PAGE_PSE | +massage_pgprot (pud_pgprot))); start +=PUD_SIZE ; cpa->pfn +=PUD_SIZE ;
Juergen Gross
maintainerPATCH v6 17/18 x86: Enable PAT to use cache mode translation tables
In reply to Juergen GrossUpdate the translation tables from cache mode to pgprot values according to the PAT settings. This enables changing the cache attributes of a PAT index in just one place without having to change at the users side.
With this change it is possible to use the same kernel with different PAT configurations, e.g. supporting Xen.
--- arch/x86/include/asm/pat.h | 1 + arch/x86/include/asm/pgtable_types.h | 4 +++ arch/x86/mm/init.c | 8 ++++++ arch/x86/mm/mm_internal.h | 2 ++ arch/x86/mm/pat.c | 50 ++++++++++++++++++++++++++++++++++-- 5 files changed, 63 insertions(+), 2 deletions(-)
arch/x86/include/asm/pat.h +1 -0
@@ -11,6 +11,7 @@ static const int pat_enabled;#endif extern void pat_init (void ); +void pat_init_cache_modes (void );extern int reserve_memtype (u64 start,u64 end,enum page_cache_mode req_pcm,enum page_cache_mode *ret_pcm);
arch/x86/include/asm/pgtable_types.h +4 -0
@@ -351,6 +351,10 @@ extern uint8_t __pte2cachemode_tbl[8]; ((((cb) >> (_PAGE_BIT_PAT -2 )) &4 ) | \ (((cb) >> (_PAGE_BIT_PCD -1 )) &2 ) | \ (((cb) >> _PAGE_BIT_PWT) & 1)) +#define __cm_idx2pte (i) \ + ((((i) & 4) << (_PAGE_BIT_PAT - 2)) | \ + (((i) & 2) << (_PAGE_BIT_PCD - 1)) | \ + (((i) & 1) << _PAGE_BIT_PWT))static inline unsigned long cachemode2protval (enum page_cache_mode pcm) {
arch/x86/mm/init.c +8 -0
@@ -716,3 +716,11 @@ void __init zone_sizes_init(void)free_area_init_nodes (max_zone_pfns); } +void update_cache_mode_entry (unsigned entry,enum page_cache_mode cache) +{ +/* entry 0 MUST be WB (hardwired to speed up translations) */ +BUG_ON (!entry && cache != _PAGE_CACHE_MODE_WB); + + __cachemode2pte_tbl[cache] =__cm_idx2pte (entry); + __pte2cachemode_tbl[entry] = cache; +}
arch/x86/mm/mm_internal.h +2 -0
@@ -16,4 +16,6 @@ void zone_sizes_init(void);extern int after_bootmem; +void update_cache_mode_entry (unsigned entry,enum page_cache_mode cache); +#endif /* __X86_MM_INTERNAL_H */
arch/x86/mm/pat.c +48 -2
68 lines not on this page · raw message
Signature · 1 line
1.8.4.5
tip-bot for Juergen Gross
merge bot[tip:x86/mm] x86: Enable PAT to use cache mode translation tables
x86: Enable PAT to use cache mode translation tables
Commit-ID: bd809af16e3ab1f8d55b3e2928c47c67e2a865d2 Gitweb: http://git.kernel.org/tip/bd809af16e3ab1f8d55b3e2928c47c67e2a865d2 Author: Juergen Gross <[email protected]> AuthorDate: Mon, 3 Nov 2014 14:02:03 +0100 Committer: Thomas Gleixner <[email protected]> CommitDate: Sun, 16 Nov 2014 11:04:26 +0100
x86: Enable PAT to use cache mode translation tables
Update the translation tables from cache mode to pgprot values according to the PAT settings. This enables changing the cache attributes of a PAT index in just one place without having to change at the users side.
With this change it is possible to use the same kernel with different PAT configurations, e.g. supporting Xen.
--- arch/x86/include/asm/pat.h | 1 + arch/x86/include/asm/pgtable_types.h | 4 +++ arch/x86/mm/init.c | 8 ++++++ arch/x86/mm/mm_internal.h | 2 ++ arch/x86/mm/pat.c | 50 ++++++++++++++++++++++++++++++++++-- 5 files changed, 63 insertions(+), 2 deletions(-)
arch/x86/include/asm/pat.h +1 -0
@@ -11,6 +11,7 @@ static const int pat_enabled;#endif extern void pat_init (void ); +void pat_init_cache_modes (void );extern int reserve_memtype (u64 start,u64 end,enum page_cache_mode req_pcm,enum page_cache_mode *ret_pcm);
arch/x86/include/asm/pgtable_types.h +4 -0
@@ -351,6 +351,10 @@ extern uint8_t __pte2cachemode_tbl[8]; ((((cb) >> (_PAGE_BIT_PAT -2 )) &4 ) | \ (((cb) >> (_PAGE_BIT_PCD -1 )) &2 ) | \ (((cb) >> _PAGE_BIT_PWT) & 1)) +#define __cm_idx2pte (i) \ + ((((i) & 4) << (_PAGE_BIT_PAT - 2)) | \ + (((i) & 2) << (_PAGE_BIT_PCD - 1)) | \ + (((i) & 1) << _PAGE_BIT_PWT))static inline unsigned long cachemode2protval (enum page_cache_mode pcm) {
arch/x86/mm/init.c +8 -0
@@ -716,3 +716,11 @@ void __init zone_sizes_init(void)free_area_init_nodes (max_zone_pfns); } +void update_cache_mode_entry (unsigned entry,enum page_cache_mode cache) +{ +/* entry 0 MUST be WB (hardwired to speed up translations) */ +BUG_ON (!entry && cache != _PAGE_CACHE_MODE_WB); + + __cachemode2pte_tbl[cache] =__cm_idx2pte (entry); + __pte2cachemode_tbl[entry] = cache; +}
arch/x86/mm/mm_internal.h +2 -0
@@ -16,4 +16,6 @@ void zone_sizes_init(void);extern int after_bootmem; +void update_cache_mode_entry (unsigned entry,enum page_cache_mode cache); +#endif /* __X86_MM_INTERNAL_H */
arch/x86/mm/pat.c +48 -2
68 lines not on this page · raw message
Juergen Gross
maintainerPATCH v6 18/18 xen: Support Xen pv-domains using PAT
In reply to Juergen GrossWith the dynamical mapping between cache modes and pgprot values it is now possible to use all cache modes via the Xen hypervisor PAT settings in a pv domain.
All to be done is to read the PAT configuration MSR and set up the translation tables accordingly.
--- arch/x86/xen/enlighten.c | 25 +++++++------------------ arch/x86/xen/mmu.c | 47 +---------------------------------------------- arch/x86/xen/xen-ops.h | 1 - 3 files changed, 8 insertions(+), 65 deletions(-)
arch/x86/xen/enlighten.c +7 -18
@@ -1100,12 +1100,6 @@ static int xen_write_msr_safe(unsigned int msr, unsigned low, unsigned high)/* Fast syscall setup is all done in hypercalls, so these are all ignored. Stub them out here to stop Xen console noise. */ -break ; - -case MSR_IA32_CR_PAT : -if (smp_processor_id () ==0 ) -xen_set_pat (((u64 )high <<32 ) | low); -break ;default : ret =native_write_msr_safe (msr, low, high);
@@ -1561,10 +1555,6 @@ asmlinkage __visible void __init xen_start_kernel(void)/* Prevent unwanted bits from being set in PTEs. */ __supported_pte_mask &= ~_PAGE_GLOBAL; -#if 0 - if (!xen_initial_domain()) -#endif - __supported_pte_mask &= ~(_PAGE_PWT | _PAGE_PCD);/* * Prevent page tables from being allocated in highmem, even
@@ -1618,14 +1608,6 @@ asmlinkage __visible void __init xen_start_kernel(void)*/ acpi_numa =-1 ;#endif -#ifdef CONFIG_X86_PAT -/* -* For right now disable the PAT. We should remove this once -* git commit 8eaffa67b43e99ae581622c5133e20b0f48bcef1 -* (xen/pat: Disable PAT support for now) is reverted. -*/ - pat_enabled =0 ; -#endif /* Don't do the full vcpu_info placement stuff until we have a possible map and a non-dummy shared_info. */ per_cpu (xen_vcpu,0 ) = &HYPERVISOR_shared_info->vcpu_info[0 ];
@@ -1636,6 +1618,13 @@ asmlinkage __visible void __init xen_start_kernel(void)xen_raw_console_write ("mapping kernel into physical memory\n" );xen_setup_kernel_pagetable ((pgd_t *)xen_start_info->pt_base, xen_start_info->nr_pages); +/* +* Modify the cache mode translation tables to match Xen's PAT +* configuration. +*/ + +pat_init_cache_modes (); +/* keep using Xen gdt for now; no urgent need to change it */ #ifdef CONFIG_X86_32
arch/x86/xen/mmu.c +1 -46
61 lines not on this page · raw message
arch/x86/xen/xen-ops.h +0 -1
@@ -33,7 +33,6 @@ extern unsigned long xen_max_p2m_pfn;void xen_mm_pin_all (void );void xen_mm_unpin_all (void ); -void xen_set_pat (u64 );char *__init xen_memory_setup (void );char *xen_auto_xlated_memory_setup (void );
Signature · 1 line
1.8.4.5
tip-bot for Juergen Gross
merge bot[tip:x86/mm] xen: Support Xen pv-domains using PAT
xen: Support Xen pv-domains using PAT
Commit-ID: 47591df505129c9774af6cca2debf283a6e56ed7 Gitweb: http://git.kernel.org/tip/47591df505129c9774af6cca2debf283a6e56ed7 Author: Juergen Gross <[email protected]> AuthorDate: Mon, 3 Nov 2014 14:02:04 +0100 Committer: Thomas Gleixner <[email protected]> CommitDate: Sun, 16 Nov 2014 11:04:26 +0100
xen: Support Xen pv-domains using PAT
With the dynamical mapping between cache modes and pgprot values it is now possible to use all cache modes via the Xen hypervisor PAT settings in a pv domain.
All to be done is to read the PAT configuration MSR and set up the translation tables accordingly.
--- arch/x86/xen/enlighten.c | 25 +++++++------------------ arch/x86/xen/mmu.c | 47 +---------------------------------------------- arch/x86/xen/xen-ops.h | 1 - 3 files changed, 8 insertions(+), 65 deletions(-)
arch/x86/xen/enlighten.c +7 -18
@@ -1100,12 +1100,6 @@ static int xen_write_msr_safe(unsigned int msr, unsigned low, unsigned high)/* Fast syscall setup is all done in hypercalls, so these are all ignored. Stub them out here to stop Xen console noise. */ -break ; - -case MSR_IA32_CR_PAT : -if (smp_processor_id () ==0 ) -xen_set_pat (((u64 )high <<32 ) | low); -break ;default : ret =native_write_msr_safe (msr, low, high);
@@ -1561,10 +1555,6 @@ asmlinkage __visible void __init xen_start_kernel(void)/* Prevent unwanted bits from being set in PTEs. */ __supported_pte_mask &= ~_PAGE_GLOBAL; -#if 0 - if (!xen_initial_domain()) -#endif - __supported_pte_mask &= ~(_PAGE_PWT | _PAGE_PCD);/* * Prevent page tables from being allocated in highmem, even
@@ -1618,14 +1608,6 @@ asmlinkage __visible void __init xen_start_kernel(void)*/ acpi_numa =-1 ;#endif -#ifdef CONFIG_X86_PAT -/* -* For right now disable the PAT. We should remove this once -* git commit 8eaffa67b43e99ae581622c5133e20b0f48bcef1 -* (xen/pat: Disable PAT support for now) is reverted. -*/ - pat_enabled =0 ; -#endif /* Don't do the full vcpu_info placement stuff until we have a possible map and a non-dummy shared_info. */ per_cpu (xen_vcpu,0 ) = &HYPERVISOR_shared_info->vcpu_info[0 ];
@@ -1636,6 +1618,13 @@ asmlinkage __visible void __init xen_start_kernel(void)xen_raw_console_write ("mapping kernel into physical memory\n" );xen_setup_kernel_pagetable ((pgd_t *)xen_start_info->pt_base, xen_start_info->nr_pages); +/* +* Modify the cache mode translation tables to match Xen's PAT +* configuration. +*/ + +pat_init_cache_modes (); +/* keep using Xen gdt for now; no urgent need to change it */ #ifdef CONFIG_X86_32
arch/x86/xen/mmu.c +1 -46
61 lines not on this page · raw message
arch/x86/xen/xen-ops.h +0 -1
@@ -33,7 +33,6 @@ extern unsigned long xen_max_p2m_pfn;void xen_mm_pin_all (void );void xen_mm_unpin_all (void ); -void xen_set_pat (u64 );char *__init xen_memory_setup (void );char *xen_auto_xlated_memory_setup (void );
Quoted 33 lines · Juergen Gross
The x86 architecture offers via the PAT (Page Attribute Table) a way to specify different caching modes in page table entries. The PAT MSR contains 8 entries each specifying one of 6 possible cache modes. A pte references one of those entries via 3 bits:
_PAGE_PAT,_PAGE_PWTand_PAGE_PCD.The Linux kernel currently supports only 4 different cache modes. The PAT MSR is set up in a way that the setting of
_PAGE_PATin a pte doesn't matter: the top 4 entries in the PAT MSR are the same as the 4 lower entries.This results in the kernel not supporting e.g. write-through mode. Especially this cache mode would speed up drivers of video cards which now have to use uncached accesses.
OTOH some old processors (Pentium) don't support PAT correctly and the Xen hypervisor has been using a different PAT MSR configuration for some time now and can't change that as this setting is part of the ABI.
This patch set abstracts the cache mode from the pte and introduces tables to translate between cache mode and pte bits (the default cache mode "write back" is hard-wired to PAT entry 0). The tables are statically initialized with values being compatible to old processors and current usage. As soon as the PAT MSR is changed (or - in case of Xen - is read at boot time) the tables are changed accordingly. Requests of mappings with special cache modes are always possible now, in case they are not supported there will be a fallback to a compatible but slower mode.
Summing it up, this patch set adds the following features:
- capability to support WT and WP cache modes on processors with full PAT support
- processors with no or uncorrect PAT support are still working as today, even if WT or WP cache mode are selected by drivers for some pages
- reduction of Xen special handling regarding cache mode
Quoted 5 lines
Changes in V6:
- add new patch 10 (x86: Remove looking for setting of_PAGE_PAT_LARGEinpageattr.c) as suggested by Thomas Gleixner
- replaced SOB of Stefan Bader by "Based-on-patch-by:" as suggested by Borislav Petkov
For patch 01/18 to 16/18:
Thanks,
-Toshi
Juergen Gross
maintainerIngo,
could you take the patches, please?
Juergen
Quoted 109 lines · Juergen Gross
The x86 architecture offers via the PAT (Page Attribute Table) a way to specify different caching modes in page table entries. The PAT MSR contains 8 entries each specifying one of 6 possible cache modes. A pte references one of those entries via 3 bits:
_PAGE_PAT,_PAGE_PWTand_PAGE_PCD.The Linux kernel currently supports only 4 different cache modes. The PAT MSR is set up in a way that the setting of
_PAGE_PATin a pte doesn't matter: the top 4 entries in the PAT MSR are the same as the 4 lower entries.This results in the kernel not supporting e.g. write-through mode. Especially this cache mode would speed up drivers of video cards which now have to use uncached accesses.
OTOH some old processors (Pentium) don't support PAT correctly and the Xen hypervisor has been using a different PAT MSR configuration for some time now and can't change that as this setting is part of the ABI.
This patch set abstracts the cache mode from the pte and introduces tables to translate between cache mode and pte bits (the default cache mode "write back" is hard-wired to PAT entry 0). The tables are statically initialized with values being compatible to old processors and current usage. As soon as the PAT MSR is changed (or - in case of Xen - is read at boot time) the tables are changed accordingly. Requests of mappings with special cache modes are always possible now, in case they are not supported there will be a fallback to a compatible but slower mode.
Summing it up, this patch set adds the following features:
- capability to support WT and WP cache modes on processors with full PAT support
- processors with no or uncorrect PAT support are still working as today, even if WT or WP cache mode are selected by drivers for some pages
- reduction of Xen special handling regarding cache modeChanges in V6:
- add new patch 10 (x86: Remove looking for setting of_PAGE_PAT_LARGEinpageattr.c) as suggested by Thomas Gleixner
- replaced SOB of Stefan Bader by "Based-on-patch-by:" as suggested by Borislav PetkovChanges in V5:
- split up first patch as requested by Ingo Molnar and Thomas Gleixner
- add a helper function inpat_init_cache_modes()as requested by Ingo MolnarChanges in V4:
- rebased to 3.18-rc2Changes in V3:
- corrected two minor nits (UC_MINUS, again) detected by Toshi KaniChanges in V2:
- simplified handling of PAT MSR write under Xen as suggested by David Vrabel
- removed resetting ofpat_enabledunder Xen
- two small corrections requested by Toshi Kani (UC_MINUScache mode in vermilion driver, fix 32 bit kernel build failure)
- correct build error on non-x86 arch by moving definition ofupdate_cache_mode_entry()to x86 specific headerChanges since RFC:
- renamed functions and variables as suggested by Toshi Kani
- corrected cache mode bits for WT and WP
- modified handling of PAT MSR write under Xen as suggested by Jan BeulichJuergen Gross (18):
x86: Make page cache mode a real type
x86: Use new cache mode type ininclude/asm/fb.h
x86: Use new cache mode type indrivers/video/fbdev/gbefb.c
x86: Use new cache mode type indrivers/video/fbdev/vermilion
x86: Use new cache mode type inarch/x86/pci
x86: Use new cache mode type inarch/x86/mm/init_64.c
x86: Use new cache mode type inasm/pgtable.h
x86: Use new cache mode type inmm/iomap_32.c
x86: Use new cache mode type intrack_pfn_remap()and
track_pfn_insert()
x86: Remove looking for setting of_PAGE_PAT_LARGEinpageattr.c
x86: Use new cache mode type in setting page attributes
x86: Use new cache mode type inmm/ioremap.c
x86: Use new cache mode type in memtype related functions
x86: Clean uppgtable_types.h
x86: Support PAT bit in pagetable dump for lower levels
x86: Respect PAT bit when copying pte values between large and normal
pages
x86: Enable PAT to use cache mode translation tables
xen: Support Xen pv-domains using PATarch/x86/include/asm/cacheflush.h | 38 ++++--- arch/x86/include/asm/fb.h | 6 +- arch/x86/include/asm/io.h | 2 +- arch/x86/include/asm/pat.h | 7 +- arch/x86/include/asm/pgtable.h | 19 ++-- arch/x86/include/asm/pgtable_types.h | 96 ++++++++++++---- arch/x86/mm/dump_pagetables.c | 24 ++-- arch/x86/mm/init.c | 37 +++++++ arch/x86/mm/init_64.c | 9 +- arch/x86/mm/iomap_32.c | 12 +- arch/x86/mm/ioremap.c | 63 ++++++----- arch/x86/mm/mm_internal.h | 2 + arch/x86/mm/pageattr.c | 84 ++++++++------ arch/x86/mm/pat.c | 176 +++++++++++++++++++----------- arch/x86/mm/pat_internal.h | 22 ++-- arch/x86/mm/pat_rbtree.c | 8 +- arch/x86/pci/i386.c | 4 +- arch/x86/xen/enlighten.c | 25 ++--- arch/x86/xen/mmu.c | 47 +------- arch/x86/xen/xen-ops.h | 1 - drivers/video/fbdev/gbefb.c | 3 +- drivers/video/fbdev/vermilion/vermilion.c | 6 +- 22 files changed, 412 insertions(+), 279 deletions(-)
Quoted 1 line · Juergen Gross
arch/x86/include/asm/cacheflush.h | 38 ++++---
FYI, this series breaks the UML build:
In file included from /home/mingo/tip/include/linux/highmem.h:11:0,
from /home/mingo/tip/include/linux/pagemap.h:10,
from /home/mingo/tip/include/linux/mempolicy.h:14,
from /home/mingo/tip/include/linux/shmem_fs.h:6,
from /home/mingo/tip/init/do_mounts.c:30:
/home/mingo/tip/arch/x86/include/asm/cacheflush.h:67:36: error: return type is an incomplete type
static inline enum page_cache_mode get_page_memtype(struct page *pg)
^/home/mingo/tip/arch/x86/include/asm/cacheflush.h: In function ‘get_page_memtype’:/home/mingo/tip/arch/x86/include/asm/cacheflush.h:69:2: warning: ‘return’ with a value, in function returning void [enabled by default]
return -1;
^/home/mingo/tip/arch/x86/include/asm/cacheflush.h: At top level:/home/mingo/tip/arch/x86/include/asm/cacheflush.h:72:30: error: parameter 2 (‘memtype’) has incomplete type
enum page_cache_mode memtype)
^/home/mingo/tip/arch/x86/include/asm/cacheflush.h:71:20: error: function declaration isn’t a prototype [-Werror=strict-prototypes]
Thanks,
Ingo
Jürgen Groß
maintainerQuoted 19 lines · Ingo Molnar
Quoted 1 line · Juergen Gross
arch/x86/include/asm/cacheflush.h | 38 ++++---FYI, this series breaks the UML build:
In file included from
/home/mingo/tip/include/linux/highmem.h:11:0,from /home/mingo/tip/include/linux/pagemap.h:10, from /home/mingo/tip/include/linux/mempolicy.h:14, from /home/mingo/tip/include/linux/shmem_fs.h:6, from /home/mingo/tip/init/do_mounts.c:30: /home/mingo/tip/arch/x86/include/asm/cacheflush.h:67:36: error: return type is an incomplete type static inline enum page_cache_mode get_page_memtype(struct page *pg) ^ /home/mingo/tip/arch/x86/include/asm/cacheflush.h: In function ‘get_page_memtype’: /home/mingo/tip/arch/x86/include/asm/cacheflush.h:69:2: warning: ‘return’ with a value, in function returning void [enabled by default] return -1; ^
Quoted 5 lines
/home/mingo/tip/arch/x86/include/asm/cacheflush.h: At top level:/home/mingo/tip/arch/x86/include/asm/cacheflush.h:72:30: error: parameter 2 (‘memtype’) has incomplete type
enumpage_cache_modememtype)
^/home/mingo/tip/arch/x86/include/asm/cacheflush.h:71:20: error: function declaration isn’t a prototype [-Werror=strict-prototypes]
Thomas already committed a fixup. Thank you, Thomas.
Juergen