x86: Full support of PAT
Subsystems (6)
[email protected]
maintainercover x86: Full support of PAT
The x86 architecture offers via the PAT (Page Attribute Table) a way to specify different caching modes in page table entries. The PAT MSR contains 8 entries each specifying one of 6 possible cache modes. A pte references one of those entries via 3 bits: _PAGE_PAT, _PAGE_PWT and _PAGE_PCD.
The Linux kernel currently supports only 4 different cache modes. The PAT MSR is set up in a way that the setting of _PAGE_PAT in a pte doesn't matter: the top 4 entries in the PAT MSR are the same as the 4 lower entries.
This results in the kernel not supporting e.g. write-through mode. Especially this cache mode would speed up drivers of video cards which now have to use uncached accesses.
OTOH some old processors (Pentium) don't support PAT and the Xen hypervisor has been using a different PAT MSR configuration for some time now and can't change that as this setting is part of the ABI.
This patch set abstracts the cache mode from the pte and introduces tables to translate between cache mode and pte bits (the default cache mode "write back" is hard-wired to PAT entry 0). The tables are statically initialized with values being compatible to old processors and current usage. As soon as the PAT MSR is changed (or - in case of Xen - is read at boot time) the tables are changed accordingly. Requests of mappings with special cache modes are always possible now, in case they are not supported there will be a fallback to a compatible but slower mode.
[PATCH RFC 1/3] x86: Make page cache mode a real type [PATCH RFC 2/3] x86: Enable PAT to use cache mode translation tables [PATCH RFC 3/3] Support Xen pv-domains using PAT
[email protected]
maintainerRFC PATCH 1/3 x86: Make page cache mode a real type
From: Juergen Gross <[email protected]>
At the moment there are a lot of places that handle setting or getting the page cache mode by treating the pgprot bits equal to the cache mode. This is only true because there are a lot of assumptions about the setup of the PAT MSR. Otherwise the cache type needs to get translated into pgprot bits and vice versa.
This patch tries to prepare for that by introducing a seperate type for the cache mode and adding functions to translate between those and pgprot values.
To avoid too much performance penalty the translation between cache mode and pgprot values is done via tables which contain the relevant information. Write-back cache mode is hard-wired to be 0, all other modes are configurable via those tables. For large pages there are translation functions as the PAT bit is located at different positions in the ptes of 4k and large pages.
--- arch/x86/include/asm/cacheflush.h | 34 +++++--- arch/x86/include/asm/fb.h | 6 +- arch/x86/include/asm/io.h | 2 +- arch/x86/include/asm/pat.h | 6 +- arch/x86/include/asm/pgtable.h | 19 ++--- arch/x86/include/asm/pgtable_types.h | 92 +++++++++++++++++----- arch/x86/mm/dump_pagetables.c | 24 +++--- arch/x86/mm/init.c | 29 +++++++ arch/x86/mm/init_64.c | 9 ++- arch/x86/mm/iomap_32.c | 15 ++-- arch/x86/mm/ioremap.c | 63 ++++++++------- arch/x86/mm/pageattr.c | 84 ++++++++++++-------- arch/x86/mm/pat.c | 126 +++++++++++++++--------------- arch/x86/mm/pat_internal.h | 20 ++--- arch/x86/mm/pat_rbtree.c | 8 +- arch/x86/pci/i386.c | 4 +- drivers/video/fbdev/gbefb.c | 3 +- drivers/video/fbdev/vermilion/vermilion.c | 6 +- 18 files changed, 339 insertions(+), 211 deletions(-)
arch/x86/include/asm/cacheflush.h +21 -13
@@ -9,10 +9,10 @@/* * X86 PAT uses page flags WC and Uncached together to keep track of * memory type of pages that have backing page struct. X86 PAT supports 3 -* different memory types, _PAGE_CACHE_WB, _PAGE_CACHE_WC and -* _PAGE_CACHE_UC_MINUS and fourth state where page's memory type has not +* different memory types, _PAGE_CACHE_MODE_WB, _PAGE_CACHE_MODE_WC and +* _PAGE_CACHE_MODE_UC_MINUS and fourth state where page's memory type has not * been changed from its default (value of -1 used to denote this). -* Note we do not support _PAGE_CACHE_UC here. +* Note we do not support _PAGE_CACHE_MODE_UC here. */ #define _PGMT_DEFAULT 0
@@ -22,34 +22,36 @@#define _PGMT_MASK (1UL << PG_uncached | 1UL << PG_arch_1)#define _PGMT_CLEAR_MASK (~_PGMT_MASK) -static inline unsigned long get_page_memtype (struct page *pg) +static inline enum page_cache_mode get_page_memtype (struct page *pg) {unsigned long pg_flags = pg->flags & _PGMT_MASK;if (pg_flags == _PGMT_DEFAULT)return -1 ;else if (pg_flags == _PGMT_WC) -return _PAGE_CACHE_WC; +return _PAGE_CACHE_MODE_WC;else if (pg_flags == _PGMT_UC_MINUS) -return _PAGE_CACHE_UC_MINUS; +return _PAGE_CACHE_MODE_UC_MINUS;else -return _PAGE_CACHE_WB; +return _PAGE_CACHE_MODE_WB; } -static inline void set_page_memtype (struct page *pg,unsigned long memtype) +static inline void set_page_memtype (struct page *pg, +enum page_cache_mode memtype) {unsigned long memtype_flags = _PGMT_DEFAULT;unsigned long old_flags;unsigned long new_flags;switch (memtype) { -case _PAGE_CACHE_WC: +case _PAGE_CACHE_MODE_WC: memtype_flags = _PGMT_WC;break ; -case _PAGE_CACHE_UC_MINUS: +case _PAGE_CACHE_MODE_UC_MINUS: memtype_flags = _PGMT_UC_MINUS;break ; -case _PAGE_CACHE_WB: +case _PAGE_CACHE_MODE_WB: +default : memtype_flags = _PGMT_WB;break ; }
@@ -60,8 +62,14 @@ static inline void set_page_memtype(struct page *pg, unsigned long memtype) } while (cmpxchg(&pg->flags, old_flags, new_flags) != old_flags); }#else -static inline unsigned long get_page_memtype (struct page *pg) {return -1 ; } -static inline void set_page_memtype (struct page *pg,unsigned long memtype) { } +static inline enum page_cache_mode get_page_memtype (struct page *pg) +{ +return -1 ; +} +static inline void set_page_memtype (struct page *pg, +enum page_cache_mode memtype) +{ +}#endif /*
arch/x86/include/asm/fb.h +5 -1
@@ -8,8 +8,12 @@static inline void fb_pgprotect (struct file *file,struct vm_area_struct *vma,unsigned long off) { +unsigned long prot; + + prot =pgprot_val (vma->vm_page_prot) & ~_PAGE_CACHE_MASK;if (boot_cpu_data.x86 >3 ) -pgprot_val (vma->vm_page_prot) |= _PAGE_PCD; +pgprot_val (vma->vm_page_prot) = + prot |protval_cachemode (_PAGE_CACHE_MODE_UC); }extern int fb_is_primary_device (struct fb_info *info);
arch/x86/include/asm/io.h +1 -1
@@ -314,7 +314,7 @@ extern void *xlate_dev_mem_ptr(unsigned long phys);extern void unxlate_dev_mem_ptr (unsigned long phys,void *addr);extern int ioremap_change_attr (unsigned long vaddr,unsigned long size, -unsigned long prot_val); +enum page_cache_mode pct);extern void __iomem *ioremap_wc (resource_size_t offset,unsigned long size);extern bool is_early_ioremap_ptep (pte_t *ptep);
arch/x86/include/asm/pat.h +3 -3
@@ -13,14 +13,14 @@ static const int pat_enabled;extern void pat_init (void );extern int reserve_memtype (u64 start,u64 end, -unsigned long req_type,unsigned long *ret_type); +enum page_cache_mode req_pct,enum page_cache_mode *ret_pct);extern int free_memtype (u64 start,u64 end);extern int kernel_map_sync_memtype (u64 base,unsigned long size, -unsigned long flag); +enum page_cache_mode pct);int io_reserve_memtype (resource_size_t start,resource_size_t end, -unsigned long *type); +enum page_cache_mode *pct);void io_free_memtype (resource_size_t start,resource_size_t end);
arch/x86/include/asm/pgtable.h +10 -9
@@ -9,9 +9,10 @@/* * Macro to mark a page protection value as UC- */ -#define pgprot_noncached (prot) \ - ((boot_cpu_data.x86 > 3) \ - ? (__pgprot(pgprot_val(prot) | _PAGE_CACHE_UC_MINUS)) \ +#define pgprot_noncached (prot) \ + ((boot_cpu_data.x86 > 3) \ + ? (__pgprot(pgprot_val(prot) | \ + protval_cachemode(_PAGE_CACHE_MODE_UC_MINUS))) \ : (prot))#ifndef __ASSEMBLY__
@@ -399,8 +400,8 @@ static inline pgprot_t pgprot_modify(pgprot_t oldprot, pgprot_t newprot)#define canon_pgprot (p) __pgprot(massage_pgprot(p))static inline int is_new_memtype_allowed (u64 paddr,unsigned long size, -unsigned long flags, -unsigned long new_flags) +enum page_cache_mode pct, +enum page_cache_mode new_pct) {/* * PAT type is always WB for untracked ranges, so no need to check.
@@ -414,10 +415,10 @@ static inline int is_new_memtype_allowed(u64 paddr, unsigned long size,* - request is uncached, return cannot be write-back * - request is write-combine, return cannot be write-back */ -if ((flags == _PAGE_CACHE_UC_MINUS && - new_flags == _PAGE_CACHE_WB) || - (flags == _PAGE_CACHE_WC && - new_flags == _PAGE_CACHE_WB)) { +if ((pct == _PAGE_CACHE_MODE_UC_MINUS && + new_pct == _PAGE_CACHE_MODE_WB) || + (pct == _PAGE_CACHE_MODE_WC && + new_pct == _PAGE_CACHE_MODE_WB)) {return 0 ; }
arch/x86/include/asm/pgtable_types.h +72 -20
130 lines not on this page · raw message
arch/x86/mm/dump_pagetables.c +11 -13
@@ -122,7 +122,7 @@ static void printk_prot(struct seq_file *m, pgprot_t prot, int level, bool dmsg)if (!pgprot_val (prot)) {/* Not present */ -pt_dump_cont_printf (m, dmsg," " ); +pt_dump_cont_printf (m, dmsg," " ); }else {if (pr & _PAGE_USER)pt_dump_cont_printf (m, dmsg,"USR " );
@@ -141,18 +141,16 @@ static void printk_prot(struct seq_file *m, pgprot_t prot, int level, bool dmsg)else pt_dump_cont_printf (m, dmsg," " ); -/* Bit 9 has a different meaning on level 3 vs 4 */ -if (level <=3 ) { -if (pr & _PAGE_PSE) -pt_dump_cont_printf (m, dmsg,"PSE " ); -else -pt_dump_cont_printf (m, dmsg," " ); - }else { -if (pr & _PAGE_PAT) -pt_dump_cont_printf (m, dmsg,"pat " ); -else -pt_dump_cont_printf (m, dmsg," " ); - } +/* Bit 7 has a different meaning on level 3 vs 4 */ +if (level <=3 && pr & _PAGE_PSE) +pt_dump_cont_printf (m, dmsg,"PSE " ); +else +pt_dump_cont_printf (m, dmsg," " ); +if ((level ==4 && pr & _PAGE_PAT) || + ((level ==3 || level ==2 ) && pr & _PAGE_PAT_LARGE)) +pt_dump_cont_printf (m, dmsg,"pat " ); +else +pt_dump_cont_printf (m, dmsg," " );if (pr & _PAGE_GLOBAL)pt_dump_cont_printf (m, dmsg,"GLB " );else
arch/x86/mm/init.c +29 -0
@@ -27,6 +27,35 @@#include "mm_internal.h" +/* +* Tables translating between page_cache_type_t and pte encoding. +* Minimal supported modes are defined statically, modified if more supported +* cache modes are available. +* Index into __cachemode2pte_tbl is the cachemode. +* Index into __pte2cachemode_tbl are the caching attribute bits of the pte +* (_PAGE_PWT, _PAGE_PCD, _PAGE_PAT) at index bit positions 0, 1, 2. +*/ +uint16_t __cachemode2pte_tbl[_PAGE_CACHE_MODE_NUM] = { + [_PAGE_CACHE_MODE_WB] =0 , + [_PAGE_CACHE_MODE_WC] = _PAGE_PWT, + [_PAGE_CACHE_MODE_UC_MINUS] = _PAGE_PCD, + [_PAGE_CACHE_MODE_UC] = _PAGE_PCD | _PAGE_PWT, + [_PAGE_CACHE_MODE_WT] = _PAGE_PWT, + [_PAGE_CACHE_MODE_WP] = _PAGE_PWT, +}; +EXPORT_SYMBOL_GPL (__cachemode2pte_tbl); +uint8_t __pte2cachemode_tbl[8 ] = { + [__pte2cm_idx (0 )] = _PAGE_CACHE_MODE_WB, + [__pte2cm_idx (_PAGE_PWT)] = _PAGE_CACHE_MODE_WC, + [__pte2cm_idx (_PAGE_PCD)] = _PAGE_CACHE_MODE_UC_MINUS, + [__pte2cm_idx (_PAGE_PWT | _PAGE_PCD)] = _PAGE_CACHE_MODE_UC, + [__pte2cm_idx (_PAGE_PAT)] = _PAGE_CACHE_MODE_WB, + [__pte2cm_idx (_PAGE_PWT | _PAGE_PAT)] = _PAGE_CACHE_MODE_WC, + [__pte2cm_idx (_PAGE_PCD | _PAGE_PAT)] = _PAGE_CACHE_MODE_UC_MINUS, + [__pte2cm_idx (_PAGE_PWT | _PAGE_PCD | _PAGE_PAT)] = _PAGE_CACHE_MODE_UC, +}; +EXPORT_SYMBOL_GPL (__pte2cachemode_tbl); +static unsigned long __initdata pgt_buf_start;static unsigned long __initdata pgt_buf_end;static unsigned long __initdata pgt_buf_top;
arch/x86/mm/init_64.c +6 -3
@@ -327,12 +327,15 @@ pte_t * __init populate_extra_pte(unsigned long vaddr)* Create large page table mappings for a range of physical addresses. */ static void __init __init_extra_mapping (unsigned long phys,unsigned long size, -pgprot_t prot) +enum page_cache_mode cache) {pgd_t *pgd;pud_t *pud;pmd_t *pmd; +pgprot_t prot; +pgprot_val (prot) =pgprot_val (PAGE_KERNEL_LARGE ) | +pgprot_val (pgprot_4k_2_large (pgprot_cachemode (cache)));BUG_ON ((phys & ~PMD_MASK ) || (size & ~PMD_MASK ));for (; size; phys +=PMD_SIZE , size -=PMD_SIZE ) { pgd =pgd_offset_k ((unsigned long )__va (phys));
@@ -355,12 +358,12 @@ static void __init __init_extra_mapping(unsigned long phys, unsigned long size,void __init init_extra_mapping_wb (unsigned long phys,unsigned long size) { -__init_extra_mapping (phys, size,PAGE_KERNEL_LARGE ); +__init_extra_mapping (phys, size, _PAGE_CACHE_MODE_WB); }void __init init_extra_mapping_uc (unsigned long phys,unsigned long size) { -__init_extra_mapping (phys, size,PAGE_KERNEL_LARGE_NOCACHE ); +__init_extra_mapping (phys, size, _PAGE_CACHE_MODE_UC); }/*
arch/x86/mm/iomap_32.c +10 -5
@@ -33,17 +33,17 @@ static int is_io_mapping_possible(resource_size_t base, unsigned long size)int iomap_create_wc (resource_size_t base,unsigned long size,pgprot_t *prot) { -unsigned long flag = _PAGE_CACHE_WC; +enum page_cache_mode pct = _PAGE_CACHE_MODE_WC;int ret;if (!is_io_mapping_possible (base, size))return -EINVAL ; - ret =io_reserve_memtype (base, base + size, &flag); + ret =io_reserve_memtype (base, base + size, &pct);if (ret)return ret; - *prot =__pgprot (__PAGE_KERNEL | flag); + *prot =__pgprot (__PAGE_KERNEL |pgprot_val (pgprot_cachemode (pct)));return 0 ; }EXPORT_SYMBOL_GPL (iomap_create_wc);
@@ -73,6 +73,9 @@ void *kmap_atomic_prot_pfn(unsigned long pfn, pgprot_t prot)/* * Map 'pfn' using protections 'prot' */ +#define __PAGE_KERNEL_WC (__PAGE_KERNEL | \ + protval_cachemode(_PAGE_CACHE_TYPE_WC)) +void __iomem *iomap_atomic_prot_pfn (unsigned long pfn,pgprot_t prot) {
@@ -82,12 +85,14 @@ iomap_atomic_prot_pfn(unsigned long pfn, pgprot_t prot)* MTRR is UC or WC. UC_MINUS gets the real intention, of the * user, which is "WC if the MTRR is WC, UC if you can't do that." */ -if (!pat_enabled &&pgprot_val (prot) ==pgprot_val (PAGE_KERNEL_WC )) - prot =PAGE_KERNEL_UC_MINUS ; +if (!pat_enabled &&pgprot_val (prot) == __PAGE_KERNEL_WC) + prot =__pgprot (__PAGE_KERNEL | +protval_pagemode (_PAGE_CACHE_TYPE_UC_MINUS));return (void __force __iomem *)kmap_atomic_prot_pfn (pfn, prot); }EXPORT_SYMBOL_GPL (iomap_atomic_prot_pfn); +#undef __PAGE_KERNEL_WCvoid iounmap_atomic (void __iomem *kvaddr)
arch/x86/mm/ioremap.c +33 -30
147 lines not on this page · raw message
arch/x86/mm/pageattr.c +50 -34
243 lines not on this page · raw message
arch/x86/mm/pat.c +65 -61
336 lines not on this page · raw message
arch/x86/mm/pat_internal.h +10 -10
@@ -10,30 +10,30 @@ struct memtype {u64 start;u64 end;u64 subtree_max_end; -unsigned long type; +enum page_cache_mode type;struct rb_node rb; }; -static inline char *cattr_name (unsigned long flags) +static inline char *cattr_name (enum page_cache_mode pct) { -switch (flags & _PAGE_CACHE_MASK) { -case _PAGE_CACHE_UC:return "uncached" ; -case _PAGE_CACHE_UC_MINUS:return "uncached-minus" ; -case _PAGE_CACHE_WB:return "write-back" ; -case _PAGE_CACHE_WC:return "write-combining" ; -default :return "broken" ; +switch (pct) { +case _PAGE_CACHE_MODE_UC:return "uncached" ; +case _PAGE_CACHE_MODE_UC_MINUS:return "uncached-minus" ; +case _PAGE_CACHE_MODE_WB:return "write-back" ; +case _PAGE_CACHE_MODE_WC:return "write-combining" ; +default :return "broken" ; } }#ifdef CONFIG_X86_PAT extern int rbt_memtype_check_insert (struct memtype *new, -unsigned long *new_type); +enum page_cache_mode *new_type);extern struct memtype *rbt_memtype_erase (u64 start,u64 end);extern struct memtype *rbt_memtype_lookup (u64 addr);extern int rbt_memtype_copy_nth_element (struct memtype *out,loff_t pos);#else static inline int rbt_memtype_check_insert (struct memtype *new, -unsigned long *new_type) +enum page_cache_mode *new_type) {return 0 ; }static inline struct memtype *rbt_memtype_erase (u64 start,u64 end) {return NULL ; }
arch/x86/mm/pat_rbtree.c +5 -3
@@ -122,11 +122,12 @@ static struct memtype *memtype_rb_exact_match(struct rb_root *root,static int memtype_rb_check_conflict (struct rb_root *root,u64 start,u64 end, -unsigned long reqtype,unsigned long *newtype) +enum page_cache_mode reqtype, +enum page_cache_mode *newtype) {struct rb_node *node;struct memtype *match; -int found_type = reqtype; +enum page_cache_mode found_type = reqtype; match =memtype_rb_lowest_match (&memtype_rbroot, start, end);if (match ==NULL )
@@ -187,7 +188,8 @@ static void memtype_rb_insert(struct rb_root *root, struct memtype *newdata)rb_insert_augmented (&newdata->rb, root, &memtype_rb_augment_cb); } -int rbt_memtype_check_insert (struct memtype *new,unsigned long *ret_type) +int rbt_memtype_check_insert (struct memtype *new, +enum page_cache_mode *ret_type) {int err =0 ;
arch/x86/pci/i386.c +2 -2
@@ -433,14 +433,14 @@ int pci_mmap_page_range(struct pci_dev *dev, struct vm_area_struct *vma,return -EINVAL ;if (pat_enabled && write_combine) - prot |= _PAGE_CACHE_WC; + prot |=protval_cachemode (_PAGE_CACHE_MODE_WC);else if (pat_enabled || boot_cpu_data.x86 >3 )/* * ioremap() and ioremap_nocache() defaults to UC MINUS for now. * To avoid attribute conflicts, request UC MINUS here * as well. */ - prot |= _PAGE_CACHE_UC_MINUS; + prot |=protval_cachemode (_PAGE_CACHE_MODE_UC_MINUS); prot |= _PAGE_IOMAP;/* creating a mapping for IO */
drivers/video/fbdev/gbefb.c +2 -1
@@ -54,7 +54,8 @@ struct gbefb_par {#endif #endif #ifdef CONFIG_X86 -#define pgprot_fb (_prot) ((_prot) | _PAGE_PCD) +#define pgprot_fb (_prot) (((_prot) & ~_PAGE_CACHE_MASK) | \ + protval_cachemode(_PAGE_CACHE_TYPE_UC))#endif /*
drivers/video/fbdev/vermilion/vermilion.c +4 -2
@@ -1004,13 +1004,15 @@ static int vmlfb_mmap(struct fb_info *info, struct vm_area_struct *vma) struct vml_info *vinfo = container_of(info, struct vml_info, info);unsigned long offset = vma->vm_pgoff <<PAGE_SHIFT ;int ret; +unsigned long prot; ret =vmlfb_vram_offset (vinfo, offset);if (ret)return -EINVAL ; -pgprot_val (vma->vm_page_prot) |= _PAGE_PCD; -pgprot_val (vma->vm_page_prot) &= ~_PAGE_PWT; + prot =pgprot_val (vma->vm_page_prot) & ~_PAGE_CACHE_MASK; +pgprot_val (vma->vm_page_prot) = + prot |protval_cachemode (_PAGE_CACHE_MODE_UC);return vm_iomap_memory (vma, vinfo->vram_start, vinfo->vram_contig_size);
Signature · 1 line
1.8.4.5
Re: [PATCH RFC 1/3] x86: Make page cache mode a real type
Quoted 12 lines · On Tue, 2014-08-19 at 15:25 +0200, [email protected] wrote:
From: Juergen Gross <[email protected]>
At the moment there are a lot of places that handle setting or getting the page cache mode by treating the pgprot bits equal to the cache mode. This is only true because there are a lot of assumptions about the setup of the PAT MSR. Otherwise the cache type needs to get translated into pgprot bits and vice versa.
This patch tries to prepare for that by introducing a seperate type for the cache mode and adding functions to translate between those and pgprot values.
Quoted 5 lines
To avoid too much performance penalty the translation between cache mode and pgprot values is done via tables which contain the relevant information. Write-back cache mode is hard-wired to be 0, all other modes are configurable via those tables. For large pages there are translation functions as the PAT bit is located at different positions in the ptes of 4k and large pages.
Hi Juergen,
Thanks for driving this. As we talked before, the changes look good to me. I will post a patchset to enable WT on top of your patchset once this is settled a bit.
I have couples of minor comments below.
arch/x86/include/asm/pgtable_types.h
Quoted 4 lines
arch/x86/include/asm/pgtable_types.h +0
-0
:
Quoted 10 lines
/* xwr */
#define__P000PAGE_NONEarch/x86/include/asm/pgtable_types.h +4
-0@@ -328,6 +331,55 @@ static inline pteval_t pte_flags(pte_t pte)#define pgprot_val (x) ((x).pgprot)#define __pgprot (x) ((pgprot_t) { (x) } ) +extern uint16_t __cachemode2pte_tbl[_PAGE_CACHE_MODE_NUM]; +extern uint8_t __pte2cachemode_tbl[8 ]; + +#define __pte2cm_idx (cb) \
arch/x86/include/asm/pgtable_types.h static inline pteval_t pte_flags(pte_t pte) · line 347
Quoted 10 lines
arch/x86/include/asm/pgtable_types.h +10
-0+ ((((cb) >> (_PAGE_BIT_PAT -2 )) &4 ) | \ + (((cb) >> (_PAGE_BIT_PCD -1 )) &2 ) | \ + (((cb) >> _PAGE_BIT_PWT) & 1)) + +static inline unsigned long protval_cachemode (enum page_cache_mode pct) +{ +if (likely (pct ==0 )) +return 0 ; +return __cachemode2pte_tbl[pct]; +}
I think this function name is not intuitive. pgprot_val() works as pgprot-to-protval, but protval_cachemode() works the other way around as cachemode-to-protval.
How about renaming to cachemode_protval()?
Also, "pct" should probably be changed to "pcm".
arch/x86/include/asm/pgtable_types.h static inline pteval_t pte_flags(pte_t pte) · line 351
Quoted 4 lines
arch/x86/include/asm/pgtable_types.h +4
-0+static inline pgprot_t pgprot_cachemode (enum page_cache_mode pct) +{ +return __pgprot (protval_cachemode (pct)); +}
Ditto.
arch/x86/include/asm/pgtable_types.h static inline pteval_t pte_flags(pte_t pte) · line 360
Quoted 9 lines
arch/x86/include/asm/pgtable_types.h +9
-0+static inline enum page_cache_mode cachemode_pgprot (pgprot_t pgprot) +{ +unsigned long masked; + + masked =pgprot_val (pgprot) & _PAGE_CACHE_MASK; +if (likely (masked ==0 )) +return 0 ; +return __pte2cachemode_tbl[__pte2cm_idx (masked)]; +}
Ditto.
Quoted 14 lines
arch/x86/mm/init.c +6
-0@@ -27,6 +27,35 @@#include "mm_internal.h" +/* +* Tables translating between page_cache_type_t and pte encoding. +* Minimal supported modes are defined statically, modified if more supported +* cache modes are available. +* Index into __cachemode2pte_tbl is the cachemode. +* Index into __pte2cachemode_tbl are the caching attribute bits of the pte
arch/x86/mm/init.c line 45
Quoted 10 lines
arch/x86/mm/init.c +10
-0+* (_PAGE_PWT, _PAGE_PCD, _PAGE_PAT) at index bit positions 0, 1, 2. +*/ +uint16_t __cachemode2pte_tbl[_PAGE_CACHE_MODE_NUM] = { + [_PAGE_CACHE_MODE_WB] =0 , + [_PAGE_CACHE_MODE_WC] = _PAGE_PWT, + [_PAGE_CACHE_MODE_UC_MINUS] = _PAGE_PCD, + [_PAGE_CACHE_MODE_UC] = _PAGE_PCD | _PAGE_PWT, + [_PAGE_CACHE_MODE_WT] = _PAGE_PWT, + [_PAGE_CACHE_MODE_WP] = _PAGE_PWT, +};
I think WT and WP should be set to _PAGE_PCD (UC_MINUS) for safe.
arch/x86/mm/init.c line 56
Quoted 11 lines
arch/x86/mm/init.c +11
-0+EXPORT_SYMBOL_GPL (__cachemode2pte_tbl); +uint8_t __pte2cachemode_tbl[8 ] = { + [__pte2cm_idx (0 )] = _PAGE_CACHE_MODE_WB, + [__pte2cm_idx (_PAGE_PWT)] = _PAGE_CACHE_MODE_WC, + [__pte2cm_idx (_PAGE_PCD)] = _PAGE_CACHE_MODE_UC_MINUS, + [__pte2cm_idx (_PAGE_PWT | _PAGE_PCD)] = _PAGE_CACHE_MODE_UC, + [__pte2cm_idx (_PAGE_PAT)] = _PAGE_CACHE_MODE_WB, + [__pte2cm_idx (_PAGE_PWT | _PAGE_PAT)] = _PAGE_CACHE_MODE_WC, + [__pte2cm_idx (_PAGE_PCD | _PAGE_PAT)] = _PAGE_CACHE_MODE_UC_MINUS, + [__pte2cm_idx (_PAGE_PWT | _PAGE_PCD | _PAGE_PAT)] = _PAGE_CACHE_MODE_UC, +};
:
Quoted 12 lines
arch/x86/mm/iomap_32.c +1
-1@@ -33,17 +33,17 @@ static int is_io_mapping_possible(resource_size_t base, unsigned long size)int iomap_create_wc (resource_size_t base,unsigned long size,pgprot_t *prot) { -unsigned long flag = _PAGE_CACHE_WC; +enum page_cache_mode pct = _PAGE_CACHE_MODE_WC;int ret;
arch/x86/mm/iomap_32.c static int is_io_mapping_possible(resource_size_t base, unsigned long size) · line 46
Quoted 10 lines
arch/x86/mm/iomap_32.c +2
-2if (!is_io_mapping_possible (base, size))return -EINVAL ; - ret =io_reserve_memtype (base, base + size, &flag); + ret =io_reserve_memtype (base, base + size, &pct);if (ret)return ret; - *prot =__pgprot (__PAGE_KERNEL | flag); + *prot =__pgprot (__PAGE_KERNEL |pgprot_val (pgprot_cachemode (pct)));
pgrot_val(pgprot_cachemode(pct)) can be simply protval_cachemode(pct). (again, this should be renamed as cachemode_protval().)
There are other places similar to this.
arch/x86/mm/iomap_32.c void *kmap_atomic_prot_pfn(unsigned long pfn, pgprot_t prot) · line 77
Quoted 9 lines
arch/x86/mm/iomap_32.c +2
-0return 0 ; }EXPORT_SYMBOL_GPL (iomap_create_wc);@@ -73,6 +73,9 @@ void *kmap_atomic_prot_pfn(unsigned long pfn, pgprot_t prot)/* * Map 'pfn' using protections 'prot' */ +#define __PAGE_KERNEL_WC (__PAGE_KERNEL | \ + protval_cachemode(_PAGE_CACHE_TYPE_WC))
_PAGE_CACHE_TYPE_WC should be _PAGE_CACHE_MODE_WC.
There are a few other places where _PAGE_CACHE_TYPE_* are still used, so please fix for that.
Thanks,
-Toshi
Juergen Gross
maintainerRe: [Xen-devel] [PATCH RFC 1/3] x86: Make page cache mode a real type
Quoted 57 lines · Toshi Kani
Quoted 12 lines · On Tue, 2014-08-19 at 15:25 +0200, [email protected] wrote:
From: Juergen Gross <[email protected]>
At the moment there are a lot of places that handle setting or getting the page cache mode by treating the pgprot bits equal to the cache mode. This is only true because there are a lot of assumptions about the setup of the PAT MSR. Otherwise the cache type needs to get translated into pgprot bits and vice versa.
This patch tries to prepare for that by introducing a seperate type for the cache mode and adding functions to translate between those and pgprot values.
Quoted 5 lines
To avoid too much performance penalty the translation between cache mode and pgprot values is done via tables which contain the relevant information. Write-back cache mode is hard-wired to be 0, all other modes are configurable via those tables. For large pages there are translation functions as the PAT bit is located at different positions in the ptes of 4k and large pages.
Hi Juergen,
Thanks for driving this. As we talked before, the changes look good to me. I will post a patchset to enable WT on top of your patchset once this is settled a bit.
I have couples of minor comments below.
arch/x86/include/asm/pgtable_types.h
Quoted 4 lines
arch/x86/include/asm/pgtable_types.h +0
-0:
Quoted 10 lines
/* xwr */ #define __P000 PAGE_NONEarch/x86/include/asm/pgtable_types.h +4
-0@@ -328,6 +331,55 @@ static inline pteval_t pte_flags(pte_t pte)#define pgprot_val (x) ((x).pgprot)#define __pgprot (x) ((pgprot_t) { (x) } ) +extern uint16_t __cachemode2pte_tbl[_PAGE_CACHE_MODE_NUM]; +extern uint8_t __pte2cachemode_tbl[8 ]; + +#define __pte2cm_idx (cb) \arch/x86/include/asm/pgtable_types.h static inline pteval_t pte_flags(pte_t pte) · line 347
Quoted 10 lines
arch/x86/include/asm/pgtable_types.h +10
-0+ ((((cb) >> (_PAGE_BIT_PAT -2 )) &4 ) | \ + (((cb) >> (_PAGE_BIT_PCD -1 )) &2 ) | \ + (((cb) >> _PAGE_BIT_PWT) & 1)) + +static inline unsigned long protval_cachemode (enum page_cache_mode pct) +{ +if (likely (pct ==0 )) +return 0 ; +return __cachemode2pte_tbl[pct]; +}I think this function name is not intuitive.
pgprot_val()works as pgprot-to-protval, butprotval_cachemode()works the other way around as cachemode-to-protval.
Quoted 1 line
How about renaming to
cachemode_protval()?
I think I'll use cachemode2protval().
Quoted 2 lines
Also, "pct" should probably be changed to "pcm".
Yeah.
Quoted 7 lines
arch/x86/include/asm/pgtable_types.h static inline pteval_t pte_flags(pte_t pte) · line 351
Quoted 5 lines
arch/x86/include/asm/pgtable_types.h +4
-0+static inline pgprot_t pgprot_cachemode (enum page_cache_mode pct) +{ +return __pgprot (protval_cachemode (pct)); +}Ditto.
Will be cachemode2pgprot().
Quoted 12 lines
arch/x86/include/asm/pgtable_types.h static inline pteval_t pte_flags(pte_t pte) · line 360
Quoted 10 lines
arch/x86/include/asm/pgtable_types.h +9
-0+static inline enum page_cache_mode cachemode_pgprot (pgprot_t pgprot) +{ +unsigned long masked; + + masked =pgprot_val (pgprot) & _PAGE_CACHE_MASK; +if (likely (masked ==0 )) +return 0 ; +return __pte2cachemode_tbl[__pte2cm_idx (masked)]; +}Ditto.
Will be pgprot2cachemode().
Quoted 19 lines
Quoted 19 lines
arch/x86/mm/init.c +10
-0@@ -27,6 +27,35 @@#include "mm_internal.h" +/* +* Tables translating between page_cache_type_t and pte encoding. +* Minimal supported modes are defined statically, modified if more supported +* cache modes are available. +* Index into __cachemode2pte_tbl is the cachemode. +* Index into __pte2cachemode_tbl are the caching attribute bits of the pte +* (_PAGE_PWT, _PAGE_PCD, _PAGE_PAT) at index bit positions 0, 1, 2. +*/ +uint16_t __cachemode2pte_tbl[_PAGE_CACHE_MODE_NUM] = { + [_PAGE_CACHE_MODE_WB] =0 ,
Quoted 8 lines
arch/x86/mm/init.c line 45
Quoted 6 lines
arch/x86/mm/init.c +6
-0+ [_PAGE_CACHE_MODE_WC] = _PAGE_PWT, + [_PAGE_CACHE_MODE_UC_MINUS] = _PAGE_PCD, + [_PAGE_CACHE_MODE_UC] = _PAGE_PCD | _PAGE_PWT, + [_PAGE_CACHE_MODE_WT] = _PAGE_PWT, + [_PAGE_CACHE_MODE_WP] = _PAGE_PWT, +};I think WT and WP should be set to
_PAGE_PCD(UC_MINUS) for safe.
Oh, you are right.
Quoted 33 lines
arch/x86/mm/init.c line 56
Quoted 12 lines
arch/x86/mm/init.c +11
-0+EXPORT_SYMBOL_GPL (__cachemode2pte_tbl); +uint8_t __pte2cachemode_tbl[8 ] = { + [__pte2cm_idx (0 )] = _PAGE_CACHE_MODE_WB, + [__pte2cm_idx (_PAGE_PWT)] = _PAGE_CACHE_MODE_WC, + [__pte2cm_idx (_PAGE_PCD)] = _PAGE_CACHE_MODE_UC_MINUS, + [__pte2cm_idx (_PAGE_PWT | _PAGE_PCD)] = _PAGE_CACHE_MODE_UC, + [__pte2cm_idx (_PAGE_PAT)] = _PAGE_CACHE_MODE_WB, + [__pte2cm_idx (_PAGE_PWT | _PAGE_PAT)] = _PAGE_CACHE_MODE_WC, + [__pte2cm_idx (_PAGE_PCD | _PAGE_PAT)] = _PAGE_CACHE_MODE_UC_MINUS, + [__pte2cm_idx (_PAGE_PWT | _PAGE_PCD | _PAGE_PAT)] = _PAGE_CACHE_MODE_UC, +};:
Quoted 20 lines
arch/x86/mm/iomap_32.c +2
-2@@ -33,17 +33,17 @@ static int is_io_mapping_possible(resource_size_t base, unsigned long size)int iomap_create_wc (resource_size_t base,unsigned long size,pgprot_t *prot) { -unsigned long flag = _PAGE_CACHE_WC; +enum page_cache_mode pct = _PAGE_CACHE_MODE_WC;int ret;if (!is_io_mapping_possible (base, size))return -EINVAL ; - ret =io_reserve_memtype (base, base + size, &flag); + ret =io_reserve_memtype (base, base + size, &pct);if (ret)return ret;
Quoted 5 lines
arch/x86/mm/iomap_32.c static int is_io_mapping_possible(resource_size_t base, unsigned long size) · line 46
Quoted 2 lines
arch/x86/mm/iomap_32.c +1
-1- *prot =__pgprot (__PAGE_KERNEL | flag); + *prot =__pgprot (__PAGE_KERNEL |pgprot_val (pgprot_cachemode (pct)));pgrot_val(pgprot_cachemode(pct)) can be simply
protval_cachemode(pct). (again, this should be renamed ascachemode_protval().)
... or cachemode2protval().
Quoted 15 lines
There are other places similar to this.
arch/x86/mm/iomap_32.c void *kmap_atomic_prot_pfn(unsigned long pfn, pgprot_t prot) · line 77
Quoted 9 lines
arch/x86/mm/iomap_32.c +2
-0return 0 ; }EXPORT_SYMBOL_GPL (iomap_create_wc);@@ -73,6 +73,9 @@ void *kmap_atomic_prot_pfn(unsigned long pfn, pgprot_t prot)/* * Map 'pfn' using protections 'prot' */ +#define __PAGE_KERNEL_WC (__PAGE_KERNEL | \ + protval_cachemode(_PAGE_CACHE_TYPE_WC))
_PAGE_CACHE_TYPE_WCshould be_PAGE_CACHE_MODE_WC.
Quoted 2 lines
There are a few other places where _PAGE_CACHE_TYPE_* are still used, so please fix for that.
Oops.
Thanks, Juergen
Re: [Xen-devel] [PATCH RFC 1/3] x86: Make page cache mode a real type
Quoted 28 lines
Quoted 1 line
Quoted 1 line · On 21.08.14 at 11:30, <[email protected]> wrote:
Quoted 17 lines · Toshi Kani
Quoted 16 lines · On Tue, 2014-08-19 at 15:25 +0200, [email protected] wrote:
arch/x86/mm/init.c +10
-0@@ -27,6 +27,35 @@#include "mm_internal.h" +/* +* Tables translating between page_cache_type_t and pte encoding. +* Minimal supported modes are defined statically, modified if more supported +* cache modes are available. +* Index into __cachemode2pte_tbl is the cachemode. +* Index into __pte2cachemode_tbl are the caching attribute bits of the pte +* (_PAGE_PWT, _PAGE_PCD, _PAGE_PAT) at index bit positions 0, 1, 2. +*/ +uint16_t __cachemode2pte_tbl[_PAGE_CACHE_MODE_NUM] = { + [_PAGE_CACHE_MODE_WB] =0 ,Quoted 8 lines
arch/x86/mm/init.c line 45
Quoted 6 lines
arch/x86/mm/init.c +6
-0+ [_PAGE_CACHE_MODE_WC] = _PAGE_PWT, + [_PAGE_CACHE_MODE_UC_MINUS] = _PAGE_PCD, + [_PAGE_CACHE_MODE_UC] = _PAGE_PCD | _PAGE_PWT, + [_PAGE_CACHE_MODE_WT] = _PAGE_PWT, + [_PAGE_CACHE_MODE_WP] = _PAGE_PWT, +};I think WT and WP should be set to
_PAGE_PCD(UC_MINUS) for safe.
Quoted 1 line
Oh, you are right.
Actually I suppose the original comment was about WC and WP; defaulting WT to _PAGE_PWT seems quite correct to me.
Jan
Re: [Xen-devel] [PATCH RFC 1/3] x86: Make page cache mode a real type
Quoted 28 lines · Jan Beulich
Quoted 28 lines
Quoted 1 line
Quoted 1 line · On 21.08.14 at 11:30, <[email protected]> wrote:
Quoted 25 lines · Toshi Kani
Quoted 12 lines · On Tue, 2014-08-19 at 15:25 +0200, [email protected] wrote:
arch/x86/mm/init.c +6
-0@@ -27,6 +27,35 @@#include "mm_internal.h" +/* +* Tables translating between page_cache_type_t and pte encoding. +* Minimal supported modes are defined statically, modified if more supported +* cache modes are available. +* Index into __cachemode2pte_tbl is the cachemode. +* Index into __pte2cachemode_tbl are the caching attribute bits of the pte arch/x86/mm/init.c line 45
Quoted 10 lines
arch/x86/mm/init.c +10
-0+* (_PAGE_PWT, _PAGE_PCD, _PAGE_PAT) at index bit positions 0, 1, 2. +*/ +uint16_t __cachemode2pte_tbl[_PAGE_CACHE_MODE_NUM] = { + [_PAGE_CACHE_MODE_WB] =0 , + [_PAGE_CACHE_MODE_WC] = _PAGE_PWT, + [_PAGE_CACHE_MODE_UC_MINUS] = _PAGE_PCD, + [_PAGE_CACHE_MODE_UC] = _PAGE_PCD | _PAGE_PWT, + [_PAGE_CACHE_MODE_WT] = _PAGE_PWT, + [_PAGE_CACHE_MODE_WP] = _PAGE_PWT, +};I think WT and WP should be set to
_PAGE_PCD(UC_MINUS) for safe.
Quoted 4 lines
Quoted 1 line
Oh, you are right.
Actually I suppose the original comment was about WC and WP; defaulting WT to
_PAGE_PWTseems quite correct to me.
My comment was about WT and WP, whose cache modes are defined here but are not supported in this patchset. _PAGE_PWT is used by WC, which has weakly ordered writes. WT and WP have strongly ordered writes, so they should be redirected to UC- for safe.
Thanks,
-Toshi
Re: [PATCH RFC 1/3] x86: Make page cache mode a real type
In reply to [email protected]Quoted 12 lines · On Tue, 2014-08-19 at 15:25 +0200, [email protected] wrote:
From: Juergen Gross <[email protected]>
At the moment there are a lot of places that handle setting or getting the page cache mode by treating the pgprot bits equal to the cache mode. This is only true because there are a lot of assumptions about the setup of the PAT MSR. Otherwise the cache type needs to get translated into pgprot bits and vice versa.
This patch tries to prepare for that by introducing a seperate type for the cache mode and adding functions to translate between those and pgprot values.
Quoted 5 lines
To avoid too much performance penalty the translation between cache mode and pgprot values is done via tables which contain the relevant information. Write-back cache mode is hard-wired to be 0, all other modes are configurable via those tables. For large pages there are translation functions as the PAT bit is located at different positions in the ptes of 4k and large pages.
One more comment below..
arch/x86/include/asm/cacheflush.h
Quoted 1 line
arch/x86/include/asm/cacheflush.h +0
-0
:
Quoted 13 lines
arch/x86/include/asm/cacheflush.h +3
-2-static inline void set_page_memtype (struct page *pg,unsigned long memtype) +static inline void set_page_memtype (struct page *pg, +enum page_cache_mode memtype) {unsigned long memtype_flags = _PGMT_DEFAULT;unsigned long old_flags;unsigned long new_flags;switch (memtype) { -case _PAGE_CACHE_WC: +case _PAGE_CACHE_MODE_WC: memtype_flags = _PGMT_WC;break ;
arch/x86/include/asm/cacheflush.h line 57
Quoted 10 lines
arch/x86/include/asm/cacheflush.h +3
-2-case _PAGE_CACHE_UC_MINUS: +case _PAGE_CACHE_MODE_UC_MINUS: memtype_flags = _PGMT_UC_MINUS;break ; -case _PAGE_CACHE_WB: +case _PAGE_CACHE_MODE_WB: +default : memtype_flags = _PGMT_WB;break ; }
Adding the "default" case handled as _PGMT_WB is not correct here. free_ram_pages_type() calls set_page_memtype() with -1, which needs to be set to _PGMT_DEFAULT.
Thanks,
-Toshi
Juergen Gross
maintainerRe: [PATCH RFC 1/3] x86: Make page cache mode a real type
Quoted 42 lines · Toshi Kani
Quoted 12 lines · On Tue, 2014-08-19 at 15:25 +0200, [email protected] wrote:
From: Juergen Gross <[email protected]>
At the moment there are a lot of places that handle setting or getting the page cache mode by treating the pgprot bits equal to the cache mode. This is only true because there are a lot of assumptions about the setup of the PAT MSR. Otherwise the cache type needs to get translated into pgprot bits and vice versa.
This patch tries to prepare for that by introducing a seperate type for the cache mode and adding functions to translate between those and pgprot values.
Quoted 5 lines
To avoid too much performance penalty the translation between cache mode and pgprot values is done via tables which contain the relevant information. Write-back cache mode is hard-wired to be 0, all other modes are configurable via those tables. For large pages there are translation functions as the PAT bit is located at different positions in the ptes of 4k and large pages.
One more comment below..
arch/x86/include/asm/cacheflush.h
Quoted 1 line
arch/x86/include/asm/cacheflush.h +0
-0:
Quoted 19 lines
arch/x86/include/asm/cacheflush.h +5
-4-static inline void set_page_memtype (struct page *pg,unsigned long memtype) +static inline void set_page_memtype (struct page *pg, +enum page_cache_mode memtype) {unsigned long memtype_flags = _PGMT_DEFAULT;unsigned long old_flags;unsigned long new_flags;switch (memtype) { -case _PAGE_CACHE_WC: +case _PAGE_CACHE_MODE_WC: memtype_flags = _PGMT_WC;break ; -case _PAGE_CACHE_UC_MINUS: +case _PAGE_CACHE_MODE_UC_MINUS: memtype_flags = _PGMT_UC_MINUS;break ; -case _PAGE_CACHE_WB: +case _PAGE_CACHE_MODE_WB:
Quoted 8 lines
arch/x86/include/asm/cacheflush.h line 57
Quoted 4 lines
arch/x86/include/asm/cacheflush.h +1
-0+default : memtype_flags = _PGMT_WB;break ; }Adding the "default" case handled as
_PGMT_WBis not correct here.free_ram_pages_type()callsset_page_memtype()with -1, which needs to be set to_PGMT_DEFAULT.
It says so in the comment above. I'll correct it, thanks.
Juergen
[email protected]
maintainerRFC PATCH 2/3 x86: Enable PAT to use cache mode translation tables
In reply to [email protected]From: Juergen Gross <[email protected]>
Update the translation tables from cache mode to pgprot values according to the PAT settings. This enables changing the cache attributes of a PAT index in just one place without having to change at the users side.
With this change it is possible to use the same kernel with different PAT configurations, e.g. supporting Xen.
--- arch/x86/include/asm/pat.h | 1 + arch/x86/include/asm/pgtable_types.h | 4 +++ arch/x86/mm/init.c | 8 +++++ arch/x86/mm/pat.c | 57 +++++++++++++++++++++++++++++++++++- include/linux/mm.h | 1 + 5 files changed, 70 insertions(+), 1 deletion(-)
arch/x86/include/asm/pat.h +1 -0
@@ -11,6 +11,7 @@ static const int pat_enabled;#endif extern void pat_init (void ); +void pat_init_cache_modes (u64 pat);extern int reserve_memtype (u64 start,u64 end,enum page_cache_mode req_pct,enum page_cache_mode *ret_pct);
arch/x86/include/asm/pgtable_types.h +4 -0
@@ -338,6 +338,10 @@ extern uint8_t __pte2cachemode_tbl[8]; ((((cb) >> (_PAGE_BIT_PAT -2 )) &4 ) | \ (((cb) >> (_PAGE_BIT_PCD -1 )) &2 ) | \ (((cb) >> _PAGE_BIT_PWT) & 1)) +#define __cm_idx2pte (i) \ + ((((i) & 4) << (_PAGE_BIT_PAT - 2)) | \ + (((i) & 2) << (_PAGE_BIT_PCD - 1)) | \ + (((i) & 1) << _PAGE_BIT_PWT))static inline unsigned long protval_cachemode (enum page_cache_mode pct) {
arch/x86/mm/init.c +8 -0
@@ -716,3 +716,11 @@ void __init zone_sizes_init(void)free_area_init_nodes (max_zone_pfns); } +void update_cache_mode_entry (unsigned entry,enum page_cache_mode cache) +{ +/* entry 0 MUST be WB (hardwired to speed up translations) */ +BUG_ON (!entry && cache != _PAGE_CACHE_MODE_WB); + + __cachemode2pte_tbl[cache] =__cm_idx2pte (entry); + __pte2cachemode_tbl[entry] = cache; +}
arch/x86/mm/pat.c +56 -1
70 lines not on this page · raw message
include/linux/mm.h +1 -0
@@ -1574,6 +1574,7 @@ extern void free_area_init(unsigned long * zones_size);extern void free_area_init_node (int nid,unsigned long * zones_size,unsigned long zone_start_pfn,unsigned long *zholes_size);extern void free_initmem (void ); +extern void update_cache_mode_entry (unsigned entry,enum page_cache_mode cache);/* * Free reserved pages within range [PAGE_ALIGN(start), end & PAGE_MASK)
Signature · 1 line
1.8.4.5
Re: [PATCH RFC 2/3] x86: Enable PAT to use cache mode translation tables
arch/x86/mm/pat.c void pat_init(void) · line 175
Quoted 12 lines
Quoted 1 line
Quoted 1 line · On 19.08.14 at 15:25, <[email protected]> wrote:
arch/x86/mm/pat.c +5
-1@@ -118,8 +167,14 @@ void pat_init(void)PAT (4 ,WB ) |PAT (5 ,WC ) |PAT (6 ,UC_MINUS ) |PAT (7 ,UC );/* Boot CPU check */ -if (!boot_pat_state) +if (!boot_pat_state) {rdmsrl (MSR_IA32_CR_PAT , boot_pat_state); +/* +* Init cache mode tables before writing MSR to give Xen a +* chance to correct the changes when doing the write. +*/
This comment seems pretty odd to me: For one, a PV guest on Xen shouldn't be trying to write PAT MSR at all under the current ABI (the write will be ignored, yes, but accompanied with a warning message, which PV kernels - by the mere fact that they're PV - should try to avoid). And then "correct the changes" both gives the impression as if they were wrong and as if some of what the kernel writes may be under the kernel's control. Hence I think this code and comment should either be consistently assuming that the kernel has no control at all, or should read back the value after having written it, and set the internal tables based on the value read back.
Jan
Quoted 5 lines
arch/x86/mm/pat.c +2
-0+pat_init_cache_modes (pat); + }wrmsrl (MSR_IA32_CR_PAT , pat);
Re: [Xen-devel] [PATCH RFC 2/3] x86: Enable PAT to use cache mode translation tables
In reply to [email protected]arch/x86/mm/pat.c void pat_init(void) · line 175
Quoted 12 lines
Quoted 1 line
Quoted 1 line · On 19.08.14 at 15:25, <[email protected]> wrote:
arch/x86/mm/pat.c +5
-1@@ -118,8 +167,14 @@ void pat_init(void)PAT (4 ,WB ) |PAT (5 ,WC ) |PAT (6 ,UC_MINUS ) |PAT (7 ,UC );/* Boot CPU check */ -if (!boot_pat_state) +if (!boot_pat_state) {rdmsrl (MSR_IA32_CR_PAT , boot_pat_state); +/* +* Init cache mode tables before writing MSR to give Xen a +* chance to correct the changes when doing the write. +*/
This comment seems pretty odd to me: For one, a PV guest on Xen shouldn't be trying to write PAT MSR at all under the current ABI (the write will be ignored, yes, but accompanied with a warning message, which PV kernels - by the mere fact that they're PV - should try to avoid). And then "correct the changes" both gives the impression as if they were wrong and as if some of what the kernel writes may be under the kernel's control. Hence I think this code and comment should either be consistently assuming that the kernel has no control at all, or should read back the value after having written it, and set the internal tables based on the value read back.
Jan
Quoted 5 lines
arch/x86/mm/pat.c +2
-0+pat_init_cache_modes (pat); + }wrmsrl (MSR_IA32_CR_PAT , pat);
Juergen Gross
maintainerRe: [Xen-devel] [PATCH RFC 2/3] x86: Enable PAT to use cache mode translation tables
Quoted 16 lines · Jan Beulich
arch/x86/mm/pat.c void pat_init(void) · line 175
Quoted 12 lines
Quoted 1 line
Quoted 1 line · On 19.08.14 at 15:25, <[email protected]> wrote:
arch/x86/mm/pat.c +5
-1@@ -118,8 +167,14 @@ void pat_init(void)PAT (4 ,WB ) |PAT (5 ,WC ) |PAT (6 ,UC_MINUS ) |PAT (7 ,UC );/* Boot CPU check */ -if (!boot_pat_state) +if (!boot_pat_state) {rdmsrl (MSR_IA32_CR_PAT , boot_pat_state); +/* +* Init cache mode tables before writing MSR to give Xen a +* chance to correct the changes when doing the write. +*/ This comment seems pretty odd to me: For one, a PV guest on Xen shouldn't be trying to write PAT MSR at all under the current ABI (the write will be ignored, yes, but accompanied with a warning
Quoted 8 lines
message, which PV kernels - by the mere fact that they're PV - should try to avoid). And then "correct the changes" both gives the impression as if they were wrong and as if some of what the kernel writes may be under the kernel's control. Hence I think this code and comment should either be consistently assuming that the kernel has no control at all, or should read back the value after having written it, and set the internal tables based on the value read back.
I think the latter alternative is the better one. I'll change the patch.
Juergen
Juergen Gross
maintainerRe: [PATCH RFC 2/3] x86: Enable PAT to use cache mode translation tables
In reply to Jan BeulichQuoted 16 lines · Jan Beulich
arch/x86/mm/pat.c void pat_init(void) · line 175
Quoted 12 lines
Quoted 1 line
Quoted 1 line · On 19.08.14 at 15:25, <[email protected]> wrote:
arch/x86/mm/pat.c +5
-1@@ -118,8 +167,14 @@ void pat_init(void)PAT (4 ,WB ) |PAT (5 ,WC ) |PAT (6 ,UC_MINUS ) |PAT (7 ,UC );/* Boot CPU check */ -if (!boot_pat_state) +if (!boot_pat_state) {rdmsrl (MSR_IA32_CR_PAT , boot_pat_state); +/* +* Init cache mode tables before writing MSR to give Xen a +* chance to correct the changes when doing the write. +*/ This comment seems pretty odd to me: For one, a PV guest on Xen shouldn't be trying to write PAT MSR at all under the current ABI (the write will be ignored, yes, but accompanied with a warning
Quoted 8 lines
message, which PV kernels - by the mere fact that they're PV - should try to avoid). And then "correct the changes" both gives the impression as if they were wrong and as if some of what the kernel writes may be under the kernel's control. Hence I think this code and comment should either be consistently assuming that the kernel has no control at all, or should read back the value after having written it, and set the internal tables based on the value read back.
I think the latter alternative is the better one. I'll change the patch.
Juergen
[email protected]
maintainerRFC PATCH 3/3 Support Xen pv-domains using PAT
In reply to [email protected]From: Juergen Gross <[email protected]>
With the dynamical mapping between cache modes and pgprot values it is now possible to use all cache modes via the Xen hypervisor PAT settings in a pv domain.
All to be done is to read the PAT configuration MSR and set up the translation tables accordingly.
--- arch/x86/xen/enlighten.c | 12 ++++++---- arch/x86/xen/mmu.c | 60 +++++++++++++++++------------------------------- arch/x86/xen/xen-ops.h | 1 + 3 files changed, 30 insertions(+), 43 deletions(-)
arch/x86/xen/enlighten.c +8 -4
@@ -1552,12 +1552,16 @@ asmlinkage __visible void __init xen_start_kernel(void)xen_init_mmu_ops (); +/* +* Modify the cache mode translation tables to match Xen's PAT +* configuration. +*/ + +if (xen_init_cache_types ()) + __supported_pte_mask &= ~(_PAGE_PWT | _PAGE_PCD); +/* Prevent unwanted bits from being set in PTEs. */ __supported_pte_mask &= ~_PAGE_GLOBAL; -#if 0 - if (!xen_initial_domain()) -#endif - __supported_pte_mask &= ~(_PAGE_PWT | _PAGE_PCD); __supported_pte_mask |= _PAGE_IOMAP;
arch/x86/xen/mmu.c +21 -39
81 lines not on this page · raw message
arch/x86/xen/xen-ops.h +1 -0
@@ -34,6 +34,7 @@ extern unsigned long xen_max_p2m_pfn;void xen_mm_pin_all (void );void xen_mm_unpin_all (void );void xen_set_pat (u64 ); +int xen_init_cache_types (void );char *__init xen_memory_setup (void );char *xen_auto_xlated_memory_setup (void );
Signature · 1 line
1.8.4.5
Quoted 7 lines
The Linux kernel currently supports only 4 different cache modes. The PAT MSR is set up in a way that the setting of
_PAGE_PATin a pte doesn't matter: the top 4 entries in the PAT MSR are the same as the 4 lower entries.This results in the kernel not supporting e.g. write-through mode. Especially this cache mode would speed up drivers of video cards which now have to use uncached accesses.
Pentium II erratum A52 (and similar on quite a few other processors)
Problem: The Page Attribute Table (PAT) contains eight entries, which must all be initialized and considered when setting up memory types for the Pentium II processor. However, in Mode B or Mode C paging, the upper four entries do not function correctly for 4-Kbyte pages. Specifically, bit seven of page table entries that translate addresses to 4-Kbyte pages should be used as the upper bit of a three-bit index to determine the PAT entry that specifies the memory type for the page. When Mode B (CR4.PSE =
1) and/or Mode C (CR4.PAE) are enabled, the processor forces this bit to zero when determining the memory type regardless of the value in the page table entry. The upper four entries of the PAT function correctly for 2-Mbyte and 4-Mbyte large pages (specified by bit 12 of the page directory entry for those translations). Implication: Only the lower four PAT entries are useful for 4 KB translations when Mode B or C paging is used. In Mode A paging (4-Kbyte pages only), all eight entries may be used. All eight entries may be used for large pages in Mode B or C paging.
Doing this stuff for Xen also IMHO makes no sense at all. We shouldn't have a kernel full of crap to deal with Xen-isms. IFF it means the changes can also implement a mix of four PAT entries on Pentium-M or earlier CPUs with PAT errata, and the full PAT on processors without errata then IMHO it becomes a whole world more interesting.
Alan
Quoted 10 lines
Quoted 8 lines
Quoted 1 line
One Thousand Gnomes <[email protected]> 08/20/14 2:08 PM >>>
The Linux kernel currently supports only 4 different cache modes. The PAT MSR is set up in a way that the setting of
_PAGE_PATin a pte doesn't matter: the top 4 entries in the PAT MSR are the same as the 4 lower entries.This results in the kernel not supporting e.g. write-through mode. Especially this cache mode would speed up drivers of video cards which now have to use uncached accesses.
Pentium II erratum A52 (and similar on quite a few other processors)
But you're aware that for the last two major releases Xen hasn't been supporting 32-bit CPUs at all anymore? I.e. why should we limit functionality under Xen based on errata on 64-bit-incapable CPUs?
Jan
H. Peter Anvin
reviewerQuoted 5 lines · Jan Beulich
But you're aware that for the last two major releases Xen hasn't been supporting 32-bit CPUs at all anymore? I.e. why should we limit functionality under Xen based on errata on 64-bit-incapable CPUs?
Because there is no fscking way we're doing this differently on 32 and 64 bits. Quite frankly the issue is if we should support this under Xen at all, but I think we can do it cleanly.
-hpa
Juergen Gross
maintainerQuoted 29 lines · One Thousand Gnomes
Quoted 7 lines
The Linux kernel currently supports only 4 different cache modes. The PAT MSR is set up in a way that the setting of
_PAGE_PATin a pte doesn't matter: the top 4 entries in the PAT MSR are the same as the 4 lower entries.This results in the kernel not supporting e.g. write-through mode. Especially this cache mode would speed up drivers of video cards which now have to use uncached accesses.
Pentium II erratum A52 (and similar on quite a few other processors)
Problem: The Page Attribute Table (PAT) contains eight entries, which must all be initialized and considered when setting up memory types for the Pentium II processor. However, in Mode B or Mode C paging, the upper four entries do not function correctly for 4-Kbyte pages. Specifically, bit seven of page table entries that translate addresses to 4-Kbyte pages should be used as the upper bit of a three-bit index to determine the PAT entry that specifies the memory type for the page. When Mode B (CR4.PSE =
1) and/or Mode C (CR4.PAE) are enabled, the processor forces this bit to zero when determining the memory type regardless of the value in the page table entry. The upper four entries of the PAT function correctly for 2-Mbyte and 4-Mbyte large pages (specified by bit 12 of the page directory entry for those translations). Implication: Only the lower four PAT entries are useful for 4 KB translations when Mode B or C paging is used. In Mode A paging (4-Kbyte pages only), all eight entries may be used. All eight entries may be used for large pages in Mode B or C paging.
Quoted 5 lines
Doing this stuff for Xen also IMHO makes no sense at all. We shouldn't have a kernel full of crap to deal with Xen-isms. IFF it means the changes can also implement a mix of four PAT entries on Pentium-M or earlier CPUs with PAT errata, and the full PAT on processors without errata then IMHO it becomes a whole world more interesting.
This is the case.
The patches as posted don't change PAT MSR settings at all. Using all 8 PAT MSR entries on processors without errata is easy while limit usage to only 4 entries is possible as well. It is possible to make the decision dynamically while booting the system. The only difference is the value written to the PAT MSR, as this value is used to modify the tables used for translating between cache modes and pte bits. If a driver wants to set write-through mode on a processor with the PAT erratum, the translation will result in uncached mode which is okay.
Regarding Xen: The patches don't introduce further Xen-isms. They reduce the Xen specialties by being able to use the same translation mechanisms on a native system as on a Xen based system. The only differences are the values in the translation tables, as Xen is using a different PAT configuration as native Linux (with my patches Linux could use the same PAT configuration as Xen, of course).
Juergen
H. Peter Anvin
reviewerQuoted 2 lines · One Thousand Gnomes
Doing this stuff for Xen also IMHO makes no sense at all. We shouldn't have a kernel full of crap to deal with Xen-isms.
We already *have* a kernel full of crap to deal with Xen-isms. We screwed that pooch some 10 years ago.
Quoted 4 lines
IFF it means the changes can also implement a mix of four PAT entries on Pentium-M or earlier CPUs with PAT errata, and the full PAT on processors without errata then IMHO it becomes a whole world more interesting.
That is one of the wins, indeed, as well as handle pre-PAT processors.
-hpa