Skip to contentSkip to conversation

x86: Full support of PAT

PATCH Superseded Juergen Gross 4 participants 14 messages Aug 26, 2014 – Aug 29, 2014

Subsystems (6)
x86 PCI MM core Framebuffer layer +2 more subsystems

v1 of 6 Superseded Latest is v6 →

lore
Share & tools
lore

Juergen Gross

author maintainer
#0 lore raw

cover x86: Full support of PAT

The x86 architecture offers via the PAT (Page Attribute Table) a way to specify different caching modes in page table entries. The PAT MSR contains 8 entries each specifying one of 6 possible cache modes. A pte references one of those entries via 3 bits: _PAGE_PAT, _PAGE_PWT and _PAGE_PCD.

The Linux kernel currently supports only 4 different cache modes. The PAT MSR is set up in a way that the setting of _PAGE_PAT in a pte doesn't matter: the top 4 entries in the PAT MSR are the same as the 4 lower entries.

This results in the kernel not supporting e.g. write-through mode. Especially this cache mode would speed up drivers of video cards which now have to use uncached accesses.

OTOH some old processors (Pentium) don't support PAT correctly and the Xen hypervisor has been using a different PAT MSR configuration for some time now and can't change that as this setting is part of the ABI.

This patch set abstracts the cache mode from the pte and introduces tables to translate between cache mode and pte bits (the default cache mode "write back" is hard-wired to PAT entry 0). The tables are statically initialized with values being compatible to old processors and current usage. As soon as the PAT MSR is changed (or - in case of Xen - is read at boot time) the tables are changed accordingly. Requests of mappings with special cache modes are always possible now, in case they are not supported there will be a fallback to a compatible but slower mode.

Summing it up, this patch set adds the following features:
- capability to support WT and WP cache modes on processors with full PAT support
- processors with no or uncorrect PAT support are still working as today, even if WT or WP cache mode are selected by drivers for some pages
- reduction of Xen special handling regarding cache mode

Changes since RFC:
- renamed functions and variables as suggested by Toshi Kani
- corrected cache mode bits for WT and WP
- modified handling of PAT MSR write under Xen as suggested by Jan Beulich

Juergen Gross (3):
x86: Make page cache mode a real type
x86: Enable PAT to use cache mode translation tables Support Xen pv-domains using PAT

 arch/x86/include/asm/cacheflush.h         |  38 ++++---
 arch/x86/include/asm/fb.h                 |   6 +-
 arch/x86/include/asm/io.h                 |   2 +-
 arch/x86/include/asm/pat.h                |   7 +-
 arch/x86/include/asm/pgtable.h            |  19 ++--
 arch/x86/include/asm/pgtable_types.h      |  96 ++++++++++++----
 arch/x86/mm/dump_pagetables.c             |  24 ++--
 arch/x86/mm/init.c                        |  37 ++++++
 arch/x86/mm/init_64.c                     |   9 +-
 arch/x86/mm/iomap_32.c                    |  15 ++-
 arch/x86/mm/ioremap.c                     |  63 ++++++-----
 arch/x86/mm/pageattr.c                    |  84 ++++++++------
 arch/x86/mm/pat.c                         | 180 +++++++++++++++++++-----------
 arch/x86/mm/pat_internal.h                |  22 ++--
 arch/x86/mm/pat_rbtree.c                  |   8 +-
 arch/x86/pci/i386.c                       |   4 +-
 arch/x86/xen/enlighten.c                  |  11 +-
 arch/x86/xen/mmu.c                        |  52 +++------
 arch/x86/xen/xen-ops.h                    |   1 +
 drivers/video/fbdev/gbefb.c               |   3 +-
 drivers/video/fbdev/vermilion/vermilion.c |   6 +-
 include/linux/mm.h                        |   1 +
 22 files changed, 431 insertions(+), 257 deletions(-)
Signature · 1 line
1.8.4.5
author maintainer
#1 lore raw

PATCH 1/3 x86: Make page cache mode a real type

At the moment there are a lot of places that handle setting or getting the page cache mode by treating the pgprot bits equal to the cache mode. This is only true because there are a lot of assumptions about the setup of the PAT MSR. Otherwise the cache type needs to get translated into pgprot bits and vice versa.

This patch tries to prepare for that by introducing a seperate type for the cache mode and adding functions to translate between those and pgprot values.

To avoid too much performance penalty the translation between cache mode and pgprot values is done via tables which contain the relevant information. Write-back cache mode is hard-wired to be 0, all other modes are configurable via those tables. For large pages there are translation functions as the PAT bit is located at different positions in the ptes of 4k and large pages.

---
 arch/x86/include/asm/cacheflush.h         |  38 +++++----
 arch/x86/include/asm/fb.h                 |   6 +-
 arch/x86/include/asm/io.h                 |   2 +-
 arch/x86/include/asm/pat.h                |   6 +-
 arch/x86/include/asm/pgtable.h            |  19 ++---
 arch/x86/include/asm/pgtable_types.h      |  92 +++++++++++++++++-----
 arch/x86/mm/dump_pagetables.c             |  24 +++---
 arch/x86/mm/init.c                        |  29 +++++++
 arch/x86/mm/init_64.c                     |   9 ++-
 arch/x86/mm/iomap_32.c                    |  15 ++--
 arch/x86/mm/ioremap.c                     |  63 ++++++++-------
 arch/x86/mm/pageattr.c                    |  84 ++++++++++++--------
 arch/x86/mm/pat.c                         | 126 +++++++++++++++---------------
 arch/x86/mm/pat_internal.h                |  22 +++---
 arch/x86/mm/pat_rbtree.c                  |   8 +-
 arch/x86/pci/i386.c                       |   4 +-
 drivers/video/fbdev/gbefb.c               |   3 +-
 drivers/video/fbdev/vermilion/vermilion.c |   6 +-
 18 files changed, 344 insertions(+), 212 deletions(-)
arch/x86/include/asm/cacheflush.h +24 -14
index 9863ee3..157644b 100644
@@ -9,10 +9,10 @@
 /*
  * X86 PAT uses page flags WC and Uncached together to keep track of
  * memory type of pages that have backing page struct. X86 PAT supports 3
- * different memory types, _PAGE_CACHE_WB, _PAGE_CACHE_WC and
- * _PAGE_CACHE_UC_MINUS and fourth state where page's memory type has not
+ * different memory types, _PAGE_CACHE_MODE_WB, _PAGE_CACHE_MODE_WC and
+ * _PAGE_CACHE_MODE_UC_MINUS and fourth state where page's memory type has not
  * been changed from its default (value of -1 used to denote this).
- * Note we do not support _PAGE_CACHE_UC here.
+ * Note we do not support _PAGE_CACHE_MODE_UC here.
  */
 
 #define _PGMT_DEFAULT		0
@@ -22,36 +22,40 @@
 #define _PGMT_MASK		(1UL << PG_uncached | 1UL << PG_arch_1)
 #define _PGMT_CLEAR_MASK	(~_PGMT_MASK)
 
-static inline unsigned long get_page_memtype(struct page *pg)
+static inline enum page_cache_mode get_page_memtype(struct page *pg)
 {
 	unsigned long pg_flags = pg->flags & _PGMT_MASK;
 
 	if (pg_flags == _PGMT_DEFAULT)
 		return -1;
 	else if (pg_flags == _PGMT_WC)
-		return _PAGE_CACHE_WC;
+		return _PAGE_CACHE_MODE_WC;
 	else if (pg_flags == _PGMT_UC_MINUS)
-		return _PAGE_CACHE_UC_MINUS;
+		return _PAGE_CACHE_MODE_UC_MINUS;
 	else
-		return _PAGE_CACHE_WB;
+		return _PAGE_CACHE_MODE_WB;
 }
 
-static inline void set_page_memtype(struct page *pg, unsigned long memtype)
+static inline void set_page_memtype(struct page *pg,
+				    enum page_cache_mode memtype)
 {
-	unsigned long memtype_flags = _PGMT_DEFAULT;
+	unsigned long memtype_flags;
 	unsigned long old_flags;
 	unsigned long new_flags;
 
 	switch (memtype) {
-	case _PAGE_CACHE_WC:
+	case _PAGE_CACHE_MODE_WC:
 		memtype_flags = _PGMT_WC;
 		break;
-	case _PAGE_CACHE_UC_MINUS:
+	case _PAGE_CACHE_MODE_UC_MINUS:
 		memtype_flags = _PGMT_UC_MINUS;
 		break;
-	case _PAGE_CACHE_WB:
+	case _PAGE_CACHE_MODE_WB:
 		memtype_flags = _PGMT_WB;
 		break;
+	default:
+		memtype_flags = _PGMT_DEFAULT;
+		break;
 	}
 
 	do {
@@ -60,8 +64,14 @@ static inline void set_page_memtype(struct page *pg, unsigned long memtype)
 	} while (cmpxchg(&pg->flags, old_flags, new_flags) != old_flags);
 }
 #else
-static inline unsigned long get_page_memtype(struct page *pg) { return -1; }
-static inline void set_page_memtype(struct page *pg, unsigned long memtype) { }
+static inline enum page_cache_mode get_page_memtype(struct page *pg)
+{
+	return -1;
+}
+static inline void set_page_memtype(struct page *pg,
+				    enum page_cache_mode memtype)
+{
+}
 #endif
 
 /*
arch/x86/include/asm/fb.h +5 -1
index 2519d06..0902a50 100644
@@ -8,8 +8,12 @@
 static inline void fb_pgprotect(struct file *file, struct vm_area_struct *vma,
 				unsigned long off)
 {
+	unsigned long prot;
+
+	prot = pgprot_val(vma->vm_page_prot) & ~_PAGE_CACHE_MASK;
 	if (boot_cpu_data.x86 > 3)
-		pgprot_val(vma->vm_page_prot) |= _PAGE_PCD;
+		pgprot_val(vma->vm_page_prot) =
+			prot | cachemode2protval(_PAGE_CACHE_MODE_UC);
 }
 
 extern int fb_is_primary_device(struct fb_info *info);
arch/x86/include/asm/io.h +1 -1
index b8237d8..71b9e65 100644
@@ -314,7 +314,7 @@ extern void *xlate_dev_mem_ptr(unsigned long phys);
 extern void unxlate_dev_mem_ptr(unsigned long phys, void *addr);
 
 extern int ioremap_change_attr(unsigned long vaddr, unsigned long size,
-				unsigned long prot_val);
+				enum page_cache_mode pcm);
 extern void __iomem *ioremap_wc(resource_size_t offset, unsigned long size);
 
 extern bool is_early_ioremap_ptep(pte_t *ptep);
arch/x86/include/asm/pat.h +3 -3
index e2c1668..150407a 100644
@@ -13,14 +13,14 @@ static const int pat_enabled;
 extern void pat_init(void);
 
 extern int reserve_memtype(u64 start, u64 end,
-		unsigned long req_type, unsigned long *ret_type);
+		enum page_cache_mode req_pcm, enum page_cache_mode *ret_pcm);
 extern int free_memtype(u64 start, u64 end);
 
 extern int kernel_map_sync_memtype(u64 base, unsigned long size,
-		unsigned long flag);
+		enum page_cache_mode pcm);
 
 int io_reserve_memtype(resource_size_t start, resource_size_t end,
-			unsigned long *type);
+			enum page_cache_mode *pcm);
 
 void io_free_memtype(resource_size_t start, resource_size_t end);
 
arch/x86/include/asm/pgtable.h +10 -9
index 0ec0560..055047f 100644
@@ -9,9 +9,10 @@
 /*
  * Macro to mark a page protection value as UC-
  */
-#define pgprot_noncached(prot)					\
-	((boot_cpu_data.x86 > 3)				\
-	 ? (__pgprot(pgprot_val(prot) | _PAGE_CACHE_UC_MINUS))	\
+#define pgprot_noncached(prot)						\
+	((boot_cpu_data.x86 > 3)					\
+	 ? (__pgprot(pgprot_val(prot) |					\
+		     cachemode2protval(_PAGE_CACHE_MODE_UC_MINUS)))	\
 	 : (prot))
 
 #ifndef __ASSEMBLY__
@@ -399,8 +400,8 @@ static inline pgprot_t pgprot_modify(pgprot_t oldprot, pgprot_t newprot)
 #define canon_pgprot(p) __pgprot(massage_pgprot(p))
 
 static inline int is_new_memtype_allowed(u64 paddr, unsigned long size,
-					 unsigned long flags,
-					 unsigned long new_flags)
+					 enum page_cache_mode pcm,
+					 enum page_cache_mode new_pcm)
 {
 	/*
 	 * PAT type is always WB for untracked ranges, so no need to check.
@@ -414,10 +415,10 @@ static inline int is_new_memtype_allowed(u64 paddr, unsigned long size,
 	 * - request is uncached, return cannot be write-back
 	 * - request is write-combine, return cannot be write-back
 	 */
-	if ((flags == _PAGE_CACHE_UC_MINUS &&
-	     new_flags == _PAGE_CACHE_WB) ||
-	    (flags == _PAGE_CACHE_WC &&
-	     new_flags == _PAGE_CACHE_WB)) {
+	if ((pcm == _PAGE_CACHE_MODE_UC_MINUS &&
+	     new_pcm == _PAGE_CACHE_MODE_WB) ||
+	    (pcm == _PAGE_CACHE_MODE_WC &&
+	     new_pcm == _PAGE_CACHE_MODE_WB)) {
 		return 0;
 	}
 
arch/x86/include/asm/pgtable_types.h +72 -20

130 lines not on this page · raw message

arch/x86/mm/dump_pagetables.c +11 -13
index 167ffca..52a336a 100644
@@ -122,7 +122,7 @@ static void printk_prot(struct seq_file *m, pgprot_t prot, int level, bool dmsg)
 
 	if (!pgprot_val(prot)) {
 		/* Not present */
-		pt_dump_cont_printf(m, dmsg, "                          ");
+		pt_dump_cont_printf(m, dmsg, "                              ");
 	} else {
 		if (pr & _PAGE_USER)
 			pt_dump_cont_printf(m, dmsg, "USR ");
@@ -141,18 +141,16 @@ static void printk_prot(struct seq_file *m, pgprot_t prot, int level, bool dmsg)
 		else
 			pt_dump_cont_printf(m, dmsg, "    ");
 
-		/* Bit 9 has a different meaning on level 3 vs 4 */
-		if (level <= 3) {
-			if (pr & _PAGE_PSE)
-				pt_dump_cont_printf(m, dmsg, "PSE ");
-			else
-				pt_dump_cont_printf(m, dmsg, "    ");
-		} else {
-			if (pr & _PAGE_PAT)
-				pt_dump_cont_printf(m, dmsg, "pat ");
-			else
-				pt_dump_cont_printf(m, dmsg, "    ");
-		}
+		/* Bit 7 has a different meaning on level 3 vs 4 */
+		if (level <= 3 && pr & _PAGE_PSE)
+			pt_dump_cont_printf(m, dmsg, "PSE ");
+		else
+			pt_dump_cont_printf(m, dmsg, "    ");
+		if ((level == 4 && pr & _PAGE_PAT) ||
+		    ((level == 3 || level == 2) && pr & _PAGE_PAT_LARGE))
+			pt_dump_cont_printf(m, dmsg, "pat ");
+		else
+			pt_dump_cont_printf(m, dmsg, "    ");
 		if (pr & _PAGE_GLOBAL)
 			pt_dump_cont_printf(m, dmsg, "GLB ");
 		else
arch/x86/mm/init.c +29 -0
index 66dba36..a9776ba 100644
@@ -27,6 +27,35 @@
 
 #include "mm_internal.h"
 
+/*
+ * Tables translating between page_cache_type_t and pte encoding.
+ * Minimal supported modes are defined statically, modified if more supported
+ * cache modes are available.
+ * Index into __cachemode2pte_tbl is the cachemode.
+ * Index into __pte2cachemode_tbl are the caching attribute bits of the pte
+ * (_PAGE_PWT, _PAGE_PCD, _PAGE_PAT) at index bit positions 0, 1, 2.
+ */
+uint16_t __cachemode2pte_tbl[_PAGE_CACHE_MODE_NUM] = {
+	[_PAGE_CACHE_MODE_WB]		= 0,
+	[_PAGE_CACHE_MODE_WC]		= _PAGE_PWT,
+	[_PAGE_CACHE_MODE_UC_MINUS]	= _PAGE_PCD,
+	[_PAGE_CACHE_MODE_UC]		= _PAGE_PCD | _PAGE_PWT,
+	[_PAGE_CACHE_MODE_WT]		= _PAGE_PCD,
+	[_PAGE_CACHE_MODE_WP]		= _PAGE_PCD,
+};
+EXPORT_SYMBOL_GPL(__cachemode2pte_tbl);
+uint8_t __pte2cachemode_tbl[8] = {
+	[__pte2cm_idx(0)] = _PAGE_CACHE_MODE_WB,
+	[__pte2cm_idx(_PAGE_PWT)] = _PAGE_CACHE_MODE_WC,
+	[__pte2cm_idx(_PAGE_PCD)] = _PAGE_CACHE_MODE_UC_MINUS,
+	[__pte2cm_idx(_PAGE_PWT | _PAGE_PCD)] = _PAGE_CACHE_MODE_UC,
+	[__pte2cm_idx(_PAGE_PAT)] = _PAGE_CACHE_MODE_WB,
+	[__pte2cm_idx(_PAGE_PWT | _PAGE_PAT)] = _PAGE_CACHE_MODE_WC,
+	[__pte2cm_idx(_PAGE_PCD | _PAGE_PAT)] = _PAGE_CACHE_MODE_UC_MINUS,
+	[__pte2cm_idx(_PAGE_PWT | _PAGE_PCD | _PAGE_PAT)] = _PAGE_CACHE_MODE_UC,
+};
+EXPORT_SYMBOL_GPL(__pte2cachemode_tbl);
+
 static unsigned long __initdata pgt_buf_start;
 static unsigned long __initdata pgt_buf_end;
 static unsigned long __initdata pgt_buf_top;
arch/x86/mm/init_64.c +6 -3
index 5621c47..201f3d0 100644
@@ -327,12 +327,15 @@ pte_t * __init populate_extra_pte(unsigned long vaddr)
  * Create large page table mappings for a range of physical addresses.
  */
 static void __init __init_extra_mapping(unsigned long phys, unsigned long size,
-						pgprot_t prot)
+					enum page_cache_mode cache)
 {
 	pgd_t *pgd;
 	pud_t *pud;
 	pmd_t *pmd;
+	pgprot_t prot;
 
+	pgprot_val(prot) = pgprot_val(PAGE_KERNEL_LARGE) |
+		pgprot_val(pgprot_4k_2_large(cachemode2pgprot(cache)));
 	BUG_ON((phys & ~PMD_MASK) || (size & ~PMD_MASK));
 	for (; size; phys += PMD_SIZE, size -= PMD_SIZE) {
 		pgd = pgd_offset_k((unsigned long)__va(phys));
@@ -355,12 +358,12 @@ static void __init __init_extra_mapping(unsigned long phys, unsigned long size,
 
 void __init init_extra_mapping_wb(unsigned long phys, unsigned long size)
 {
-	__init_extra_mapping(phys, size, PAGE_KERNEL_LARGE);
+	__init_extra_mapping(phys, size, _PAGE_CACHE_MODE_WB);
 }
 
 void __init init_extra_mapping_uc(unsigned long phys, unsigned long size)
 {
-	__init_extra_mapping(phys, size, PAGE_KERNEL_LARGE_NOCACHE);
+	__init_extra_mapping(phys, size, _PAGE_CACHE_MODE_UC);
 }
 
 /*
arch/x86/mm/iomap_32.c +10 -5
index 7b179b4..eb879cf 100644
@@ -33,17 +33,17 @@ static int is_io_mapping_possible(resource_size_t base, unsigned long size)
 
 int iomap_create_wc(resource_size_t base, unsigned long size, pgprot_t *prot)
 {
-	unsigned long flag = _PAGE_CACHE_WC;
+	enum page_cache_mode pcm = _PAGE_CACHE_MODE_WC;
 	int ret;
 
 	if (!is_io_mapping_possible(base, size))
 		return -EINVAL;
 
-	ret = io_reserve_memtype(base, base + size, &flag);
+	ret = io_reserve_memtype(base, base + size, &pcm);
 	if (ret)
 		return ret;
 
-	*prot = __pgprot(__PAGE_KERNEL | flag);
+	*prot = __pgprot(__PAGE_KERNEL | cachemode2protval(pcm));
 	return 0;
 }
 EXPORT_SYMBOL_GPL(iomap_create_wc);
@@ -73,6 +73,9 @@ void *kmap_atomic_prot_pfn(unsigned long pfn, pgprot_t prot)
 /*
  * Map 'pfn' using protections 'prot'
  */
+#define __PAGE_KERNEL_WC	(__PAGE_KERNEL | \
+				 cachemode2protval(_PAGE_CACHE_MODE_WC))
+
 void __iomem *
 iomap_atomic_prot_pfn(unsigned long pfn, pgprot_t prot)
 {
@@ -82,12 +85,14 @@ iomap_atomic_prot_pfn(unsigned long pfn, pgprot_t prot)
 	 * MTRR is UC or WC.  UC_MINUS gets the real intention, of the
 	 * user, which is "WC if the MTRR is WC, UC if you can't do that."
 	 */
-	if (!pat_enabled && pgprot_val(prot) == pgprot_val(PAGE_KERNEL_WC))
-		prot = PAGE_KERNEL_UC_MINUS;
+	if (!pat_enabled && pgprot_val(prot) == __PAGE_KERNEL_WC)
+		prot = __pgprot(__PAGE_KERNEL |
+				protval_pagemode(_PAGE_CACHE_MODE_UC_MINUS));
 
 	return (void __force __iomem *) kmap_atomic_prot_pfn(pfn, prot);
 }
 EXPORT_SYMBOL_GPL(iomap_atomic_prot_pfn);
+#undef __PAGE_KERNEL_WC
 
 void
 iounmap_atomic(void __iomem *kvaddr)
arch/x86/mm/ioremap.c +33 -30

147 lines not on this page · raw message

arch/x86/mm/pageattr.c +50 -34

243 lines not on this page · raw message

arch/x86/mm/pat.c +65 -61

336 lines not on this page · raw message

arch/x86/mm/pat_internal.h +12 -10
index 77e5ba1..f641162 100644
@@ -10,30 +10,32 @@ struct memtype {
 	u64			start;
 	u64			end;
 	u64			subtree_max_end;
-	unsigned long		type;
+	enum page_cache_mode	type;
 	struct rb_node		rb;
 };
 
-static inline char *cattr_name(unsigned long flags)
+static inline char *cattr_name(enum page_cache_mode pcm)
 {
-	switch (flags & _PAGE_CACHE_MASK) {
-	case _PAGE_CACHE_UC:		return "uncached";
-	case _PAGE_CACHE_UC_MINUS:	return "uncached-minus";
-	case _PAGE_CACHE_WB:		return "write-back";
-	case _PAGE_CACHE_WC:		return "write-combining";
-	default:			return "broken";
+	switch (pcm) {
+	case _PAGE_CACHE_MODE_UC:		return "uncached";
+	case _PAGE_CACHE_MODE_UC_MINUS:		return "uncached-minus";
+	case _PAGE_CACHE_MODE_WB:		return "write-back";
+	case _PAGE_CACHE_MODE_WC:		return "write-combining";
+	case _PAGE_CACHE_MODE_WT:		return "write-through";
+	case _PAGE_CACHE_MODE_WP:		return "write-protected";
+	default:				return "broken";
 	}
 }
 
 #ifdef CONFIG_X86_PAT
 extern int rbt_memtype_check_insert(struct memtype *new,
-					unsigned long *new_type);
+					enum page_cache_mode *new_type);
 extern struct memtype *rbt_memtype_erase(u64 start, u64 end);
 extern struct memtype *rbt_memtype_lookup(u64 addr);
 extern int rbt_memtype_copy_nth_element(struct memtype *out, loff_t pos);
 #else
 static inline int rbt_memtype_check_insert(struct memtype *new,
-					unsigned long *new_type)
+					enum page_cache_mode *new_type)
 { return 0; }
 static inline struct memtype *rbt_memtype_erase(u64 start, u64 end)
 { return NULL; }
arch/x86/mm/pat_rbtree.c +5 -3
index 415f6c4..6582adc 100644
@@ -122,11 +122,12 @@ static struct memtype *memtype_rb_exact_match(struct rb_root *root,
 
 static int memtype_rb_check_conflict(struct rb_root *root,
 				u64 start, u64 end,
-				unsigned long reqtype, unsigned long *newtype)
+				enum page_cache_mode reqtype,
+				enum page_cache_mode *newtype)
 {
 	struct rb_node *node;
 	struct memtype *match;
-	int found_type = reqtype;
+	enum page_cache_mode found_type = reqtype;
 
 	match = memtype_rb_lowest_match(&memtype_rbroot, start, end);
 	if (match == NULL)
@@ -187,7 +188,8 @@ static void memtype_rb_insert(struct rb_root *root, struct memtype *newdata)
 	rb_insert_augmented(&newdata->rb, root, &memtype_rb_augment_cb);
 }
 
-int rbt_memtype_check_insert(struct memtype *new, unsigned long *ret_type)
+int rbt_memtype_check_insert(struct memtype *new,
+			     enum page_cache_mode *ret_type)
 {
 	int err = 0;
 
arch/x86/pci/i386.c +2 -2
index 2ae525e..9d6c9d3 100644
@@ -433,14 +433,14 @@ int pci_mmap_page_range(struct pci_dev *dev, struct vm_area_struct *vma,
 		return -EINVAL;
 
 	if (pat_enabled && write_combine)
-		prot |= _PAGE_CACHE_WC;
+		prot |= cachemode2protval(_PAGE_CACHE_MODE_WC);
 	else if (pat_enabled || boot_cpu_data.x86 > 3)
 		/*
 		 * ioremap() and ioremap_nocache() defaults to UC MINUS for now.
 		 * To avoid attribute conflicts, request UC MINUS here
 		 * as well.
 		 */
-		prot |= _PAGE_CACHE_UC_MINUS;
+		prot |= cachemode2protval(_PAGE_CACHE_MODE_UC_MINUS);
 
 	prot |= _PAGE_IOMAP;	/* creating a mapping for IO */
 
drivers/video/fbdev/gbefb.c +2 -1
index 4aa56ba..b1cc7a6 100644
@@ -54,7 +54,8 @@ struct gbefb_par {
 #endif
 #endif
 #ifdef CONFIG_X86
-#define pgprot_fb(_prot) ((_prot) | _PAGE_PCD)
+#define pgprot_fb(_prot) (((_prot) & ~_PAGE_CACHE_MASK) |	\
+			  cachemode2protval(_PAGE_CACHE_MODE_UC))
 #endif
 
 /*
drivers/video/fbdev/vermilion/vermilion.c +4 -2
index 048a666..6bbc559 100644
@@ -1004,13 +1004,15 @@ static int vmlfb_mmap(struct fb_info *info, struct vm_area_struct *vma)
 	struct vml_info *vinfo = container_of(info, struct vml_info, info);
 	unsigned long offset = vma->vm_pgoff << PAGE_SHIFT;
 	int ret;
+	unsigned long prot;
 
 	ret = vmlfb_vram_offset(vinfo, offset);
 	if (ret)
 		return -EINVAL;
 
-	pgprot_val(vma->vm_page_prot) |= _PAGE_PCD;
-	pgprot_val(vma->vm_page_prot) &= ~_PAGE_PWT;
+	prot = pgprot_val(vma->vm_page_prot) & ~_PAGE_CACHE_MASK;
+	pgprot_val(vma->vm_page_prot) =
+		prot | cachemode2protval(_PAGE_CACHE_MODE_UC);
 
 	return vm_iomap_memory(vma, vinfo->vram_start,
 			vinfo->vram_contig_size);
Signature · 1 line
1.8.4.5
#2 lore raw

Re: [PATCH 1/3] x86: Make page cache mode a real type

Quoted 10 lines · Juergen Gross

At the moment there are a lot of places that handle setting or getting the page cache mode by treating the pgprot bits equal to the cache mode. This is only true because there are a lot of assumptions about the setup of the PAT MSR. Otherwise the cache type needs to get translated into pgprot bits and vice versa.

This patch tries to prepare for that by introducing a seperate type for the cache mode and adding functions to translate between those and pgprot values.

Quoted 8 lines

To avoid too much performance penalty the translation between cache mode and pgprot values is done via tables which contain the relevant information. Write-back cache mode is hard-wired to be 0, all other modes are configurable via those tables. For large pages there are translation functions as the PAT bit is located at different positions in the ptes of 4k and large pages.

Hi Juergen,

Thanks for the updates! A few comments below...

Quoted 11 lines
arch/x86/mm/iomap_32.c +3 -0
@@ -73,6 +73,9 @@ void *kmap_atomic_prot_pfn(unsigned long pfn, pgprot_t prot)
 /*
  * Map 'pfn' using protections 'prot'
  */
+#define __PAGE_KERNEL_WC	(__PAGE_KERNEL | \
+				 cachemode2protval(_PAGE_CACHE_MODE_WC))
+
 void __iomem *
 iomap_atomic_prot_pfn(unsigned long pfn, pgprot_t prot)
 {

arch/x86/mm/iomap_32.c iomap_atomic_prot_pfn(unsigned long pfn, pgprot_t prot) · line 90

Quoted 8 lines
arch/x86/mm/iomap_32.c +3 -2
@@ -82,12 +85,14 @@ iomap_atomic_prot_pfn(unsigned long pfn, pgprot_t prot)
 	 * MTRR is UC or WC.  UC_MINUS gets the real intention, of the
 	 * user, which is "WC if the MTRR is WC, UC if you can't do that."
 	 */
-	if (!pat_enabled && pgprot_val(prot) == pgprot_val(PAGE_KERNEL_WC))
-		prot = PAGE_KERNEL_UC_MINUS;
+	if (!pat_enabled && pgprot_val(prot) == __PAGE_KERNEL_WC)
+		prot = __pgprot(__PAGE_KERNEL |
+				protval_pagemode(_PAGE_CACHE_MODE_UC_MINUS));

protval_pagemode() should be cachemode2protval().

Quoted 10 lines
(quoted diff) +0 -0
 /*
drivers/video/fbdev/vermilion/vermilion.c +1 -0
index 048a666..6bbc559 100644
@@ -1004,13 +1004,15 @@ static int vmlfb_mmap(struct fb_info *info, struct vm_area_struct *vma)
 	struct vml_info *vinfo = container_of(info, struct vml_info, info);
 	unsigned long offset = vma->vm_pgoff << PAGE_SHIFT;
 	int ret;
+	unsigned long prot;

drivers/video/fbdev/vermilion/vermilion.c static int vmlfb_mmap(struct fb_info *info, struct vm_area_struct *vma) · line 1015

Quoted 10 lines
drivers/video/fbdev/vermilion/vermilion.c +3 -2
 
 	ret = vmlfb_vram_offset(vinfo, offset);
 	if (ret)
 		return -EINVAL;
 
-	pgprot_val(vma->vm_page_prot) |= _PAGE_PCD;
-	pgprot_val(vma->vm_page_prot) &= ~_PAGE_PWT;
+	prot = pgprot_val(vma->vm_page_prot) & ~_PAGE_CACHE_MASK;
+	pgprot_val(vma->vm_page_prot) =
+		prot | cachemode2protval(_PAGE_CACHE_MODE_UC);

This cache mode should be _PAGE_CACHE_MODE_UC_MINUS as the original code only sets the PCD bit.

Thanks,
-Toshi

author maintainer
#3 lore raw

Re: [Xen-devel] [PATCH 1/3] x86: Make page cache mode a real type

Quoted 37 lines · Toshi Kani
Quoted 10 lines · Juergen Gross

At the moment there are a lot of places that handle setting or getting the page cache mode by treating the pgprot bits equal to the cache mode. This is only true because there are a lot of assumptions about the setup of the PAT MSR. Otherwise the cache type needs to get translated into pgprot bits and vice versa.

This patch tries to prepare for that by introducing a seperate type for the cache mode and adding functions to translate between those and pgprot values.

Quoted 8 lines

To avoid too much performance penalty the translation between cache mode and pgprot values is done via tables which contain the relevant information. Write-back cache mode is hard-wired to be 0, all other modes are configurable via those tables. For large pages there are translation functions as the PAT bit is located at different positions in the ptes of 4k and large pages.

Hi Juergen,

Thanks for the updates! A few comments below...

Quoted 13 lines
arch/x86/mm/iomap_32.c +3 -0
@@ -73,6 +73,9 @@ void *kmap_atomic_prot_pfn(unsigned long pfn, pgprot_t prot)
  /*
   * Map 'pfn' using protections 'prot'
   */
+#define __PAGE_KERNEL_WC	(__PAGE_KERNEL | \
+				 cachemode2protval(_PAGE_CACHE_MODE_WC))
+
  void __iomem *
  iomap_atomic_prot_pfn(unsigned long pfn, pgprot_t prot)
  {
@@ -82,12 +85,14 @@ iomap_atomic_prot_pfn(unsigned long pfn, pgprot_t prot)
  	 * MTRR is UC or WC.  UC_MINUS gets the real intention, of the
  	 * user, which is "WC if the MTRR is WC, UC if you can't do that."
Quoted 8 lines

arch/x86/mm/iomap_32.c iomap_atomic_prot_pfn(unsigned long pfn, pgprot_t prot) · line 90

Quoted 6 lines
arch/x86/mm/iomap_32.c +3 -2
  	 */
-	if (!pat_enabled && pgprot_val(prot) == pgprot_val(PAGE_KERNEL_WC))
-		prot = PAGE_KERNEL_UC_MINUS;
+	if (!pat_enabled && pgprot_val(prot) == __PAGE_KERNEL_WC)
+		prot = __pgprot(__PAGE_KERNEL |
+				protval_pagemode(_PAGE_CACHE_MODE_UC_MINUS));

protval_pagemode() should be cachemode2protval().

Obviously, yes.

Quoted 16 lines
Quoted 16 lines
(quoted diff) +0 -0
  /*
drivers/video/fbdev/vermilion/vermilion.c +1 -0
index 048a666..6bbc559 100644
@@ -1004,13 +1004,15 @@ static int vmlfb_mmap(struct fb_info *info, struct vm_area_struct *vma)
  	struct vml_info *vinfo = container_of(info, struct vml_info, info);
  	unsigned long offset = vma->vm_pgoff << PAGE_SHIFT;
  	int ret;
+	unsigned long prot;
 
  	ret = vmlfb_vram_offset(vinfo, offset);
  	if (ret)
  		return -EINVAL;
 
Quoted 8 lines

drivers/video/fbdev/vermilion/vermilion.c static int vmlfb_mmap(struct fb_info *info, struct vm_area_struct *vma) · line 1015

Quoted 5 lines
drivers/video/fbdev/vermilion/vermilion.c +3 -2
-	pgprot_val(vma->vm_page_prot) |= _PAGE_PCD;
-	pgprot_val(vma->vm_page_prot) &= ~_PAGE_PWT;
+	prot = pgprot_val(vma->vm_page_prot) & ~_PAGE_CACHE_MASK;
+	pgprot_val(vma->vm_page_prot) =
+		prot | cachemode2protval(_PAGE_CACHE_MODE_UC);

This cache mode should be _PAGE_CACHE_MODE_UC_MINUS as the original code only sets the PCD bit.

I'll change it.

Thanks,

Juergen

author maintainer
#4 lore raw

PATCH 2/3 x86: Enable PAT to use cache mode translation tables

In reply to Juergen Gross

Update the translation tables from cache mode to pgprot values according to the PAT settings. This enables changing the cache attributes of a PAT index in just one place without having to change at the users side.

With this change it is possible to use the same kernel with different PAT configurations, e.g. supporting Xen.

---
 arch/x86/include/asm/pat.h           |  1 +
 arch/x86/include/asm/pgtable_types.h |  4 +++
 arch/x86/mm/init.c                   |  8 ++++++
 arch/x86/mm/pat.c                    | 54 ++++++++++++++++++++++++++++++++++--
 include/linux/mm.h                   |  1 +
 5 files changed, 66 insertions(+), 2 deletions(-)
arch/x86/include/asm/pat.h +1 -0
index 150407a..91bc4ba 100644
@@ -11,6 +11,7 @@ static const int pat_enabled;
 #endif
 
 extern void pat_init(void);
+void pat_init_cache_modes(void);
 
 extern int reserve_memtype(u64 start, u64 end,
 		enum page_cache_mode req_pcm, enum page_cache_mode *ret_pcm);
arch/x86/include/asm/pgtable_types.h +4 -0
index 0d38511..bd2f50f 100644
@@ -338,6 +338,10 @@ extern uint8_t __pte2cachemode_tbl[8];
 	((((cb) >> (_PAGE_BIT_PAT - 2)) & 4) |		\
 	 (((cb) >> (_PAGE_BIT_PCD - 1)) & 2) |		\
 	 (((cb) >> _PAGE_BIT_PWT) & 1))
+#define __cm_idx2pte(i)					\
+	((((i) & 4) << (_PAGE_BIT_PAT - 2)) |		\
+	 (((i) & 2) << (_PAGE_BIT_PCD - 1)) |		\
+	 (((i) & 1) << _PAGE_BIT_PWT))
 
 static inline unsigned long cachemode2protval(enum page_cache_mode pcm)
 {
arch/x86/mm/init.c +8 -0
index a9776ba..82b41d5 100644
@@ -716,3 +716,11 @@ void __init zone_sizes_init(void)
 	free_area_init_nodes(max_zone_pfns);
 }
 
+void update_cache_mode_entry(unsigned entry, enum page_cache_mode cache)
+{
+	/* entry 0 MUST be WB (hardwired to speed up translations) */
+	BUG_ON(!entry && cache != _PAGE_CACHE_MODE_WB);
+
+	__cachemode2pte_tbl[cache] = __cm_idx2pte(entry);
+	__pte2cachemode_tbl[entry] = cache;
+}
arch/x86/mm/pat.c +52 -2

66 lines not on this page · raw message

include/linux/mm.h +1 -0
index 8981cc8..d7bf551 100644
@@ -1574,6 +1574,7 @@ extern void free_area_init(unsigned long * zones_size);
 extern void free_area_init_node(int nid, unsigned long * zones_size,
 		unsigned long zone_start_pfn, unsigned long *zholes_size);
 extern void free_initmem(void);
+extern void update_cache_mode_entry(unsigned entry, enum page_cache_mode cache);
 
 /*
  * Free reserved pages within range [PAGE_ALIGN(start), end & PAGE_MASK)
Signature · 1 line
1.8.4.5
Reviewed-by on v1 2/3
#5 lore raw

Re: [PATCH 2/3] x86: Enable PAT to use cache mode translation tables

Quoted 8 lines · Juergen Gross

Update the translation tables from cache mode to pgprot values according to the PAT settings. This enables changing the cache attributes of a PAT index in just one place without having to change at the users side.

With this change it is possible to use the same kernel with different PAT configurations, e.g. supporting Xen.

Thanks,
-Toshi

#6 lore raw

Re: [PATCH 2/3] x86: Enable PAT to use cache mode translation tables

In reply to Juergen Gross
Quoted 8 lines · Juergen Gross

Update the translation tables from cache mode to pgprot values according to the PAT settings. This enables changing the cache attributes of a PAT index in just one place without having to change at the users side.

With this change it is possible to use the same kernel with different PAT configurations, e.g. supporting Xen.

I know I sent my review-by before, but my build test failed...

include/linux/mm.h extern void free_area_init(unsigned long * zones_size); · line 1577

Quoted 9 lines
include/linux/mm.h +1 -0
index 8981cc8..d7bf551 100644
@@ -1574,6 +1574,7 @@ extern void free_area_init(unsigned long * zones_size);
 extern void free_area_init_node(int nid, unsigned long * zones_size,
 		unsigned long zone_start_pfn, unsigned long *zholes_size);
 extern void free_initmem(void);
+extern void update_cache_mode_entry(unsigned entry, enum page_cache_mode cache);

This new interface cannot be defined in this common header file since page_cache_mode is only defined in x86. It should be defined in x86 header, such as pat.h.

Thanks,
-Toshi

author maintainer
#7 lore raw

Re: [PATCH 2/3] x86: Enable PAT to use cache mode translation tables

Quoted 12 lines · Toshi Kani
Quoted 8 lines · Juergen Gross

Update the translation tables from cache mode to pgprot values according to the PAT settings. This enables changing the cache attributes of a PAT index in just one place without having to change at the users side.

With this change it is possible to use the same kernel with different PAT configurations, e.g. supporting Xen.

I know I sent my review-by before, but my build test failed...

Quoted 13 lines

include/linux/mm.h extern void free_area_init(unsigned long * zones_size); · line 1577

Quoted 9 lines
include/linux/mm.h +1 -0
index 8981cc8..d7bf551 100644
@@ -1574,6 +1574,7 @@ extern void free_area_init(unsigned long * zones_size);
  extern void free_area_init_node(int nid, unsigned long * zones_size,
  		unsigned long zone_start_pfn, unsigned long *zholes_size);
  extern void free_initmem(void);
+extern void update_cache_mode_entry(unsigned entry, enum page_cache_mode cache);

This new interface cannot be defined in this common header file since page_cache_mode is only defined in x86. It should be defined in x86 header, such as pat.h.

Oh, right you are.

Thanks,

Juergen

author maintainer
#8 lore raw

PATCH 3/3 Support Xen pv-domains using PAT

In reply to Juergen Gross

With the dynamical mapping between cache modes and pgprot values it is now possible to use all cache modes via the Xen hypervisor PAT settings in a pv domain.

All to be done is to read the PAT configuration MSR and set up the translation tables accordingly.

---
 arch/x86/xen/enlighten.c | 11 ++++++----
 arch/x86/xen/mmu.c       | 52 ++++++++++++------------------------------------
 arch/x86/xen/xen-ops.h   |  1 +
 3 files changed, 21 insertions(+), 43 deletions(-)
arch/x86/xen/enlighten.c +7 -4
index c0cb11f..113c01c 100644
@@ -1552,12 +1552,15 @@ asmlinkage __visible void __init xen_start_kernel(void)
 
 	xen_init_mmu_ops();
 
+	/*
+	 * Modify the cache mode translation tables to match Xen's PAT
+	 * configuration.
+	 */
+
+	xen_init_cache_types();
+
 	/* Prevent unwanted bits from being set in PTEs. */
 	__supported_pte_mask &= ~_PAGE_GLOBAL;
-#if 0
-	if (!xen_initial_domain())
-#endif
-		__supported_pte_mask &= ~(_PAGE_PWT | _PAGE_PCD);
 
 	__supported_pte_mask |= _PAGE_IOMAP;
 
arch/x86/xen/mmu.c +13 -39

73 lines not on this page · raw message

arch/x86/xen/xen-ops.h +1 -0
index 28c7e0b..348e68aa 100644
@@ -34,6 +34,7 @@ extern unsigned long xen_max_p2m_pfn;
 void xen_mm_pin_all(void);
 void xen_mm_unpin_all(void);
 void xen_set_pat(u64);
+void xen_init_cache_types(void);
 
 char * __init xen_memory_setup(void);
 char * xen_auto_xlated_memory_setup(void);
Signature · 1 line
1.8.4.5
#9 lore raw

Re: [Xen-devel] [PATCH 3/3] Support Xen pv-domains using PAT

Quoted 6 lines · Juergen Gross

With the dynamical mapping between cache modes and pgprot values it is now possible to use all cache modes via the Xen hypervisor PAT settings in a pv domain.

All to be done is to read the PAT configuration MSR and set up the translation tables accordingly.

[...]

arch/x86/xen/mmu.c __visible pgdval_t xen_pgd_val(pgd_t pgd) · line 462

Quoted 4 lines
arch/x86/xen/mmu.c +4 -0
+void xen_init_cache_types(void)
+{
+	pat_init_cache_modes();
+}

No need to introduce xen_init_cache_types(), just call pat_init_cache_modes() directly.

arch/x86/xen/mmu.c __visible pgdval_t xen_pgd_val(pgd_t pgd) · line 467

Quoted 7 lines

void xen_set_pat(u64 pat)
{

arch/x86/xen/mmu.c +1 -3
-	/* We expect Linux to use a PAT setting of
-	 * UC UC- WC WB (ignoring the PAT flag) */
-	WARN_ON(pat != 0x0007010600070106ull);
+	xen_init_cache_types();
 }

Remove xen_set_pat() and remove the special-cased handling of the PAT MSR from xen_write_msr_safe().

David

author maintainer
#10 lore raw

Re: [Xen-devel] [PATCH 3/3] Support Xen pv-domains using PAT

Quoted 7 lines · David Vrabel
Quoted 6 lines · Juergen Gross

With the dynamical mapping between cache modes and pgprot values it is now possible to use all cache modes via the Xen hypervisor PAT settings in a pv domain.

All to be done is to read the PAT configuration MSR and set up the translation tables accordingly.

Quoted 8 lines

[...]

arch/x86/xen/mmu.c __visible pgdval_t xen_pgd_val(pgd_t pgd) · line 462

Quoted 4 lines
arch/x86/xen/mmu.c +4 -0
+void xen_init_cache_types(void)
+{
+	pat_init_cache_modes();
+}

No need to introduce xen_init_cache_types(), just call pat_init_cache_modes() directly.

Indeed. This was a leftover from a previous version.

Quoted 11 lines

arch/x86/xen/mmu.c __visible pgdval_t xen_pgd_val(pgd_t pgd) · line 467

Quoted 8 lines
  void xen_set_pat(u64 pat)
  {
arch/x86/xen/mmu.c +1 -3
-	/* We expect Linux to use a PAT setting of
-	 * UC UC- WC WB (ignoring the PAT flag) */
-	WARN_ON(pat != 0x0007010600070106ull);
+	xen_init_cache_types();
  }

Remove xen_set_pat() and remove the special-cased handling of the PAT MSR from xen_write_msr_safe().

Hmm, why not. Reduces Xen specific handling even more.

Juergen

#11 lore raw
In reply to Juergen Gross
Quoted 17 lines · Juergen Gross

The x86 architecture offers via the PAT (Page Attribute Table) a way to specify different caching modes in page table entries. The PAT MSR contains 8 entries each specifying one of 6 possible cache modes. A pte references one of those entries via 3 bits: _PAGE_PAT, _PAGE_PWT and _PAGE_PCD.

The Linux kernel currently supports only 4 different cache modes. The PAT MSR is set up in a way that the setting of _PAGE_PAT in a pte doesn't matter: the top 4 entries in the PAT MSR are the same as the 4 lower entries.

This results in the kernel not supporting e.g. write-through mode. Especially this cache mode would speed up drivers of video cards which now have to use uncached accesses.

OTOH some old processors (Pentium) don't support PAT correctly and the Xen hypervisor has been using a different PAT MSR configuration for some time now and can't change that as this setting is part of the ABI.

Quoted 8 lines

This patch set abstracts the cache mode from the pte and introduces tables to translate between cache mode and pte bits (the default cache mode "write back" is hard-wired to PAT entry 0). The tables are statically initialized with values being compatible to old processors and current usage. As soon as the PAT MSR is changed (or - in case of Xen - is read at boot time) the tables are changed accordingly. Requests of mappings with special cache modes are always possible now, in case they are not supported there will be a fallback to a compatible but slower mode.

I feel like I'm missing something here. Where's the support for the high PAT bit on huge pages? Once you start using the top four entries, you'll need that.

Also, this probably needs errata handling. IIRC there are a handful of CPUs that support PAT but don't work correctly if the high bit is set.

--Andy

#12 lore raw
Quoted 27 lines · Andy Lutomirski
Quoted 17 lines · Juergen Gross

The x86 architecture offers via the PAT (Page Attribute Table) a way to specify different caching modes in page table entries. The PAT MSR contains 8 entries each specifying one of 6 possible cache modes. A pte references one of those entries via 3 bits: _PAGE_PAT, _PAGE_PWT and _PAGE_PCD.

The Linux kernel currently supports only 4 different cache modes. The PAT MSR is set up in a way that the setting of _PAGE_PAT in a pte doesn't matter: the top 4 entries in the PAT MSR are the same as the 4 lower entries.

This results in the kernel not supporting e.g. write-through mode. Especially this cache mode would speed up drivers of video cards which now have to use uncached accesses.

OTOH some old processors (Pentium) don't support PAT correctly and the Xen hypervisor has been using a different PAT MSR configuration for some time now and can't change that as this setting is part of the ABI.

Quoted 8 lines

This patch set abstracts the cache mode from the pte and introduces tables to translate between cache mode and pte bits (the default cache mode "write back" is hard-wired to PAT entry 0). The tables are statically initialized with values being compatible to old processors and current usage. As soon as the PAT MSR is changed (or - in case of Xen - is read at boot time) the tables are changed accordingly. Requests of mappings with special cache modes are always possible now, in case they are not supported there will be a fallback to a compatible but slower mode.

Quoted 3 lines

I feel like I'm missing something here. Where's the support for the high PAT bit on huge pages? Once you start using the top four entries, you'll need that.

pgprot_4k_2_large() and pgprot_large_2_4k() provide the conversion of the PAT bit.

Quoted 2 lines

Also, this probably needs errata handling. IIRC there are a handful of CPUs that support PAT but don't work correctly if the high bit is set.

This patchset provides the infrastructure, but does not actually use the upper four entries. I am working on additional patchset on top of this, which enables WT with the PAT bit except on the following Intel processors. If I missed some processors affected, please let me know.

errata cpuid
--------------------------------------

  Pentium 2, A52   family 0x6, model 0x5
  Pentium 3, E27   family 0x6, model 0x7
  Pentium M, Y26   family 0x6, model 0x9
  Pentium 4, N46   family 0xf, model 0x0

Thanks,
-Toshi

#13 lore raw
Quoted 28 lines · Toshi Kani
Quoted 27 lines · Andy Lutomirski
Quoted 25 lines · Juergen Gross

The x86 architecture offers via the PAT (Page Attribute Table) a way to specify different caching modes in page table entries. The PAT MSR contains 8 entries each specifying one of 6 possible cache modes. A pte references one of those entries via 3 bits: _PAGE_PAT, _PAGE_PWT and _PAGE_PCD.

The Linux kernel currently supports only 4 different cache modes. The PAT MSR is set up in a way that the setting of _PAGE_PAT in a pte doesn't matter: the top 4 entries in the PAT MSR are the same as the 4 lower entries.

This results in the kernel not supporting e.g. write-through mode. Especially this cache mode would speed up drivers of video cards which now have to use uncached accesses.

OTOH some old processors (Pentium) don't support PAT correctly and the Xen hypervisor has been using a different PAT MSR configuration for some time now and can't change that as this setting is part of the ABI.

This patch set abstracts the cache mode from the pte and introduces tables to translate between cache mode and pte bits (the default cache mode "write back" is hard-wired to PAT entry 0). The tables are statically initialized with values being compatible to old processors and current usage. As soon as the PAT MSR is changed (or - in case of Xen - is read at boot time) the tables are changed accordingly. Requests of mappings with special cache modes are always possible now, in case they are not supported there will be a fallback to a compatible but slower mode.

Quoted 6 lines
Quoted 3 lines

I feel like I'm missing something here. Where's the support for the high PAT bit on huge pages? Once you start using the top four entries, you'll need that.

pgprot_4k_2_large() and pgprot_large_2_4k() provide the conversion of the PAT bit.

Whoops, I missed that.

Quoted 11 lines
Quoted 3 lines

Also, this probably needs errata handling. IIRC there are a handful of CPUs that support PAT but don't work correctly if the high bit is set.

This patchset provides the infrastructure, but does not actually use the upper four entries. I am working on additional patchset on top of this, which enables WT with the PAT bit except on the following Intel processors. If I missed some processors affected, please let me know.

errata cpuid
--------------------------------------

Quoted 7 lines
  Pentium 2, A52   family 0x6, model 0x5
  Pentium 3, E27   family 0x6, model 0x7
  Pentium M, Y26   family 0x6, model 0x9
  Pentium 4, N46   family 0xf, model 0x0

From very vague memory, that sounds correct.

--Andy