Skip to contentSkip to conversation

x86, PAT, CPA: Cleanups and minor bug fixes

PATCH In review [email protected] 4 participants 18 messages Apr 9, 2009 – Apr 13, 2009

Subsystems (3)
lore
Share & tools
lore
#1 lore raw

PATCH 1/6 x86, CPA: Change idmap attribute before ioremap attribute setup

From: Suresh Siddha <[email protected]>
Subject: [patch 1/6] x86, CPA: Change idmap attribute before ioremap attribute setup

Change the identity mapping with the requested attribute first, before we setup the virtual memory mapping with the new requested attribute. This makes sure that there is no window when identity map'ed attribute may disagree with ioremap range on the attribute type.
This also avoids doing cpa on the ioremap'ed address twice (first in ioremap_page_range and then in ioremap_change_attr using vaddr), and should improve ioremap performance a bit.

---
 arch/x86/mm/ioremap.c |    7 ++++---
 1 files changed, 4 insertions(+), 3 deletions(-)
arch/x86/mm/ioremap.c +4 -3
index 0dfa09d..329387e 100644
@@ -280,15 +280,16 @@ static void __iomem *__ioremap_caller(resource_size_t phys_addr,
 		return NULL;
 	area->phys_addr = phys_addr;
 	vaddr = (unsigned long) area->addr;
-	if (ioremap_page_range(vaddr, vaddr + size, phys_addr, prot)) {
+
+	if (kernel_map_sync_memtype(phys_addr, size, prot_val)) {
 		free_memtype(phys_addr, phys_addr + size);
 		free_vm_area(area);
 		return NULL;
 	}
 
-	if (ioremap_change_attr(vaddr, size, prot_val) < 0) {
+	if (ioremap_page_range(vaddr, vaddr + size, phys_addr, prot)) {
 		free_memtype(phys_addr, phys_addr + size);
-		vunmap(area->addr);
+		free_vm_area(area);
 		return NULL;
 	}
 
Signature · 3 lines
1.6.0.6

-- 
#2 lore raw

[tip:x86/pat] x86, CPA: Change idmap attribute before ioremap attribute setup

Commit-ID:  43a432b1559798d33970261f710030f787770231
Gitweb:     http://git.kernel.org/tip/43a432b1559798d33970261f710030f787770231
Author:     Suresh Siddha <[email protected]>
AuthorDate: Thu, 9 Apr 2009 14:26:47 -0700
Committer:  Ingo Molnar <[email protected]>
CommitDate: Fri, 10 Apr 2009 13:55:46 +0200

x86, CPA: Change idmap attribute before ioremap attribute setup

Change the identity mapping with the requested attribute first, before we setup the virtual memory mapping with the new requested attribute.

This makes sure that there is no window when identity map'ed attribute may disagree with ioremap range on the attribute type.

This also avoids doing cpa on the ioremap'ed address twice (first in ioremap_page_range and then in ioremap_change_attr using vaddr), and should improve ioremap performance a bit.

LKML-Reference: <[email protected]>

---
 arch/x86/mm/ioremap.c |    7 ++++---
 1 files changed, 4 insertions(+), 3 deletions(-)
arch/x86/mm/ioremap.c +4 -3
index 0dfa09d..329387e 100644
@@ -280,15 +280,16 @@ static void __iomem *__ioremap_caller(resource_size_t phys_addr,
 		return NULL;
 	area->phys_addr = phys_addr;
 	vaddr = (unsigned long) area->addr;
-	if (ioremap_page_range(vaddr, vaddr + size, phys_addr, prot)) {
+
+	if (kernel_map_sync_memtype(phys_addr, size, prot_val)) {
 		free_memtype(phys_addr, phys_addr + size);
 		free_vm_area(area);
 		return NULL;
 	}
 
-	if (ioremap_change_attr(vaddr, size, prot_val) < 0) {
+	if (ioremap_page_range(vaddr, vaddr + size, phys_addr, prot)) {
 		free_memtype(phys_addr, phys_addr + size);
-		vunmap(area->addr);
+		free_vm_area(area);
 		return NULL;
 	}
 
#3 lore raw

PATCH 2/6 x86, PAT: Change order of cpa and free in set_memory_wb

In reply to [email protected]

To be free of aliasing due to races, set_memory_* interfaces should follow ordering of reserving, changing memtype to UC/WC, changing memtype back to WB followed by free.

---
 arch/x86/mm/pageattr.c |   11 +++++++----
 1 files changed, 7 insertions(+), 4 deletions(-)
arch/x86/mm/pageattr.c +7 -4
index 1224865..38dc61f 100644
@@ -1007,15 +1007,19 @@ int _set_memory_wb(unsigned long addr, int numpages)
 
 int set_memory_wb(unsigned long addr, int numpages)
 {
+	int ret = _set_memory_wb(addr, numpages);
 	free_memtype(__pa(addr), __pa(addr) + numpages * PAGE_SIZE);
-
-	return _set_memory_wb(addr, numpages);
+	return ret;
 }
 EXPORT_SYMBOL(set_memory_wb);
 
 int set_memory_array_wb(unsigned long *addr, int addrinarray)
 {
 	int i;
+	int ret;
+
+	ret = change_page_attr_clear(addr, addrinarray,
+				      __pgprot(_PAGE_CACHE_MASK), 1);
 
 	for (i = 0; i < addrinarray; i++) {
 		unsigned long start = __pa(addr[i]);
@@ -1028,8 +1032,7 @@ int set_memory_array_wb(unsigned long *addr, int addrinarray)
 		}
 		free_memtype(start, end);
 	}
-	return change_page_attr_clear(addr, addrinarray,
-				      __pgprot(_PAGE_CACHE_MASK), 1);
+	return ret;
 }
 EXPORT_SYMBOL(set_memory_array_wb);
 
Signature · 3 lines
1.6.0.6

-- 
#4 lore raw

[tip:x86/pat] x86, PAT: Change order of cpa and free in set_memory_wb

Commit-ID:  a5593e0b329a14dea41ea173380dbf1533de2bd2
Gitweb:     http://git.kernel.org/tip/a5593e0b329a14dea41ea173380dbf1533de2bd2
Author:     [email protected] <[email protected]>
AuthorDate: Thu, 9 Apr 2009 14:26:48 -0700
Committer:  Ingo Molnar <[email protected]>
CommitDate: Fri, 10 Apr 2009 13:55:46 +0200

x86, PAT: Change order of cpa and free in set_memory_wb

To be free of aliasing due to races, set_memory_* interfaces should follow ordering of reserving, changing memtype to UC/WC, changing memtype back to WB followed by free.

LKML-Reference: <[email protected]>

---
 arch/x86/mm/pageattr.c |   11 +++++++----
 1 files changed, 7 insertions(+), 4 deletions(-)
arch/x86/mm/pageattr.c +7 -4
index d71e1b6..d487eaa 100644
@@ -1021,15 +1021,19 @@ int _set_memory_wb(unsigned long addr, int numpages)
 
 int set_memory_wb(unsigned long addr, int numpages)
 {
+	int ret = _set_memory_wb(addr, numpages);
 	free_memtype(__pa(addr), __pa(addr) + numpages * PAGE_SIZE);
-
-	return _set_memory_wb(addr, numpages);
+	return ret;
 }
 EXPORT_SYMBOL(set_memory_wb);
 
 int set_memory_array_wb(unsigned long *addr, int addrinarray)
 {
 	int i;
+	int ret;
+
+	ret = change_page_attr_clear(addr, addrinarray,
+				      __pgprot(_PAGE_CACHE_MASK), 1);
 
 	for (i = 0; i < addrinarray; i++) {
 		unsigned long start = __pa(addr[i]);
@@ -1042,8 +1046,7 @@ int set_memory_array_wb(unsigned long *addr, int addrinarray)
 		}
 		free_memtype(start, end);
 	}
-	return change_page_attr_clear(addr, addrinarray,
-				      __pgprot(_PAGE_CACHE_MASK), 1);
+	return ret;
 }
 EXPORT_SYMBOL(set_memory_array_wb);
 
#5 lore raw

PATCH 3/6 x86, PAT: Handle faults cleanly in set_memory_ APIs

In reply to [email protected]

Handle faults and do proper cleanups in set_memory_*() functions. In some cases, these functions were not doing proper free on failure paths.

With the changes to tracking memtype of RAM pages in struct page instead of pat list, we do not need the changes in commits c5e147. This patch reverts that change.

---
 arch/x86/mm/pageattr.c |  113 +++++++++++++++++++++++++++--------------------
 1 files changed, 65 insertions(+), 48 deletions(-)
arch/x86/mm/pageattr.c +65 -48
index 38dc61f..3226504 100644
@@ -931,52 +931,56 @@ int _set_memory_uc(unsigned long addr, int numpages)
 
 int set_memory_uc(unsigned long addr, int numpages)
 {
+	int ret;
+
 	/*
 	 * for now UC MINUS. see comments in ioremap_nocache()
 	 */
-	if (reserve_memtype(__pa(addr), __pa(addr) + numpages * PAGE_SIZE,
-			    _PAGE_CACHE_UC_MINUS, NULL))
-		return -EINVAL;
+	ret = reserve_memtype(__pa(addr), __pa(addr) + numpages * PAGE_SIZE,
+			    _PAGE_CACHE_UC_MINUS, NULL);
+	if (ret)
+		goto out_err;
+
+	ret = _set_memory_uc(addr, numpages);
+	if (ret)
+		goto out_free;
+
+	return 0;
 
-	return _set_memory_uc(addr, numpages);
+out_free:
+	free_memtype(__pa(addr), __pa(addr) + numpages * PAGE_SIZE);
+out_err:
+	return ret;
 }
 EXPORT_SYMBOL(set_memory_uc);
 
 int set_memory_array_uc(unsigned long *addr, int addrinarray)
 {
-	unsigned long start;
-	unsigned long end;
-	int i;
+	int i, j;
+	int ret;
+
 	/*
 	 * for now UC MINUS. see comments in ioremap_nocache()
 	 */
 	for (i = 0; i < addrinarray; i++) {
-		start = __pa(addr[i]);
-		for (end = start + PAGE_SIZE; i < addrinarray - 1; end += PAGE_SIZE) {
-			if (end != __pa(addr[i + 1]))
-				break;
-			i++;
-		}
-		if (reserve_memtype(start, end, _PAGE_CACHE_UC_MINUS, NULL))
-			goto out;
+		ret = reserve_memtype(__pa(addr[i]), __pa(addr[i]) + PAGE_SIZE,
+					_PAGE_CACHE_UC_MINUS, NULL);
+		if (ret)
+			goto out_free;
 	}
 
-	return change_page_attr_set(addr, addrinarray,
+	ret = change_page_attr_set(addr, addrinarray,
 				    __pgprot(_PAGE_CACHE_UC_MINUS), 1);
-out:
-	for (i = 0; i < addrinarray; i++) {
-		unsigned long tmp = __pa(addr[i]);
-
-		if (tmp == start)
-			break;
-		for (end = tmp + PAGE_SIZE; i < addrinarray - 1; end += PAGE_SIZE) {
-			if (end != __pa(addr[i + 1]))
-				break;
-			i++;
-		}
-		free_memtype(tmp, end);
-	}
-	return -EINVAL;
+	if (ret)
+		goto out_free;
+
+	return 0;
+
+out_free:
+	for (j = 0; j < i; j++)
+		free_memtype(__pa(addr[j]), __pa(addr[j]) + PAGE_SIZE);
+
+	return ret;
 }
 EXPORT_SYMBOL(set_memory_array_uc);
 
@@ -988,14 +992,26 @@ int _set_memory_wc(unsigned long addr, int numpages)
 
 int set_memory_wc(unsigned long addr, int numpages)
 {
+	int ret;
+
 	if (!pat_enabled)
 		return set_memory_uc(addr, numpages);
 
-	if (reserve_memtype(__pa(addr), __pa(addr) + numpages * PAGE_SIZE,
-		_PAGE_CACHE_WC, NULL))
-		return -EINVAL;
+	ret = reserve_memtype(__pa(addr), __pa(addr) + numpages * PAGE_SIZE,
+		_PAGE_CACHE_WC, NULL);
+	if (ret)
+		goto out_err;
 
-	return _set_memory_wc(addr, numpages);
+	ret = _set_memory_wc(addr, numpages);
+	if (ret)
+		goto out_free;
+
+	return 0;
+
+out_free:
+	free_memtype(__pa(addr), __pa(addr) + numpages * PAGE_SIZE);
+out_err:
+	return ret;
 }
 EXPORT_SYMBOL(set_memory_wc);
 
@@ -1007,9 +1023,14 @@ int _set_memory_wb(unsigned long addr, int numpages)
 
 int set_memory_wb(unsigned long addr, int numpages)
 {
-	int ret = _set_memory_wb(addr, numpages);
+	int ret;
+
+	ret = _set_memory_wb(addr, numpages);
+	if (ret)
+		return ret;
+
 	free_memtype(__pa(addr), __pa(addr) + numpages * PAGE_SIZE);
-	return ret;
+	return 0;
 }
 EXPORT_SYMBOL(set_memory_wb);
 
@@ -1020,19 +1041,13 @@ int set_memory_array_wb(unsigned long *addr, int addrinarray)
 
 	ret = change_page_attr_clear(addr, addrinarray,
 				      __pgprot(_PAGE_CACHE_MASK), 1);
+	if (ret)
+		return ret;
 
-	for (i = 0; i < addrinarray; i++) {
-		unsigned long start = __pa(addr[i]);
-		unsigned long end;
+	for (i = 0; i < addrinarray; i++)
+		free_memtype(__pa(addr[i]), __pa(addr[i]) + PAGE_SIZE);
 
-		for (end = start + PAGE_SIZE; i < addrinarray - 1; end += PAGE_SIZE) {
-			if (end != __pa(addr[i + 1]))
-				break;
-			i++;
-		}
-		free_memtype(start, end);
-	}
-	return ret;
+	return 0;
 }
 EXPORT_SYMBOL(set_memory_array_wb);
 
@@ -1125,6 +1140,8 @@ int set_pages_array_wb(struct page **pages, int addrinarray)
 
 	retval = cpa_clear_pages_array(pages, addrinarray,
 			__pgprot(_PAGE_CACHE_MASK));
+	if (retval)
+		return retval;
 
 	for (i = 0; i < addrinarray; i++) {
 		start = (unsigned long)page_address(pages[i]);
@@ -1132,7 +1149,7 @@ int set_pages_array_wb(struct page **pages, int addrinarray)
 		free_memtype(start, end);
 	}
 
-	return retval;
+	return 0;
 }
 EXPORT_SYMBOL(set_pages_array_wb);
 
Signature · 3 lines
1.6.0.6

-- 
#6 lore raw

[tip:x86/pat] x86, PAT: Handle faults cleanly in set_memory_ APIs

Commit-ID:  9fa3ab390abfc8b49fc0dd7c845b0ad224ec429f
Gitweb:     http://git.kernel.org/tip/9fa3ab390abfc8b49fc0dd7c845b0ad224ec429f
Author:     [email protected] <[email protected]>
AuthorDate: Thu, 9 Apr 2009 14:26:49 -0700
Committer:  Ingo Molnar <[email protected]>
CommitDate: Fri, 10 Apr 2009 13:55:47 +0200

x86, PAT: Handle faults cleanly in set_memory_ APIs

Handle faults and do proper cleanups in set_memory_*() functions. In some cases, these functions were not doing proper free on failure paths.

With the changes to tracking memtype of RAM pages in struct page instead of pat list, we do not need the changes in commits c5e147. This patch reverts that change.

LKML-Reference: <[email protected]>

---
 arch/x86/mm/pageattr.c |  113 +++++++++++++++++++++++++++--------------------
 1 files changed, 65 insertions(+), 48 deletions(-)
arch/x86/mm/pageattr.c +65 -48
index d487eaa..985eef8 100644
@@ -945,52 +945,56 @@ int _set_memory_uc(unsigned long addr, int numpages)
 
 int set_memory_uc(unsigned long addr, int numpages)
 {
+	int ret;
+
 	/*
 	 * for now UC MINUS. see comments in ioremap_nocache()
 	 */
-	if (reserve_memtype(__pa(addr), __pa(addr) + numpages * PAGE_SIZE,
-			    _PAGE_CACHE_UC_MINUS, NULL))
-		return -EINVAL;
+	ret = reserve_memtype(__pa(addr), __pa(addr) + numpages * PAGE_SIZE,
+			    _PAGE_CACHE_UC_MINUS, NULL);
+	if (ret)
+		goto out_err;
+
+	ret = _set_memory_uc(addr, numpages);
+	if (ret)
+		goto out_free;
+
+	return 0;
 
-	return _set_memory_uc(addr, numpages);
+out_free:
+	free_memtype(__pa(addr), __pa(addr) + numpages * PAGE_SIZE);
+out_err:
+	return ret;
 }
 EXPORT_SYMBOL(set_memory_uc);
 
 int set_memory_array_uc(unsigned long *addr, int addrinarray)
 {
-	unsigned long start;
-	unsigned long end;
-	int i;
+	int i, j;
+	int ret;
+
 	/*
 	 * for now UC MINUS. see comments in ioremap_nocache()
 	 */
 	for (i = 0; i < addrinarray; i++) {
-		start = __pa(addr[i]);
-		for (end = start + PAGE_SIZE; i < addrinarray - 1; end += PAGE_SIZE) {
-			if (end != __pa(addr[i + 1]))
-				break;
-			i++;
-		}
-		if (reserve_memtype(start, end, _PAGE_CACHE_UC_MINUS, NULL))
-			goto out;
+		ret = reserve_memtype(__pa(addr[i]), __pa(addr[i]) + PAGE_SIZE,
+					_PAGE_CACHE_UC_MINUS, NULL);
+		if (ret)
+			goto out_free;
 	}
 
-	return change_page_attr_set(addr, addrinarray,
+	ret = change_page_attr_set(addr, addrinarray,
 				    __pgprot(_PAGE_CACHE_UC_MINUS), 1);
-out:
-	for (i = 0; i < addrinarray; i++) {
-		unsigned long tmp = __pa(addr[i]);
-
-		if (tmp == start)
-			break;
-		for (end = tmp + PAGE_SIZE; i < addrinarray - 1; end += PAGE_SIZE) {
-			if (end != __pa(addr[i + 1]))
-				break;
-			i++;
-		}
-		free_memtype(tmp, end);
-	}
-	return -EINVAL;
+	if (ret)
+		goto out_free;
+
+	return 0;
+
+out_free:
+	for (j = 0; j < i; j++)
+		free_memtype(__pa(addr[j]), __pa(addr[j]) + PAGE_SIZE);
+
+	return ret;
 }
 EXPORT_SYMBOL(set_memory_array_uc);
 
@@ -1002,14 +1006,26 @@ int _set_memory_wc(unsigned long addr, int numpages)
 
 int set_memory_wc(unsigned long addr, int numpages)
 {
+	int ret;
+
 	if (!pat_enabled)
 		return set_memory_uc(addr, numpages);
 
-	if (reserve_memtype(__pa(addr), __pa(addr) + numpages * PAGE_SIZE,
-		_PAGE_CACHE_WC, NULL))
-		return -EINVAL;
+	ret = reserve_memtype(__pa(addr), __pa(addr) + numpages * PAGE_SIZE,
+		_PAGE_CACHE_WC, NULL);
+	if (ret)
+		goto out_err;
 
-	return _set_memory_wc(addr, numpages);
+	ret = _set_memory_wc(addr, numpages);
+	if (ret)
+		goto out_free;
+
+	return 0;
+
+out_free:
+	free_memtype(__pa(addr), __pa(addr) + numpages * PAGE_SIZE);
+out_err:
+	return ret;
 }
 EXPORT_SYMBOL(set_memory_wc);
 
@@ -1021,9 +1037,14 @@ int _set_memory_wb(unsigned long addr, int numpages)
 
 int set_memory_wb(unsigned long addr, int numpages)
 {
-	int ret = _set_memory_wb(addr, numpages);
+	int ret;
+
+	ret = _set_memory_wb(addr, numpages);
+	if (ret)
+		return ret;
+
 	free_memtype(__pa(addr), __pa(addr) + numpages * PAGE_SIZE);
-	return ret;
+	return 0;
 }
 EXPORT_SYMBOL(set_memory_wb);
 
@@ -1034,19 +1055,13 @@ int set_memory_array_wb(unsigned long *addr, int addrinarray)
 
 	ret = change_page_attr_clear(addr, addrinarray,
 				      __pgprot(_PAGE_CACHE_MASK), 1);
+	if (ret)
+		return ret;
 
-	for (i = 0; i < addrinarray; i++) {
-		unsigned long start = __pa(addr[i]);
-		unsigned long end;
+	for (i = 0; i < addrinarray; i++)
+		free_memtype(__pa(addr[i]), __pa(addr[i]) + PAGE_SIZE);
 
-		for (end = start + PAGE_SIZE; i < addrinarray - 1; end += PAGE_SIZE) {
-			if (end != __pa(addr[i + 1]))
-				break;
-			i++;
-		}
-		free_memtype(start, end);
-	}
-	return ret;
+	return 0;
 }
 EXPORT_SYMBOL(set_memory_array_wb);
 
@@ -1139,6 +1154,8 @@ int set_pages_array_wb(struct page **pages, int addrinarray)
 
 	retval = cpa_clear_pages_array(pages, addrinarray,
 			__pgprot(_PAGE_CACHE_MASK));
+	if (retval)
+		return retval;
 
 	for (i = 0; i < addrinarray; i++) {
 		start = (unsigned long)page_address(pages[i]);
@@ -1146,7 +1163,7 @@ int set_pages_array_wb(struct page **pages, int addrinarray)
 		free_memtype(start, end);
 	}
 
-	return retval;
+	return 0;
 }
 EXPORT_SYMBOL(set_pages_array_wb);
 
#7 lore raw

PATCH 4/6 x86, PAT: Changing memtype to WC ensuring no WB alias

In reply to [email protected]

As per SDM, there should not be any aliasing of a WC with any cacheable type across CPUs. That is if one CPU is changing the identity map memtype to _WC, no other CPU at the time of this change should not have a TLB for this page that carries a WB attribute. SDM suggests to make the page not present. But for that we will have to handle any page faults that can potentially happen due to these pages being not present.

Other way to deal with this without having any WB mapping is to change the page first to UC and then to WC. This ensures that we meet the SDM requirement of no cacheable alais to WC page. This also has same or lower overhead than marking the page not present and making it present later.

---

arch/x86/mm/pageattr.c +8 -1
index 3226504..4fa8996 100644
@@ -986,8 +986,15 @@ EXPORT_SYMBOL(set_memory_array_uc);
 
 int _set_memory_wc(unsigned long addr, int numpages)
 {
-	return change_page_attr_set(&addr, numpages,
+	int ret;
+	ret = change_page_attr_set(&addr, numpages,
+				    __pgprot(_PAGE_CACHE_UC_MINUS), 0);
+
+	if (!ret) {
+		ret = change_page_attr_set(&addr, numpages,
 				    __pgprot(_PAGE_CACHE_WC), 0);
+	}
+	return ret;
 }
 
 int set_memory_wc(unsigned long addr, int numpages)
Signature · 3 lines
1.6.0.6

-- 
#8 lore raw

[tip:x86/pat] x86, PAT: Changing memtype to WC ensuring no WB alias

Commit-ID:  3869c4aa18835c8c61b44bd0f3ace36e9d3b5bd0
Gitweb:     http://git.kernel.org/tip/3869c4aa18835c8c61b44bd0f3ace36e9d3b5bd0
Author:     [email protected] <[email protected]>
AuthorDate: Thu, 9 Apr 2009 14:26:50 -0700
Committer:  Ingo Molnar <[email protected]>
CommitDate: Fri, 10 Apr 2009 13:55:47 +0200

x86, PAT: Changing memtype to WC ensuring no WB alias

As per SDM, there should not be any aliasing of a WC with any cacheable type across CPUs. That is if one CPU is changing the identity map memtype to _WC, no other CPU at the time of this change should not have a TLB for this page that carries a WB attribute. SDM suggests to make the page not present. But for that we will have to handle any page faults that can potentially happen due to these pages being not present.

Other way to deal with this without having any WB mapping is to change the page first to UC and then to WC. This ensures that we meet the SDM requirement of no cacheable alais to WC page. This also has same or lower overhead than marking the page not present and making it present later.

LKML-Reference: <[email protected]>

---
 arch/x86/mm/pageattr.c |    9 ++++++++-
 1 files changed, 8 insertions(+), 1 deletions(-)
arch/x86/mm/pageattr.c +8 -1
index 985eef8..797f9f1 100644
@@ -1000,8 +1000,15 @@ EXPORT_SYMBOL(set_memory_array_uc);
 
 int _set_memory_wc(unsigned long addr, int numpages)
 {
-	return change_page_attr_set(&addr, numpages,
+	int ret;
+	ret = change_page_attr_set(&addr, numpages,
+				    __pgprot(_PAGE_CACHE_UC_MINUS), 0);
+
+	if (!ret) {
+		ret = change_page_attr_set(&addr, numpages,
 				    __pgprot(_PAGE_CACHE_WC), 0);
+	}
+	return ret;
 }
 
 int set_memory_wc(unsigned long addr, int numpages)
#9 lore raw

PATCH 5/6 x86, PAT: Consolidate code in pat_x_mtrr_type() and reserve_memtype()

In reply to [email protected]

From: Suresh Siddha <[email protected]>
Subject: [patch 5/6] x86, PAT: Consolidate code in pat_x_mtrr_type() and reserve_memtype()

Fix pat_x_mtrr_type() to use UC_MINUS when the mtrr type return UC. This is to be consistent with ioremap() and ioremap_nocache() which uses UC_MINUS.

Consolidate the code such that reserve_memtype() also uses pat_x_mtrr_type() when the caller doesn't specify any special attribute (non WB attribute).

---
 arch/x86/mm/ioremap.c |    3 ++-
 arch/x86/mm/pat.c     |   35 +++++++++++++----------------------
 2 files changed, 15 insertions(+), 23 deletions(-)
arch/x86/mm/ioremap.c +2 -1
index 329387e..d4c4b2c 100644
@@ -375,7 +375,8 @@ static void __iomem *ioremap_default(resource_size_t phys_addr,
 	 * - UC_MINUS for non-WB-able memory with no other conflicting mappings
 	 * - Inherit from confliting mappings otherwise
 	 */
-	err = reserve_memtype(phys_addr, phys_addr + size, -1, &flags);
+	err = reserve_memtype(phys_addr, phys_addr + size,
+				_PAGE_CACHE_WB, &flags);
 	if (err < 0)
 		return NULL;
 
arch/x86/mm/pat.c +13 -22
index 95d3b1a..3c8624c 100644
@@ -182,10 +182,10 @@ static unsigned long pat_x_mtrr_type(u64 start, u64 end, unsigned long req_type)
 		u8 mtrr_type;
 
 		mtrr_type = mtrr_type_lookup(start, end);
-		if (mtrr_type == MTRR_TYPE_UNCACHABLE)
-			return _PAGE_CACHE_UC;
-		if (mtrr_type == MTRR_TYPE_WRCOMB)
-			return _PAGE_CACHE_WC;
+		if (mtrr_type != MTRR_TYPE_WRBACK)
+			return _PAGE_CACHE_UC_MINUS;
+
+		return _PAGE_CACHE_WB;
 	}
 
 	return req_type;
@@ -352,23 +352,13 @@ int reserve_memtype(u64 start, u64 end, unsigned long req_type,
 		return 0;
 	}
 
-	if (req_type == -1) {
-		/*
-		 * Call mtrr_lookup to get the type hint. This is an
-		 * optimization for /dev/mem mmap'ers into WB memory (BIOS
-		 * tools and ACPI tools). Use WB request for WB memory and use
-		 * UC_MINUS otherwise.
-		 */
-		u8 mtrr_type = mtrr_type_lookup(start, end);
-
-		if (mtrr_type == MTRR_TYPE_WRBACK)
-			actual_type = _PAGE_CACHE_WB;
-		else
-			actual_type = _PAGE_CACHE_UC_MINUS;
-	} else {
-		actual_type = pat_x_mtrr_type(start, end,
-					      req_type & _PAGE_CACHE_MASK);
-	}
+	/*
+	 * Call mtrr_lookup to get the type hint. This is an
+	 * optimization for /dev/mem mmap'ers into WB memory (BIOS
+	 * tools and ACPI tools). Use WB request for WB memory and use
+	 * UC_MINUS otherwise.
+	 */
+	actual_type = pat_x_mtrr_type(start, end, req_type & _PAGE_CACHE_MASK);
 
 	if (new_type)
 		*new_type = actual_type;
@@ -587,7 +577,8 @@ int phys_mem_access_prot_allowed(struct file *file, unsigned long pfn,
 	if (flags != -1) {
 		retval = reserve_memtype(offset, offset + size, flags, NULL);
 	} else {
-		retval = reserve_memtype(offset, offset + size, -1, &flags);
+		retval = reserve_memtype(offset, offset + size,
+					_PAGE_CACHE_WB, &flags);
 	}
 
 	if (retval < 0)
Signature · 3 lines
1.6.0.6

-- 
#10 lore raw

[tip:x86/pat] x86, PAT: Consolidate code in pat_x_mtrr_type() and reserve_memtype()

Commit-ID:  b6ff32d9aaeeeecf98f9a852d715569183585312
Gitweb:     http://git.kernel.org/tip/b6ff32d9aaeeeecf98f9a852d715569183585312
Author:     Suresh Siddha <[email protected]>
AuthorDate: Thu, 9 Apr 2009 14:26:51 -0700
Committer:  Ingo Molnar <[email protected]>
CommitDate: Fri, 10 Apr 2009 13:55:48 +0200

x86, PAT: Consolidate code in pat_x_mtrr_type() and reserve_memtype()

Fix pat_x_mtrr_type() to use UC_MINUS when the mtrr type return UC. This is to be consistent with ioremap() and ioremap_nocache() which uses UC_MINUS.

Consolidate the code such that reserve_memtype() also uses pat_x_mtrr_type() when the caller doesn't specify any special attribute (non WB attribute).

LKML-Reference: <[email protected]>

---
 arch/x86/mm/ioremap.c |    3 ++-
 arch/x86/mm/pat.c     |   35 +++++++++++++----------------------
 2 files changed, 15 insertions(+), 23 deletions(-)
arch/x86/mm/ioremap.c +2 -1
index 329387e..d4c4b2c 100644
@@ -375,7 +375,8 @@ static void __iomem *ioremap_default(resource_size_t phys_addr,
 	 * - UC_MINUS for non-WB-able memory with no other conflicting mappings
 	 * - Inherit from confliting mappings otherwise
 	 */
-	err = reserve_memtype(phys_addr, phys_addr + size, -1, &flags);
+	err = reserve_memtype(phys_addr, phys_addr + size,
+				_PAGE_CACHE_WB, &flags);
 	if (err < 0)
 		return NULL;
 
arch/x86/mm/pat.c +13 -22
index 640339e..8d3de95 100644
@@ -182,10 +182,10 @@ static unsigned long pat_x_mtrr_type(u64 start, u64 end, unsigned long req_type)
 		u8 mtrr_type;
 
 		mtrr_type = mtrr_type_lookup(start, end);
-		if (mtrr_type == MTRR_TYPE_UNCACHABLE)
-			return _PAGE_CACHE_UC;
-		if (mtrr_type == MTRR_TYPE_WRCOMB)
-			return _PAGE_CACHE_WC;
+		if (mtrr_type != MTRR_TYPE_WRBACK)
+			return _PAGE_CACHE_UC_MINUS;
+
+		return _PAGE_CACHE_WB;
 	}
 
 	return req_type;
@@ -352,23 +352,13 @@ int reserve_memtype(u64 start, u64 end, unsigned long req_type,
 		return 0;
 	}
 
-	if (req_type == -1) {
-		/*
-		 * Call mtrr_lookup to get the type hint. This is an
-		 * optimization for /dev/mem mmap'ers into WB memory (BIOS
-		 * tools and ACPI tools). Use WB request for WB memory and use
-		 * UC_MINUS otherwise.
-		 */
-		u8 mtrr_type = mtrr_type_lookup(start, end);
-
-		if (mtrr_type == MTRR_TYPE_WRBACK)
-			actual_type = _PAGE_CACHE_WB;
-		else
-			actual_type = _PAGE_CACHE_UC_MINUS;
-	} else {
-		actual_type = pat_x_mtrr_type(start, end,
-					      req_type & _PAGE_CACHE_MASK);
-	}
+	/*
+	 * Call mtrr_lookup to get the type hint. This is an
+	 * optimization for /dev/mem mmap'ers into WB memory (BIOS
+	 * tools and ACPI tools). Use WB request for WB memory and use
+	 * UC_MINUS otherwise.
+	 */
+	actual_type = pat_x_mtrr_type(start, end, req_type & _PAGE_CACHE_MASK);
 
 	if (new_type)
 		*new_type = actual_type;
@@ -587,7 +577,8 @@ int phys_mem_access_prot_allowed(struct file *file, unsigned long pfn,
 	if (flags != -1) {
 		retval = reserve_memtype(offset, offset + size, flags, NULL);
 	} else {
-		retval = reserve_memtype(offset, offset + size, -1, &flags);
+		retval = reserve_memtype(offset, offset + size,
+					_PAGE_CACHE_WB, &flags);
 	}
 
 	if (retval < 0)
#11 lore raw

PATCH 6/6 x86, PAT: Remove duplicate memtype reserve in devmem mmap

In reply to [email protected]

From: Suresh Siddha <[email protected]>
Subject: [patch 6/6] x86, PAT: Remove duplicate memtype reserve in devmem mmap

/dev/mem mmap code was doing memtype reserve/free for a while now. Recently we added memtype tracking in remap_pfn_range, and /dev/mem mmap uses it indirectly. So, we don't need seperate tracking in /dev/mem code any more. That means another ~100 lines of code removed :-).

---
 arch/x86/include/asm/pat.h |    4 ---
 arch/x86/mm/pat.c          |   60 +------------------------------------------
 drivers/char/mem.c         |   27 -------------------
 3 files changed, 2 insertions(+), 89 deletions(-)
arch/x86/include/asm/pat.h +0 -4
index 2cd07b9..7af14e5 100644
@@ -18,9 +18,5 @@ extern int free_memtype(u64 start, u64 end);
 
 extern int kernel_map_sync_memtype(u64 base, unsigned long size,
 		unsigned long flag);
-extern void map_devmem(unsigned long pfn, unsigned long size,
-		       struct pgprot vma_prot);
-extern void unmap_devmem(unsigned long pfn, unsigned long size,
-			 struct pgprot vma_prot);
 
 #endif /* _ASM_X86_PAT_H */
arch/x86/mm/pat.c +2 -58

84 lines not on this page · raw message

drivers/char/mem.c +0 -27
index 3586b3b..8f05c38 100644
@@ -301,33 +301,7 @@ static inline int private_mapping_ok(struct vm_area_struct *vma)
 }
 #endif
 
-void __attribute__((weak))
-map_devmem(unsigned long pfn, unsigned long len, pgprot_t prot)
-{
-	/* nothing. architectures can override. */
-}
-
-void __attribute__((weak))
-unmap_devmem(unsigned long pfn, unsigned long len, pgprot_t prot)
-{
-	/* nothing. architectures can override. */
-}
-
-static void mmap_mem_open(struct vm_area_struct *vma)
-{
-	map_devmem(vma->vm_pgoff,  vma->vm_end - vma->vm_start,
-			vma->vm_page_prot);
-}
-
-static void mmap_mem_close(struct vm_area_struct *vma)
-{
-	unmap_devmem(vma->vm_pgoff,  vma->vm_end - vma->vm_start,
-			vma->vm_page_prot);
-}
-
 static struct vm_operations_struct mmap_mem_ops = {
-	.open  = mmap_mem_open,
-	.close = mmap_mem_close,
 #ifdef CONFIG_HAVE_IOREMAP_PROT
 	.access = generic_access_phys
 #endif
@@ -362,7 +336,6 @@ static int mmap_mem(struct file * file, struct vm_area_struct * vma)
 			    vma->vm_pgoff,
 			    size,
 			    vma->vm_page_prot)) {
-		unmap_devmem(vma->vm_pgoff, size, vma->vm_page_prot);
 		return -EAGAIN;
 	}
 	return 0;
Signature · 3 lines
1.6.0.6

-- 
#12 lore raw

[tip:x86/pat] x86, PAT: Remove duplicate memtype reserve in devmem mmap

Commit-ID:  0c3c8a18361a636069f5a5d9d0d0f9c2124e6b94
Gitweb:     http://git.kernel.org/tip/0c3c8a18361a636069f5a5d9d0d0f9c2124e6b94
Author:     Suresh Siddha <[email protected]>
AuthorDate: Thu, 9 Apr 2009 14:26:52 -0700
Committer:  Ingo Molnar <[email protected]>
CommitDate: Fri, 10 Apr 2009 13:55:48 +0200

x86, PAT: Remove duplicate memtype reserve in devmem mmap

/dev/mem mmap code was doing memtype reserve/free for a while now. Recently we added memtype tracking in remap_pfn_range, and /dev/mem mmap uses it indirectly. So, we don't need seperate tracking in /dev/mem code any more. That means another ~100 lines of code removed :-).

LKML-Reference: <[email protected]>

---
 arch/x86/include/asm/pat.h |    4 ---
 arch/x86/mm/pat.c          |   60 +------------------------------------------
 drivers/char/mem.c         |   27 -------------------
 3 files changed, 2 insertions(+), 89 deletions(-)
arch/x86/include/asm/pat.h +0 -4
index 2cd07b9..7af14e5 100644
@@ -18,9 +18,5 @@ extern int free_memtype(u64 start, u64 end);
 
 extern int kernel_map_sync_memtype(u64 base, unsigned long size,
 		unsigned long flag);
-extern void map_devmem(unsigned long pfn, unsigned long size,
-		       struct pgprot vma_prot);
-extern void unmap_devmem(unsigned long pfn, unsigned long size,
-			 struct pgprot vma_prot);
 
 #endif /* _ASM_X86_PAT_H */
arch/x86/mm/pat.c +2 -58

84 lines not on this page · raw message

drivers/char/mem.c +0 -27
index 3586b3b..8f05c38 100644
@@ -301,33 +301,7 @@ static inline int private_mapping_ok(struct vm_area_struct *vma)
 }
 #endif
 
-void __attribute__((weak))
-map_devmem(unsigned long pfn, unsigned long len, pgprot_t prot)
-{
-	/* nothing. architectures can override. */
-}
-
-void __attribute__((weak))
-unmap_devmem(unsigned long pfn, unsigned long len, pgprot_t prot)
-{
-	/* nothing. architectures can override. */
-}
-
-static void mmap_mem_open(struct vm_area_struct *vma)
-{
-	map_devmem(vma->vm_pgoff,  vma->vm_end - vma->vm_start,
-			vma->vm_page_prot);
-}
-
-static void mmap_mem_close(struct vm_area_struct *vma)
-{
-	unmap_devmem(vma->vm_pgoff,  vma->vm_end - vma->vm_start,
-			vma->vm_page_prot);
-}
-
 static struct vm_operations_struct mmap_mem_ops = {
-	.open  = mmap_mem_open,
-	.close = mmap_mem_close,
 #ifdef CONFIG_HAVE_IOREMAP_PROT
 	.access = generic_access_phys
 #endif
@@ -362,7 +336,6 @@ static int mmap_mem(struct file * file, struct vm_area_struct * vma)
 			    vma->vm_pgoff,
 			    size,
 			    vma->vm_page_prot)) {
-		unmap_devmem(vma->vm_pgoff, size, vma->vm_page_prot);
 		return -EAGAIN;
 	}
 	return 0;
#13 lore raw
In reply to [email protected]
Quoted 6 lines · * [email protected] <[email protected]> wrote:

This patchset contains cleanups and minor bug fixes in x86 PAT and CPA related code. The bugs were mostly found by code inspection. There should not be any functionality changes with this patchset.

Great, this series looks really nice!

Does this solve the problems reported in the "2.6.29 git master and PAT problems" thread and addressed via an earlier patch of yours:

Subject: [PATCH] x86, PAT: Remove page granularity tracking for vm_insert_pfn maps

i am worried about this particular patch - it looks more like a workaround than a true realization of a bug.

Ingo

#14 lore raw
Quoted 11 lines · Ingo Molnar
Quoted 6 lines · * [email protected] <[email protected]> wrote:

This patchset contains cleanups and minor bug fixes in x86 PAT and CPA related code. The bugs were mostly found by code inspection. There should not be any functionality changes with this patchset.

Great, this series looks really nice!

Quoted 8 lines

Does this solve the problems reported in the "2.6.29 git master and PAT problems" thread and addressed via an earlier patch of yours:

Subject: [PATCH] x86, PAT: Remove page granularity tracking for vm_insert_pfn maps

i am worried about this particular patch - it looks more like a workaround than a true realization of a bug.

No. This patchset does not fix that problem. "Remove page granularity tracking" patch is still needed with this patchset.

Yes. That patch is more of a workaround or a direction change about how we want to handle vm_insert_pfn with PAT.
When we added the page level tracking, there were no in kernel users of vm_insert_pfn() and we did not consider the usages where same address space/vma will be used to map different physical addresses over time with unmap_mapping_range() and re -inserting different pfns to the same vma. That is what X 915 driver is doing now.
You are right in saying that the patch is not handling the bug of "freeing invalid memtype" errors. They are happening due to unbalanced reserve/free (more free than reserve) in the code path of tracking vm_insert_pfn pages. I couldn't really reproduce the problem where we are getting those unbalanced frees as reported in that bug report. But, that bug report also points to another issue. The issue of 1000s of single page UC_MINUS or WC mappings from X driver. Even though it is not a functionality problem, it will have major performance impact tracking thousands of memtypes. We don't have to really track memtypes of such small chunks and they want WC (or UC_MINUS) for all those mappings. So, the plan is to forget about tracking of individual pages. That automatically takes care of the unbalanced reserve/free problem.

We are adding a new API where driver can ask for a type for a big address range, something like the entire PCI map range. And then used the type for each individual page that they may map on demand. This way we don't have to keep track of individual pages that driver may map and unmap. This is still in works, I should have a patch for this soon (a week or so).

Thanks,
Venki

#15 lore raw
In reply to [email protected]
Quoted 3 lines · On Thu, 2009-04-09 at 14:26 -0700, [email protected] wrote:

This patchset contains cleanups and minor bug fixes in x86 PAT and CPA related code. The bugs were mostly found by code inspection. There should not be any functionality changes with this patchset.

I've been curious, what are you using to test PAT changes for regressions? I've got some regression tests at:

http://cgit.freedesktop.org/xorg/app/intel-gpu-tools/

Requires KMS enabled and master of libdrm, but after that you can sudo make check, and it tests execution of several DRM paths without requiring X. In benchmarks/ there are a few microbenchmarks of various mapping types, which has been useful in making sure that we're ending up with the right PTEs.

It should work on all Intel GPUs from the 830 through the GM45, though I haven't tested pre-915 much.

Signature · 2 lines
Eric Anholt
[email protected]                         [email protected]
#16 lore raw
Quoted 8 lines · Eric Anholt
Quoted 3 lines · On Thu, 2009-04-09 at 14:26 -0700, [email protected] wrote:

This patchset contains cleanups and minor bug fixes in x86 PAT and CPA related code. The bugs were mostly found by code inspection. There should not be any functionality changes with this patchset.

I've been curious, what are you using to test PAT changes for regressions? I've got some regression tests at:

Quoted 7 lines

http://cgit.freedesktop.org/xorg/app/intel-gpu-tools/

Requires KMS enabled and master of libdrm, but after that you can sudo make check, and it tests execution of several DRM paths without requiring X. In benchmarks/ there are a few microbenchmarks of various mapping types, which has been useful in making sure that we're ending up with the right PTEs.

Looks really nice! Regarding libdrm, is there a version cutoff from where it is expected to work just fine? I've got this version:
libdrm-2.4.6-3.fc11.x86_64.

Ingo

#17 lore raw
Quoted 18 lines · Ingo Molnar
Quoted 8 lines · Eric Anholt
Quoted 3 lines · On Thu, 2009-04-09 at 14:26 -0700, [email protected] wrote:

This patchset contains cleanups and minor bug fixes in x86 PAT and CPA related code. The bugs were mostly found by code inspection. There should not be any functionality changes with this patchset.

I've been curious, what are you using to test PAT changes for regressions? I've got some regression tests at:

Quoted 7 lines

http://cgit.freedesktop.org/xorg/app/intel-gpu-tools/

Requires KMS enabled and master of libdrm, but after that you can sudo make check, and it tests execution of several DRM paths without requiring X. In benchmarks/ there are a few microbenchmarks of various mapping types, which has been useful in making sure that we're ending up with the right PTEs.

Quoted 3 lines

Looks really nice! Regarding libdrm, is there a version cutoff from where it is expected to work just fine? I've got this version:
libdrm-2.4.6-3.fc11.x86_64.

I should keep the pkgconfig check up to date, but in the worst case it doesn't compile and you go get new libdrm. The 2.4.6 check right now appears to be correct.

Signature · 2 lines
Eric Anholt
[email protected]                         [email protected]