use MTRR for write combining if PAT is not available
RFC PATCH 1/1 use MTRR for write combining if PAT is not available
Hi,
I've found a problem with X.org not setting up MTRR for the framebuffer memory. After I investigated I think this is not a X.org problem, but a kernel issue.
X.org uses libpciaccess to map the framebuffer memory. This library opens /sys/bus/pci/devices/*/resource0_wc and mmaps the memory. Unfortunately, the kernel only enables write combining if PAT is enabled, if it is not, the memory is mmapped uncached. But Xorg (respectively libpciaccess) thinks it was successfully mapped with write combining enabled and thus does not additionally set up MTRR entries.
The corresponding libpciaccess code can be found here:
http://cgit.freedesktop.org/xorg/lib/libpciaccess/tree/src/linux_sysfs.c#n501
If the kernel behavior is intentional and X.org should always set up MTRR entries, why should it use /sys/.../resource0_wc at all? I think there are 2 possibilities to make the kernel behavior consistent:
1. The mmap_wc should fail if PAT is not enabled. (libpciaccess will then map the framebuffer uncached and set up MTRR entries)
2. Use MTRR to enable write combining if PAT is not available.
In an earlier thread about ioremap_wc, Arjan van de Ven wrote that option 2 is preferred over option 1:
http://lkml.indiana.edu/hypermail/linux/kernel/0805.3/2925.html
So, I've created the attached patch implementing option 2. For me this solves the problem with the slow Video playback due to not correctly set up MTRR entries.
Kind regards,
Thomas Schlichter
From 19fb39061825a0110d1a4feb3f83dfa3f09fb738 Mon Sep 17 00:00:00 2001
From: Thomas Schlichter <[email protected]>
Date: Thu, 8 Oct 2009 21:24:07 +0200
Subject: [PATCH] Use MTRR for write combining mmap/ioremap if PAT is not available
X.org uses libpciaccess which tries to mmap with write combining enabled via /sys/bus/pci/devices/*/resource0_wc. Currently, when PAT is not enabled, we fall back to uncached mmap. Then libpciaccess thinks it succeeded mapping with write combining anabled and does not set up suited MTRR entries. ;-(
So when falling back to uncached mapping, we better try to set up MTRR entries automatically. To match this modified PCI mmap behavior, also ioremap_wc and set_memory_wc are adjusted.
--- arch/x86/mm/ioremap.c | 14 +++++++++----- arch/x86/mm/pageattr.c | 10 ++++++++-- arch/x86/pci/i386.c | 6 ++++++ 3 files changed, 23 insertions(+), 7 deletions(-)
arch/x86/mm/ioremap.c +9 -5
@@ -21,6 +21,7 @@#include <asm/tlbflush.h> #include <asm/pgalloc.h> #include <asm/pat.h> +#include <asm/mtrr.h> #include "physaddr.h"
@@ -268,11 +269,14 @@ EXPORT_SYMBOL(ioremap_nocache);*/ void __iomem *ioremap_wc (resource_size_t phys_addr,unsigned long size) { -if (pat_enabled) -return __ioremap_caller (phys_addr, size, _PAGE_CACHE_WC, -__builtin_return_address (0 )); -else -return ioremap_nocache (phys_addr, size); +if (!pat_enabled) { +void __iomem *ret =ioremap_nocache (phys_addr, size); +if (ret) +mtrr_add (phys_addr, size,MTRR_TYPE_WRCOMB , false); +return ret; + } +return __ioremap_caller (phys_addr, size, _PAGE_CACHE_WC, +__builtin_return_address (0 )); }EXPORT_SYMBOL (ioremap_wc);
arch/x86/mm/pageattr.c +8 -2
@@ -23,6 +23,7 @@#include <asm/pgalloc.h> #include <asm/proto.h> #include <asm/pat.h> +#include <asm/mtrr.h> /* * The current flushing context - we pass it instead of 5 arguments:
@@ -1010,8 +1011,13 @@ int set_memory_wc(unsigned long addr, int numpages) {int ret; -if (!pat_enabled) -return set_memory_uc (addr, numpages); +if (!pat_enabled) { + ret =set_memory_uc (addr, numpages); +if (!ret) +mtrr_add (__pa (addr), numpages *PAGE_SIZE , +MTRR_TYPE_WRCOMB , false); +return ret; + } ret =reserve_memtype (__pa (addr),__pa (addr) + numpages *PAGE_SIZE , _PAGE_CACHE_WC,NULL );
arch/x86/pci/i386.c +6 -0
@@ -33,6 +33,7 @@#include <linux/bootmem.h> #include <asm/pat.h> +#include <asm/mtrr.h> #include <asm/e820.h> #include <asm/pci_x86.h> #include <asm/io_apic.h>
@@ -301,5 +302,10 @@ int pci_mmap_page_range(struct pci_dev *dev, struct vm_area_struct *vma, vma->vm_ops = &pci_mmap_ops; +if (!pat_enabled && write_combine) +mtrr_add_page (vma->vm_pgoff, + (vma->vm_end - vma->vm_start) >>PAGE_SHIFT , +MTRR_TYPE_WRCOMB , false); +return 0 ; }
Signature · 1 line
1.6.4.4
Quoted 3 lines · Thomas Schlichter
Hi,
I've found a problem with X.org not setting up MTRR for the framebuffer memory. After I investigated I think this is not a X.org problem, but a kernel issue.
is there any CPU left that does not support PAT ?
eg in other words, should we try instead to get PAT working for you?
Am Samstag 10 Oktober 2009 06:24:41 schrieb Arjan van de Ven:
Quoted 10 lines
Quoted 5 lines · Thomas Schlichter
Hi,
I've found a problem with X.org not setting up MTRR for the framebuffer memory. After I investigated I think this is not a X.org problem, but a kernel issue.
is there any CPU left that does not support PAT ?
eg in other words, should we try instead to get PAT working for you?
Hmm, my CPU (VIA Nano) seems to support PAT, but Ubuntu disables this feature in its kernel config. I'm using 9.04, but even 9.10 will have PAT disabled:
http://kernel.ubuntu.com/git?p=ubuntu/ubuntu-
karmic.git;a=blob;f=debian.master/config/config.common.ubuntu;h=a43cbabd02bfdf5412de6fea232db86aa5d3e742;hb=HEAD
The patch I sent does nothing if PAT is enabled, so it should be quite safe for this case. And for the case when PAT is not enabled, it simply tries to set up MTRR entries, if this fails it will still behave like it does today...
So, convincing Ubuntu to enable PAT would be a workaround, but I think the problem would still exist and should be fixed.
Kind regards,
Thomas
Quoted 3 lines · Thomas Schlichter
Hmm, my CPU (VIA Nano) seems to support PAT, but Ubuntu disables this feature in its kernel config. I'm using 9.04, but even 9.10 will have PAT disabled:
that sounds like a bad idea. But to each distro their own I suppose.
Quoted 6 lines
http://kernel.ubuntu.com/git?p=ubuntu/ubuntu-
karmic.git;a=blob;f=debian.master/config/config.common.ubuntu;h=a43cbabd02bfdf5412de6fea232db86aa5d3e742;hb=HEADSo, convincing Ubuntu to enable PAT would be a workaround, but I think the problem would still exist and should be fixed.
I think it's time to get rid of CONFIG_PAT at all.. and just have it be there. Making 10+ year old CPU features that are essential functionality a config option... ... not a good idea to be honest.
Quoted 6 lines
Quoted 4 lines
Hmm, my CPU (VIA Nano) seems to support PAT, but Ubuntu disables this feature in its kernel config. I'm using 9.04, but even 9.10 will have PAT disabled:
that sounds like a bad idea. But to each distro their own I suppose.
Seems that Ubuntu has caught this issue, and it will probably be
resolved: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/446480
- R.
Quoted 13 lines · Arjan van de Ven
Quoted 3 lines · Thomas Schlichter
Hmm, my CPU (VIA Nano) seems to support PAT, but Ubuntu disables this feature in its kernel config. I'm using 9.04, but even 9.10 will have PAT disabled:
that sounds like a bad idea. But to each distro their own I suppose.
Quoted 3 lines
http://kernel.ubuntu.com/git?p=ubuntu/ubuntu-
karmic.git;a=blob;f=debian.master/config/config.common.ubuntu;h=a43cbabd02bfdf5412de6fea232db86aa5d3e742;hb=HEAD
Quoted 6 lines
Quoted 2 lines
So, convincing Ubuntu to enable PAT would be a workaround, but I think the problem would still exist and should be fixed.
I think it's time to get rid of
CONFIG_PATat all.. and just have it be there. Making 10+ year old CPU features that are essential functionality a config option... ... not a good idea to be honest.
Mind sending a patch that makes the configurability of PAT dependent on EMBEDDED or so? There's still the nopat boot option so it can still be configured by a distro if it wishes to.
Ingo
Quoted 7 lines · Arjan van de Ven
Quoted 2 lines · Thomas Schlichter
So, convincing Ubuntu to enable PAT would be a workaround, but I think the problem would still exist and should be fixed.
I think it's time to get rid of
CONFIG_PATat all.. and just have it be
there. Making 10+ year old CPU features that are essential functionality a config option... ... not a good idea to be honest.
I do completely agree with you in this point.
But should the current behavior of not using MTRR for write combining when PAT is disabled (for whatever reason) stay? I think it is more consistent to do whatever is necessary to enable write combining when we are asked to.
Kind regards,
Thomas
Quoted 5 lines · Arjan van de Ven
Quoted 3 lines
I've found a problem with X.org not setting up MTRR for the framebuffer memory. After I investigated I think this is not a X.org problem, but a kernel issue.
is there any CPU left that does not support PAT ?
A few million of them. Like every Centrino laptop out there, unless the kernel blacklist for PAT on Intel CPUs is wrong.
Quoted 1 line
eg in other words, should we try instead to get PAT working for you?
Is it even possible? Can you get Intel to fix my Pentium M? I'd be quite happy to test any microcode updates you send my way :)
As things stand right now, you can assume PAT will be there in 2015 or thereabouts. Machine models that can run Linux have a service life of around 15 years.
Signature · 4 lines
"One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie." -- The Silicon Valley Tarot Henrique Holschuh
Quoted 8 lines · Henrique de Moraes Holschuh
Quoted 4 lines · Arjan van de Ven
Quoted 3 lines
I've found a problem with X.org not setting up MTRR for the framebuffer memory. After I investigated I think this is not a X.org problem, but a kernel issue.
is there any CPU left that does not support PAT ?
A few million of them. Like every Centrino laptop out there, unless the kernel blacklist for PAT on Intel CPUs is wrong.
afaik it is extremely conservative right now. FAR too much so.
Quoted 11 lines · Arjan van de Ven
Quoted 8 lines · Henrique de Moraes Holschuh
Quoted 4 lines · Arjan van de Ven
Quoted 3 lines
I've found a problem with X.org not setting up MTRR for the framebuffer memory. After I investigated I think this is not a X.org problem, but a kernel issue.
is there any CPU left that does not support PAT ?
A few million of them. Like every Centrino laptop out there, unless the kernel blacklist for PAT on Intel CPUs is wrong.
afaik it is extremely conservative right now. FAR too much so.
That *still* means we need the patch in this thread.
Signature · 4 lines
"One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie." -- The Silicon Valley Tarot Henrique Holschuh
Quoted 6 lines · Henrique de Moraes Holschuh
Quoted 5 lines · Arjan van de Ven
Quoted 4 lines · Henrique de Moraes Holschuh
Quoted 3 lines · Arjan van de Ven
Quoted 3 lines
I've found a problem with X.org not setting up MTRR for the framebuffer memory. After I investigated I think this is not a X.org problem, but a kernel issue.
Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
is there any CPU left that does not support PAT ?
A few million of them. Like every Centrino laptop out there, unless the kernel blacklist for PAT on Intel CPUs is wrong.
afaik it is extremely conservative right now. FAR too much so.
That *still* means we need the patch in this thread.
I don't know that it is too conservative - the erratum which justifies the disablement (Core Solo/Duo erratum AE7, Pentium M erratum Y31, likely other IDs for it too) states that "A page whose PAT memory type is USWC while the relevant MTRR memory type is UC, the consolidated memory type may be treated as UC (rather than WC as specified in IA-32 Intel® Architecture Software Developer's Manual). When this erratum occurs, the memory page may be as UC (rather than WC). This may have a negative performance impact." This is exactly the case being discussed here with the framebuffer which we're trying to set as WC with existing MTRRs set (or defaulting to) UC. This not only affects Pentium M but also the original Core Solo and Core Duo CPUs which surely is millions of existing laptops. It seems pretty clear-cut and there's no workaround identified, so I think we do indeed need a patch like this one.