use MTRR for write combining if PAT is not available
A message that is not in the archive (sent off-list or not yet received): [email protected]
Hi!
One problem with this patch is that it conflicts with the way graphics drivers traditionally handles the situation, namely
1) Set up mtrr
2) Map. If fallback to uncached minus we will still have write-combined access.
I think mtrr-add used in this fashion will typically fail due to the alignment constraints. In particular, for set_memory_wc() the typical usage pattern is a large number of pages in a fragmented physical address space.
So if we were to fix the problem with libpciaccess in the kernel, I think the best option would be to fail the user-space mapping when we can't make it write-combined.
Thanks,
Thomas
Quoted 36 lines · Thomas Schlichter
Hi,
when I first sent this E-Mail on Saturday, I unfortunately forgot to CC many people. Now I used get_maintainer.pl to get the list of people that may want to contribute to this topic.
Because of this topic, there is already a patch from Arjan in the -tip tree to make PAT and MTRR options only configurable if EMBEDDED and enabled by default. I think this is a step in the right direction, but at least Henrique, Robert and I seem to think something like the attached patch is still required. What do you think?
Kind regards,
Thomas-----------------------------------------------------------------------------
Hi,
I've found a problem with X.org not setting up MTRR for the framebuffer memory. After I investigated I think this is not a X.org problem, but a kernel issue.
X.org uses libpciaccess to map the framebuffer memory. This library opens
/sys/bus/pci/devices/*/resource0_wcand mmaps the memory. Unfortunately, the kernel only enables write combining if PAT is enabled, if it is not, the memory is mmapped uncached. But Xorg (respectively libpciaccess) thinks it was successfully mapped with write combining enabled and thus does not additionally set up MTRR entries.The corresponding libpciaccess code can be found here:
http://cgit.freedesktop.org/xorg/lib/libpciaccess/tree/src/linux_sysfs.c#n501If the kernel behavior is intentional and X.org should always set up MTRR entries, why should it use
/sys/.../resource0_wcat all? I think there are 2 possibilities to make the kernel behavior consistent:1. The
mmap_wcshould fail if PAT is not enabled. (libpciaccess will then map the framebuffer uncached and set up MTRR entries)
2. Use MTRR to enable write combining if PAT is not available.In an earlier thread about
ioremap_wc, Arjan van de Ven wrote that option 2 is preferred over option 1:http://lkml.indiana.edu/hypermail/linux/kernel/0805.3/2925.htmlSo, I've created the attached patch implementing option 2. For me this solves the problem with the slow Video playback due to not correctly set up MTRR entries.
Kind regards,
Thomas Schlichter
Quoted 6 lines · Thomas Hellstrom
Hi!
One problem with this patch is that it conflicts with the way graphics drivers traditionally handles
the situation, namely
Quoted 8 lines
1) Set up mtrr
2) Map. If fallback to uncached minus we will still have write-combined access.I think mtrr-add used in this fashion will typically fail due to the alignment constraints. In particular,
forset_memory_wc()the typical usage pattern is a large number of pages in a fragmented physical address space.
Yes, maybe this patch tries to change current behavior too less. Indeed, if setting up MTRR entries it simply behaves as today, and userspace does not see that write combining is not correctly enabled.
Quoted 3 lines
So if we were to fix the problem with libpciaccess in the kernel, I think the best option would be to fail the user-space mapping when we can't make it write-combined.
One idea to do this would be the attached patch. It simply returns an error if PAT is not available. It does not even try to use MTRR on its own. But maybe even better would be to combine both patches to something like this:
1. try to use PAT
2. if this fails try to set up MTRR
3. if this also fails, return error
Kind regards,
Thomas
From afb48e1a1ef035c4580a5ce59a956b54a56a5c18 Mon Sep 17 00:00:00 2001
From: Thomas Schlichter <[email protected]>
Date: Thu, 8 Oct 2009 00:42:47 +0200
Subject: [PATCH] Do not mmap/ioremap uncached when WC is requested
X.org uses libpciaccess which tries to mmap with write combining enabled via /sys/bus/pci/devices/*/resource0_wc. Currently, when PAT is not enabled, we fall back to uncached mmap. Then libpciaccess thinks it succeeded mapping with write combining anabled and does not set up suited MTRR entries. ;-(
So instead of silently falling back to uncached mapping, we better fail. In this case libpciaccess mmaps via /sys/bus/pci/devices/*/resource0 and correctly sets up MTRR entries.
Aditionally modify ioremap_wc and set_memory_wc to match this behavior.
--- arch/x86/mm/ioremap.c | 10 +++++----- arch/x86/mm/pageattr.c | 2 +- arch/x86/pci/i386.c | 6 ++++++ 3 files changed, 12 insertions(+), 6 deletions(-)
arch/x86/mm/ioremap.c +5 -5
@@ -268,11 +268,11 @@ EXPORT_SYMBOL(ioremap_nocache);*/ void __iomem *ioremap_wc (resource_size_t phys_addr,unsigned long size) { -if (pat_enabled) -return __ioremap_caller (phys_addr, size, _PAGE_CACHE_WC, -__builtin_return_address (0 )); -else -return ioremap_nocache (phys_addr, size); +if (!pat_enabled) +return NULL ; + +return __ioremap_caller (phys_addr, size, _PAGE_CACHE_WC, +__builtin_return_address (0 )); }EXPORT_SYMBOL (ioremap_wc);
arch/x86/mm/pageattr.c +1 -1
@@ -1011,7 +1011,7 @@ int set_memory_wc(unsigned long addr, int numpages)int ret;if (!pat_enabled) -return set_memory_uc (addr, numpages); +return -EINVAL ; ret =reserve_memtype (__pa (addr),__pa (addr) + numpages *PAGE_SIZE , _PAGE_CACHE_WC,NULL );
arch/x86/pci/i386.c +6 -0
@@ -281,6 +281,12 @@ int pci_mmap_page_range(struct pci_dev *dev, struct vm_area_struct *vma,if (mmap_state == pci_mmap_io)return -EINVAL ; +/* We cannot mmap write combining (WC) without PAT enabled. +* So better fail and let the user map without WC and use MTRR. +*/ +if (!pat_enabled && write_combine) +return -EINVAL ; + prot =pgprot_val (vma->vm_page_prot);if (pat_enabled && write_combine) prot |= _PAGE_CACHE_WC;
Signature · 1 line
1.6.4.4
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 8 lines · Eric Anholt
Quoted 3 lines · Thomas Schlichter
Yes, maybe this patch tries to change current behavior too less. Indeed, if setting up MTRR entries it simply behaves as today, and userspace does not see that write combining is not correctly enabled.
I'm uncomfortable with this patch because it doesn't appear to cover any callers of these functions inside of the kernel. Have you audited them to make sure they can handle NULL being returned?
No, I haven't. And to be honest, I think the earlier patch that adds MTRR entries should be more safe, as it modifies behavior only slightly.
Quoted 2 lines
Seems like we should install an MTRR instead. Requiring userland to set up the MTRR on the kernel's behalf is backwards.
Exactly.
Quoted 2 lines
With modern drivers we're installing any required MTRRs at module load in the kernel, and that's where things should be moving.
I think in general this is not possible, because not for all graphics chips there are kernel drivers (required). E.g. for my VIA VX800 based notebook, there is no kernel module that should be responsible to set-up the MTRR entries. Here it's up to the (userspace) X.org driver. It opens the /sys/bus/pci/devices/.../resource0_wc device and mmaps the framebuffer memory. With PAT this will set up a write combining memory area, and with my first patch this should also happen without PAT with MTRR.
Quoted 3 lines
As long as
this doesn't interfere with them, I'm OK. And if some kernel driver is failing to install its MTRR, well, let's fix it.
Well, I think the mtrr_add inside mmap should not do any harm...
Kind regards,
Thomas