Skip to contentSkip to conversation

PAGE_CACHE_WC strikes again

PATCH Eric Anholt 5 participants 11 messages Apr 1, 2009

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

I just tracked down what was cutting performance 10x on one of my systems on a microbenchmark I'd just written:

drivers/gpu/drm/drm_gem.c +0 -0
@@ -540,7 +540,7 @@ int drm_gem_mmap(struct file *filp, struct

vm_area_struct *vma)
/* FIXME: use pgprot_writecombine when available */ prot = pgprot_val(vma->vm_page_prot);
#ifdef CONFIG_X86
- prot |= _PAGE_CACHE_WC;
+ /*prot |= _PAGE_CACHE_WC;*/
#endif
vma->vm_page_prot = __pgprot(prot);

Turns out that setting PAGE_CACHE_WC disables the WC effect of the MTRR on my non-PAT (disabled due to CPU errata) 945GM system, and this workaround took GTT-mapped writes from 120MB/s to 1180MB/s.

What's the right way to be setting our PTEs? This is similar to the workaround in ef5fa0ab24b87646c7bc98645acbb4b51fc2acd4, but to do it in the driver as well means exporting pat_enabled, and it really seems like PAT presence shouldn't be something the driver has to worry about -- I've got a WC MTRR and I'm asking for a WC mapping and I'm getting uncached. If I failed to init the MTRR, the state of the aperture I'm mapping would be uncached.

Test code is at
git://anongit.freedesktop.org/git/xorg/app/intel-gpu-tools
under benchmarks/intel_upload_blit_large_gtt but requires libdrm master until we get a release out in the next week or so. Also includes a few standalone regression tests, which may be useful for people making changes that may affect i915.

Signature · 2 lines
Eric Anholt
[email protected]                         [email protected]
#1 lore raw
Quoted 15 lines · On Tue, 31 Mar 2009 17:10:47 -0700 Eric Anholt <[email protected]> wrote:

I just tracked down what was cutting performance 10x on one of my systems on a microbenchmark I'd just written:

drivers/gpu/drm/drm_gem.c +1 -1
@@ -540,7 +540,7 @@ int drm_gem_mmap(struct file *filp, struct
 vm_area_struct *vma)
 	/* FIXME: use pgprot_writecombine when available */
 	prot = pgprot_val(vma->vm_page_prot);
 #ifdef CONFIG_X86
-	prot |= _PAGE_CACHE_WC;
+	/*prot |= _PAGE_CACHE_WC;*/
 #endif
 	vma->vm_page_prot = __pgprot(prot);
Quoted 3 lines

Turns out that setting PAGE_CACHE_WC disables the WC effect of the MTRR on my non-PAT (disabled due to CPU errata) 945GM system, and this workaround took GTT-mapped writes from 120MB/s to 1180MB/s.

What the... There's a pgprot_writecombine now, but it basically does the same thing. Why is WC so broken? Venki is the fix for this covered in your last patchset?

Signature · 1 line
Jesse Barnes, Intel Open Source Technology Center
#2 lore raw
Quoted 27 lines

-----Original Message-----
From: Jesse Barnes [mailto:[email protected]]
Sent: Tuesday, March 31, 2009 5:15 PM
To: Eric Anholt
Cc: lkml; Pallipadi, Venkatesh
Subject: Re: PAGE_CACHE_WC strikes again

Quoted 15 lines · On Tue, 31 Mar 2009 17:10:47 -0700 Eric Anholt <[email protected]> wrote:

I just tracked down what was cutting performance 10x on one of my systems on a microbenchmark I'd just written:

drivers/gpu/drm/drm_gem.c +1 -1
@@ -540,7 +540,7 @@ int drm_gem_mmap(struct file *filp, struct
 vm_area_struct *vma)
 	/* FIXME: use pgprot_writecombine when available */
 	prot = pgprot_val(vma->vm_page_prot);
 #ifdef CONFIG_X86
-	prot |= _PAGE_CACHE_WC;
+	/*prot |= _PAGE_CACHE_WC;*/
 #endif
 	vma->vm_page_prot = __pgprot(prot);
Quoted 9 lines
Quoted 2 lines

Turns out that setting PAGE_CACHE_WC disables the WC effect of the MTRR on my non-PAT (disabled due to CPU errata) 945GM

system, and this

Quoted 1 line

workaround took GTT-mapped writes from 120MB/s to 1180MB/s.

What the... There's a pgprot_writecombine now, but it basically does the same thing. Why is WC so broken? Venki is the fix for this covered in your last patchset?

The key point here is

Quoted 2 lines

setting PAGE_CACHE_WC disables the WC effect of the MTRR on my non-PAT (disabled due to CPU errata)

When PAT is disabled, the default setting in PAT MSR is 00 - WB
01 - WT
10 - UC_MINUS
11 - UC

There is no way to set WC with PAT. By hardcoding _PAGE_CACHE_WC (which is 01) the driver is basically selecting write-through!

And when MTRR says WC and PAT says WT, effective type is UC.

Basically, no one should be hard-coding the memory type. Please use pgprot_writecombine() which does the right thing by using WC (when PAT is enabled) or UC_MINUS (when PAT is disabled).

Thanks,
Venki

#3 lore raw
Quoted 48 lines · Venkatesh
Quoted 27 lines

-----Original Message-----
From: Jesse Barnes [mailto:[email protected]]
Sent: Tuesday, March 31, 2009 5:15 PM
To: Eric Anholt
Cc: lkml; Pallipadi, Venkatesh
Subject: Re: PAGE_CACHE_WC strikes again

Quoted 15 lines · On Tue, 31 Mar 2009 17:10:47 -0700 Eric Anholt <[email protected]> wrote:

I just tracked down what was cutting performance 10x on one of my systems on a microbenchmark I'd just written:

drivers/gpu/drm/drm_gem.c +1 -1
@@ -540,7 +540,7 @@ int drm_gem_mmap(struct file *filp, struct
 vm_area_struct *vma)
 	/* FIXME: use pgprot_writecombine when available */
 	prot = pgprot_val(vma->vm_page_prot);
 #ifdef CONFIG_X86
-	prot |= _PAGE_CACHE_WC;
+	/*prot |= _PAGE_CACHE_WC;*/
 #endif
 	vma->vm_page_prot = __pgprot(prot);
Quoted 9 lines
Quoted 2 lines

Turns out that setting PAGE_CACHE_WC disables the WC effect of the MTRR on my non-PAT (disabled due to CPU errata) 945GM

system, and this

Quoted 1 line

workaround took GTT-mapped writes from 120MB/s to 1180MB/s.

What the... There's a pgprot_writecombine now, but it basically does the same thing. Why is WC so broken? Venki is the fix for this covered in your last patchset?

The key point here is

Quoted 2 lines

setting PAGE_CACHE_WC disables the WC effect of the MTRR on my non-PAT (disabled due to CPU errata)

When PAT is disabled, the default setting in PAT MSR is 00 - WB
01 - WT
10 - UC_MINUS
11 - UC

Quoted 8 lines

There is no way to set WC with PAT. By hardcoding _PAGE_CACHE_WC (which is 01) the driver is basically selecting write-through!

And when MTRR says WC and PAT says WT, effective type is UC.

Basically, no one should be hard-coding the memory type. Please use pgprot_writecombine() which does the right thing by using WC (when PAT is enabled) or UC_MINUS (when PAT is disabled).

Ah ok, that makes sense. I'm glad we have pgprot_writecombine now (sorry Eric I should have sent that patch along with my patch containing the vm_insert_pfn -EINVAL handling).

Signature · 1 line
Jesse Barnes, Intel Open Source Technology Center
#4 lore raw
In reply to Pallipadi, Venkatesh
Quoted 11 lines · Venkatesh

The key point here is

Quoted 2 lines

setting PAGE_CACHE_WC disables the WC effect of the MTRR on my non-PAT (disabled due to CPU errata)

When PAT is disabled, the default setting in PAT MSR is 00 - WB
01 - WT
10 - UC_MINUS
11 - UC

Quoted 8 lines

There is no way to set WC with PAT. By hardcoding _PAGE_CACHE_WC (which is 01) the driver is basically selecting write-through!

And when MTRR says WC and PAT says WT, effective type is UC.

Basically, no one should be hard-coding the memory type. Please use pgprot_writecombine() which does the right thing by using WC (when PAT is enabled) or UC_MINUS (when PAT is disabled).

And the driver should use right API to track the underlying page frame thats getting mapped by this vma, with the corresponding attribute. API's like remap_pfn_range(), vm_insert_pfn() will setup the PTE's aswell as track the pfn's attributes.

API's like set_memory_uc/wc() will explicitly setup the page attributes.

Jesse, As far as I see, the drm GEM fault handler routines don't seem to do any of this. Am I missing something? We need to fix this so that we can avoid potential aliasing issues.

thanks,
suresh

#5 lore raw
Quoted 26 lines · On Tue, 31 Mar 2009 17:03:10 -0800 Suresh Siddha <[email protected]>…
Quoted 11 lines · Venkatesh

The key point here is

Quoted 2 lines

setting PAGE_CACHE_WC disables the WC effect of the MTRR on my non-PAT (disabled due to CPU errata)

When PAT is disabled, the default setting in PAT MSR is 00 - WB
01 - WT
10 - UC_MINUS
11 - UC

Quoted 8 lines

There is no way to set WC with PAT. By hardcoding _PAGE_CACHE_WC (which is 01) the driver is basically selecting write-through!

And when MTRR says WC and PAT says WT, effective type is UC.

Basically, no one should be hard-coding the memory type. Please use pgprot_writecombine() which does the right thing by using WC (when PAT is enabled) or UC_MINUS (when PAT is disabled).

And the driver should use right API to track the underlying page frame thats getting mapped by this vma, with the corresponding attribute. API's like remap_pfn_range(), vm_insert_pfn() will setup the PTE's aswell as track the pfn's attributes.

Quoted 6 lines

API's like set_memory_uc/wc() will explicitly setup the page attributes.

Jesse, As far as I see, the drm GEM fault handler routines don't seem to do any of this. Am I missing something? We need to fix this so that we can avoid potential aliasing issues.

Right, the drm driver code went in before we had pgprot_writecombine. Now that it's available we should definitely use it. I'm not sure about the set_memory_* routines though; we create io mappings in i915_dma.c at init time, and I thought we took care of things in i915_gem.c but we may need updates there.

Signature · 1 line
Jesse Barnes, Intel Open Source Technology Center
#6 lore raw
Quoted 5 lines · Jesse Barnes

Right, the drm driver code went in before we had pgprot_writecombine. Now that it's available we should definitely use it. I'm not sure about the set_memory_* routines though; we create io mappings in i915_dma.c at init time, and I thought we took care of things in i915_gem.c but we may need updates there.

Jesse, yes i915_gem.c seems to be doing the right thing.

What about various fault handlers in drm_vm.c like drm_do_vm_shm_fault() etc. None of these fault handlers require special attributes like wc/uc etc?

thanks,
suresh

#7 lore raw
Quoted 7 lines · On Tue, 31 Mar 2009 17:30:24 -0800 Suresh Siddha <[email protected]>…
Quoted 5 lines · Jesse Barnes

Right, the drm driver code went in before we had pgprot_writecombine. Now that it's available we should definitely use it. I'm not sure about the set_memory_* routines though; we create io mappings in i915_dma.c at init time, and I thought we took care of things in i915_gem.c but we may need updates there.

Quoted 5 lines

Jesse, yes i915_gem.c seems to be doing the right thing.

What about various fault handlers in drm_vm.c like drm_do_vm_shm_fault() etc. None of these fault handlers require special attributes like wc/uc etc?

Those are legacy. I don't think they need anything changed... Dave could correct me though.

Signature · 1 line
Jesse Barnes, Intel Open Source Technology Center
maintainer
#8 lore raw
Quoted 10 lines · Jesse Barnes
Quoted 7 lines · On Tue, 31 Mar 2009 17:30:24 -0800 Suresh Siddha <[email protected]>…
Quoted 5 lines · Jesse Barnes

Right, the drm driver code went in before we had pgprot_writecombine. Now that it's available we should definitely use it.  I'm not sure about the set_memory_* routines though; we create io mappings in i915_dma.c at init time, and I thought we took care of things in i915_gem.c but we may need updates there.

Quoted 8 lines
Quoted 5 lines

Jesse, yes i915_gem.c seems to be doing the right thing.

What about various fault handlers in drm_vm.c like drm_do_vm_shm_fault() etc. None of these fault handlers require special attributes like wc/uc etc?

Those are legacy.  I don't think they need anything changed...  Dave could correct me though.

Yeah just don't look at the top of drm_vm.c:drm_io_prot.

the attribute handling in there is one of the bits of the drm I hide away from.

and I think we can just move away from that with memory managed drivers doing it properly.

Dave.

Quoted 4 lines

--
Jesse Barnes, Intel Open Source Technology Center

#9 lore raw
In reply to Jesse Barnes
Quoted 46 lines

-----Original Message-----
From: Jesse Barnes [mailto:[email protected]]
Sent: Tuesday, March 31, 2009 6:13 PM
To: Siddha, Suresh B
Cc: Pallipadi, Venkatesh; Eric Anholt; lkml
Subject: Re: PAGE_CACHE_WC strikes again

Quoted 15 lines · On Tue, 31 Mar 2009 17:03:10 -0800 Suresh Siddha <[email protected]>…
Quoted 14 lines · Venkatesh

The key point here is

Quoted 2 lines

setting PAGE_CACHE_WC disables the WC effect of the MTRR on my non-PAT (disabled due to CPU errata)

When PAT is disabled, the default setting in PAT MSR is 00 - WB
01 - WT
10 - UC_MINUS
11 - UC

There is no way to set WC with PAT. By hardcoding _PAGE_CACHE_WC (which is 01) the driver is basically selecting write-through!

Quoted 7 lines
Quoted 5 lines

And when MTRR says WC and PAT says WT, effective type is UC.

Basically, no one should be hard-coding the memory type. Please use pgprot_writecombine() which does the right thing by using WC (when PAT is enabled) or UC_MINUS (when PAT is disabled).

And the driver should use right API to track the underlying

page frame

Quoted 10 lines

thats getting mapped by this vma, with the corresponding attribute. API's like remap_pfn_range(), vm_insert_pfn() will setup the PTE's aswell as track the pfn's attributes.

API's like set_memory_uc/wc() will explicitly setup the page attributes.

Jesse, As far as I see, the drm GEM fault handler routines don't seem to do any of this. Am I missing something? We need to fix this so that we can avoid potential aliasing issues.

Quoted 6 lines

Right, the drm driver code went in before we had pgprot_writecombine. Now that it's available we should definitely use it. I'm not sure about the set_memory_* routines though; we create io mappings in i915_dma.c at init time, and I thought we took care of things in i915_gem.c but we may need updates there.

There is one problem of too many single page mappings, most likely coming from gem vm_insert_pfn. As in this mail
http://marc.info/?l=linux-kernel&m=123845244713183&w=2

The actual 'freeing invalid memtype' is a bug that I still trying to zero in on. Regardless of that, having so many small WC mappings in IO region will result it pretty slow vm_insert_pfn. I think we should switch to io_mapping_create_wc kind of API where we do bulk reserve and have a vm_insert_pfn_quick().

Thanks,
Venki

#10 lore raw
In reply to Suresh Siddha
Quoted 26 lines · On Tue, 31 Mar 2009 17:03:10 -0800 Suresh Siddha <[email protected]>…
Quoted 11 lines · Venkatesh

The key point here is

Quoted 2 lines

setting PAGE_CACHE_WC disables the WC effect of the MTRR on my non-PAT (disabled due to CPU errata)

When PAT is disabled, the default setting in PAT MSR is 00 - WB
01 - WT
10 - UC_MINUS
11 - UC

Quoted 8 lines

There is no way to set WC with PAT. By hardcoding _PAGE_CACHE_WC (which is 01) the driver is basically selecting write-through!

And when MTRR says WC and PAT says WT, effective type is UC.

Basically, no one should be hard-coding the memory type. Please use pgprot_writecombine() which does the right thing by using WC (when PAT is enabled) or UC_MINUS (when PAT is disabled).

And the driver should use right API to track the underlying page frame thats getting mapped by this vma, with the corresponding attribute. API's like remap_pfn_range(), vm_insert_pfn() will setup the PTE's aswell as track the pfn's attributes.

Quoted 6 lines

API's like set_memory_uc/wc() will explicitly setup the page attributes.

Jesse, As far as I see, the drm GEM fault handler routines don't seem to do any of this. Am I missing something? We need to fix this so that we can avoid potential aliasing issues.

Ok so we use set_memory_wc/wb in the physical mapping stuff, which I think is fine. We also use io_mapping_create_wc on the whole aperture at load time. That will take care of any pages we map there right? So along with this fix I think we'll be ok?

Signature · 1 line
Jesse Barnes, Intel Open Source Technology Center
drivers/gpu/drm/drm_gem.c +1 -5
index c1173d8..fa828a8 100644
@@ -538,11 +538,7 @@ int drm_gem_mmap(struct file *filp, struct vm_area_struct *vma)
 	vma->vm_ops = obj->dev->driver->gem_vm_ops;
 	vma->vm_private_data = map->handle;
 	/* FIXME: use pgprot_writecombine when available */
-	prot = pgprot_val(vma->vm_page_prot);
-#ifdef CONFIG_X86
-	prot |= _PAGE_CACHE_WC;
-#endif
-	vma->vm_page_prot = __pgprot(prot);
+	vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot);
 
 	/* Take a ref for this mapping of the object, so that the fault
 	 * handler can dereference the mmap offset's pointer to the object.