Skip to contentSkip to conversation

2.6.29 git master and PAT problems

PATCH Arkadiusz Miskiewicz 5 participants 29 messages Mar 30, 2009 – Apr 16, 2009 1 message not in the archive Page 2 of 2

Subsystems (6)
Memory management x86 PCI MM core +2 more subsystems
lore
Share & tools
lore
#9 lore raw
In reply to Yinghai Lu
Quoted 104 lines · Yinghai Lu

On Tue, Mar 31, 2009 at 12:44 AM, Arkadiusz Miskiewicz

Quoted 92 lines · <[email protected]> wrote:
Quoted 25 lines · Venkatesh
Quoted 12 lines · Arkadiusz Miskiewicz
Quoted 5 lines · Arkadiusz Miskiewicz

On Monday 30 of March 2009, Pallipadi, Venkatesh wrote:

More info follows. Now I've switched to e1c502482853f84606928f5a2f2eb6da1993cda1 which contains latest drm fixes and now I get much lower numbers of PAT errors but still.

Also when I switch t400 into discrete mode (radeon hd 3400 instead of integrated intel GM45) I get such errors (probably unrelated to these seen when using intel):

[  419.187657] X:10550 conflicting memory types cfff0000-d0000000
Quoted 11 lines

uncached<->uncached-minus [ 419.187670] reserve_memtype failed 0xcfff0000-0xd0000000, track uncached, req write-back [ 419.553914] X:10550 conflicting memory types cfff0000-d0000000 uncached<->uncached-minus [ 419.553923] reserve_memtype failed 0xcfff0000-0xd0000000, track uncached, req write-back [ 419.813592] X:10550 conflicting memory types cfff0000-d0000000 uncached<->uncached-minus [ 419.813601] reserve_memtype failed 0xcfff0000-0xd0000000, track uncached, req write-back [ 420.100102] X:10550 conflicting memory types cfff0000-d0000000 uncached<->uncached-minus [ 420.100111] reserve_memtype failed 0xcfff0000-0xd0000000, track uncached, req write-back

Quoted 3 lines

Yes. This is a different problem than the freeing invalid type one. Are these errors also with latest git kernel? Can you try the patch below (which is a part of a bigger cleanup patch I have lined up).

It's a latest git kernel as of today morning (latest commit is 15f7176eb1cccec0a332541285ee752b935c1c85)
+ your patch. Problem persists:

[   74.696353] [drm] Setting GART location based on new memory map
[   74.711520] [drm] Loading RV620 CP Microcode
[   74.711792] [drm] Loading RV620 PFP Microcode
[   74.726719] [drm] Resetting GPU
[   74.726776] [drm] writeback test succeeded in 1 usecs
[   75.256034] X:5366 conflicting memory types d0000000-e0000000

uncached-minus<->write-combining [ 75.256043] reserve_memtype failed
0xd0000000-0xe0000000, track uncached-minus, req write-back [ 75.849951] X:5366 conflicting memory types d0000000-e0000000
uncached-minus<->write-combining [ 75.849960] reserve_memtype failed
0xd0000000-0xe0000000, track uncached-minus, req write-back [ 76.054374] X:5366 conflicting memory types d0000000-e0000000
uncached-minus<->write-combining [ 76.054377] reserve_memtype failed
0xd0000000-0xe0000000, track uncached-minus, req write-back [ 76.074481] X:5378 freeing invalid memtype d0000000-e0000000

[   76.176881] X:5366 conflicting memory types d0000000-e0000000

uncached-minus<->write-combining [ 76.176885] reserve_memtype failed
0xd0000000-0xe0000000, track uncached-minus, req write-back [ 76.207734] X:5380 freeing invalid memtype d0000000-e0000000

cat /debug/x86/pat_memtype_list
PAT memtype list:
uncached-minus @ 0xbfad1000-0xbfad2000
uncached-minus @ 0xbfad2000-0xbfad3000
uncached-minus @ 0xbfad3000-0xbfad4000
uncached-minus @ 0xbfb06000-0xbfb07000
uncached-minus @ 0xbfb07000-0xbfb08000
uncached-minus @ 0xbfd6a000-0xbfd6b000
uncached-minus @ 0xbfd6a000-0xbfd6b000
uncached-minus @ 0xbfd6a000-0xbfd7a000
uncached-minus @ 0xbfd79000-0xbfd7a000
uncached-minus @ 0xbfd79000-0xbfd7a000
uncached-minus @ 0xbfd79000-0xbfd7a000
uncached-minus @ 0xbfd79000-0xbfd7a000
uncached-minus @ 0xbfd79000-0xbfd7a000
uncached-minus @ 0xbfd79000-0xbfd7a000
uncached-minus @ 0xbfd79000-0xbfd7a000
uncached-minus @ 0xbfd7b000-0xbfd7c000
uncached-minus @ 0xbfd8d000-0xbfd8e000
uncached-minus @ 0xbfd8e000-0xbfd8f000
uncached-minus @ 0xbfd8e000-0xbfd8f000
uncached-minus @ 0xcfff0000-0xd0000000
uncached-minus @ 0xcfff0000-0xd0000000
uncached-minus @ 0xcfff0000-0xd0000000
uncached-minus @ 0xcfff0000-0xd0000000
uncached-minus @ 0xcfff0000-0xd0000000
write-combining @ 0xdfff0000-0xe0000000
uncached-minus @ 0xe0000000-0xe4000000
uncached-minus @ 0xf4200000-0xf4202000
uncached-minus @ 0xf4300000-0xf4301000
uncached-minus @ 0xf4301000-0xf4302000
uncached-minus @ 0xf4301000-0xf4302000
uncached-minus @ 0xfc200000-0xfc220000
uncached-minus @ 0xfc220000-0xfc224000
uncached-minus @ 0xfc225000-0xfc226000
uncached-minus @ 0xfc226000-0xfc227000
uncached-minus @ 0xfc226000-0xfc227000
uncached-minus @ 0xfc227000-0xfc228000

Quoted 8 lines

uncached-minus @ 0xfed00000-0xfed01000
uncached-minus @ 0xfed1f000-0xfed20000
cat /proc/mtrr
reg00: base=0x13c000000 ( 5056MB), size= 64MB, count=1: uncachable
reg01: base=0x000000000 ( 0MB), size= 2048MB, count=1: write-back
reg02: base=0x080000000 ( 2048MB), size= 1024MB, count=1: write-back
reg03: base=0x100000000 ( 4096MB), size= 1024MB, count=1: write-back
reg04: base=0x0d0000000 ( 3328MB), size= 256MB, count=1: write-combining

Quoted 3 lines

please try on attached patch on top on tip/master

http://people.redhat.com/mingo/tip.git/readme.txt

The issue still persists on tip kernel + patch:

Apr  1 15:07:32 t400 kernel: [   52.531195] sysctl net.netfilter.nf_conntrack_acct=1 to enable it.
Apr  1 13:38:51 t400 kernel: [ 1930.778773] X:4365 freeing invalid memtype d8c17000-d8c18000
Apr  1 13:38:51 t400 kernel: [ 1930.778901] X:4365 freeing invalid memtype d8c18000-d8c19000
Apr  1 13:38:51 t400 kernel: [ 1930.779021] X:4365 freeing invalid memtype d8c19000-d8c1a000
Apr  1 13:38:51 t400 kernel: [ 1930.779140] X:4365 freeing invalid memtype d8c1a000-d8c1b000
Apr  1 13:38:51 t400 kernel: [ 1930.779268] X:4365 freeing invalid memtype d8c1b000-d8c1c000
Apr  1 13:38:51 t400 kernel: [ 1930.779393] X:4365 freeing invalid memtype d8c1c000-d8c1d000
Apr  1 13:38:51 t400 kernel: [ 1930.779522] X:4365 freeing invalid memtype d8c1d000-d8c1e000
Apr  1 13:38:51 t400 kernel: [ 1930.779641] X:4365 freeing invalid memtype d8c1e000-d8c1f000
Apr  1 13:38:51 t400 kernel: [ 1930.779771] X:4365 freeing invalid memtype d8c1f000-d8c20000
Apr  1 13:38:51 t400 kernel: [ 1930.779897] X:4365 freeing invalid memtype d8c20000-d8c21000
Apr  1 13:38:51 t400 kernel: [ 1930.780024] X:4365 freeing invalid memtype d8c21000-d8c22000
Apr  1 13:38:51 t400 kernel: [ 1930.780155] X:4365 freeing invalid memtype d8c22000-d8c23000
Apr  1 13:38:51 t400 kernel: [ 1930.780274] X:4365 freeing invalid memtype d8c23000-d8c24000
Apr  1 13:38:51 t400 kernel: [ 1930.780393] X:4365 freeing invalid memtype d8c24000-d8c25000
Apr  1 13:38:51 t400 kernel: [ 1930.780512] X:4365 freeing invalid memtype d8c25000-d8c26000
Apr  1 13:38:51 t400 kernel: [ 1930.780631] X:4365 freeing invalid memtype d8c26000-d8c27000
Apr  1 13:38:51 t400 kernel: [ 1930.780751] X:4365 freeing invalid memtype d8c27000-d8c28000
Apr  1 13:38:51 t400 kernel: [ 1930.780870] X:4365 freeing invalid memtype d8c28000-d8c29000
Apr  1 13:38:51 t400 kernel: [ 1930.781019] X:4365 freeing invalid memtype d8c29000-d8c2a000
Apr  1 13:38:51 t400 kernel: [ 1930.781150] X:4365 freeing invalid memtype d8c2a000-d8c2b000
Apr  1 13:38:51 t400 kernel: [ 1930.781279] X:4365 freeing invalid memtype d8c2b000-d8c2c000
Apr  1 13:38:51 t400 kernel: [ 1930.781407] X:4365 freeing invalid memtype d8c2c000-d8c2d000
Apr  1 13:38:51 t400 kernel: [ 1930.781535] X:4365 freeing invalid memtype d8c2d000-d8c2e000
Apr  1 13:38:51 t400 kernel: [ 1930.781662] X:4365 freeing invalid memtype d8c2e000-d8c2f000
Apr  1 13:38:51 t400 kernel: [ 1930.781789] X:4365 freeing invalid memtype d8c2f000-d8c30000
Apr  1 13:38:51 t400 kernel: [ 1930.781909] X:4365 freeing invalid memtype d8c30000-d8c31000
Apr  1 13:38:51 t400 kernel: [ 1930.782028] X:4365 freeing invalid memtype d8c31000-d8c32000
Apr  1 13:38:51 t400 kernel: [ 1930.782147] X:4365 freeing invalid memtype d8c32000-d8c33000
Apr  1 13:38:51 t400 kernel: [ 1930.782266] X:4365 freeing invalid memtype d8c33000-d8c34000

104707 of these logged in one hour of kde4 session

13,941 lines not shown · raw message

Part of this message is not shown here. View the raw message.

#12 lore raw
In reply to Arkadiusz Miskiewicz
Quoted 65 lines · Arkadiusz Miskiewicz
Quoted 64 lines · Venkatesh
Quoted 49 lines · Arkadiusz Miskiewicz
Quoted 25 lines · Venkatesh
Quoted 12 lines · Arkadiusz Miskiewicz
Quoted 5 lines · Arkadiusz Miskiewicz

On Monday 30 of March 2009, Pallipadi, Venkatesh wrote:

More info follows. Now I've switched to e1c502482853f84606928f5a2f2eb6da1993cda1 which contains latest drm fixes and now I get much lower numbers of PAT errors but still.

Also when I switch t400 into discrete mode (radeon hd 3400 instead of integrated intel GM45) I get such errors (probably unrelated to these seen when using intel):

[  419.187657] X:10550 conflicting memory types cfff0000-d0000000
Quoted 11 lines

uncached<->uncached-minus [ 419.187670] reserve_memtype failed 0xcfff0000-0xd0000000, track uncached, req write-back [ 419.553914] X:10550 conflicting memory types cfff0000-d0000000 uncached<->uncached-minus [ 419.553923] reserve_memtype failed 0xcfff0000-0xd0000000, track uncached, req write-back [ 419.813592] X:10550 conflicting memory types cfff0000-d0000000 uncached<->uncached-minus [ 419.813601] reserve_memtype failed 0xcfff0000-0xd0000000, track uncached, req write-back [ 420.100102] X:10550 conflicting memory types cfff0000-d0000000 uncached<->uncached-minus [ 420.100111] reserve_memtype failed 0xcfff0000-0xd0000000, track uncached, req write-back

Quoted 3 lines

Yes. This is a different problem than the freeing invalid type one. Are these errors also with latest git kernel? Can you try the patch below (which is a part of a bigger cleanup patch I have lined up).

It's a latest git kernel as of today morning (latest commit is 15f7176eb1cccec0a332541285ee752b935c1c85)
+ your patch. Problem persists:

[   74.696353] [drm] Setting GART location based on new memory map
[   74.711520] [drm] Loading RV620 CP Microcode
[   74.711792] [drm] Loading RV620 PFP Microcode
[   74.726719] [drm] Resetting GPU
[   74.726776] [drm] writeback test succeeded in 1 usecs
[   75.256034] X:5366 conflicting memory types d0000000-e0000000

uncached-minus<->write-combining [ 75.256043] reserve_memtype failed
0xd0000000-0xe0000000, track uncached-minus, req write-back [ 75.849951] X:5366 conflicting memory types d0000000-e0000000
uncached-minus<->write-combining [ 75.849960] reserve_memtype failed
0xd0000000-0xe0000000, track uncached-minus, req write-back [ 76.054374] X:5366 conflicting memory types d0000000-e0000000
uncached-minus<->write-combining [ 76.054377] reserve_memtype failed
0xd0000000-0xe0000000, track uncached-minus, req write-back [ 76.074481] X:5378 freeing invalid memtype d0000000-e0000000

Quoted 4 lines
[   76.176881] X:5366 conflicting memory types d0000000-e0000000

uncached-minus<->write-combining [ 76.176885] reserve_memtype failed
0xd0000000-0xe0000000, track uncached-minus, req write-back [ 76.207734] X:5380 freeing invalid memtype d0000000-e0000000

OK. We now have a theory on what is going wrong here.

The problem seems to be pci mmap uses vm_page_prot flag to remember the memtype for this region. Looks like that memtype is somehow getting cleared in this case. We still don't know where it is getting cleared. But, with debug patch below we can be sure that it is indeed getting cleared, which is causing problems on fork() as child wont know the memtype that parent got.

Quoted 5 lines
Quoted 2 lines

Can you please try the below debug patch over upstream git and check whether you indeed hit the warnon.

warns triggered:

OK. Below patch should resolve this problem.

Thanks,
Venki

From: Suresh Siddha <[email protected]>
Subject: [PATCH] x86, PAT: Remove duplicate memtype reserve in pci mmap

pci mmap code was doing memtype reserve for a while now. Recently we added memtype tracking in remap_pfn_range, and pci code indirectly calls remap_pfn_range. So, we don't need seperate tracking in pci code anymore. Which means a patch that removes ~50 lines of code :-).

Also, recently we found out that the pci tracking is not working as we expect it to work in some cases. Specifically, userlevel X mmap of pci, with some recent version of X, is having a problem with vm_page_prot getting reset. The pci tracking uses vm_page_prot to pass on the protection type from parent to child during fork.
a) Parent does a pci mmap
b) We look at PAT and get either UC_MINUS or WC mapping for parent
c) Store that mapping type in vma vm_page_prot for future use
d) This thread does a fork
e) Fork results in mmap_ops ->open for the child process
f) We get the vm_page_prot from vma and reserve that type for the child process

But, between c) and e) above, the vma vm_page_prot is getting reset to zero. This results in PAT reserve failing at the time of fork as in here.
http://marc.info/?l=linux-kernel&m=123858163103240&w=2

This cleanup makes the above problem go away as we do not depend on vm_page_prot in our PAT code anymore.

---
 arch/x86/pci/i386.c |   46 ----------------------------------------------
 1 file changed, 46 deletions(-)
arch/x86/pci/i386.c +0 -46
Index: linux-2.6/arch/x86/pci/i386.c
===================================================================
@@ -258,24 +258,7 @@ void pcibios_set_master(struct pci_dev *
 	pci_write_config_byte(dev, PCI_LATENCY_TIMER, lat);
 }
 
-static void pci_unmap_page_range(struct vm_area_struct *vma)
-{
-	u64 addr = (u64)vma->vm_pgoff << PAGE_SHIFT;
-	free_memtype(addr, addr + vma->vm_end - vma->vm_start);
-}
-
-static void pci_track_mmap_page_range(struct vm_area_struct *vma)
-{
-	u64 addr = (u64)vma->vm_pgoff << PAGE_SHIFT;
-	unsigned long flags = pgprot_val(vma->vm_page_prot)
-						& _PAGE_CACHE_MASK;
-
-	reserve_memtype(addr, addr + vma->vm_end - vma->vm_start, flags, NULL);
-}
-
 static struct vm_operations_struct pci_mmap_ops = {
-	.open  = pci_track_mmap_page_range,
-	.close = pci_unmap_page_range,
 	.access = generic_access_phys,
 };
 
@@ -283,11 +266,6 @@ int pci_mmap_page_range(struct pci_dev *
 			enum pci_mmap_state mmap_state, int write_combine)
 {
 	unsigned long prot;
-	u64 addr = vma->vm_pgoff << PAGE_SHIFT;
-	unsigned long len = vma->vm_end - vma->vm_start;
-	unsigned long flags;
-	unsigned long new_flags;
-	int retval;
 
 	/* I/O space cannot be accessed via normal processor loads and
 	 * stores on this platform.
@@ -308,30 +286,6 @@ int pci_mmap_page_range(struct pci_dev *
 
 	vma->vm_page_prot = __pgprot(prot);
 
-	flags = pgprot_val(vma->vm_page_prot) & _PAGE_CACHE_MASK;
-	retval = reserve_memtype(addr, addr + len, flags, &new_flags);
-	if (retval)
-		return retval;
-
-	if (flags != new_flags) {
-		if (!is_new_memtype_allowed(flags, new_flags)) {
-			free_memtype(addr, addr+len);
-			return -EINVAL;
-		}
-		flags = new_flags;
-		vma->vm_page_prot = __pgprot(
-			(pgprot_val(vma->vm_page_prot) & ~_PAGE_CACHE_MASK) |
-			flags);
-	}
-
-	if (((vma->vm_pgoff < max_low_pfn_mapped) ||
-	     (vma->vm_pgoff >= (1UL<<(32 - PAGE_SHIFT)) &&
-	      vma->vm_pgoff < max_pfn_mapped)) &&
-	    ioremap_change_attr((unsigned long)__va(addr), len, flags)) {
-		free_memtype(addr, addr + len);
-		return -EINVAL;
-	}
-
 	if (io_remap_pfn_range(vma, vma->vm_start, vma->vm_pgoff,
 			       vma->vm_end - vma->vm_start,
 			       vma->vm_page_prot))
#15 lore raw
In reply to Pallipadi, Venkatesh
Quoted 34 lines · Venkatesh
Quoted 17 lines · Arkadiusz Miskiewicz

On Monday 30 of March 2009, Pallipadi, Venkatesh wrote:

More info follows. Now I've switched to e1c502482853f84606928f5a2f2eb6da1993cda1 which contains latest drm fixes and now I get much lower numbers of PAT errors but still.

Quoted 11 lines · Arkadiusz Miskiewicz
Quoted 10 lines · Venkatesh
Quoted 5 lines

Patch here should get rid of these errors.

http://marc.info/?l=linux-kernel&m=123788806506230&w=2

The patch is in tip and on its way to upstream.

The problem is that kernel I'm running already contains this patch (it's merged already). Other ideas?

Quoted 15 lines
Quoted 5 lines
Quoted 1 line

ratelimiting that error is good IMO anyway.

Rate limiting will just work around the problem here. Ideally we should never see these errors. So, it will be better if we can narrow down on the bug resulting in these error messages.

Of course it's better. I'm saying that when these messages "fire" then it's hard to do anything else on the system for a while until these stop.

Quoted 5 lines

Can you please send me the output of

# cat /debug/x86/pat_memtype_list
with debugfs mounted.
and
# cat /proc/mtrr
Quoted 8 lines

There seems to be two different problems here.
- We should not have that many single page ranges reserved. That will cause a performance problem with drm even without the "freeing invalid type" error.
- "freeing invalid type" error itself. Seems to be caused due to some unbalanced free along the drm path. We tried to find anything obvious in the code that may be causing problem here. But, haven't found anything so far. Will try to reproduce the problem internally and debug it further.

OK. I think we have root caused the thinko that was resulting in "freeing invalid type" error. Can you try the below test patch. Patch is not final version and may need some cleanup.

arch/x86/mm/pat.c +9 -5
index 640339e..c161700 100644
@@ -847,7 +847,8 @@ cleanup_ret:
  * can be for the entire vma (in which case size can be zero).
  */
 void untrack_pfn_vma(struct vm_area_struct *vma, unsigned long pfn,
-			unsigned long size)
+			unsigned long size,
+			unsigned long vstart, unsigned long vend)
 {
 	unsigned long i;
 	resource_size_t paddr;
@@ -866,7 +867,7 @@ void untrack_pfn_vma(struct vm_area_struct *vma, unsigned long pfn,
 		return;
 	}
 
-	if (size != 0 && size != vma_size) {
+	if (size != 0) {
 		/* free page by page, using pfn and size */
 		paddr = (resource_size_t)pfn << PAGE_SHIFT;
 		for (i = 0; i < size; i += PAGE_SIZE) {
@@ -874,9 +875,12 @@ void untrack_pfn_vma(struct vm_area_struct *vma, unsigned long pfn,
 			free_pfn_range(paddr, PAGE_SIZE);
 		}
 	} else {
-		/* free entire vma, page by page, using the pfn from pte */
-		for (i = 0; i < vma_size; i += PAGE_SIZE) {
-			if (follow_phys(vma, vma_start + i, 0, &prot, &paddr))
+		/*
+		 * free vma range from vstart to end, page by page
+		 * using the pfn from pte
+		 */
+		for (i = vstart; i < vend; i += PAGE_SIZE) {
+			if (follow_phys(vma, i, 0, &prot, &paddr))
 				continue;
 
 			free_pfn_range(paddr, PAGE_SIZE);
include/asm-generic/pgtable.h +4 -2
index 8e6d0ca..a325dc1 100644
@@ -328,7 +328,8 @@ static inline int track_pfn_vma_copy(struct vm_area_struct *vma)
  * can be for the entire vma (in which case size can be zero).
  */
 static inline void untrack_pfn_vma(struct vm_area_struct *vma,
-					unsigned long pfn, unsigned long size)
+				unsigned long pfn, unsigned long size,
+				unsigned long vstart, unsigned long vend)
 {
 }
 #else
@@ -336,7 +337,8 @@ extern int track_pfn_vma_new(struct vm_area_struct *vma, pgprot_t *prot,
 				unsigned long pfn, unsigned long size);
 extern int track_pfn_vma_copy(struct vm_area_struct *vma);
 extern void untrack_pfn_vma(struct vm_area_struct *vma, unsigned long pfn,
-				unsigned long size);
+				unsigned long size,
+				unsigned long vstart, unsigned long vend);
 #endif
 
 #endif /* !__ASSEMBLY__ */
mm/memory.c +3 -3
index cf6873e..6e111c5 100644
@@ -983,7 +983,7 @@ unsigned long unmap_vmas(struct mmu_gather **tlbp,
 			*nr_accounted += (end - start) >> PAGE_SHIFT;
 
 		if (unlikely(is_pfn_mapping(vma)))
-			untrack_pfn_vma(vma, 0, 0);
+			untrack_pfn_vma(vma, 0, 0, start, end);
 
 		while (start != end) {
 			if (!tlb_start_valid) {
@@ -1537,7 +1537,7 @@ int vm_insert_pfn(struct vm_area_struct *vma, unsigned long addr,
 	ret = insert_pfn(vma, addr, pfn, pgprot);
 
 	if (ret)
-		untrack_pfn_vma(vma, pfn, PAGE_SIZE);
+		untrack_pfn_vma(vma, pfn, PAGE_SIZE, 0, 0);
 
 	return ret;
 }
@@ -1702,7 +1702,7 @@ int remap_pfn_range(struct vm_area_struct *vma, unsigned long addr,
 	} while (pgd++, addr = next, addr != end);
 
 	if (err)
-		untrack_pfn_vma(vma, pfn, PAGE_ALIGN(size));
+		untrack_pfn_vma(vma, pfn, PAGE_ALIGN(size), 0, 0);
 
 	return err;
 }
#13 lore raw
In reply to Pallipadi, Venkatesh
Quoted 5 lines · Venkatesh
Quoted 4 lines · Arkadiusz Miskiewicz
Quoted 3 lines · Venkatesh
Quoted 2 lines · Arkadiusz Miskiewicz
Quoted 1 line · Venkatesh

On Mon, Mar 30, 2009 at 04:25:11PM -0700, Arkadiusz Miskiewicz

Quoted 62 lines · wrote:
Quoted 62 lines
Quoted 62 lines
Quoted 48 lines
Quoted 25 lines
Quoted 13 lines · Arkadiusz Miskiewicz
Quoted 6 lines · Venkatesh

More info follows. Now I've switched to e1c502482853f84606928f5a2f2eb6da1993cda1 which contains latest drm fixes and now I get much lower numbers of PAT errors but still.

Also when I switch t400 into discrete mode (radeon hd 3400 instead of integrated intel GM45) I get such errors (probably unrelated to these seen when using intel):

[  419.187657] X:10550 conflicting memory types cfff0000-d0000000
Quoted 11 lines

uncached<->uncached-minus [ 419.187670] reserve_memtype failed 0xcfff0000-0xd0000000, track uncached, req write-back [ 419.553914] X:10550 conflicting memory types cfff0000-d0000000 uncached<->uncached-minus [ 419.553923] reserve_memtype failed 0xcfff0000-0xd0000000, track uncached, req write-back [ 419.813592] X:10550 conflicting memory types cfff0000-d0000000 uncached<->uncached-minus [ 419.813601] reserve_memtype failed 0xcfff0000-0xd0000000, track uncached, req write-back [ 420.100102] X:10550 conflicting memory types cfff0000-d0000000 uncached<->uncached-minus [ 420.100111] reserve_memtype failed 0xcfff0000-0xd0000000, track uncached, req write-back

Quoted 3 lines

Yes. This is a different problem than the freeing invalid type one. Are these errors also with latest git kernel? Can you try the patch below (which is a part of a bigger cleanup patch I have lined up).

It's a latest git kernel as of today morning (latest commit is 15f7176eb1cccec0a332541285ee752b935c1c85)
+ your patch. Problem persists:

[   74.696353] [drm] Setting GART location based on new memory map
[   74.711520] [drm] Loading RV620 CP Microcode
[   74.711792] [drm] Loading RV620 PFP Microcode
[   74.726719] [drm] Resetting GPU
[   74.726776] [drm] writeback test succeeded in 1 usecs
[   75.256034] X:5366 conflicting memory types d0000000-e0000000

uncached-minus<->write-combining [ 75.256043] reserve_memtype
failed 0xd0000000-0xe0000000, track uncached-minus, req write-back [ 75.849951] X:5366 conflicting memory types d0000000-e0000000
uncached-minus<->write-combining [ 75.849960] reserve_memtype
failed 0xd0000000-0xe0000000, track uncached-minus, req write-back [ 76.054374] X:5366 conflicting memory types d0000000-e0000000
uncached-minus<->write-combining [ 76.054377] reserve_memtype
failed 0xd0000000-0xe0000000, track uncached-minus, req write-back [ 76.074481] X:5378 freeing invalid memtype d0000000-e0000000

Quoted 4 lines
[   76.176881] X:5366 conflicting memory types d0000000-e0000000

uncached-minus<->write-combining [ 76.176885] reserve_memtype
failed 0xd0000000-0xe0000000, track uncached-minus, req write-back [ 76.207734] X:5380 freeing invalid memtype d0000000-e0000000

OK. We now have a theory on what is going wrong here.

The problem seems to be pci mmap uses vm_page_prot flag to remember the memtype for this region. Looks like that memtype is somehow getting cleared in this case. We still don't know where it is getting cleared. But, with debug patch below we can be sure that it is indeed getting cleared, which is causing problems on fork() as child wont know the memtype that parent got.

Quoted 6 lines
Quoted 4 lines
Quoted 2 lines

Can you please try the below debug patch over upstream git and check whether you indeed hit the warnon.

warns triggered:

OK. Below patch should resolve this problem.

The issue is gone with this patch. Thanks!

Quoted 110 lines

Thanks,
Venki

From: Suresh Siddha <[email protected]>
Subject: [PATCH] x86, PAT: Remove duplicate memtype reserve in pci mmap

pci mmap code was doing memtype reserve for a while now. Recently we added memtype tracking in remap_pfn_range, and pci code indirectly calls remap_pfn_range. So, we don't need seperate tracking in pci code anymore. Which means a patch that removes ~50 lines of code :-).

Also, recently we found out that the pci tracking is not working as we expect it to work in some cases. Specifically, userlevel X mmap of pci, with some recent version of X, is having a problem with vm_page_prot getting reset. The pci tracking uses vm_page_prot to pass on the protection type from parent to child during fork.
a) Parent does a pci mmap
b) We look at PAT and get either UC_MINUS or WC mapping for parent
c) Store that mapping type in vma vm_page_prot for future use
d) This thread does a fork
e) Fork results in mmap_ops ->open for the child process
f) We get the vm_page_prot from vma and reserve that type for the child process

But, between c) and e) above, the vma vm_page_prot is getting reset to zero. This results in PAT reserve failing at the time of fork as in here.
http://marc.info/?l=linux-kernel&m=123858163103240&w=2

This cleanup makes the above problem go away as we do not depend on vm_page_prot in our PAT code anymore.

---
 arch/x86/pci/i386.c |   46 ----------------------------------------------
 1 file changed, 46 deletions(-)

Index: linux-2.6/arch/x86/pci/i386.c
===================================================================

arch/x86/pci/i386.c +0 -46
@@ -258,24 +258,7 @@ void pcibios_set_master(struct pci_dev *
 	pci_write_config_byte(dev, PCI_LATENCY_TIMER, lat);
 }
 
-static void pci_unmap_page_range(struct vm_area_struct *vma)
-{
-	u64 addr = (u64)vma->vm_pgoff << PAGE_SHIFT;
-	free_memtype(addr, addr + vma->vm_end - vma->vm_start);
-}
-
-static void pci_track_mmap_page_range(struct vm_area_struct *vma)
-{
-	u64 addr = (u64)vma->vm_pgoff << PAGE_SHIFT;
-	unsigned long flags = pgprot_val(vma->vm_page_prot)
-						& _PAGE_CACHE_MASK;
-
-	reserve_memtype(addr, addr + vma->vm_end - vma->vm_start, flags, NULL);
-}
-
 static struct vm_operations_struct pci_mmap_ops = {
-	.open  = pci_track_mmap_page_range,
-	.close = pci_unmap_page_range,
 	.access = generic_access_phys,
 };
 
@@ -283,11 +266,6 @@ int pci_mmap_page_range(struct pci_dev *
 			enum pci_mmap_state mmap_state, int write_combine)
 {
 	unsigned long prot;
-	u64 addr = vma->vm_pgoff << PAGE_SHIFT;
-	unsigned long len = vma->vm_end - vma->vm_start;
-	unsigned long flags;
-	unsigned long new_flags;
-	int retval;
 
 	/* I/O space cannot be accessed via normal processor loads and
 	 * stores on this platform.
@@ -308,30 +286,6 @@ int pci_mmap_page_range(struct pci_dev *
 
 	vma->vm_page_prot = __pgprot(prot);
 
-	flags = pgprot_val(vma->vm_page_prot) & _PAGE_CACHE_MASK;
-	retval = reserve_memtype(addr, addr + len, flags, &new_flags);
-	if (retval)
-		return retval;
-
-	if (flags != new_flags) {
-		if (!is_new_memtype_allowed(flags, new_flags)) {
-			free_memtype(addr, addr+len);
-			return -EINVAL;
-		}
-		flags = new_flags;
-		vma->vm_page_prot = __pgprot(
-			(pgprot_val(vma->vm_page_prot) & ~_PAGE_CACHE_MASK) |
-			flags);
-	}
-
-	if (((vma->vm_pgoff < max_low_pfn_mapped) ||
-	     (vma->vm_pgoff >= (1UL<<(32 - PAGE_SHIFT)) &&
-	      vma->vm_pgoff < max_pfn_mapped)) &&
-	    ioremap_change_attr((unsigned long)__va(addr), len, flags)) {
-		free_memtype(addr, addr + len);
-		return -EINVAL;
-	}
-
 	if (io_remap_pfn_range(vma, vma->vm_start, vma->vm_pgoff,
 			       vma->vm_end - vma->vm_start,
 			       vma->vm_page_prot))
Signature · 2 lines
Arkadiusz Miśkiewicz        PLD/Linux Team
arekm / maven.pl            http://ftp.pld-linux.org/
#16 lore raw
In reply to Pallipadi, Venkatesh
Quoted 137 lines · Venkatesh
Quoted 34 lines · Venkatesh
Quoted 17 lines · Arkadiusz Miskiewicz

On Monday 30 of March 2009, Pallipadi, Venkatesh wrote:

More info follows. Now I've switched to e1c502482853f84606928f5a2f2eb6da1993cda1 which contains latest drm fixes and now I get much lower numbers of PAT errors but still.

Quoted 11 lines · Arkadiusz Miskiewicz
Quoted 10 lines · Venkatesh
Quoted 5 lines

Patch here should get rid of these errors.

http://marc.info/?l=linux-kernel&m=123788806506230&w=2

The patch is in tip and on its way to upstream.

The problem is that kernel I'm running already contains this patch (it's merged already). Other ideas?

Quoted 15 lines
Quoted 5 lines
Quoted 1 line

ratelimiting that error is good IMO anyway.

Rate limiting will just work around the problem here. Ideally we should never see these errors. So, it will be better if we can narrow down on the bug resulting in these error messages.

Of course it's better. I'm saying that when these messages "fire" then it's hard to do anything else on the system for a while until these stop.

Quoted 5 lines

Can you please send me the output of

# cat /debug/x86/pat_memtype_list
with debugfs mounted.
and
# cat /proc/mtrr
Quoted 8 lines

There seems to be two different problems here.
- We should not have that many single page ranges reserved. That will cause a performance problem with drm even without the "freeing invalid type" error.
- "freeing invalid type" error itself. Seems to be caused due to some unbalanced free along the drm path. We tried to find anything obvious in the code that may be causing problem here. But, haven't found anything so far. Will try to reproduce the problem internally and debug it further.

OK. I think we have root caused the thinko that was resulting in "freeing invalid type" error. Can you try the below test patch. Patch is not final version and may need some cleanup.

arch/x86/mm/pat.c +9 -5
index 640339e..c161700 100644
@@ -847,7 +847,8 @@ cleanup_ret:
  * can be for the entire vma (in which case size can be zero).
  */
  void untrack_pfn_vma(struct vm_area_struct *vma, unsigned long pfn,
-                       unsigned long size)
+                       unsigned long size,
+                       unsigned long vstart, unsigned long vend)
  {
        unsigned long i;
        resource_size_t paddr;
@@ -866,7 +867,7 @@ void untrack_pfn_vma(struct vm_area_struct *vma, unsigned long pfn,
                return;
        }
 
-       if (size != 0 && size != vma_size) {
+       if (size != 0) {
                /* free page by page, using pfn and size */
                paddr = (resource_size_t)pfn << PAGE_SHIFT;
                for (i = 0; i < size; i += PAGE_SIZE) {
@@ -874,9 +875,12 @@ void untrack_pfn_vma(struct vm_area_struct *vma, unsigned long pfn,
                        free_pfn_range(paddr, PAGE_SIZE);
                }
        } else {
-               /* free entire vma, page by page, using the pfn from pte */
-               for (i = 0; i < vma_size; i += PAGE_SIZE) {
-                       if (follow_phys(vma, vma_start + i, 0, &prot, &paddr))
+               /*
+                * free vma range from vstart to end, page by page
+                * using the pfn from pte
+                */
+               for (i = vstart; i < vend; i += PAGE_SIZE) {
+                       if (follow_phys(vma, i, 0, &prot, &paddr))
                                continue;
 
                        free_pfn_range(paddr, PAGE_SIZE);
include/asm-generic/pgtable.h +4 -2
index 8e6d0ca..a325dc1 100644
@@ -328,7 +328,8 @@ static inline int track_pfn_vma_copy(struct vm_area_struct *vma)
  * can be for the entire vma (in which case size can be zero).
  */
  static inline void untrack_pfn_vma(struct vm_area_struct *vma,
-                                       unsigned long pfn, unsigned long size)
+                               unsigned long pfn, unsigned long size,
+                               unsigned long vstart, unsigned long vend)
  {
  }
  #else
@@ -336,7 +337,8 @@ extern int track_pfn_vma_new(struct vm_area_struct *vma, pgprot_t *prot,
                                unsigned long pfn, unsigned long size);
  extern int track_pfn_vma_copy(struct vm_area_struct *vma);
  extern void untrack_pfn_vma(struct vm_area_struct *vma, unsigned long pfn,
-                               unsigned long size);
+                               unsigned long size,
+                               unsigned long vstart, unsigned long vend);
  #endif
 
  #endif /* !__ASSEMBLY__ */
mm/memory.c +2 -2
index cf6873e..6e111c5 100644
@@ -983,7 +983,7 @@ unsigned long unmap_vmas(struct mmu_gather **tlbp,
                        *nr_accounted += (end - start) >> PAGE_SHIFT;
 
                if (unlikely(is_pfn_mapping(vma)))
-                       untrack_pfn_vma(vma, 0, 0);
+                       untrack_pfn_vma(vma, 0, 0, start, end);
 
                while (start != end) {
                        if (!tlb_start_valid) {
@@ -1537,7 +1537,7 @@ int vm_insert_pfn(struct vm_area_struct *vma, unsigned long addr,
        ret = insert_pfn(vma, addr, pfn, pgprot);
 
        if (ret)
-               untrack_pfn_vma(vma, pfn, PAGE_SIZE);
+               untrack_pfn_vma(vma, pfn, PAGE_SIZE, 0, 0);
 
        return ret;
  }

mm/memory.c int remap_pfn_range(struct vm_area_struct *vma, unsigned long addr, · line 1708

Quoted 8 lines
mm/memory.c +1 -1
@@ -1702,7 +1702,7 @@ int remap_pfn_range(struct vm_area_struct *vma, unsigned long addr,
        } while (pgd++, addr = next, addr != end);
 
        if (err)
-               untrack_pfn_vma(vma, pfn, PAGE_ALIGN(size));
+               untrack_pfn_vma(vma, pfn, PAGE_ALIGN(size), 0, 0);
 
        return err;
  }

2.6.29-git9 plus the above patch still doesn't fix my Dell E6400 running
Fedora 10 x86_64:

[asuardi@duff ~]$ dmesg|grep type |grep mem
X:3509 conflicting memory types e0000000-f0000000 uncached<->write-combining reserve_memtype failed 0xe0000000-0xf0000000, track uncached, req write-back X:3509 conflicting memory types f6e00000-f7000000 uncached<->uncached-minus reserve_memtype failed 0xf6e00000-0xf7000000, track uncached, req write-back X:3509 conflicting memory types f6c00000-f6c80000 uncached<->uncached-minus reserve_memtype failed 0xf6c00000-0xf6c80000, track uncached, req write-back X:3509 conflicting memory types e0000000-f0000000 uncached<->write-combining reserve_memtype failed 0xe0000000-0xf0000000, track uncached, req write-back X:3509 conflicting memory types f6e00000-f7000000 uncached<->uncached-minus reserve_memtype failed 0xf6e00000-0xf7000000, track uncached, req write-back X:3509 conflicting memory types f6c00000-f6c80000 uncached<->uncached-minus reserve_memtype failed 0xf6c00000-0xf6c80000, track uncached, req write-back X:3509 conflicting memory types e0000000-f0000000 uncached<->write-combining reserve_memtype failed 0xe0000000-0xf0000000, track uncached, req write-back X:3509 conflicting memory types f6e00000-f7000000 uncached<->uncached-minus reserve_memtype failed 0xf6e00000-0xf7000000, track uncached, req write-back X:3509 conflicting memory types f6c00000-f6c80000 uncached<->uncached-minus reserve_memtype failed 0xf6c00000-0xf6c80000, track uncached, req write-back X:3509 conflicting memory types e0000000-f0000000 uncached<->write-combining reserve_memtype failed 0xe0000000-0xf0000000, track uncached, req write-back X:3509 conflicting memory types f6e00000-f7000000 uncached<->uncached-minus reserve_memtype failed 0xf6e00000-0xf7000000, track uncached, req write-back X:3509 conflicting memory types f6c00000-f6c80000 uncached<->uncached-minus reserve_memtype failed 0xf6c00000-0xf6c80000, track uncached, req write-back X:3561 freeing invalid memtype e0000000-f0000000
X:3561 freeing invalid memtype f6e00000-f7000000
X:3509 conflicting memory types e0000000-f0000000 uncached<->write-combining reserve_memtype failed 0xe0000000-0xf0000000, track uncached, req write-back X:3509 conflicting memory types f6e00000-f7000000 uncached<->uncached-minus reserve_memtype failed 0xf6e00000-0xf7000000, track uncached, req write-back X:3509 conflicting memory types f6c00000-f6c80000 uncached<->uncached-minus reserve_memtype failed 0xf6c00000-0xf6c80000, track uncached, req write-back X:3562 freeing invalid memtype e0000000-f0000000
X:3562 freeing invalid memtype f6e00000-f7000000
X:3562 freeing invalid memtype f6c00000-f6c80000
X:3509 conflicting memory types e0000000-f0000000 uncached<->write-combining reserve_memtype failed 0xe0000000-0xf0000000, track uncached, req write-back X:3509 conflicting memory types f6e00000-f7000000 uncached<->uncached-minus reserve_memtype failed 0xf6e00000-0xf7000000, track uncached, req write-back X:3509 conflicting memory types f6c00000-f6c80000 uncached<->uncached-minus reserve_memtype failed 0xf6c00000-0xf6c80000, track uncached, req write-back X:3563 freeing invalid memtype e0000000-f0000000
X:3563 freeing invalid memtype f6e00000-f7000000
X:3563 freeing invalid memtype f6c00000-f6c80000
X:3509 conflicting memory types e0000000-f0000000 uncached<->write-combining reserve_memtype failed 0xe0000000-0xf0000000, track uncached, req write-back X:3509 conflicting memory types f6e00000-f7000000 uncached<->uncached-minus reserve_memtype failed 0xf6e00000-0xf7000000, track uncached, req write-back X:3509 conflicting memory types f6c00000-f6c80000 uncached<->uncached-minus reserve_memtype failed 0xf6c00000-0xf6c80000, track uncached, req write-back X:3564 freeing invalid memtype e0000000-f0000000
X:3564 freeing invalid memtype f6e00000-f7000000
X:3564 freeing invalid memtype f6c00000-f6c80000
X:3509 conflicting memory types e0000000-f0000000 uncached<->write-combining reserve_memtype failed 0xe0000000-0xf0000000, track uncached, req write-back X:3509 conflicting memory types f6e00000-f7000000 uncached<->uncached-minus reserve_memtype failed 0xf6e00000-0xf7000000, track uncached, req write-back X:3509 conflicting memory types f6c00000-f6c80000 uncached<->uncached-minus reserve_memtype failed 0xf6c00000-0xf6c80000, track uncached, req write-back X:822 freeing invalid memtype e0000000-f0000000
X:822 freeing invalid memtype f6e00000-f7000000
X:822 freeing invalid memtype f6c00000-f6c80000

I also get a corrupted display exiting X to tty console (monitor flashes
wildly, looking like it's redisplaying the shell prompt but on top instead of on bottom), and if I try to Alt-Fn from X to a tty console, the laptop locks up hard - Caps doesn't light its led anymore, and I have to keep the power button pressed for 10" to power off the box.

The F10 kernel has the same display corruption exiting X to console,
haven't tried yet Alt-Fn.

I don't have earlier kernel comparisons as I installed F10 a week ago
on this brand new laptop, 2.6.29-git5 was the first I built; but I could try building 2.6.28 if deemed useful; right now I'm building up 2.6.29.1 and will report later.

Any details - just ask. Thanks,

--alessandro

"Sun keeps rising in the west / I keep on waking fully confused"

(The Replacements, "Within Your Reach")

#17 lore raw
Quoted 183 lines

-----Original Message-----
From: Alessandro Suardi [mailto:[email protected]]
Sent: Friday, April 03, 2009 2:54 AM
To: Pallipadi, Venkatesh
Cc: Arkadiusz Miskiewicz; Siddha, Suresh B; [email protected]; Jesse Barnes
Subject: Re: 2.6.29 git master and PAT problems

Quoted 2 lines · Venkatesh
Quoted 1 line · Venkatesh

On Mon, Mar 30, 2009 at 03:31:09PM -0700, Arkadiusz

Quoted 4 lines · Miskiewicz
Quoted 4 lines
Quoted 4 lines · Venkatesh

More info follows. Now I've switched to e1c502482853f84606928f5a2f2eb6da1993cda1 which contains

latest drm fixes and

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

now I get much lower numbers of PAT errors but still.

Quoted 9 lines · Arkadiusz Miskiewicz
Quoted 8 lines · Venkatesh
Quoted 5 lines

Patch here should get rid of these errors.

http://marc.info/?l=linux-kernel&m=123788806506230&w=2

The patch is in tip and on its way to upstream.

The problem is that kernel I'm running already

contains this patch (it's

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines

merged already). Other ideas?

ratelimiting that error is good IMO anyway.

Rate limiting will just work around the problem here.

Ideally we should

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

never see these errors. So, it will be better if we can

narrow down on

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

the bug resulting in these error messages.

Of course it's better. I'm saying that when these

messages "fire" then it's

Quoted 1 line
Quoted 1 line
Quoted 1 line

hard to do anything else on the system for a while until

these stop.

Quoted 10 lines
Quoted 10 lines
Quoted 7 lines
Quoted 6 lines

Can you please send me the output of

# cat /debug/x86/pat_memtype_list
with debugfs mounted.
and
# cat /proc/mtrr

There seems to be two different problems here.
- We should not have that many single page ranges reserved.

That will cause a

Quoted 1 line
Quoted 1 line

performance problem with drm even without the "freeing

invalid type" error.

Quoted 1 line
Quoted 1 line

- "freeing invalid type" error itself. Seems to be caused

due to some

Quoted 1 line
Quoted 1 line

unbalanced free along the drm path. We tried to find

anything obvious in the

Quoted 1 line
Quoted 1 line

code that may be causing problem here. But, haven't found

anything so far.

arch/x86/mm/pat.c void untrack_pfn_vma(struct

Quoted 25 lines
Quoted 2 lines

Will try to reproduce the problem internally and debug it further.

OK. I think we have root caused the thinko that was resulting in "freeing invalid type" error. Can you try the below test patch. Patch is not final version and may need some cleanup.

arch/x86/mm/pat.c +2 -1
index 640339e..c161700 100644
@@ -847,7 +847,8 @@ cleanup_ret:
  * can be for the entire vma (in which case size can be zero).
  */
  void untrack_pfn_vma(struct vm_area_struct *vma, unsigned long pfn,
-                       unsigned long size)
+                       unsigned long size,
+                       unsigned long vstart, unsigned long vend)
  {
        unsigned long i;
        resource_size_t paddr;
@@ -866,7 +867,7 @@ void untrack_pfn_vma(struct 

vm_area_struct *vma, unsigned long pfn,

arch/x86/mm/pat.c void untrack_pfn_vma(struct

Quoted 9 lines
arch/x86/mm/pat.c +1 -1
                return;
        }
 
-       if (size != 0 && size != vma_size) {
+       if (size != 0) {
                /* free page by page, using pfn and size */
                paddr = (resource_size_t)pfn << PAGE_SHIFT;
                for (i = 0; i < size; i += PAGE_SIZE) {
@@ -874,9 +875,12 @@ void untrack_pfn_vma(struct 

vm_area_struct *vma, unsigned long pfn,

arch/x86/mm/pat.c void untrack_pfn_vma(struct vm_area_struct *vma, unsigned long pfn,

Quoted 4 lines
arch/x86/mm/pat.c +0 -1
                        free_pfn_range(paddr, PAGE_SIZE);
                }
        } else {
-               /* free entire vma, page by page, using the 

pfn from pte */

arch/x86/mm/pat.c void untrack_pfn_vma(struct · line 879

Quoted 2 lines
arch/x86/mm/pat.c +0 -2
-               for (i = 0; i < vma_size; i += PAGE_SIZE) {
-                       if (follow_phys(vma, vma_start + i, 

0, &prot, &paddr))

Review

Quoted 10 lines
arch/x86/mm/pat.c +6 -0
+               /*
+                * free vma range from vstart to end, page by page
+                * using the pfn from pte
+                */
+               for (i = vstart; i < vend; i += PAGE_SIZE) {
+                       if (follow_phys(vma, i, 0, &prot, &paddr))
                                continue;
 
                        free_pfn_range(paddr, PAGE_SIZE);
(quoted diff) +0 -0

b/include/asm-generic/pgtable.h

include/asm-generic/pgtable.h static inline int

Quoted 4 lines

index 8e6d0ca..a325dc1 100644

include/asm-generic/pgtable.h +0 -0
@@ -328,7 +328,8 @@ static inline int 

track_pfn_vma_copy(struct vm_area_struct *vma)

include/asm-generic/pgtable.h static inline int track_pfn_vma_copy(struct vm_area_struct *vma)

Quoted 4 lines
include/asm-generic/pgtable.h +0 -1
  * can be for the entire vma (in which case size can be zero).
  */
  static inline void untrack_pfn_vma(struct vm_area_struct *vma,
-                                       unsigned long pfn, 

unsigned long size)

include/asm-generic/pgtable.h static inline int · line 331

Quoted 1 line
include/asm-generic/pgtable.h +1 -0
+                               unsigned long pfn, unsigned 

long size,

include/asm-generic/pgtable.h static inline int · line 332

Quoted 1 line
include/asm-generic/pgtable.h +1 -0
+                               unsigned long vstart, 

unsigned long vend)

include/asm-generic/pgtable.h extern int track_pfn_vma_new(struct

Quoted 4 lines
include/asm-generic/pgtable.h +0 -0
  {
  }
  #else
@@ -336,7 +337,8 @@ extern int track_pfn_vma_new(struct 

vm_area_struct *vma, pgprot_t *prot,

Quoted 1 line

                               unsigned long pfn, unsigned

long size);

Quoted 2 lines

 extern int track_pfn_vma_copy(struct vm_area_struct *vma); extern void untrack_pfn_vma(struct vm_area_struct *vma,

unsigned long pfn,

Quoted 3 lines

-                               unsigned long size);
+                               unsigned long size,
+                               unsigned long vstart,

unsigned long vend);

mm/memory.c unsigned long unmap_vmas(struct

Quoted 8 lines

 #endif

 #endif /* !__ASSEMBLY__ */

mm/memory.c +0 -0
index cf6873e..6e111c5 100644
@@ -983,7 +983,7 @@ unsigned long unmap_vmas(struct 

mmu_gather **tlbp,

mm/memory.c int vm_insert_pfn(struct

Quoted 9 lines
mm/memory.c +1 -1
                        *nr_accounted += (end - start) >> PAGE_SHIFT;
 
                if (unlikely(is_pfn_mapping(vma)))
-                       untrack_pfn_vma(vma, 0, 0);
+                       untrack_pfn_vma(vma, 0, 0, start, end);
 
                while (start != end) {
                        if (!tlb_start_valid) {
@@ -1537,7 +1537,7 @@ int vm_insert_pfn(struct 

vm_area_struct *vma, unsigned long addr,

mm/memory.c int remap_pfn_range(struct

Quoted 9 lines
mm/memory.c +1 -1
        ret = insert_pfn(vma, addr, pfn, pgprot);
 
        if (ret)
-               untrack_pfn_vma(vma, pfn, PAGE_SIZE);
+               untrack_pfn_vma(vma, pfn, PAGE_SIZE, 0, 0);
 
        return ret;
  }
@@ -1702,7 +1702,7 @@ int remap_pfn_range(struct 

vm_area_struct *vma, unsigned long addr,

Quoted 6 lines
mm/memory.c +1 -1
        } while (pgd++, addr = next, addr != end);
 
        if (err)
-               untrack_pfn_vma(vma, pfn, PAGE_ALIGN(size));
+               untrack_pfn_vma(vma, pfn, PAGE_ALIGN(size), 0, 0);
 
Quoted 6 lines

mm/memory.c int remap_pfn_range(struct vm_area_struct *vma, unsigned long addr, · line 1708

Quoted 2 lines
mm/memory.c +0 -0
        return err;
  }

2.6.29-git9 plus the above patch still doesn't fix my Dell E6400 running
Fedora 10 x86_64:

The patch here should get rid of the below msgs.
http://marc.info/?l=linux-kernel&m=123862741914199&w=2

Can you please test and check whether that resolves the problems?

The patch inline in this mail above was a test patch to debug the another problem that only happens with recent X and that one we are still debugging. So you can ignore the above patch for now.

Thanks,
Venki

#18 lore raw
Quoted 195 lines · Venkatesh
Quoted 183 lines

-----Original Message-----
From: Alessandro Suardi [mailto:[email protected]]
Sent: Friday, April 03, 2009 2:54 AM
To: Pallipadi, Venkatesh
Cc: Arkadiusz Miskiewicz; Siddha, Suresh B; [email protected]; Jesse Barnes
Subject: Re: 2.6.29 git master and PAT problems

Quoted 2 lines · Venkatesh
Quoted 1 line · Venkatesh

On Mon, Mar 30, 2009 at 03:31:09PM -0700, Arkadiusz

Quoted 4 lines · Miskiewicz
Quoted 4 lines
Quoted 4 lines · Venkatesh

More info follows. Now I've switched to e1c502482853f84606928f5a2f2eb6da1993cda1 which contains

latest drm fixes and

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

now I get much lower numbers of PAT errors but still.

Quoted 9 lines · Arkadiusz Miskiewicz
Quoted 8 lines · Venkatesh
Quoted 5 lines

Patch here should get rid of these errors.

http://marc.info/?l=linux-kernel&m=123788806506230&w=2

The patch is in tip and on its way to upstream.

The problem is that kernel I'm running already

contains this patch (it's

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines

merged already). Other ideas?

ratelimiting that error is good IMO anyway.

Rate limiting will just work around the problem here.

Ideally we should

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

never see these errors. So, it will be better if we can

narrow down on

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

the bug resulting in these error messages.

Of course it's better. I'm saying that when these

messages "fire" then it's

Quoted 1 line
Quoted 1 line
Quoted 1 line

hard to do anything else on the system for a while until

these stop.

Quoted 10 lines
Quoted 10 lines
Quoted 7 lines
Quoted 6 lines

Can you please send me the output of

# cat /debug/x86/pat_memtype_list
with debugfs mounted.
and
# cat /proc/mtrr

There seems to be two different problems here.
- We should not have that many single page ranges reserved.

That will cause a

Quoted 1 line
Quoted 1 line

performance problem with drm even without the "freeing

invalid type" error.

Quoted 1 line
Quoted 1 line

- "freeing invalid type" error itself. Seems to be caused

due to some

Quoted 1 line
Quoted 1 line

unbalanced free along the drm path. We tried to find

anything obvious in the

Quoted 1 line
Quoted 1 line

code that may be causing problem here. But, haven't found

anything so far.

arch/x86/mm/pat.c void untrack_pfn_vma(struct

Quoted 25 lines
Quoted 2 lines

Will try to reproduce the problem internally and debug it further.

OK. I think we have root caused the thinko that was resulting in "freeing invalid type" error. Can you try the below test patch. Patch is not final version and may need some cleanup.

arch/x86/mm/pat.c +2 -1
index 640339e..c161700 100644
@@ -847,7 +847,8 @@ cleanup_ret:
  * can be for the entire vma (in which case size can be zero).
  */
  void untrack_pfn_vma(struct vm_area_struct *vma, unsigned long pfn,
-                       unsigned long size)
+                       unsigned long size,
+                       unsigned long vstart, unsigned long vend)
  {
        unsigned long i;
        resource_size_t paddr;
@@ -866,7 +867,7 @@ void untrack_pfn_vma(struct

vm_area_struct *vma, unsigned long pfn,

arch/x86/mm/pat.c void untrack_pfn_vma(struct

Quoted 9 lines
arch/x86/mm/pat.c +1 -1
                return;
        }
 
-       if (size != 0 && size != vma_size) {
+       if (size != 0) {
                /* free page by page, using pfn and size */
                paddr = (resource_size_t)pfn << PAGE_SHIFT;
                for (i = 0; i < size; i += PAGE_SIZE) {
@@ -874,9 +875,12 @@ void untrack_pfn_vma(struct

vm_area_struct *vma, unsigned long pfn,

arch/x86/mm/pat.c void untrack_pfn_vma(struct vm_area_struct *vma, unsigned long pfn,

Quoted 4 lines
arch/x86/mm/pat.c +0 -1
                        free_pfn_range(paddr, PAGE_SIZE);
                }
        } else {
-               /* free entire vma, page by page, using the

pfn from pte */

arch/x86/mm/pat.c void untrack_pfn_vma(struct · line 879

Quoted 2 lines
arch/x86/mm/pat.c +0 -2
-               for (i = 0; i < vma_size; i += PAGE_SIZE) {
-                       if (follow_phys(vma, vma_start + i,

0, &prot, &paddr))

Review

Quoted 10 lines
arch/x86/mm/pat.c +6 -0
+               /*
+                * free vma range from vstart to end, page by page
+                * using the pfn from pte
+                */
+               for (i = vstart; i < vend; i += PAGE_SIZE) {
+                       if (follow_phys(vma, i, 0, &prot, &paddr))
                                continue;
 
                        free_pfn_range(paddr, PAGE_SIZE);
(quoted diff) +0 -0

b/include/asm-generic/pgtable.h

include/asm-generic/pgtable.h static inline int

Quoted 4 lines

index 8e6d0ca..a325dc1 100644

include/asm-generic/pgtable.h +0 -0
@@ -328,7 +328,8 @@ static inline int

track_pfn_vma_copy(struct vm_area_struct *vma)

include/asm-generic/pgtable.h static inline int track_pfn_vma_copy(struct vm_area_struct *vma)

Quoted 4 lines
include/asm-generic/pgtable.h +0 -1
  * can be for the entire vma (in which case size can be zero).
  */
  static inline void untrack_pfn_vma(struct vm_area_struct *vma,
-                                       unsigned long pfn,

unsigned long size)

include/asm-generic/pgtable.h static inline int · line 331

Quoted 1 line
include/asm-generic/pgtable.h +1 -0
+                               unsigned long pfn, unsigned

long size,

include/asm-generic/pgtable.h static inline int · line 332

Quoted 1 line
include/asm-generic/pgtable.h +1 -0
+                               unsigned long vstart,

unsigned long vend)

include/asm-generic/pgtable.h extern int track_pfn_vma_new(struct

Quoted 4 lines
include/asm-generic/pgtable.h +0 -0
  {
  }
  #else
@@ -336,7 +337,8 @@ extern int track_pfn_vma_new(struct

vm_area_struct *vma, pgprot_t *prot,

Quoted 1 line

                               unsigned long pfn, unsigned

long size);

Quoted 2 lines

 extern int track_pfn_vma_copy(struct vm_area_struct *vma); extern void untrack_pfn_vma(struct vm_area_struct *vma,

unsigned long pfn,

Quoted 3 lines

-                               unsigned long size);
+                               unsigned long size,
+                               unsigned long vstart,

unsigned long vend);

mm/memory.c unsigned long unmap_vmas(struct

Quoted 8 lines

 #endif

 #endif /* !__ASSEMBLY__ */

mm/memory.c +0 -0
index cf6873e..6e111c5 100644
@@ -983,7 +983,7 @@ unsigned long unmap_vmas(struct

mmu_gather **tlbp,

mm/memory.c int vm_insert_pfn(struct

Quoted 9 lines
mm/memory.c +1 -1
                        *nr_accounted += (end - start) >> PAGE_SHIFT;
 
                if (unlikely(is_pfn_mapping(vma)))
-                       untrack_pfn_vma(vma, 0, 0);
+                       untrack_pfn_vma(vma, 0, 0, start, end);
 
                while (start != end) {
                        if (!tlb_start_valid) {
@@ -1537,7 +1537,7 @@ int vm_insert_pfn(struct

vm_area_struct *vma, unsigned long addr,

mm/memory.c int remap_pfn_range(struct

Quoted 9 lines
mm/memory.c +1 -1
        ret = insert_pfn(vma, addr, pfn, pgprot);
 
        if (ret)
-               untrack_pfn_vma(vma, pfn, PAGE_SIZE);
+               untrack_pfn_vma(vma, pfn, PAGE_SIZE, 0, 0);
 
        return ret;
  }
@@ -1702,7 +1702,7 @@ int remap_pfn_range(struct

vm_area_struct *vma, unsigned long addr,

Quoted 6 lines
mm/memory.c +1 -1
        } while (pgd++, addr = next, addr != end);
 
        if (err)
-               untrack_pfn_vma(vma, pfn, PAGE_ALIGN(size));
+               untrack_pfn_vma(vma, pfn, PAGE_ALIGN(size), 0, 0);
 
Quoted 6 lines

mm/memory.c int remap_pfn_range(struct vm_area_struct *vma, unsigned long addr, · line 1708

Quoted 2 lines
mm/memory.c +0 -0
        return err;
  }

2.6.29-git9 plus the above patch still doesn't fix my Dell E6400 running
Fedora 10 x86_64:

The patch here should get rid of the below msgs.
http://marc.info/?l=linux-kernel&m=123862741914199&w=2

Can you please test and check whether that resolves the problems?

Quoted 7 lines

The patch inline in this mail above was a test patch to debug the another problem that only happens with recent X and that one we are still debugging. So you can ignore the above patch for now.

Thanks,
Venki

Thanks Venki - the messages are gone, but I appear to be far from
having my graphical environment problems fixed...

- I still get

Apr  6 11:59:39 localhost kernel: [drm] Initialized drm 1.1.0 20060810
Apr  6 11:59:39 localhost kernel: pci 0000:00:02.0: PCI INT A -> GSI
16 (level, low) -> IRQ 16
Apr  6 11:59:39 localhost kernel: mtrr: type mismatch for
e0000000,10000000 old: write-back new: write-combining
Apr  6 11:59:39 localhost kernel: [drm] MTRR allocation failed.
Graphics performance may suffer.
Apr  6 11:59:39 localhost kernel: [drm] Initialized i915 1.6.0
20080730 for 0000:00:02.0 on minor 0
Apr  6 11:59:39 localhost kernel: mtrr: type mismatch for

e0000000,10000000 old: write-back new: write-combining

- I can't switch from X to VT (Alt-Fn) as my screen gets blank, no Caps Lock key led flashing when I hit the key, nothing else than a poweroff works

- I can't exit from X to VT as my screen gets flashing and scrolls what appears to be a prompt on the top of the screen instead of the bottom

F10 kernels behave the same way. 2.6.29.1 as well. Will test
2.6.28 or earlier and report in a separate thread with full config details.

Thanks,

--alessandro

"Sun keeps rising in the west / I keep on waking fully confused"

(The Replacements, "Within Your Reach")

#19 lore raw
Quoted 47 lines

-----Original Message-----
From: Alessandro Suardi [mailto:[email protected]]
Sent: Monday, April 06, 2009 8:37 AM
To: Pallipadi, Venkatesh
Cc: Arkadiusz Miskiewicz; Siddha, Suresh B; [email protected]; Jesse Barnes
Subject: Re: 2.6.29 git master and PAT problems

Quoted 6 lines · Venkatesh

The patch here should get rid of the below msgs.
http://marc.info/?l=linux-kernel&m=123862741914199&w=2

Can you please test and check whether that resolves the problems?

Quoted 7 lines

The patch inline in this mail above was a test patch to debug the another problem that only happens with recent X and that one we are still debugging. So you can ignore the above patch for now.

Thanks,
Venki

Thanks Venki - the messages are gone, but I appear to be far from having my graphical environment problems fixed...

- I still get

Apr  6 11:59:39 localhost kernel: [drm] Initialized drm 1.1.0 20060810
Apr  6 11:59:39 localhost kernel: pci 0000:00:02.0: PCI INT A -> GSI
16 (level, low) -> IRQ 16
Apr  6 11:59:39 localhost kernel: mtrr: type mismatch for
e0000000,10000000 old: write-back new: write-combining
Apr  6 11:59:39 localhost kernel: [drm] MTRR allocation failed.
Graphics performance may suffer.
Apr  6 11:59:39 localhost kernel: [drm] Initialized i915 1.6.0
20080730 for 0000:00:02.0 on minor 0
Apr  6 11:59:39 localhost kernel: mtrr: type mismatch for

e0000000,10000000 old: write-back new: write-combining

- I can't switch from X to VT (Alt-Fn) as my screen gets blank, no Caps Lock key led flashing when I hit the key, nothing else than a poweroff works

Quoted 8 lines

- I can't exit from X to VT as my screen gets flashing and scrolls what appears to be a prompt on the top of the screen instead of the bottom

F10 kernels behave the same way. 2.6.29.1 as well. Will test 2.6.28 or earlier and report in a separate thread with full config details.

Also test with and without CONFIG_DRM_* for your graphics card.

MTRR messages looks like are due to the conflict between BIOS MTRR setting and what X is trying to set. Booting into run level 3 and output of cat /proc/mtrr would be useful data as well.

Thanks,
Venki

#21 lore raw
In reply to a message not in the archive
Quoted 53 lines · Arkadiusz Miskiewicz
Quoted 44 lines · Venkatesh
Quoted 34 lines · Venkatesh
Quoted 19 lines · Arkadiusz Miskiewicz

On Monday 30 of March 2009, Pallipadi, Venkatesh wrote:

More info follows. Now I've switched to e1c502482853f84606928f5a2f2eb6da1993cda1 which contains latest drm fixes and now I get much lower numbers of PAT errors but still.

Quoted 13 lines · Arkadiusz Miskiewicz
Quoted 11 lines · Venkatesh
Quoted 5 lines

Patch here should get rid of these errors.

http://marc.info/?l=linux-kernel&m=123788806506230&w=2

The patch is in tip and on its way to upstream.

The problem is that kernel I'm running already contains this patch (it's merged already). Other ideas?

ratelimiting that error is good IMO anyway.

Quoted 13 lines
Quoted 3 lines

Rate limiting will just work around the problem here. Ideally we should never see these errors. So, it will be better if we can narrow down on the bug resulting in these error messages.

Of course it's better. I'm saying that when these messages "fire" then it's hard to do anything else on the system for a while until these stop.

Quoted 5 lines

Can you please send me the output of

# cat /debug/x86/pat_memtype_list
with debugfs mounted.
and
# cat /proc/mtrr
Quoted 8 lines

There seems to be two different problems here.
- We should not have that many single page ranges reserved. That will cause a performance problem with drm even without the "freeing invalid type" error. - "freeing invalid type" error itself. Seems to be caused due to some unbalanced free along the drm path. We tried to find anything obvious in the code that may be causing problem here. But, haven't found anything so far. Will try to reproduce the problem internally and debug it further.

Quoted 3 lines

OK. I think we have root caused the thinko that was resulting in "freeing invalid type" error. Can you try the below test patch. Patch is not final version and may need some cleanup.

Was testing on linus git as of today +
[PATCH] x86, PAT: Remove duplicate memtype reserve in pci mmap
+ patch from this thread.

Quoted 5 lines

It doesn't fix the problem. At least I'm able to reproduce "Freeint invalid memtype" by just running http://www.tremulous.net/ game. It also happened when watching youtube with opera. _Maybe_ things are little better because number of "freeing invalid memtype" messages is much lower than before but there is posibility that I simply didn't trigger it fully.

Arkadiusz,

I was finally able to reproduce the problem of "freeing invalid memtype" with upstream git kernel (commit 0221c81b1b) + latest xf86 intel driver. But, with upstream + the patch I had sent you earlier in this thread
(http://marc.info/?l=linux-kernel&m=123863345520617&w=2) I don't see those freeing invalid memtype errors anymore.

Can you please double check with current git and that patch and let me know if you are still seeing the problem.

Thanks,
Venki

#22 lore raw
Quoted 63 lines · Venkatesh
Quoted 53 lines · Arkadiusz Miskiewicz
Quoted 44 lines · Venkatesh
Quoted 34 lines · Venkatesh
Quoted 19 lines · Arkadiusz Miskiewicz

On Monday 30 of March 2009, Pallipadi, Venkatesh wrote:

More info follows. Now I've switched to e1c502482853f84606928f5a2f2eb6da1993cda1 which contains latest drm fixes and now I get much lower numbers of PAT errors but still.

Quoted 13 lines · Arkadiusz Miskiewicz
Quoted 11 lines · Venkatesh
Quoted 5 lines

Patch here should get rid of these errors.

http://marc.info/?l=linux-kernel&m=123788806506230&w=2

The patch is in tip and on its way to upstream.

The problem is that kernel I'm running already contains this patch (it's merged already). Other ideas?

ratelimiting that error is good IMO anyway.

Quoted 13 lines
Quoted 3 lines

Rate limiting will just work around the problem here. Ideally we should never see these errors. So, it will be better if we can narrow down on the bug resulting in these error messages.

Of course it's better. I'm saying that when these messages "fire" then it's hard to do anything else on the system for a while until these stop.

Quoted 5 lines

Can you please send me the output of

# cat /debug/x86/pat_memtype_list
with debugfs mounted.
and
# cat /proc/mtrr
Quoted 8 lines

There seems to be two different problems here.
- We should not have that many single page ranges reserved. That will cause a performance problem with drm even without the "freeing invalid type" error. - "freeing invalid type" error itself. Seems to be caused due to some unbalanced free along the drm path. We tried to find anything obvious in the code that may be causing problem here. But, haven't found anything so far. Will try to reproduce the problem internally and debug it further.

Quoted 3 lines

OK. I think we have root caused the thinko that was resulting in "freeing invalid type" error. Can you try the below test patch. Patch is not final version and may need some cleanup.

Was testing on linus git as of today +
[PATCH] x86, PAT: Remove duplicate memtype reserve in pci mmap
+ patch from this thread.

Quoted 6 lines

It doesn't fix the problem. At least I'm able to reproduce "Freeint invalid memtype" by just running http://www.tremulous.net/ game. It also happened when watching youtube with opera. _Maybe_ things are little better because number of "freeing invalid memtype" messages is much lower than before but there is posibility that I simply didn't trigger it fully.

Arkadiusz,

Quoted 8 lines

I was finally able to reproduce the problem of "freeing invalid memtype" with upstream git kernel (commit 0221c81b1b) + latest xf86 intel driver. But, with upstream + the patch I had sent you earlier in this thread
(http://marc.info/?l=linux-kernel&m=123863345520617&w=2) I don't see those freeing invalid memtype errors anymore.

Can you please double check with current git and that patch and let me know if you are still seeing the problem.

Latest linus tree + that patch (it's really applied here), xserver 1.6, libdrm from git master, intel driver from git master, previously mesa 7.4 (and 7.5 snap currently), tremolous.net 1.1.0 game (tremolous-smp binary), GM45 gpu.

To reproduce I just need to run tremolous-smp and connect to some map. When map finishes loading I instantly get:

      1 [  132.341378] tremulous-smp:5554 freeing invalid memtype d570d000-
d570e000
      1 [  132.341394] tremulous-smp:5554 freeing invalid memtype d570e000-
d570f000
      1 [  132.341409] tremulous-smp:5554 freeing invalid memtype d570f000-
d5710000
      1 [  139.323677] X:5238 freeing invalid memtype d6168000-d6169000            
      1 [  139.323698] X:5238 freeing invalid memtype d6169000-d616a000            
      1 [  139.323722] X:5238 freeing invalid memtype d616a000-d616b000            
      1 [  139.323742] X:5238 freeing invalid memtype d616b000-d616c000       
$ dmesg|grep "freeing invalid" | wc -l
6643
Quoted 2 lines

Thanks,
Venki

Signature · 2 lines
Arkadiusz Miśkiewicz        PLD/Linux Team
arekm / maven.pl            http://ftp.pld-linux.org/
#23 lore raw
Quoted 30 lines · Arkadiusz Miskiewicz
Quoted 10 lines · Venkatesh

On Thu, 2009-04-02 at 00:12 -0700, Arkadiusz Miskiewicz wrote:

I was finally able to reproduce the problem of "freeing invalid memtype" with upstream git kernel (commit 0221c81b1b) + latest xf86 intel driver. But, with upstream + the patch I had sent you earlier in this thread
(http://marc.info/?l=linux-kernel&m=123863345520617&w=2) I don't see those freeing invalid memtype errors anymore.

Can you please double check with current git and that patch and let me know if you are still seeing the problem.

Latest linus tree + that patch (it's really applied here), xserver 1.6, libdrm from git master, intel driver from git master, previously mesa 7.4 (and 7.5 snap currently), tremolous.net 1.1.0 game (tremolous-smp binary), GM45 gpu.

To reproduce I just need to run tremolous-smp and connect to some map. When map finishes loading I instantly get:

      1 [  132.341378] tremulous-smp:5554 freeing invalid memtype d570d000-
d570e000
      1 [  132.341394] tremulous-smp:5554 freeing invalid memtype d570e000-
d570f000
      1 [  132.341409] tremulous-smp:5554 freeing invalid memtype d570f000-
d5710000
      1 [  139.323677] X:5238 freeing invalid memtype d6168000-d6169000            
      1 [  139.323698] X:5238 freeing invalid memtype d6169000-d616a000            
      1 [  139.323722] X:5238 freeing invalid memtype d616a000-d616b000            
      1 [  139.323742] X:5238 freeing invalid memtype d616b000-d616c000       
Quoted 3 lines
$ dmesg|grep "freeing invalid" | wc -l
6643

OK. One more test patch below, applies over linus's git and you can ignore the earlier patch. The patch below should get rid of the problem and as it removes the track/untrack of vm_insert_pfn completely. This will also eliminate the overhead of hundreds or thousands of entries in pat_memtype_list. Can you please test it.

Thanks,
Venki

---
 arch/x86/mm/pat.c |  124 +++++++++++------------------------------------------
 mm/memory.c       |   14 +-----
 2 files changed, 29 insertions(+), 109 deletions(-)
arch/x86/mm/pat.c +26 -98
index 640339e..5992af2 100644
@@ -732,15 +732,11 @@ static void free_pfn_range(u64 paddr, unsigned long size)
  * track_pfn_vma_copy is called when vma that is covering the pfnmap gets
  * copied through copy_page_range().
  *
- * If the vma has a linear pfn mapping for the entire range, we get the prot
- * from pte and reserve the entire vma range with single reserve_pfn_range call.
- * Otherwise, we reserve the entire vma range, my ging through the PTEs page
- * by page to get physical address and protection.
+ * We get the prot from pte and reserve the entire vma range with single
+ * reserve_pfn_range call.
  */
 int track_pfn_vma_copy(struct vm_area_struct *vma)
 {
-	int retval = 0;
-	unsigned long i, j;
 	resource_size_t paddr;
 	unsigned long prot;
 	unsigned long vma_start = vma->vm_start;
@@ -751,94 +747,35 @@ int track_pfn_vma_copy(struct vm_area_struct *vma)
 	if (!pat_enabled)
 		return 0;
 
-	if (is_linear_pfn_mapping(vma)) {
-		/*
-		 * reserve the whole chunk covered by vma. We need the
-		 * starting address and protection from pte.
-		 */
-		if (follow_phys(vma, vma_start, 0, &prot, &paddr)) {
-			WARN_ON_ONCE(1);
-			return -EINVAL;
-		}
-		pgprot = __pgprot(prot);
-		return reserve_pfn_range(paddr, vma_size, &pgprot, 1);
-	}
-
-	/* reserve entire vma page by page, using pfn and prot from pte */
-	for (i = 0; i < vma_size; i += PAGE_SIZE) {
-		if (follow_phys(vma, vma_start + i, 0, &prot, &paddr))
-			continue;
-
-		pgprot = __pgprot(prot);
-		retval = reserve_pfn_range(paddr, PAGE_SIZE, &pgprot, 1);
-		if (retval)
-			goto cleanup_ret;
-	}
-	return 0;
-
-cleanup_ret:
-	/* Reserve error: Cleanup partial reservation and return error */
-	for (j = 0; j < i; j += PAGE_SIZE) {
-		if (follow_phys(vma, vma_start + j, 0, &prot, &paddr))
-			continue;
-
-		free_pfn_range(paddr, PAGE_SIZE);
+	/*
+	 * reserve the whole chunk covered by vma. We need the
+	 * starting address and protection from pte.
+	 */
+	if (follow_phys(vma, vma_start, 0, &prot, &paddr)) {
+		WARN_ON_ONCE(1);
+		return -EINVAL;
 	}
-
-	return retval;
+	pgprot = __pgprot(prot);
+	return reserve_pfn_range(paddr, vma_size, &pgprot, 1);
 }
 
 /*
  * track_pfn_vma_new is called when a _new_ pfn mapping is being established
  * for physical range indicated by pfn and size.
  *
- * prot is passed in as a parameter for the new mapping. If the vma has a
- * linear pfn mapping for the entire range reserve the entire vma range with
- * single reserve_pfn_range call.
- * Otherwise, we look t the pfn and size and reserve only the specified range
- * page by page.
- *
- * Note that this function can be called with caller trying to map only a
- * subrange/page inside the vma.
+ * prot is passed in as a parameter for the new mapping.
  */
 int track_pfn_vma_new(struct vm_area_struct *vma, pgprot_t *prot,
 			unsigned long pfn, unsigned long size)
 {
-	int retval = 0;
-	unsigned long i, j;
-	resource_size_t base_paddr;
 	resource_size_t paddr;
-	unsigned long vma_start = vma->vm_start;
-	unsigned long vma_end = vma->vm_end;
-	unsigned long vma_size = vma_end - vma_start;
 
 	if (!pat_enabled)
 		return 0;
 
-	if (is_linear_pfn_mapping(vma)) {
-		/* reserve the whole chunk starting from vm_pgoff */
-		paddr = (resource_size_t)vma->vm_pgoff << PAGE_SHIFT;
-		return reserve_pfn_range(paddr, vma_size, prot, 0);
-	}
-
-	/* reserve page by page using pfn and size */
-	base_paddr = (resource_size_t)pfn << PAGE_SHIFT;
-	for (i = 0; i < size; i += PAGE_SIZE) {
-		paddr = base_paddr + i;
-		retval = reserve_pfn_range(paddr, PAGE_SIZE, prot, 0);
-		if (retval)
-			goto cleanup_ret;
-	}
-	return 0;
-
-cleanup_ret:
-	/* Reserve error: Cleanup partial reservation and return error */
-	for (j = 0; j < i; j += PAGE_SIZE) {
-		paddr = base_paddr + j;
-		free_pfn_range(paddr, PAGE_SIZE);
-	}
-
-	return retval;
+	/* reserve the whole chunk starting from pfn */
+	paddr = (resource_size_t)pfn << PAGE_SHIFT;
+	return reserve_pfn_range(paddr, vma->vm_end - vma->vm_start, prot, 0);
 }
 
 /*
@@ -849,7 +786,6 @@ cleanup_ret:
 void untrack_pfn_vma(struct vm_area_struct *vma, unsigned long pfn,
 			unsigned long size)
 {
-	unsigned long i;
 	resource_size_t paddr;
 	unsigned long prot;
 	unsigned long vma_start = vma->vm_start;
@@ -859,29 +795,21 @@ void untrack_pfn_vma(struct vm_area_struct *vma, unsigned long pfn,
 	if (!pat_enabled)
 		return;
 
-	if (is_linear_pfn_mapping(vma)) {
-		/* free the whole chunk starting from vm_pgoff */
-		paddr = (resource_size_t)vma->vm_pgoff << PAGE_SHIFT;
-		free_pfn_range(paddr, vma_size);
+	if (pfn) {
+		paddr = (resource_size_t)pfn << PAGE_SHIFT;
+		free_pfn_range(paddr, size);
 		return;
 	}
 
-	if (size != 0 && size != vma_size) {
-		/* free page by page, using pfn and size */
-		paddr = (resource_size_t)pfn << PAGE_SHIFT;
-		for (i = 0; i < size; i += PAGE_SIZE) {
-			paddr = paddr + i;
-			free_pfn_range(paddr, PAGE_SIZE);
-		}
-	} else {
-		/* free entire vma, page by page, using the pfn from pte */
-		for (i = 0; i < vma_size; i += PAGE_SIZE) {
-			if (follow_phys(vma, vma_start + i, 0, &prot, &paddr))
-				continue;
-
-			free_pfn_range(paddr, PAGE_SIZE);
-		}
+	/*
+	 * reserve the whole chunk covered by vma. We need the
+	 * starting address pte.
+	 */
+	if (follow_phys(vma, vma_start, 0, &prot, &paddr)) {
+		WARN_ON_ONCE(1);
+		return;
 	}
+	free_pfn_range(paddr, vma_size);
 }
 
 pgprot_t pgprot_writecombine(pgprot_t prot)
mm/memory.c +3 -11
index cf6873e..4cae7e0 100644
@@ -719,7 +719,7 @@ int copy_page_range(struct mm_struct *dst_mm, struct mm_struct *src_mm,
 	if (is_vm_hugetlb_page(vma))
 		return copy_hugetlb_page_range(dst_mm, src_mm, vma);
 
-	if (unlikely(is_pfn_mapping(vma))) {
+	if (unlikely(is_linear_pfn_mapping(vma))) {
 		/*
 		 * We do not free on error cases below as remove_vma
 		 * gets called on error from higher level routine
@@ -982,7 +982,7 @@ unsigned long unmap_vmas(struct mmu_gather **tlbp,
 		if (vma->vm_flags & VM_ACCOUNT)
 			*nr_accounted += (end - start) >> PAGE_SHIFT;
 
-		if (unlikely(is_pfn_mapping(vma)))
+		if (unlikely(is_linear_pfn_mapping(vma)))
 			untrack_pfn_vma(vma, 0, 0);
 
 		while (start != end) {
@@ -1515,7 +1515,6 @@ out:
 int vm_insert_pfn(struct vm_area_struct *vma, unsigned long addr,
 			unsigned long pfn)
 {
-	int ret;
 	pgprot_t pgprot = vma->vm_page_prot;
 	/*
 	 * Technically, architectures with pte_special can avoid all these
@@ -1531,15 +1530,8 @@ int vm_insert_pfn(struct vm_area_struct *vma, unsigned long addr,
 
 	if (addr < vma->vm_start || addr >= vma->vm_end)
 		return -EFAULT;
-	if (track_pfn_vma_new(vma, &pgprot, pfn, PAGE_SIZE))
-		return -EINVAL;
-
-	ret = insert_pfn(vma, addr, pfn, pgprot);
 
-	if (ret)
-		untrack_pfn_vma(vma, pfn, PAGE_SIZE);
-
-	return ret;
+	return insert_pfn(vma, addr, pfn, pgprot);
 }
 EXPORT_SYMBOL(vm_insert_pfn);
 
Signature · 1 line
1.6.0.6
#24 lore raw
Quoted 20 lines · Venkatesh
Quoted 19 lines · Arkadiusz Miskiewicz
Quoted 10 lines · Venkatesh

On Thu, 2009-04-02 at 00:12 -0700, Arkadiusz Miskiewicz wrote:

I was finally able to reproduce the problem of "freeing invalid memtype" with upstream git kernel (commit 0221c81b1b) + latest xf86 intel driver. But, with upstream + the patch I had sent you earlier in this thread (http://marc.info/?l=linux-kernel&m=123863345520617&w=2) I don't see those freeing invalid memtype errors anymore.

Can you please double check with current git and that patch and let me know if you are still seeing the problem.

Latest linus tree + that patch (it's really applied here), xserver 1.6, libdrm from git master, intel driver from git master, previously mesa 7.4 (and 7.5 snap currently), tremolous.net 1.1.0 game (tremolous-smp binary), GM45 gpu.

To reproduce I just need to run tremolous-smp and connect to some map. When map finishes loading I instantly get:

[...]

Quoted 5 lines

OK. One more test patch below, applies over linus's git and you can ignore the earlier patch. The patch below should get rid of the problem and as it removes the track/untrack of vm_insert_pfn completely. This will also eliminate the overhead of hundreds or thousands of entries in pat_memtype_list. Can you please test it.

With this patch I'm no longer able to reproduce problem. Thanks!

Things look like this now:

# cat /debug/x86/pat_memtype_list 
PAT memtype list:                              
uncached-minus @ 0xbd6d1000-0xbd6d2000         
uncached-minus @ 0xbd6d2000-0xbd6d3000
uncached-minus @ 0xbd6d3000-0xbd6d4000
uncached-minus @ 0xbd706000-0xbd707000
uncached-minus @ 0xbd707000-0xbd708000
uncached-minus @ 0xbd96a000-0xbd96b000
uncached-minus @ 0xbd96a000-0xbd96b000
uncached-minus @ 0xbd96a000-0xbd97a000
uncached-minus @ 0xbd979000-0xbd97a000
uncached-minus @ 0xbd979000-0xbd97a000
uncached-minus @ 0xbd979000-0xbd97a000
uncached-minus @ 0xbd979000-0xbd97a000
uncached-minus @ 0xbd979000-0xbd97a000
uncached-minus @ 0xbd979000-0xbd97a000
uncached-minus @ 0xbd979000-0xbd97a000
uncached-minus @ 0xbd97b000-0xbd97c000
uncached-minus @ 0xbd98b000-0xbd98d000
uncached-minus @ 0xbd98b000-0xbd98e000
uncached-minus @ 0xbd98d000-0xbd98e000
uncached-minus @ 0xbd98e000-0xbd98f000
uncached-minus @ 0xbd98e000-0xbd98f000
uncached-minus @ 0xc2000000-0xc2001000
write-combining @ 0xd0000000-0xe0000000
write-combining @ 0xd2000000-0xd2020000
write-combining @ 0xd2020000-0xd2512000
uncached-minus @ 0xe0000000-0xe4000000
uncached-minus @ 0xf4400000-0xf4800000
uncached-minus @ 0xf4400000-0xf4480000
uncached-minus @ 0xf4600000-0xf4800000
uncached-minus @ 0xf4800000-0xf4801000
uncached-minus @ 0xf4801000-0xf4802000
uncached-minus @ 0xf4801000-0xf4802000
uncached-minus @ 0xfc000000-0xfc020000
uncached-minus @ 0xfc020000-0xfc024000
uncached-minus @ 0xfc025000-0xfc026000
uncached-minus @ 0xfc226000-0xfc227000
uncached-minus @ 0xfc226000-0xfc227000
uncached-minus @ 0xfc227000-0xfc228000
uncached-minus @ 0xfed00000-0xfed01000
uncached-minus @ 0xfed1f000-0xfed20000
# cat /proc/mtrr
reg00: base=0x13c000000 ( 5056MB), size=   64MB, count=1: uncachable
reg01: base=0x0be000000 ( 3040MB), size=   32MB, count=1: uncachable
reg02: base=0x000000000 (    0MB), size= 2048MB, count=1: write-back
reg03: base=0x080000000 ( 2048MB), size= 1024MB, count=1: write-back
reg04: base=0x100000000 ( 4096MB), size= 1024MB, count=1: write-back
reg05: base=0x0bde00000 ( 3038MB), size=    2MB, count=1: uncachable
reg06: base=0x0d0000000 ( 3328MB), size=  256MB, count=1: write-combining
Quoted 237 lines

Thanks,
Venki

---
 arch/x86/mm/pat.c |  124
+++++++++++------------------------------------------ mm/memory.c       |  
14 +-----
 2 files changed, 29 insertions(+), 109 deletions(-)
arch/x86/mm/pat.c +0 -0
index 640339e..5992af2 100644
@@ -732,15 +732,11 @@ static void free_pfn_range(u64 paddr, unsigned long

size) * track_pfn_vma_copy is called when vma that is covering the pfnmap gets * copied through copy_page_range().
*
- * If the vma has a linear pfn mapping for the entire range, we get the prot - * from pte and reserve the entire vma range with single reserve_pfn_range call. - * Otherwise, we reserve the entire vma range, my ging through the PTEs page - * by page to get physical address and protection.
+ * We get the prot from pte and reserve the entire vma range with single
+ * reserve_pfn_range call.
*/
int track_pfn_vma_copy(struct vm_area_struct *vma)
{

arch/x86/mm/pat.c +9 -36
-	int retval = 0;
-	unsigned long i, j;
 	resource_size_t paddr;
 	unsigned long prot;
 	unsigned long vma_start = vma->vm_start;
@@ -751,94 +747,35 @@ int track_pfn_vma_copy(struct vm_area_struct *vma)
 	if (!pat_enabled)
 		return 0;
 
-	if (is_linear_pfn_mapping(vma)) {
-		/*
-		 * reserve the whole chunk covered by vma. We need the
-		 * starting address and protection from pte.
-		 */
-		if (follow_phys(vma, vma_start, 0, &prot, &paddr)) {
-			WARN_ON_ONCE(1);
-			return -EINVAL;
-		}
-		pgprot = __pgprot(prot);
-		return reserve_pfn_range(paddr, vma_size, &pgprot, 1);
-	}
-
-	/* reserve entire vma page by page, using pfn and prot from pte */
-	for (i = 0; i < vma_size; i += PAGE_SIZE) {
-		if (follow_phys(vma, vma_start + i, 0, &prot, &paddr))
-			continue;
-
-		pgprot = __pgprot(prot);
-		retval = reserve_pfn_range(paddr, PAGE_SIZE, &pgprot, 1);
-		if (retval)
-			goto cleanup_ret;
-	}
-	return 0;
-
-cleanup_ret:
-	/* Reserve error: Cleanup partial reservation and return error */
-	for (j = 0; j < i; j += PAGE_SIZE) {
-		if (follow_phys(vma, vma_start + j, 0, &prot, &paddr))
-			continue;
-
-		free_pfn_range(paddr, PAGE_SIZE);
+	/*
+	 * reserve the whole chunk covered by vma. We need the
+	 * starting address and protection from pte.
+	 */
+	if (follow_phys(vma, vma_start, 0, &prot, &paddr)) {
+		WARN_ON_ONCE(1);
+		return -EINVAL;
 	}
-
-	return retval;
+	pgprot = __pgprot(prot);
+	return reserve_pfn_range(paddr, vma_size, &pgprot, 1);
 }
 
 /*
  * track_pfn_vma_new is called when a _new_ pfn mapping is being

established * for physical range indicated by pfn and size.
*
- * prot is passed in as a parameter for the new mapping. If the vma has a
- * linear pfn mapping for the entire range reserve the entire vma range with - * single reserve_pfn_range call.
- * Otherwise, we look t the pfn and size and reserve only the specified range - * page by page.
- *
- * Note that this function can be called with caller trying to map only a
- * subrange/page inside the vma.
+ * prot is passed in as a parameter for the new mapping.
*/
int track_pfn_vma_new(struct vm_area_struct *vma, pgprot_t *prot,

arch/x86/mm/pat.c +14 -50
 			unsigned long pfn, unsigned long size)
 {
-	int retval = 0;
-	unsigned long i, j;
-	resource_size_t base_paddr;
 	resource_size_t paddr;
-	unsigned long vma_start = vma->vm_start;
-	unsigned long vma_end = vma->vm_end;
-	unsigned long vma_size = vma_end - vma_start;
 
 	if (!pat_enabled)
 		return 0;
 
-	if (is_linear_pfn_mapping(vma)) {
-		/* reserve the whole chunk starting from vm_pgoff */
-		paddr = (resource_size_t)vma->vm_pgoff << PAGE_SHIFT;
-		return reserve_pfn_range(paddr, vma_size, prot, 0);
-	}
-
-	/* reserve page by page using pfn and size */
-	base_paddr = (resource_size_t)pfn << PAGE_SHIFT;
-	for (i = 0; i < size; i += PAGE_SIZE) {
-		paddr = base_paddr + i;
-		retval = reserve_pfn_range(paddr, PAGE_SIZE, prot, 0);
-		if (retval)
-			goto cleanup_ret;
-	}
-	return 0;
-
-cleanup_ret:
-	/* Reserve error: Cleanup partial reservation and return error */
-	for (j = 0; j < i; j += PAGE_SIZE) {
-		paddr = base_paddr + j;
-		free_pfn_range(paddr, PAGE_SIZE);
-	}
-
-	return retval;
+	/* reserve the whole chunk starting from pfn */
+	paddr = (resource_size_t)pfn << PAGE_SHIFT;
+	return reserve_pfn_range(paddr, vma->vm_end - vma->vm_start, prot, 0);
 }
 
 /*
@@ -849,7 +786,6 @@ cleanup_ret:
 void untrack_pfn_vma(struct vm_area_struct *vma, unsigned long pfn,
 			unsigned long size)
 {
-	unsigned long i;
 	resource_size_t paddr;
 	unsigned long prot;
 	unsigned long vma_start = vma->vm_start;
@@ -859,29 +795,21 @@ void untrack_pfn_vma(struct vm_area_struct *vma,
 unsigned long pfn, if (!pat_enabled)
 		return;
 
-	if (is_linear_pfn_mapping(vma)) {
-		/* free the whole chunk starting from vm_pgoff */
-		paddr = (resource_size_t)vma->vm_pgoff << PAGE_SHIFT;
-		free_pfn_range(paddr, vma_size);
+	if (pfn) {
+		paddr = (resource_size_t)pfn << PAGE_SHIFT;
+		free_pfn_range(paddr, size);
 		return;
 	}
 
-	if (size != 0 && size != vma_size) {
-		/* free page by page, using pfn and size */
-		paddr = (resource_size_t)pfn << PAGE_SHIFT;
-		for (i = 0; i < size; i += PAGE_SIZE) {
-			paddr = paddr + i;
-			free_pfn_range(paddr, PAGE_SIZE);
-		}
-	} else {
-		/* free entire vma, page by page, using the pfn from pte */
-		for (i = 0; i < vma_size; i += PAGE_SIZE) {
-			if (follow_phys(vma, vma_start + i, 0, &prot, &paddr))
-				continue;
-
-			free_pfn_range(paddr, PAGE_SIZE);
-		}
+	/*
+	 * reserve the whole chunk covered by vma. We need the
+	 * starting address pte.
+	 */
+	if (follow_phys(vma, vma_start, 0, &prot, &paddr)) {
+		WARN_ON_ONCE(1);
+		return;
 	}
+	free_pfn_range(paddr, vma_size);
 }
 
 pgprot_t pgprot_writecombine(pgprot_t prot)
mm/memory.c +3 -11
index cf6873e..4cae7e0 100644
@@ -719,7 +719,7 @@ int copy_page_range(struct mm_struct *dst_mm, struct
 mm_struct *src_mm, if (is_vm_hugetlb_page(vma))
 		return copy_hugetlb_page_range(dst_mm, src_mm, vma);
 
-	if (unlikely(is_pfn_mapping(vma))) {
+	if (unlikely(is_linear_pfn_mapping(vma))) {
 		/*
 		 * We do not free on error cases below as remove_vma
 		 * gets called on error from higher level routine
@@ -982,7 +982,7 @@ unsigned long unmap_vmas(struct mmu_gather **tlbp,
 		if (vma->vm_flags & VM_ACCOUNT)
 			*nr_accounted += (end - start) >> PAGE_SHIFT;
 
-		if (unlikely(is_pfn_mapping(vma)))
+		if (unlikely(is_linear_pfn_mapping(vma)))
 			untrack_pfn_vma(vma, 0, 0);
 
 		while (start != end) {
@@ -1515,7 +1515,6 @@ out:
 int vm_insert_pfn(struct vm_area_struct *vma, unsigned long addr,
 			unsigned long pfn)
 {
-	int ret;
 	pgprot_t pgprot = vma->vm_page_prot;
 	/*
 	 * Technically, architectures with pte_special can avoid all these
@@ -1531,15 +1530,8 @@ int vm_insert_pfn(struct vm_area_struct *vma,
 unsigned long addr,
 
 	if (addr < vma->vm_start || addr >= vma->vm_end)
 		return -EFAULT;
-	if (track_pfn_vma_new(vma, &pgprot, pfn, PAGE_SIZE))
-		return -EINVAL;
-
-	ret = insert_pfn(vma, addr, pfn, pgprot);
 
-	if (ret)
-		untrack_pfn_vma(vma, pfn, PAGE_SIZE);
-
-	return ret;
+	return insert_pfn(vma, addr, pfn, pgprot);
 }
 EXPORT_SYMBOL(vm_insert_pfn);
Signature · 2 lines
Arkadiusz Miśkiewicz        PLD/Linux Team
arekm / maven.pl            http://ftp.pld-linux.org/
#25 lore raw
Quoted 23 lines · Arkadiusz Miskiewicz
Quoted 20 lines · Venkatesh
Quoted 19 lines · Arkadiusz Miskiewicz
Quoted 10 lines · Venkatesh

On Thu, 2009-04-02 at 00:12 -0700, Arkadiusz Miskiewicz wrote:

I was finally able to reproduce the problem of "freeing invalid memtype" with upstream git kernel (commit 0221c81b1b) + latest xf86 intel driver. But, with upstream + the patch I had sent you earlier in this thread (http://marc.info/?l=linux-kernel&m=123863345520617&w=2) I don't see those freeing invalid memtype errors anymore.

Can you please double check with current git and that patch and let me know if you are still seeing the problem.

Latest linus tree + that patch (it's really applied here), xserver 1.6, libdrm from git master, intel driver from git master, previously mesa 7.4 (and 7.5 snap currently), tremolous.net 1.1.0 game (tremolous-smp binary), GM45 gpu.

To reproduce I just need to run tremolous-smp and connect to some map. When map finishes loading I instantly get:

[...]

Quoted 7 lines
Quoted 5 lines

OK. One more test patch below, applies over linus's git and you can ignore the earlier patch. The patch below should get rid of the problem and as it removes the track/untrack of vm_insert_pfn completely. This will also eliminate the overhead of hundreds or thousands of entries in pat_memtype_list. Can you please test it.

With this patch I'm no longer able to reproduce problem. Thanks!

Great, thanks!

Venki, mind sending a patch with a proper changelog, Reported-by, Tested-by tags, with Nick and Andrew Cc:-ed for the memory.c bits, etc.?

Ingo

Tested-by Arkadiusz Miskiewicz
#26 lore raw
Quoted 32 lines · Ingo Molnar
Quoted 29 lines · Arkadiusz Miskiewicz
Quoted 20 lines · Venkatesh
Quoted 19 lines · Arkadiusz Miskiewicz
Quoted 10 lines · Venkatesh

On Thu, 2009-04-02 at 00:12 -0700, Arkadiusz Miskiewicz wrote:

I was finally able to reproduce the problem of "freeing invalid memtype" with upstream git kernel (commit 0221c81b1b) + latest xf86 intel driver. But, with upstream + the patch I had sent you earlier in this thread (http://marc.info/?l=linux-kernel&m=123863345520617&w=2) I don't see those freeing invalid memtype errors anymore.

Can you please double check with current git and that patch and let me know if you are still seeing the problem.

Latest linus tree + that patch (it's really applied here), xserver 1.6, libdrm from git master, intel driver from git master, previously mesa 7.4 (and 7.5 snap currently), tremolous.net 1.1.0 game (tremolous-smp binary), GM45 gpu.

To reproduce I just need to run tremolous-smp and connect to some map. When map finishes loading I instantly get:

[...]

Quoted 5 lines

OK. One more test patch below, applies over linus's git and you can ignore the earlier patch. The patch below should get rid of the problem and as it removes the track/untrack of vm_insert_pfn completely. This will also eliminate the overhead of hundreds or thousands of entries in pat_memtype_list. Can you please test it.

Quoted 7 lines
Quoted 1 line

With this patch I'm no longer able to reproduce problem. Thanks!

Great, thanks!

Venki, mind sending a patch with a proper changelog, Reported-by, Tested-by tags, with Nick and Andrew Cc:-ed for the memory.c bits, etc.?

Ingo,

Below is the cleaner version of the patch. It does not have any changes in mm as we are only removing the tracking inside the PAT code. I have left the generic mm interface as is, until we fully resolve the issue in future.

Thanks,
Venki

Subject: [PATCH] x86, PAT: Remove page granularity tracking for vm_insert_pfn maps

Remove page level granularity track and untrack of vm_insert_pfn. memtype tracking at page granularity does not scale and cleaner approach would be for the driver to request a type for a bigger IO address range or PCI io memory range for that device, either at mmap time or driver init time and just use that type during vm_insert_pfn.

This patch just removes the track/untrack of vm_insert_pfn. That means we will be in same state as 2.6.28, with respect to these APIs. Newer APIs for the drivers to request a memtype for a bigger region is TBD and coming soon.

This change resolves the problem of too many single page entries in pat_memtype_list and "freeing invalid memtype" errors with i915, reported here.
http://marc.info/?l=linux-kernel&m=123845244713183&w=2

---
 arch/x86/mm/pat.c |   98 ++++++++++------------------------------------------
 1 files changed, 19 insertions(+), 79 deletions(-)
arch/x86/mm/pat.c +19 -79
index 640339e..a8365c8 100644
@@ -734,29 +734,28 @@ static void free_pfn_range(u64 paddr, unsigned long size)
  *
  * If the vma has a linear pfn mapping for the entire range, we get the prot
  * from pte and reserve the entire vma range with single reserve_pfn_range call.
- * Otherwise, we reserve the entire vma range, my ging through the PTEs page
- * by page to get physical address and protection.
  */
 int track_pfn_vma_copy(struct vm_area_struct *vma)
 {
-	int retval = 0;
-	unsigned long i, j;
 	resource_size_t paddr;
 	unsigned long prot;
-	unsigned long vma_start = vma->vm_start;
-	unsigned long vma_end = vma->vm_end;
-	unsigned long vma_size = vma_end - vma_start;
+	unsigned long vma_size = vma->vm_end - vma->vm_start;
 	pgprot_t pgprot;
 
 	if (!pat_enabled)
 		return 0;
 
+	/*
+	 * For now, only handle remap_pfn_range() vmas where
+	 * is_linear_pfn_mapping() == TRUE. Handling of
+	 * vm_insert_pfn() is TBD.
+	 */
 	if (is_linear_pfn_mapping(vma)) {
 		/*
 		 * reserve the whole chunk covered by vma. We need the
 		 * starting address and protection from pte.
 		 */
-		if (follow_phys(vma, vma_start, 0, &prot, &paddr)) {
+		if (follow_phys(vma, vma->vm_start, 0, &prot, &paddr)) {
 			WARN_ON_ONCE(1);
 			return -EINVAL;
 		}
@@ -764,28 +763,7 @@ int track_pfn_vma_copy(struct vm_area_struct *vma)
 		return reserve_pfn_range(paddr, vma_size, &pgprot, 1);
 	}
 
-	/* reserve entire vma page by page, using pfn and prot from pte */
-	for (i = 0; i < vma_size; i += PAGE_SIZE) {
-		if (follow_phys(vma, vma_start + i, 0, &prot, &paddr))
-			continue;
-
-		pgprot = __pgprot(prot);
-		retval = reserve_pfn_range(paddr, PAGE_SIZE, &pgprot, 1);
-		if (retval)
-			goto cleanup_ret;
-	}
 	return 0;
-
-cleanup_ret:
-	/* Reserve error: Cleanup partial reservation and return error */
-	for (j = 0; j < i; j += PAGE_SIZE) {
-		if (follow_phys(vma, vma_start + j, 0, &prot, &paddr))
-			continue;
-
-		free_pfn_range(paddr, PAGE_SIZE);
-	}
-
-	return retval;
 }
 
 /*
@@ -795,50 +773,28 @@ cleanup_ret:
  * prot is passed in as a parameter for the new mapping. If the vma has a
  * linear pfn mapping for the entire range reserve the entire vma range with
  * single reserve_pfn_range call.
- * Otherwise, we look t the pfn and size and reserve only the specified range
- * page by page.
- *
- * Note that this function can be called with caller trying to map only a
- * subrange/page inside the vma.
  */
 int track_pfn_vma_new(struct vm_area_struct *vma, pgprot_t *prot,
 			unsigned long pfn, unsigned long size)
 {
-	int retval = 0;
-	unsigned long i, j;
-	resource_size_t base_paddr;
 	resource_size_t paddr;
-	unsigned long vma_start = vma->vm_start;
-	unsigned long vma_end = vma->vm_end;
-	unsigned long vma_size = vma_end - vma_start;
+	unsigned long vma_size = vma->vm_end - vma->vm_start;
 
 	if (!pat_enabled)
 		return 0;
 
+	/*
+	 * For now, only handle remap_pfn_range() vmas where
+	 * is_linear_pfn_mapping() == TRUE. Handling of
+	 * vm_insert_pfn() is TBD.
+	 */
 	if (is_linear_pfn_mapping(vma)) {
 		/* reserve the whole chunk starting from vm_pgoff */
 		paddr = (resource_size_t)vma->vm_pgoff << PAGE_SHIFT;
 		return reserve_pfn_range(paddr, vma_size, prot, 0);
 	}
 
-	/* reserve page by page using pfn and size */
-	base_paddr = (resource_size_t)pfn << PAGE_SHIFT;
-	for (i = 0; i < size; i += PAGE_SIZE) {
-		paddr = base_paddr + i;
-		retval = reserve_pfn_range(paddr, PAGE_SIZE, prot, 0);
-		if (retval)
-			goto cleanup_ret;
-	}
 	return 0;
-
-cleanup_ret:
-	/* Reserve error: Cleanup partial reservation and return error */
-	for (j = 0; j < i; j += PAGE_SIZE) {
-		paddr = base_paddr + j;
-		free_pfn_range(paddr, PAGE_SIZE);
-	}
-
-	return retval;
 }
 
 /*
@@ -849,39 +805,23 @@ cleanup_ret:
 void untrack_pfn_vma(struct vm_area_struct *vma, unsigned long pfn,
 			unsigned long size)
 {
-	unsigned long i;
 	resource_size_t paddr;
-	unsigned long prot;
-	unsigned long vma_start = vma->vm_start;
-	unsigned long vma_end = vma->vm_end;
-	unsigned long vma_size = vma_end - vma_start;
+	unsigned long vma_size = vma->vm_end - vma->vm_start;
 
 	if (!pat_enabled)
 		return;
 
+	/*
+	 * For now, only handle remap_pfn_range() vmas where
+	 * is_linear_pfn_mapping() == TRUE. Handling of
+	 * vm_insert_pfn() is TBD.
+	 */
 	if (is_linear_pfn_mapping(vma)) {
 		/* free the whole chunk starting from vm_pgoff */
 		paddr = (resource_size_t)vma->vm_pgoff << PAGE_SHIFT;
 		free_pfn_range(paddr, vma_size);
 		return;
 	}
-
-	if (size != 0 && size != vma_size) {
-		/* free page by page, using pfn and size */
-		paddr = (resource_size_t)pfn << PAGE_SHIFT;
-		for (i = 0; i < size; i += PAGE_SIZE) {
-			paddr = paddr + i;
-			free_pfn_range(paddr, PAGE_SIZE);
-		}
-	} else {
-		/* free entire vma, page by page, using the pfn from pte */
-		for (i = 0; i < vma_size; i += PAGE_SIZE) {
-			if (follow_phys(vma, vma_start + i, 0, &prot, &paddr))
-				continue;
-
-			free_pfn_range(paddr, PAGE_SIZE);
-		}
-	}
 }
 
 pgprot_t pgprot_writecombine(pgprot_t prot)
Signature · 1 line
1.6.0.6
#27 lore raw
Quoted 47 lines · Venkatesh
Quoted 32 lines · Ingo Molnar
Quoted 31 lines · Arkadiusz Miskiewicz
Quoted 21 lines · Venkatesh
Quoted 20 lines · Arkadiusz Miskiewicz
Quoted 11 lines · Venkatesh

On Thu, 2009-04-02 at 00:12 -0700, Arkadiusz Miskiewicz wrote:

I was finally able to reproduce the problem of "freeing invalid memtype" with upstream git kernel (commit 0221c81b1b) + latest xf86 intel driver. But, with upstream + the patch I had sent you earlier in this thread
(http://marc.info/?l=linux-kernel&m=123863345520617&w=2) I don't see those freeing invalid memtype errors anymore.

Can you please double check with current git and that patch and let me know if you are still seeing the problem.

Latest linus tree + that patch (it's really applied here), xserver 1.6, libdrm from git master, intel driver from git master, previously mesa 7.4 (and 7.5 snap currently), tremolous.net 1.1.0 game (tremolous-smp binary), GM45 gpu.

To reproduce I just need to run tremolous-smp and connect to some map. When map finishes loading I instantly get:

[...]

Quoted 5 lines

OK. One more test patch below, applies over linus's git and you can ignore the earlier patch. The patch below should get rid of the problem and as it removes the track/untrack of vm_insert_pfn completely. This will also eliminate the overhead of hundreds or thousands of entries in pat_memtype_list. Can you please test it.

Quoted 7 lines
Quoted 1 line

With this patch I'm no longer able to reproduce problem. Thanks!

Great, thanks!

Venki, mind sending a patch with a proper changelog, Reported-by, Tested-by tags, with Nick and Andrew Cc:-ed for the memory.c bits, etc.?

Ingo,

Below is the cleaner version of the patch. It does not have any changes in mm as we are only removing the tracking inside the PAT code. I have left the generic mm interface as is, until we fully resolve the issue in future.

Quoted 5 lines

Thanks,
Venki

Subject: [PATCH] x86, PAT: Remove page granularity tracking for vm_insert_pfn maps

I wonder what's with these patches that they missed rc2 ?

This one and Subject: [PATCH] x86, PAT: Remove duplicate memtype reserve in pci mmap

Signature · 2 lines
Arkadiusz Miśkiewicz        PLD/Linux Team
arekm / maven.pl            http://ftp.pld-linux.org/
#28 lore raw
Quoted 77 lines

-----Original Message-----
From: Arkadiusz Miskiewicz [mailto:[email protected]]
Sent: Wednesday, April 15, 2009 10:46 AM
To: Pallipadi, Venkatesh
Cc: Ingo Molnar; Thomas Gleixner; H. Peter Anvin; Siddha, Suresh B; [email protected]; Jesse Barnes
Subject: Re: 2.6.29 git master and PAT problems

Quoted 4 lines · Venkatesh
Quoted 3 lines · Ingo Molnar
Quoted 2 lines · Arkadiusz Miskiewicz
Quoted 1 line · Venkatesh

On Tue, Apr 07, 2009 at 02:12:28AM -0700, Arkadiusz

Quoted 2 lines · Miskiewicz
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines · Venkatesh
Quoted 1 line

On Thu, 2009-04-02 at 00:12 -0700, Arkadiusz

Quoted 2 lines · Miskiewicz
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

I was finally able to reproduce the problem of

"freeing invalid

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

memtype" with upstream git kernel (commit

0221c81b1b) + latest

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

xf86 intel driver. But, with upstream + the patch

I had sent you

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

earlier in this thread

(http://marc.info/?l=linux-kernel&m=123863345520617&w=2) I don't

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

see those freeing invalid memtype errors anymore.

Can you please double check with current git and

that patch and

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

let me know if you are still seeing the problem.

Latest linus tree + that patch (it's really applied

here), xserver

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

1.6, libdrm from git master, intel driver from git master, previously mesa 7.4 (and 7.5 snap currently),

tremolous.net 1.1.0

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

game (tremolous-smp binary), GM45 gpu.

To reproduce I just need to run tremolous-smp and

connect to some

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line

map. When map finishes loading I instantly get:

[...]

Quoted 1 line

OK. One more test patch below, applies over linus's

git and you can

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

ignore the earlier patch. The patch below should get rid of the problem and as it removes the track/untrack of vm_insert_pfn completely. This will also eliminate the overhead of

hundreds or

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

thousands of entries in pat_memtype_list. Can you

please test it.

Quoted 5 lines
Quoted 5 lines
Quoted 2 lines

With this patch I'm no longer able to reproduce problem. Thanks!

Great, thanks!

Quoted 7 lines
Quoted 3 lines

Venki, mind sending a patch with a proper changelog, Reported-by, Tested-by tags, with Nick and Andrew Cc:-ed for the memory.c bits, etc.?

Ingo,

Below is the cleaner version of the patch. It does not have

any changes in

Quoted 1 line

mm as we are only removing the tracking inside the PAT code.

I have left

Quoted 1 line

the generic mm interface as is, until we fully resolve the

issue in future.

Quoted 4 lines

Thanks,
Venki

Quoted 9 lines
Quoted 2 lines

Subject: [PATCH] x86, PAT: Remove page granularity tracking for vm_insert_pfn maps

I wonder what's with these patches that they missed rc2 ?

This one and Subject: [PATCH] x86, PAT: Remove duplicate memtype reserve in
pci mmap

I do see the above patch in rc2. Here
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=5a3ae276057840f0e60664c12fc3ef80aa59d1d4

But, the patch below has not yet made it.
http://marc.info/?l=linux-kernel&m=123923026401648&w=2

Ingo: Can you push this along?

Thanks,
Venki

#29 lore raw

[tip:x86/urgent] x86, PAT: Remove page granularity tracking for vm_insert_pfn maps

In reply to Pallipadi, Venkatesh
Commit-ID:  4b065046273afa01ec8e3de7da407e8d3599251d
Gitweb:     http://git.kernel.org/tip/4b065046273afa01ec8e3de7da407e8d3599251d
Author:     Pallipadi, Venkatesh <[email protected]>
AuthorDate: Wed, 8 Apr 2009 15:37:16 -0700
Committer:  Ingo Molnar <[email protected]>
CommitDate: Fri, 17 Apr 2009 00:44:22 +0200

x86, PAT: Remove page granularity tracking for vm_insert_pfn maps

This change resolves the problem of too many single page entries in pat_memtype_list and "freeing invalid memtype" errors with i915, reported here:

http://marc.info/?l=linux-kernel&m=123845244713183&w=2

Remove page level granularity track and untrack of vm_insert_pfn. memtype tracking at page granularity does not scale and cleaner approach would be for the driver to request a type for a bigger IO address range or PCI io memory range for that device, either at mmap time or driver init time and just use that type during vm_insert_pfn.

This patch just removes the track/untrack of vm_insert_pfn. That means we will be in same state as 2.6.28, with respect to these APIs.

Newer APIs for the drivers to request a memtype for a bigger region is coming soon.

[ Impact: fix Xorg startup warnings and hangs ]

LKML-Reference: <[email protected]>

---
 arch/x86/mm/pat.c |   98 ++++++++++------------------------------------------
 1 files changed, 19 insertions(+), 79 deletions(-)
arch/x86/mm/pat.c +19 -79
index cc5e0e2..41c8057 100644
@@ -669,29 +669,28 @@ static void free_pfn_range(u64 paddr, unsigned long size)
  *
  * If the vma has a linear pfn mapping for the entire range, we get the prot
  * from pte and reserve the entire vma range with single reserve_pfn_range call.
- * Otherwise, we reserve the entire vma range, my ging through the PTEs page
- * by page to get physical address and protection.
  */
 int track_pfn_vma_copy(struct vm_area_struct *vma)
 {
-	int retval = 0;
-	unsigned long i, j;
 	resource_size_t paddr;
 	unsigned long prot;
-	unsigned long vma_start = vma->vm_start;
-	unsigned long vma_end = vma->vm_end;
-	unsigned long vma_size = vma_end - vma_start;
+	unsigned long vma_size = vma->vm_end - vma->vm_start;
 	pgprot_t pgprot;
 
 	if (!pat_enabled)
 		return 0;
 
+	/*
+	 * For now, only handle remap_pfn_range() vmas where
+	 * is_linear_pfn_mapping() == TRUE. Handling of
+	 * vm_insert_pfn() is TBD.
+	 */
 	if (is_linear_pfn_mapping(vma)) {
 		/*
 		 * reserve the whole chunk covered by vma. We need the
 		 * starting address and protection from pte.
 		 */
-		if (follow_phys(vma, vma_start, 0, &prot, &paddr)) {
+		if (follow_phys(vma, vma->vm_start, 0, &prot, &paddr)) {
 			WARN_ON_ONCE(1);
 			return -EINVAL;
 		}
@@ -699,28 +698,7 @@ int track_pfn_vma_copy(struct vm_area_struct *vma)
 		return reserve_pfn_range(paddr, vma_size, &pgprot, 1);
 	}
 
-	/* reserve entire vma page by page, using pfn and prot from pte */
-	for (i = 0; i < vma_size; i += PAGE_SIZE) {
-		if (follow_phys(vma, vma_start + i, 0, &prot, &paddr))
-			continue;
-
-		pgprot = __pgprot(prot);
-		retval = reserve_pfn_range(paddr, PAGE_SIZE, &pgprot, 1);
-		if (retval)
-			goto cleanup_ret;
-	}
 	return 0;
-
-cleanup_ret:
-	/* Reserve error: Cleanup partial reservation and return error */
-	for (j = 0; j < i; j += PAGE_SIZE) {
-		if (follow_phys(vma, vma_start + j, 0, &prot, &paddr))
-			continue;
-
-		free_pfn_range(paddr, PAGE_SIZE);
-	}
-
-	return retval;
 }
 
 /*
@@ -730,50 +708,28 @@ cleanup_ret:
  * prot is passed in as a parameter for the new mapping. If the vma has a
  * linear pfn mapping for the entire range reserve the entire vma range with
  * single reserve_pfn_range call.
- * Otherwise, we look t the pfn and size and reserve only the specified range
- * page by page.
- *
- * Note that this function can be called with caller trying to map only a
- * subrange/page inside the vma.
  */
 int track_pfn_vma_new(struct vm_area_struct *vma, pgprot_t *prot,
 			unsigned long pfn, unsigned long size)
 {
-	int retval = 0;
-	unsigned long i, j;
-	resource_size_t base_paddr;
 	resource_size_t paddr;
-	unsigned long vma_start = vma->vm_start;
-	unsigned long vma_end = vma->vm_end;
-	unsigned long vma_size = vma_end - vma_start;
+	unsigned long vma_size = vma->vm_end - vma->vm_start;
 
 	if (!pat_enabled)
 		return 0;
 
+	/*
+	 * For now, only handle remap_pfn_range() vmas where
+	 * is_linear_pfn_mapping() == TRUE. Handling of
+	 * vm_insert_pfn() is TBD.
+	 */
 	if (is_linear_pfn_mapping(vma)) {
 		/* reserve the whole chunk starting from vm_pgoff */
 		paddr = (resource_size_t)vma->vm_pgoff << PAGE_SHIFT;
 		return reserve_pfn_range(paddr, vma_size, prot, 0);
 	}
 
-	/* reserve page by page using pfn and size */
-	base_paddr = (resource_size_t)pfn << PAGE_SHIFT;
-	for (i = 0; i < size; i += PAGE_SIZE) {
-		paddr = base_paddr + i;
-		retval = reserve_pfn_range(paddr, PAGE_SIZE, prot, 0);
-		if (retval)
-			goto cleanup_ret;
-	}
 	return 0;
-
-cleanup_ret:
-	/* Reserve error: Cleanup partial reservation and return error */
-	for (j = 0; j < i; j += PAGE_SIZE) {
-		paddr = base_paddr + j;
-		free_pfn_range(paddr, PAGE_SIZE);
-	}
-
-	return retval;
 }
 
 /*
@@ -784,39 +740,23 @@ cleanup_ret:
 void untrack_pfn_vma(struct vm_area_struct *vma, unsigned long pfn,
 			unsigned long size)
 {
-	unsigned long i;
 	resource_size_t paddr;
-	unsigned long prot;
-	unsigned long vma_start = vma->vm_start;
-	unsigned long vma_end = vma->vm_end;
-	unsigned long vma_size = vma_end - vma_start;
+	unsigned long vma_size = vma->vm_end - vma->vm_start;
 
 	if (!pat_enabled)
 		return;
 
+	/*
+	 * For now, only handle remap_pfn_range() vmas where
+	 * is_linear_pfn_mapping() == TRUE. Handling of
+	 * vm_insert_pfn() is TBD.
+	 */
 	if (is_linear_pfn_mapping(vma)) {
 		/* free the whole chunk starting from vm_pgoff */
 		paddr = (resource_size_t)vma->vm_pgoff << PAGE_SHIFT;
 		free_pfn_range(paddr, vma_size);
 		return;
 	}
-
-	if (size != 0 && size != vma_size) {
-		/* free page by page, using pfn and size */
-		paddr = (resource_size_t)pfn << PAGE_SHIFT;
-		for (i = 0; i < size; i += PAGE_SIZE) {
-			paddr = paddr + i;
-			free_pfn_range(paddr, PAGE_SIZE);
-		}
-	} else {
-		/* free entire vma, page by page, using the pfn from pte */
-		for (i = 0; i < vma_size; i += PAGE_SIZE) {
-			if (follow_phys(vma, vma_start + i, 0, &prot, &paddr))
-				continue;
-
-			free_pfn_range(paddr, PAGE_SIZE);
-		}
-	}
 }
 
 pgprot_t pgprot_writecombine(pgprot_t prot)