Skip to contentSkip to conversation

Memory management patches from Blackfin team

PATCH In review Bryan Wu 5 participants 15 messages May 12, 2008 – May 15, 2008

Subsystems (3)
lore
Share & tools
lore
author
#1 lore raw

PATCH 1/4 buddy page allocator: add tunable big order allocation

---
 init/Kconfig    |    9 +++++++++
 mm/page_alloc.c |    2 +-
 2 files changed, 10 insertions(+), 1 deletions(-)
init/Kconfig +9 -0
index 6135d07..b6ff75b 100644
@@ -742,6 +742,15 @@ config SLUB_DEBUG
 	  SLUB sysfs support. /sys/slab will not exist and there will be
 	  no support for cache validation etc.
 
+config BIG_ORDER_ALLOC_NOFAIL_MAGIC
+	int "Big Order Allocation No FAIL Magic"
+	depends on EMBEDDED
+	range 3 10
+	default 3
+	help
+	  Let big-order allocations loop until memory gets free. Specified Value
+	  expresses the order.
+
 choice
 	prompt "Choose SLAB allocator"
 	default SLUB
mm/page_alloc.c +1 -1
index bdd5c43..71b09b4 100644
@@ -1631,7 +1631,7 @@ nofail_alloc:
 	pages_reclaimed += did_some_progress;
 	do_retry = 0;
 	if (!(gfp_mask & __GFP_NORETRY)) {
-		if (order <= PAGE_ALLOC_COSTLY_ORDER) {
+		if (order <= CONFIG_BIG_ORDER_ALLOC_NOFAIL_MAGIC) {
 			do_retry = 1;
 		} else {
 			if (gfp_mask & __GFP_REPEAT &&
Signature · 7 lines
1.5.5

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#2 lore raw

Re: [PATCH 1/4] [mm] buddy page allocator: add tunable big order allocation

Quoted 4 lines · On Mon, 12 May 2008 18:32:02 +0800 Bryan Wu <[email protected]> wrote:

From: Michael Hennerich <[email protected]>

Does this really solve your problem ? possible hang-up is better than page allocation failure ?

Quoted 12 lines
---
 init/Kconfig    |    9 +++++++++
 mm/page_alloc.c |    2 +-
 2 files changed, 10 insertions(+), 1 deletions(-)
init/Kconfig +0 -0
index 6135d07..b6ff75b 100644
@@ -742,6 +742,15 @@ config SLUB_DEBUG
 	  SLUB sysfs support. /sys/slab will not exist and there will be
 	  no support for cache validation etc.

init/Kconfig config SLUB_DEBUG · line 753

Quoted 10 lines
init/Kconfig +9 -0
 
+config BIG_ORDER_ALLOC_NOFAIL_MAGIC
+	int "Big Order Allocation No FAIL Magic"
+	depends on EMBEDDED
+	range 3 10
+	default 3
+	help
+	  Let big-order allocations loop until memory gets free. Specified Value
+	  expresses the order.
+

I think it's better to add a text to explain why this is for EMBEDED.

Thanks,
-Kame

#3 lore raw

RE: [PATCH 1/4] [mm] buddy page allocator: add tunable big order allocation

Quoted 6 lines

-----Original Message-----
From: KAMEZAWA Hiroyuki [mailto:[email protected]]
Sent: Dienstag, 13. Mai 2008 04:09
To: Bryan Wu
Cc: [email protected]; [email protected];

[email protected];

Quoted 2 lines

Michael Hennerich
Subject: Re: [PATCH 1/4] [mm] buddy page allocator: add tunable big

order

Quoted 5 lines

allocation

On Mon, 12 May 2008 18:32:02 +0800
Bryan Wu <[email protected]> wrote:

Quoted 7 lines
Quoted 4 lines

From: Michael Hennerich <[email protected]>

Does this really solve your problem ? possible hang-up is better than page allocation failure ?

On nommu this helped quite a bit, when we run out of memory, eaten up by the page cache. But yes - with this option it's likely that we sit there and wait form memory that might never get available.

We now use a better workaround for freeing up "available" memory currently used as page cache.

I think we should drop this patch.

Best regards,
Michael

Quoted 14 lines
Quoted 14 lines
---
 init/Kconfig    |    9 +++++++++
 mm/page_alloc.c |    2 +-
 2 files changed, 10 insertions(+), 1 deletions(-)
init/Kconfig +0 -0
index 6135d07..b6ff75b 100644
@@ -742,6 +742,15 @@ config SLUB_DEBUG
 	  SLUB sysfs support. /sys/slab will not exist and there will be
 	  no support for cache validation etc.
 
Quoted 7 lines

init/Kconfig config SLUB_DEBUG

Quoted 7 lines
init/Kconfig +7 -0
+config BIG_ORDER_ALLOC_NOFAIL_MAGIC
+	int "Big Order Allocation No FAIL Magic"
+	depends on EMBEDDED
+	range 3 10
+	default 3
+	help
+	  Let big-order allocations loop until memory gets free.

Specified

Quoted 7 lines

Value

init/Kconfig config SLUB_DEBUG · line 753

Quoted 2 lines
init/Kconfig +2 -0
+	  expresses the order.
+

I think it's better to add a text to explain why this is for EMBEDED.

Thanks,
-Kame

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

author
#4 lore raw

Re: [PATCH 1/4] [mm] buddy page allocator: add tunable big order allocation

Quoted 24 lines · Michael
Quoted 7 lines

-----Original Message-----
From: KAMEZAWA Hiroyuki [mailto:[email protected]]
Sent: Dienstag, 13. Mai 2008 04:09
To: Bryan Wu
Cc: [email protected]; [email protected];

[email protected];

Quoted 2 lines

Michael Hennerich
Subject: Re: [PATCH 1/4] [mm] buddy page allocator: add tunable big

order

Quoted 5 lines

allocation

On Mon, 12 May 2008 18:32:02 +0800
Bryan Wu <[email protected]> wrote:

Quoted 7 lines
Quoted 4 lines

From: Michael Hennerich <[email protected]>

Does this really solve your problem ? possible hang-up is better than page allocation failure ?

Quoted 9 lines

On nommu this helped quite a bit, when we run out of memory, eaten up by the page cache. But yes - with this option it's likely that we sit there and wait form memory that might never get available.

We now use a better workaround for freeing up "available" memory currently used as page cache.

I think we should drop this patch.

OK, I dropped it. And do you think the limited page_cache patch is the replacement of this patch?

-Bryan

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#5 lore raw

Re: [PATCH 1/4] [mm] buddy page allocator: add tunable big order allocation

Quoted 30 lines · On Wed, 14 May 2008 12:22:35 +0800 "Bryan Wu" <[email protected]> wrote:
Quoted 28 lines · Michael
Quoted 7 lines

-----Original Message-----
From: KAMEZAWA Hiroyuki [mailto:[email protected]]
Sent: Dienstag, 13. Mai 2008 04:09
To: Bryan Wu
Cc: [email protected]; [email protected];

[email protected];

Quoted 2 lines

Michael Hennerich
Subject: Re: [PATCH 1/4] [mm] buddy page allocator: add tunable big

order

Quoted 5 lines

allocation

On Mon, 12 May 2008 18:32:02 +0800
Bryan Wu <[email protected]> wrote:

Quoted 7 lines
Quoted 4 lines

From: Michael Hennerich <[email protected]>

Does this really solve your problem ? possible hang-up is better than page allocation failure ?

On nommu this helped quite a bit, when we run out of memory, eaten up by the page cache. But yes - with this option it's likely that we sit there and wait form memory that might never get available.

Quoted 9 lines
Quoted 5 lines

We now use a better workaround for freeing up "available" memory currently used as page cache.

I think we should drop this patch.

OK, I dropped it. And do you think the limited page_cache patch is the replacement of this patch?

I'm not so familiar with nommu environments but have some thoughts.

one idea is
- use memory resource controller.
but this eats much amount of GFP_KERNEL memory and maybe not useful ;)
- use ZONE_MOVABLE and set lowmem_reserve_ratio value to be suitable value. then, the page cache just uses MOVABLE zone...(maybe)

Thanks,
-Kame

author
#6 lore raw

PATCH 2/4 : include the problematic mapping in the munmap warning

In reply to Bryan Wu
---
 mm/nommu.c |    5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)
mm/nommu.c +3 -2
index ef8c62c..c11e5cc 100644
@@ -1117,8 +1117,9 @@ int do_munmap(struct mm_struct *mm, unsigned long addr, size_t len)
 			goto found;
 	}
 
-	printk("munmap of non-mmaped memory by process %d (%s): %p\n",
-	       current->pid, current->comm, (void *) addr);
+	printk(KERN_NOTICE "munmap of non-mmaped memory [%p-%p] by process %d (%s)\n",
+	       (void *)addr, (void *)addr+len, current->pid, current->comm);
+
 	return -EINVAL;
 
  found:
Signature · 7 lines
1.5.5

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
author
#7 lore raw

Re: [PATCH 2/4] [NOMMU]: include the problematic mapping in the munmap warning

IMO, this is useful for nommu stuff.
Is there any chance to merge this?

-Bryan

Quoted 25 lines · Bryan Wu

From: Mike Frysinger <[email protected]>

---
 mm/nommu.c |    5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)
mm/nommu.c +3 -3
index ef8c62c..c11e5cc 100644
@@ -1117,8 +1117,9 @@ int do_munmap(struct mm_struct *mm, unsigned long addr, size_t len)
                       goto found;
       }
 
-       printk("munmap of non-mmaped memory by process %d (%s): %p\n",
-              current->pid, current->comm, (void *) addr);
+       printk(KERN_NOTICE "munmap of non-mmaped memory [%p-%p] by process %d (%s)\n",
+              (void *)addr, (void *)addr+len, current->pid, current->comm);
+
       return -EINVAL;
 
 found:
--
Quoted 3 lines

1.5.5

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

author
#8 lore raw

PATCH 3/4 : use copy_to_user_page to call flush icache for [#811] toolchain old bug

In reply to Bryan Wu

access_process_vm in mm/memory.c uses copy_to_user_page and copy_from_user_page. So for !MMU we'd better do the same thing. Other archs with mmu do the cache flush in copy_to_user_page. It gives me hint that copy_to_user_page is designed to flush the cache. On other side, no archs do the cache flush ptrace.

---
 mm/nommu.c |    6 ++++--
 1 files changed, 4 insertions(+), 2 deletions(-)
mm/nommu.c +4 -2
index c11e5cc..56bb447 100644
@@ -1458,9 +1458,11 @@ int access_process_vm(struct task_struct *tsk, unsigned long addr, void *buf, in
 
 		/* only read or write mappings where it is permitted */
 		if (write && vma->vm_flags & VM_MAYWRITE)
-			len -= copy_to_user((void *) addr, buf, len);
+			copy_to_user_page(vma, NULL, NULL,
+					  (void *) addr, buf, len);
 		else if (!write && vma->vm_flags & VM_MAYREAD)
-			len -= copy_from_user(buf, (void *) addr, len);
+			copy_from_user_page(vma, NULL, NULL,
+					    buf, (void *) addr, len);
 		else
 			len = 0;
 	} else {
Signature · 7 lines
1.5.5

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
author
#9 lore raw

Re: [PATCH 3/4] [mm/nommu]: use copy_to_user_page to call flush icache for [#811] toolchain old bug

Is this patch OK for the nommu, David?

Thanks
-Bryan

Quoted 33 lines · Bryan Wu

From: Jie Zhang <[email protected]>

access_process_vm in mm/memory.c uses copy_to_user_page and copy_from_user_page. So for !MMU we'd better do the same thing. Other archs with mmu do the cache flush in copy_to_user_page. It gives me hint that copy_to_user_page is designed to flush the cache. On other side, no archs do the cache flush ptrace.

---
 mm/nommu.c |    6 ++++--
 1 files changed, 4 insertions(+), 2 deletions(-)
mm/nommu.c +4 -3
index c11e5cc..56bb447 100644
@@ -1458,9 +1458,11 @@ int access_process_vm(struct task_struct *tsk, unsigned long addr, void *buf, in
 
               /* only read or write mappings where it is permitted */
               if (write && vma->vm_flags & VM_MAYWRITE)
-                       len -= copy_to_user((void *) addr, buf, len);
+                       copy_to_user_page(vma, NULL, NULL,
+                                         (void *) addr, buf, len);
               else if (!write && vma->vm_flags & VM_MAYREAD)
-                       len -= copy_from_user(buf, (void *) addr, len);
+                       copy_from_user_page(vma, NULL, NULL,
+                                           buf, (void *) addr, len);
               else
                       len = 0;
       } else {
--
Quoted 3 lines

1.5.5

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

author
#10 lore raw

PATCH 4/4 : Export two symbols in nommu.c for mmap test

In reply to Bryan Wu

http://blackfin.uclinux.org/gf/project/uclinux-dist/tracker/?action=TrackerItemEdit&tracker_item_id=2312

---
 mm/nommu.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)
mm/nommu.c +2 -0
index 56bb447..cef9d75 100644
@@ -47,6 +47,8 @@ int heap_stack_gap = 0;
 
 EXPORT_SYMBOL(mem_map);
 EXPORT_SYMBOL(num_physpages);
+EXPORT_SYMBOL(high_memory);
+EXPORT_SYMBOL(max_mapnr);
 
 /* list of shareable VMAs */
 struct rb_root nommu_vma_tree = RB_ROOT;
Signature · 7 lines
1.5.5

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#11 lore raw

Re: [PATCH 4/4] [MM/NOMMU]: Export two symbols in nommu.c for mmap test

Quoted 3 lines · Bryan Wu

From: Vivi Li <[email protected]>

http://blackfin.uclinux.org/gf/project/uclinux-dist/tracker/?action=TrackerItemEdit&tracker_item_id=2312

i dont think URLs to our tracker is a good substitute for log messages. our tracker URLs have known to break in the past (due to gforge changes), and it's a pain for people reading changelogs to open up a browser just to see what the issue is about.
-mike

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#12 lore raw

Re: [PATCH 4/4] [MM/NOMMU]: Export two symbols in nommu.c for mmap test

Quoted 9 lines · Mike Frysinger
Quoted 3 lines · Bryan Wu

From: Vivi Li <[email protected]>

http://blackfin.uclinux.org/gf/project/uclinux-dist/tracker/?action=TrackerItemEdit&tracker_item_id=2312

i dont think URLs to our tracker is a good substitute for log messages. our tracker URLs have known to break in the past (due to gforge changes), and it's a pain for people reading changelogs to open up a browser just to see what the issue is about.

Yeah, even the url pointer to there is totally unreadable. I still don't understand why you want to export this symbols. Also please make sure to always submit the code actually using the exports in the same patchkit, that makes it a lot easier to figure.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#13 lore raw

Re: [PATCH 4/4] [MM/NOMMU]: Export two symbols in nommu.c for mmap test

Quoted 4 lines · Christoph Hellwig

Yeah, even the url pointer to there is totally unreadable. I still don't understand why you want to export this symbols. Also please make sure to always submit the code actually using the exports in the same patchkit, that makes it a lot easier to figure.

the symbols are exported for MMU already. this brings the no-MMU code in line. -mike

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

author
#14 lore raw

Re: [PATCH 4/4] [MM/NOMMU]: Export two symbols in nommu.c for mmap test

Quoted 9 lines · Mike Frysinger
Quoted 4 lines · Christoph Hellwig

Yeah, even the url pointer to there is totally unreadable. I still don't understand why you want to export this symbols. Also please make sure to always submit the code actually using the exports in the same patchkit, that makes it a lot easier to figure.

the symbols are exported for MMU already. this brings the no-MMU code in line. -mike

Right, the same symbols are exported in mm/memory.c for MMU arch. While for NOMMU arch, Vivi (one of our tester) asked for this exported symbols for some mmap test case.
And maybe they are useful for others.

Sorry for the misleading URL, I add to wrong patches. If you think this patch is OK, I will rewrite the git log.

Thanks
-Bryan

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>