nommu arch dont zero the anonymous mapping by adding UNINITIALIZE flag
Subsystems (7)
PATCH 1/1 nommu arch dont zero the anonymous mapping by adding UNINITIALIZE flag
On architectures with MMU, malloc takes about the same speed, indepentant of malloc size, while on the Blackfin (NOMMU), as the malloc size increases, the time that malloc consumes grows....
This small application, which does a bunch of mallocs, and times them with gettimeofday():
=============================================
#include <stdio.h>
#include <unistd.h>
#include <sys/time.h>
int main(int argc, char* argv[]) {
unsigned int i, * j, k, a, t1, t2, max, ave, min; struct timeval tim1, tim2;
k = 4 ;
for (i = 0; i <= 16*1024 ; i+=k ) {
max = 0;
ave = 0;
min = 0xFFFFFFFF;
for (a = 0; a < 128 ; a ++ ) {
gettimeofday(&tim2, NULL);
j = (int *)malloc (i*1024);
gettimeofday(&tim1, NULL);
free (j);
t1 = (tim1.tv_sec-tim2.tv_sec) * 1000000 + tim1.tv_usec-tim2.tv_usec;
if ( max < t1 ) max = t1;
if ( min > t1 ) min = t1;
ave = ave + t1;
}
printf("%05ik : 0x%08x %06i %06i %06i\n", i, j, min, ave/128, max);
k = 1024;
if ( i < 1024 ) k = 128;
if ( i < 128 ) k = 4;
} return 0;
};
===========================================
Summary is, when I run the app "time test",
on x86:
real 0m0.066s user 0m0.008s sys 0m0.058s
on Blackfin:
real 3m 37.69s user 0m 0.04s sys 3m 37.58s
when run on x86, it produces the below output - indicating that it is pretty constant: (min of arouns 2uS, aver of 3, and max of 30-15)
00000k : 0x0804a008 000002 000003 000030 00004k : 0x0804a008 000002 000003 000011 00008k : 0x0804a008 000002 000002 000007 00012k : 0x0804a008 000002 000003 000012 00016k : 0x0804a008 000002 000002 000008 00020k : 0x0804a008 000002 000002 000011 00024k : 0x0804a008 000002 000002 000009 00028k : 0x0804a008 000002 000002 000007 00032k : 0x0804a008 000002 000002 000007 00036k : 0x0804a008 000002 000003 000017 00040k : 0x0804a008 000002 000003 000015 00044k : 0x0804a008 000002 000002 000009 00048k : 0x0804a008 000002 000002 000011 00052k : 0x0804a008 000002 000003 000017 00056k : 0x0804a008 000002 000002 000009 00060k : 0x0804a008 000002 000002 000012 00064k : 0x0804a008 000002 000003 000014 00068k : 0x0804a008 000002 000003 000011 00072k : 0x0804a008 000002 000002 000009 00076k : 0x0804a008 000002 000003 000016 00080k : 0x0804a008 000002 000003 000013 00084k : 0x0804a008 000002 000002 000009 00088k : 0x0804a008 000002 000003 000018 00092k : 0x0804a008 000002 000003 000017 00096k : 0x0804a008 000002 000003 000012 00100k : 0x0804a008 000002 000003 000014 00104k : 0x0804a008 000002 000003 000012 00108k : 0x0804a008 000002 000002 000009 00112k : 0x0804a008 000002 000002 000011 00116k : 0x0804a008 000002 000002 000008 00120k : 0x0804a008 000002 000002 000009 00124k : 0x0804a008 000002 000002 000011 00128k : 0x0804a008 000002 000003 000011 00256k : 0x40150008 000006 000007 000019 00384k : 0x40150008 000006 000007 000019 00512k : 0x40150008 000006 000007 000018 00640k : 0x40150008 000006 000007 000019 00768k : 0x40150008 000007 000007 000013 00896k : 0x40150008 000006 000007 000013 01024k : 0x40150008 000006 000007 000019 02048k : 0x40150008 000006 000007 000018 03072k : 0x40150008 000006 000007 000013 04096k : 0x40150008 000006 000007 000012 05120k : 0x40150008 000006 000007 000018 06144k : 0x40150008 000006 000007 000015 07168k : 0x40150008 000006 000007 000014 08192k : 0x40150008 000006 000007 000017 09216k : 0x40150008 000006 000007 000010 10240k : 0x40150008 000006 000007 000015 11264k : 0x40150008 000006 000007 000012 12288k : 0x40150008 000006 000007 000012 13312k : 0x40150008 000006 000007 000019 14336k : 0x40150008 000006 000007 000019 15360k : 0x40150008 000006 000007 000010 16384k : 0x40150008 000006 000007 000019
When compiled with bfin-uclinux-gcc -Wl,-elf2flt -O3 ./test.c -o ./test
On blackfin, the time increases, as the malloc size increases.
00000k : 0x00000000 000005 000005 000011 00004k : 0x002e4000 000034 000035 000089 00008k : 0x0300c000 000056 000059 000136 00012k : 0x00284000 000081 000083 000189 00016k : 0x00284000 000113 000116 000180 00020k : 0x034c0000 000140 000143 000286 00024k : 0x034c0000 000162 000165 000220 00028k : 0x034c0000 000188 000191 000239 00032k : 0x034c0000 000221 000225 000301 00036k : 0x03010000 000288 000295 000460 00040k : 0x03010000 000360 000367 000516 00044k : 0x03010000 000424 000433 000563 00048k : 0x03010000 000495 000502 000556 00052k : 0x03010000 000565 000575 000645 00056k : 0x03010000 000637 000647 000699 00060k : 0x03010000 000706 000715 000769 00064k : 0x03010000 000772 000781 000874 00068k : 0x002c0000 000819 000828 000885 00072k : 0x002c0000 000866 000874 000930 00076k : 0x002c0000 000912 000921 000977 00080k : 0x002c0000 000959 000968 001023 00084k : 0x002c0000 001005 001017 001163 00088k : 0x002c0000 001052 001062 001152 00092k : 0x002c0000 001099 001112 001471 00096k : 0x002c0000 001145 001156 001210 00100k : 0x002c0000 001192 001204 001287 00104k : 0x002c0000 001238 001251 001339 00108k : 0x002c0000 001285 001299 001441 00112k : 0x002c0000 001331 001345 001397 00116k : 0x002c0000 001378 001392 001443 00120k : 0x002c0000 001425 001439 001521 00124k : 0x002c0000 001471 001486 001534 00128k : 0x002c0000 001518 001533 001580 00256k : 0x03040000 003011 003040 003163 00384k : 0x03080000 004536 004546 004874 00512k : 0x03080000 006027 006046 006174 00640k : 0x00300000 007520 007552 007850 00768k : 0x00300000 009038 009050 009178 00896k : 0x00300000 010529 010556 010886 01024k : 0x00300000 012046 012057 012354 02048k : 0x03200000 024056 024073 024347 03072k : 0x00400000 036069 036098 036531 04096k : 0x00400000 048081 048106 048327 05120k : 0x00800000 060094 060140 061698 06144k : 0x00800000 072104 072144 072352 07168k : 0x00800000 084113 084155 084369 08192k : 0x00800000 096126 096179 096829 09216k : 0x02000000 108144 108221 111188 10240k : 0x02000000 120157 120198 120487 11264k : 0x02000000 132163 132215 132490 12288k : 0x02000000 144174 144230 144450 13312k : 0x02000000 156185 156245 156592 14336k : 0x02000000 168195 168257 168627 15360k : 0x02000000 180206 180295 181205 16384k : 0x02000000 192217 192285 192573
It takes lots of time in malloc()->mmap()-> do_mmap_private()->memset(). When malloc a big area, memset() the area to zero makes the performance very bad.
Here is a patch for this and libc malloc() should be modified too.
--- include/asm-arm/mman.h | 2 ++ include/asm-blackfin/mman.h | 2 +- include/asm-frv/mman.h | 2 ++ include/asm-m68k/mman.h | 2 ++ include/asm-sh/mman.h | 2 ++ include/linux/mm.h | 2 ++ mm/nommu.c | 6 +++++- 7 files changed, 16 insertions(+), 2 deletions(-)
include/asm-arm/mman.h +2 -0
@@ -10,6 +10,8 @@#define MAP_NORESERVE 0x4000/* don't check for reservations */ #define MAP_POPULATE 0x8000/* populate (prefault) page tables */ #define MAP_NONBLOCK 0x10000/* do not block on IO */ +#define MAP_UNINITIALIZE 0x4000000/* For anonymous mmap, memory could +be uninitialized. */ #define MCL_CURRENT 1/* lock all current mappings */ #define MCL_FUTURE 2/* lock all future mappings */
include/asm-blackfin/mman.h +1 -1
@@ -23,7 +23,7 @@#define MAP_POPULATE 0x8000/* populate (prefault) pagetables */ #define MAP_NONBLOCK 0x10000/* do not block on IO */ #define MAP_UNINITIALIZE 0x4000000/* For anonymous mmap, memory could -be uninitialized. */ +be uninitialized. */ #define MS_ASYNC 1/* sync memory asynchronously */ #define MS_INVALIDATE 2/* invalidate the caches */
include/asm-frv/mman.h +2 -0
@@ -10,6 +10,8 @@#define MAP_NORESERVE 0x4000/* don't check for reservations */ #define MAP_POPULATE 0x8000/* populate (prefault) pagetables */ #define MAP_NONBLOCK 0x10000/* do not block on IO */ +#define MAP_UNINITIALIZE 0x4000000/* For anonymous mmap, memory could +be uninitialized. */ #define MCL_CURRENT 1/* lock all current mappings */ #define MCL_FUTURE 2/* lock all future mappings */
include/asm-m68k/mman.h +2 -0
@@ -10,6 +10,8 @@#define MAP_NORESERVE 0x4000/* don't check for reservations */ #define MAP_POPULATE 0x8000/* populate (prefault) pagetables */ #define MAP_NONBLOCK 0x10000/* do not block on IO */ +#define MAP_UNINITIALIZE 0x4000000/* For anonymous mmap, memory could +be uninitialized. */ #define MCL_CURRENT 1/* lock all current mappings */ #define MCL_FUTURE 2/* lock all future mappings */
include/asm-sh/mman.h +2 -0
@@ -10,6 +10,8 @@#define MAP_NORESERVE 0x4000/* don't check for reservations */ #define MAP_POPULATE 0x8000/* populate (prefault) page tables */ #define MAP_NONBLOCK 0x10000/* do not block on IO */ +#define MAP_UNINITIALIZE 0x4000000/* For anonymous mmap, memory could +be uninitialized. */ #define MCL_CURRENT 1/* lock all current mappings */ #define MCL_FUTURE 2/* lock all future mappings */
include/linux/mm.h +2 -0
@@ -171,6 +171,8 @@ extern int do_mprotect(unsigned long start, size_t len, unsigned long prot);#define VM_MAPPED_COPY 0x01000000/* T if mapped copy of data (nommu mmap) */ #define VM_INSERTPAGE 0x02000000/* The vma has had "vm_insert_page()" done on it */ #define VM_ALWAYSDUMP 0x04000000/* Always include in core dumps */ +#define VM_UNINITIALIZE 0x08000000/* For anonymous mmap, memory could +be uninitialized. */ #define VM_CAN_INVALIDATE 0x08000000/* The mapping may be invalidated, * eg. truncate or invalidate_inode_*.
mm/nommu.c +5 -1
@@ -680,6 +680,9 @@ static unsigned long determine_vm_flags(struct file *file,#endif vm_flags &= ~VM_MAYSHARE ; +if (flags &MAP_UNINITIALIZE ) + vm_flags |=VM_UNINITIALIZE ; +return vm_flags; }
@@ -766,7 +769,8 @@ static int do_mmap_private(struct vm_area_struct *vma, unsigned long len) }else {/* if it's an anonymous mapping, then just clear it */ -memset (base,0 , len); +if (!(vma->vm_flags &VM_UNINITIALIZE )) +memset (base,0 , len); }return 0 ;
Signature · 1 line
1.5.0.5
Quoted 3 lines · Bryan
On architectures with MMU, malloc takes about the same speed, indepentant of malloc size, while on the Blackfin (NOMMU), as the malloc size increases, the time that malloc consumes grows....
err, this is not the direction we wanted to go ... this requires changes to the user<->kernel boundary which certainly will be rejected by the glibc maintainers since it implies breakage in POSIX behavior and is specific to no-mmu only ...
the plan is to hold onto this cruft in Blackfin until we get this bug sorted out:
http://blackfin.uclinux.org/gf/project/uclinux-dist/tracker/?action=TrackerItemEdit&tracker_item_id=3027
as the thing we'll send to mainline will be sane (a no-mmu specific kconfig option to control the memset in the kernel malloc)
the memset in the kernel right now for no-mmu is by design, not defect ... we can chat about this more via Blackfin channels if you like, or just wait till we meet up in Shanghai
-mike
Quoted 2 lines · Bryan
It takes lots of time in malloc()->mmap()->do_mmap_private()->memset(). When malloc a big area,
memset()the area to zero makes the performance very bad.
Ummm...
How do you then cope with attempting to run that same application under MMU-mode Linux? Won't MMU mmap() give EINVAL? If so, then that'd be grounds for NAK'ing this patch in the form given.
My theory is that NOMMU binaries should run just as well under an MMU-mode kernel as under a NOMMU kernel.
What you're asking for is also a security risk - though obviously on NOMMU-mode one that's fairly irrelevant. It might even make a lot of sense there to move the clearance into uClibc where possible rather than doing it in the kernel.
On MMU-mode kernels, the option should just be ignored.
I'd also recommend you stick a 'D' on the end of 'MAP_UNINITIALIZE' or may be call it MAP_UNCLEARED'. But that's a minor point, but you're not telling mmap() to go and uninitialise the memory...
Lastly, why do you actually need VM_UNINITIALIZE at all? The flag is only used in a place where MAP_UNINITIALIZE is still available (okay, you'll have to hand it down as an extra argument). That looks like a waste of a VM_xxx flag, and we don't have that many to spare.
David
Quoted 20 lines · David Howells
Quoted 2 lines · Bryan
It takes lots of time in malloc()->mmap()->do_mmap_private()->memset(). When malloc a big area,
memset()the area to zero makes the performance very bad.Ummm...
How do you then cope with attempting to run that same application under MMU-mode Linux? Won't MMU
mmap()give EINVAL? If so, then that'd be grounds for NAK'ing this patch in the form given.My theory is that NOMMU binaries should run just as well under an MMU-mode kernel as under a NOMMU kernel.
What you're asking for is also a security risk - though obviously on NOMMU-mode one that's fairly irrelevant. It might even make a lot of sense there to move the clearance into uClibc where possible rather than doing it in the kernel.
On MMU-mode kernels, the option should just be ignored.
Quoted 9 lines
I'd also recommend you stick a 'D' on the end of '
MAP_UNINITIALIZE' or may be call itMAP_UNCLEARED'. But that's a minor point, but you're not tellingmmap()to go and uninitialise the memory...Lastly, why do you actually need
VM_UNINITIALIZEat all? The flag is only used in a place whereMAP_UNINITIALIZEis still available (okay, you'll have to hand it down as an extra argument). That looks like a waste of aVM_xxxflag, and we don't have that many to spare.
Well, I don't understand why we have to clear the allocation memory. I suggest we just remove the memset(addr, 0, len) operation at all. Read the malloc manual, you'll get
--------malloc() allocates size bytes and returns a pointer to the allocated memory. ****The memory is not cleared.****
--------
So, if not clearing the memory causes one application hang/crash, that's the bug of that application and it need to be fixed.
The patch in my option is:
------------------------------------
mm/nommu.c +0 -4
@@ -759,10 +759,6 @@ static int do_mmap_private(struct vm_are/* clear the last little bit */ if (ret < len)memset (base + ret,0 , len - ret); - - }else { -/* if it's an anonymous mapping, then just clear it */ -memset (base,0 , len); }return 0 ;
----------------------------------------
-Aubrey
Quoted 2 lines · Aubrey Li
malloc()allocates size bytes and returns a pointer to the allocated memory. ****The memory is not cleared.****
But this is *not* malloc(). It's mmap(). Are you prepared to guarantee that there are no applications out there that don't rely on anon mmap() giving zeroed memory?
The MMU-mode clearing is done for security reasons - there shouldn't be any leakage between processes, and because the zero page can just be faulted in.
Personally, I'd prefer to maintain compatibility with MMU-mode wherever possible, but I'm happy with overrides like the MAP_UNINITIALISED flag suggested.
David
Quoted 8 lines · David Howells
Quoted 2 lines · Aubrey Li
malloc()allocates size bytes and returns a pointer to the allocated memory. ****The memory is not cleared.****But this is *not*
malloc(). It'smmap(). Are you prepared to guarantee that there are no applications out there that don't rely on anonmmap()giving zeroed memory?
I can't find mmap must give zeroed memory in the mmap manual. Is there any reason relying on anon mmap() giving zerod memory? Can you show me an example?
Quoted 4 lines
The MMU-mode clearing is done for security reasons - there shouldn't be any leakage between processes, and because the zero page can just be faulted in.
NO-MMU can't do this clearing. Performance is down.
Quoted 4 lines
Personally, I'd prefer to maintain compatibility with MMU-mode wherever possible, but I'm happy with overrides like the
MAP_UNINITIALISEDflag suggested.
Not necessary IMHO.
-Aubrey
Quoted 2 lines
I can't find mmap must give zeroed memory in the mmap manual. Is there any reason relying on anon
mmap()giving zerod memory?
Its how all Unix/Linux like systems behave. You have to clear the memory to something to deal with security on any kind of real system, and zero is a good a value as any
Quoted 5 lines
Quoted 4 lines
Personally, I'd prefer to maintain compatibility with MMU-mode wherever possible, but I'm happy with overrides like the
MAP_UNINITIALISEDflag suggested.Not necessary IMHO.
mmap() for anonymous memory pools should not normally be a hot path, because the C library malloc is supposed to show some brains. If you need special behaviour (eg for performance hacks) then create yourself a /dev/zero like device which just maps uncleared pages. Its not much code and it keeps special cases out of the core kernel.
Quoted 4 lines · Alan Cox
Quoted 2 lines
I can't find mmap must give zeroed memory in the mmap manual. Is there any reason relying on anon
mmap()giving zerod memory?Its how all Unix/Linux like systems behave.
Fair enough.
Quoted 8 lines
You have to clear the memory
to something to deal with security on any kind of real system, and zero is a good a value as anyQuoted 4 lines
Quoted 4 lines
Personally, I'd prefer to maintain compatibility with MMU-mode wherever possible, but I'm happy with overrides like the
MAP_UNINITIALISEDflag suggested.
Quoted 8 lines
Quoted 1 line
Not necessary IMHO.
mmap()for anonymous memory pools should not normally be a hot path, because the C library malloc is supposed to show some brains. If you need special behaviour (eg for performance hacks) then create yourself a/dev/zerolike device which just maps uncleared pages. Its not much code and it keeps special cases out of the core kernel.
On MMU-mode, /dev/zero's-->mmap gives zerod memory.
On NO-MMU-mode, we have to do memset(,,,) to give zerod memory. So the different malloc size has the different time cost.
So Bryan's test case shows the terrible performance result.
---------------------------------------------------------------
Summary is, when I run the app "time test",
on x86:
real 0m0.066s user 0m0.008s sys 0m0.058s
on Blackfin:
real 3m 37.69s user 0m 0.04s sys 3m 37.58s
---------------------------------------------------------------
So now the question is,
Keep the same behave as MMU but with bad performance, or keep the same performance as MMU but without the same behave, Which one is more important?
-Aubrey
Quoted 11 lines · Aubrey Li
Summary is, when I run the app "time test",
on x86:
real 0m0.066s user 0m0.008s sys 0m0.058son Blackfin:
real 3m 37.69s user 0m 0.04s sys 3m 37.58s
That's not a good comparison: you're comparing two different machines of two different archs.
Quoted 3 lines
Keep the same behave as MMU but with bad performance, or keep the same performance as MMU but without the same behave, Which one is more important?
The thing to do is to pass a flag to mmap() to suppress the memset (as suggested). This can then be used by the uClibc malloc() implementation and does not impact any application that does an anon mmap() expecting the memory returned to be zeroed.
David
Quoted 11 lines · David Howells
Quoted 3 lines · Aubrey Li
Keep the same behave as MMU but with bad performance, or keep the same performance as MMU but without the same behave, Which one is more important?
The thing to do is to pass a flag to
mmap()to suppress the memset (as suggested). This can then be used by the uClibcmalloc()implementation and does not impact any application that does an anonmmap()expecting the memory returned to be zeroed.
As Mike mentioned, it implies breakage in POSIX behavior and is specific to no-mmu only.
What kind of application does an anonymous mmap() expecting the memory returned to be zeroed?
-Aubrey
PATCH 1/1 NOMMU: fix malloc performance by adding uninitialized flag
In reply to David HowellsFrom: Jie Zhang <[email protected]>
The no-mmu code currently clears all anonymous mmap-ed memory. While this is what we want in the default case, all memory allocation from userspace under no-mmu has to go through this interface, including malloc() which is allowed to return uninitialized memory. This can easily be a significant performance slow down. So for constrained embedded systems were security is irrelevant, allow people to avoid unnecessarily clearing memory.
--- Documentation/nommu-mmap.txt | 21 +++++++++++++++++++++ fs/binfmt_elf_fdpic.c | 2 +- include/asm-generic/mman-common.h | 5 +++++ init/Kconfig | 16 ++++++++++++++++ mm/nommu.c | 7 ++++--- 5 files changed, 47 insertions(+), 4 deletions(-)
Documentation/nommu-mmap.txt +21 -0
@@ -16,6 +16,27 @@ the CLONE_VM flag. The behaviour is similar between the MMU and no-MMU cases, but not identical; and it's also much more restricted in the latter case: + (*) Anonymous mappings - general case + + Anonymous mappings are not backed by any file, and according to the + Linux man pages (ver 2.22 or later) contents are initialized to zero. + + In the MMU case, regions are backed by arbitrary virtual pages, and the + contents are only mapped with physical pages and initialized to zero + when a read or write happens in that specific page. This spreads out + the time it takes to initialize the contents depending on the + read/write usage of the map. + + In the no-MMU case, anonymous mappings are backed by physical pages, + and the entire map is initialized to zero at allocation time. This + can cause significant delays in userspace during malloc() as the C + library does an anonymous mapping, and the kernel is doing a memset + for the entire map. Since malloc's memory is not required to be + cleared, an (optional) flag MAP_UNINITIALIZE can be passed to the + kernel's do_mmap, which will not initialize the contents to zero. + + uClibc supports this to provide a pretty significant speedup for malloc(). + (*) Anonymous mapping, MAP_PRIVATE In the MMU case: VM regions backed by arbitrary pages; copy-on-write
fs/binfmt_elf_fdpic.c +1 -1
@@ -380,7 +380,7 @@ static int load_elf_fdpic_binary(struct linux_binprm *bprm,down_write (¤t->mm->mmap_sem); current->mm->start_brk =do_mmap (NULL ,0 , stack_size,PROT_READ |PROT_WRITE |PROT_EXEC , -MAP_PRIVATE |MAP_ANONYMOUS |MAP_GROWSDOWN , +MAP_PRIVATE |MAP_ANONYMOUS |MAP_UNINITIALIZE |MAP_GROWSDOWN ,0 );if (IS_ERR_VALUE (current->mm->start_brk)) {
include/asm-generic/mman-common.h +5 -0
@@ -19,6 +19,11 @@#define MAP_TYPE 0x0f/* Mask for type of mapping */ #define MAP_FIXED 0x10/* Interpret addr exactly */ #define MAP_ANONYMOUS 0x20/* don't use a file */ +#ifdef CONFIG_MMAP_ALLOW_UNINITIALIZE +# define MAP_UNINITIALIZE 0x4000000/* For anonymous mmap, memory could be uninitialized */ +#else +# define MAP_UNINITIALIZE 0x0/* Don't support this flag */ +#endif #define MS_ASYNC 1/* sync memory asynchronously */ #define MS_INVALIDATE 2/* invalidate the caches */
init/Kconfig +16 -0
@@ -1069,6 +1069,22 @@ config SLOB endchoice +config MMAP_ALLOW_UNINITIALIZE + bool "Allow mmaped anonymous memory to be un-initialized" + depends on EMBEDDED && ! MMU + default n + help + Normally (and according to the Linux spec) mmap'ed MAP_ANONYMOUS + memory has it's contents initialized to zero. This kernel option + gives you the option of not doing that by adding a MAP_UNINITIALIZE + mmap flag (which uClibc's malloc() takes takes advantage of) + which provides a huge performance boost. + + Because of the obvious security issues, this option should only be + enabled on embedded devices which you control what is run in + userspace. Since that isn't a problem on no-MMU systems, it is + normally safe to say Y here. + config PROFILING bool "Profiling support (EXPERIMENTAL)" help
mm/nommu.c +4 -3
@@ -1143,9 +1143,6 @@ static int do_mmap_private(struct vm_area_struct *vma,if (ret < rlen)memset (base + ret,0 , rlen - ret); - }else { -/* if it's an anonymous mapping, then just clear it */ -memset (base,0 , rlen); }return 0 ;
@@ -1343,6 +1340,10 @@ unsigned long do_mmap_pgoff(struct file *file, goto error_just_free;add_nommu_region (region); +/* clear anonymous mappings that don't ask for un-initialized data */ +if (!(vma->vm_file) && !(flags &MAP_UNINITIALIZE )) +memset ((void *)region->vm_start,0 , region->vm_end - region->vm_start); +/* okay... we have a mapping; now we have to register it */ result = vma->vm_start;
Signature · 1 line
1.6.5
Re: [PATCH] NOMMU: fix malloc performance by adding uninitialized flag
This seems reasonable. MAP_UNINITIALIZE definitely needs adding to the general list so that the MMU folks don't steal the bit. I would also call it MAP_UNINITIALIZED (well, actually, I'd call it MAP_UNINITIALISED:-), otherwise it looks like you're asking mmap() to uninitialise the memory for you. Similarly for CONFIG_MMAP_ALLOW_UNINITIALIZE - I'd add a terminal 'D'.
I've also re-written the documenation somewhat and expanded the patch description to mention the changes to ELF-FDPIC.
David
---
From: Jie Zhang <[email protected]>
Subject: [PATCH] NOMMU: Fix malloc performance by adding uninitialized flag
The NOMMU code currently clears all anonymous mmapped memory. While this is what we want in the default case, all memory allocation from userspace under NOMMU has to go through this interface, including malloc() which is allowed to return uninitialized memory. This can easily be a significant performance penalty. So for constrained embedded systems were security is irrelevant, allow people to avoid clearing memory unnecessarily.
This also alters the ELF-FDPIC binfmt such that it obtains uninitialised memory for the brk and stack region.
---
Documentation/nommu-mmap.txt | 26 ++++++++++++++++++++++++++ fs/binfmt_elf_fdpic.c | 3 ++- include/asm-generic/mman-common.h | 5 +++++ init/Kconfig | 22 ++++++++++++++++++++++ mm/nommu.c | 8 +++++--- 5 files changed, 60 insertions(+), 4 deletions(-)
Documentation/nommu-mmap.txt +26 -0
@@ -119,6 +119,32 @@ FURTHER NOTES ON NO-MMU MMAP granule but will only discard the excess if appropriately configured as this has an effect on fragmentation. + (*) The memory allocated by a request for an anonymous mapping will normally + be cleared by the kernel before being returned in accordance with the + Linux man pages (ver 2.22 or later) + + In the MMU case this can be achieved with reasonable performance as + regions are backed by virtual pages, with the contents only being mapped + to cleared physical pages when a write happens on that specific page + (prior to which, the pages are effectively mapped to the global zero page + from which reads can take place). This spreads out the time it takes to + initialize the contents of a page - depending on the write-usage of the + mapping. + + In the no-MMU case, however, anonymous mappings are backed by physical + pages, and the entire map is cleared at allocation time. This can cause + significant delays during a userspace malloc() as the C library does an + anonymous mapping and the kernel then does a memset for the entire map. + + However, for memory that isn't required to be precleared - such as that + returned by malloc() - mmap() can take a MAP_UNINITIALIZE flag to indicate + to the kernel that it shouldn't bother clearing the memory before + returning it. Note that CONFIG_MMAP_ALLOW_UNINITIALIZE must be enabled to + permit this, otherwise the flag will be ignored. + + uClibc uses this to speed up malloc(), and the ELF-FDPIC binfmt uses this + to allocate the brk and stack region. + (*) A list of all the private copy and anonymous mappings on the system is visible through /proc/maps in no-MMU mode.
fs/binfmt_elf_fdpic.c +2 -1
@@ -380,7 +380,8 @@ static int load_elf_fdpic_binary(struct linux_binprm *bprm,down_write (¤t->mm->mmap_sem); current->mm->start_brk =do_mmap (NULL ,0 , stack_size,PROT_READ |PROT_WRITE |PROT_EXEC , -MAP_PRIVATE |MAP_ANONYMOUS |MAP_GROWSDOWN , +MAP_PRIVATE |MAP_ANONYMOUS | +MAP_UNINITIALIZE |MAP_GROWSDOWN ,0 );if (IS_ERR_VALUE (current->mm->start_brk)) {
include/asm-generic/mman-common.h +5 -0
@@ -19,6 +19,11 @@#define MAP_TYPE 0x0f/* Mask for type of mapping */ #define MAP_FIXED 0x10/* Interpret addr exactly */ #define MAP_ANONYMOUS 0x20/* don't use a file */ +#ifdef CONFIG_MMAP_ALLOW_UNINITIALIZE +# define MAP_UNINITIALIZE 0x4000000/* For anonymous mmap, memory could be uninitialized */ +#else +# define MAP_UNINITIALIZE 0x0/* Don't support this flag */ +#endif #define MS_ASYNC 1/* sync memory asynchronously */ #define MS_INVALIDATE 2/* invalidate the caches */
init/Kconfig +22 -0
@@ -1069,6 +1069,28 @@ config SLOB endchoice +config MMAP_ALLOW_UNINITIALIZE + bool "Allow mmapped anonymous memory to be un-initialized" + depends on EMBEDDED && !MMU + default n + help + Normally, and according to the Linux spec, anonymous memory obtained + from mmap() has it's contents cleared before it is passed to + userspace. Enabling this config option allows you to request that + mmap() skip that if it is given an MAP_UNINITIALIZE flag, thus + providing a huge performance boost. If this option is not enabled, + then the flag will be ignored. + + This is taken advantage of by uClibc's malloc(), and also by + ELF-FDPIC binfmt's brk and stack allocator. + + Because of the obvious security issues, this option should only be + enabled on embedded devices where you control what is run in + userspace. Since that isn't generally a problem on no-MMU systems, + it is normally safe to say Y here. + + See Documentation/nommu-mmap.txt for more information. + config PROFILING bool "Profiling support (EXPERIMENTAL)" help
mm/nommu.c +5 -3
@@ -1143,9 +1143,6 @@ static int do_mmap_private(struct vm_area_struct *vma,if (ret < rlen)memset (base + ret,0 , rlen - ret); - }else { -/* if it's an anonymous mapping, then just clear it */ -memset (base,0 , rlen); }return 0 ;
@@ -1343,6 +1340,11 @@ unsigned long do_mmap_pgoff(struct file *file, goto error_just_free;add_nommu_region (region); +/* clear anonymous mappings that don't ask for uninitialized data */ +if (!vma->vm_file && !(flags &MAP_UNINITIALIZE )) +memset ((void *)region->vm_start,0 , + region->vm_end - region->vm_start); +/* okay... we have a mapping; now we have to register it */ result = vma->vm_start;
PATCH v2 1/1 NOMMU: fix malloc performance by adding uninitialized flag
From: Jie Zhang <[email protected]>
The NOMMU code currently clears all anonymous mmapped memory. While this is what we want in the default case, all memory allocation from userspace under NOMMU has to go through this interface, including malloc() which is allowed to return uninitialized memory. This can easily be a significant performance penalty. So for constrained embedded systems were security is irrelevant, allow people to avoid clearing memory unnecessarily.
This also alters the ELF-FDPIC binfmt such that it obtains uninitialised memory for the brk and stack region.
---
v2
- add a 'D' suffix in define names
David: i think i slurped all your changes into this and changed all the
occurrences to have a D suffix ...
Documentation/nommu-mmap.txt | 26 ++++++++++++++++++++++++++ fs/binfmt_elf_fdpic.c | 3 ++- include/asm-generic/mman-common.h | 5 +++++ init/Kconfig | 22 ++++++++++++++++++++++ mm/nommu.c | 8 +++++--- 5 files changed, 60 insertions(+), 4 deletions(-)
Documentation/nommu-mmap.txt +26 -0
@@ -119,6 +119,32 @@ FURTHER NOTES ON NO-MMU MMAP granule but will only discard the excess if appropriately configured as this has an effect on fragmentation. + (*) The memory allocated by a request for an anonymous mapping will normally + be cleared by the kernel before being returned in accordance with the + Linux man pages (ver 2.22 or later). + + In the MMU case this can be achieved with reasonable performance as + regions are backed by virtual pages, with the contents only being mapped + to cleared physical pages when a write happens on that specific page + (prior to which, the pages are effectively mapped to the global zero page + from which reads can take place). This spreads out the time it takes to + initialize the contents of a page - depending on the write-usage of the + mapping. + + In the no-MMU case, however, anonymous mappings are backed by physical + pages, and the entire map is cleared at allocation time. This can cause + significant delays during a userspace malloc() as the C library does an + anonymous mapping and the kernel then does a memset for the entire map. + + However, for memory that isn't required to be precleared - such as that + returned by malloc() - mmap() can take a MAP_UNINITIALIZED flag to + indicate to the kernel that it shouldn't bother clearing the memory before + returning it. Note that CONFIG_MMAP_ALLOW_UNINITIALIZED must be enabled + to permit this, otherwise the flag will be ignored. + + uClibc uses this to speed up malloc(), and the ELF-FDPIC binfmt uses this + to allocate the brk and stack region. + (*) A list of all the private copy and anonymous mappings on the system is visible through /proc/maps in no-MMU mode.
fs/binfmt_elf_fdpic.c +2 -1
@@ -380,7 +380,8 @@ static int load_elf_fdpic_binary(struct linux_binprm *bprm,down_write (¤t->mm->mmap_sem); current->mm->start_brk =do_mmap (NULL ,0 , stack_size,PROT_READ |PROT_WRITE |PROT_EXEC , -MAP_PRIVATE |MAP_ANONYMOUS |MAP_GROWSDOWN , +MAP_PRIVATE |MAP_ANONYMOUS | +MAP_UNINITIALIZED |MAP_GROWSDOWN ,0 );if (IS_ERR_VALUE (current->mm->start_brk)) {
include/asm-generic/mman-common.h +5 -0
@@ -19,6 +19,11 @@#define MAP_TYPE 0x0f/* Mask for type of mapping */ #define MAP_FIXED 0x10/* Interpret addr exactly */ #define MAP_ANONYMOUS 0x20/* don't use a file */ +#ifdef CONFIG_MMAP_ALLOW_UNINITIALIZED +# define MAP_UNINITIALIZED 0x4000000/* For anonymous mmap, memory could be uninitialized */ +#else +# define MAP_UNINITIALIZED 0x0/* Don't support this flag */ +#endif #define MS_ASYNC 1/* sync memory asynchronously */ #define MS_INVALIDATE 2/* invalidate the caches */
init/Kconfig +22 -0
@@ -1069,6 +1069,28 @@ config SLOB endchoice +config MMAP_ALLOW_UNINITIALIZED + bool "Allow mmaped anonymous memory to be un-initialized" + depends on EMBEDDED && !MMU + default n + help + Normally, and according to the Linux spec, anonymous memory obtained + from mmap() has it's contents cleared before it is passed to + userspace. Enabling this config option allows you to request that + mmap() skip that if it is given an MAP_UNINITIALIZED flag, thus + providing a huge performance boost. If this option is not enabled, + then the flag will be ignored. + + This is taken advantage of by uClibc's malloc(), and also by + ELF-FDPIC binfmt's brk and stack allocator. + + Because of the obvious security issues, this option should only be + enabled on embedded devices where you control what is run in + userspace. Since that isn't generally a problem on no-MMU systems, + it is normally safe to say Y here. + + See Documentation/nommu-mmap.txt for more information. + config PROFILING bool "Profiling support (EXPERIMENTAL)" help
mm/nommu.c +5 -3
@@ -1143,9 +1143,6 @@ static int do_mmap_private(struct vm_area_struct *vma,if (ret < rlen)memset (base + ret,0 , rlen - ret); - }else { -/* if it's an anonymous mapping, then just clear it */ -memset (base,0 , rlen); }return 0 ;
@@ -1343,6 +1340,11 @@ unsigned long do_mmap_pgoff(struct file *file, goto error_just_free;add_nommu_region (region); +/* clear anonymous mappings that don't ask for uninitialized data */ +if (!vma->vm_file && !(flags &MAP_UNINITIALIZED )) +memset ((void *)region->vm_start,0 , + region->vm_end - region->vm_start); +/* okay... we have a mapping; now we have to register it */ result = vma->vm_start;
Signature · 1 line
1.6.5
Re: [PATCH v2] NOMMU: fix malloc performance by adding uninitialized flag
Quoted 9 lines · Mike Frysinger
From: Jie Zhang <[email protected]>
The NOMMU code currently clears all anonymous mmapped memory. While this is what we want in the default case, all memory allocation from userspace under NOMMU has to go through this interface, including
malloc()which is allowed to return uninitialized memory. This can easily be a significant performance penalty. So for constrained embedded systems were security is irrelevant, allow people to avoid clearing memory unnecessarily.
Quoted 7 lines
This also alters the ELF-FDPIC binfmt such that it obtains uninitialised memory for the brk and stack region.
Signed-off-by Jie Zhang <[email protected]>Signed-off-by Robin Getz <[email protected]>Signed-off-by Mike Frysinger <[email protected]>Signed-off-by David Howells <[email protected]>
Re: [PATCH v2] NOMMU: fix malloc performance by adding uninitialized flag
In reply to Mike Frysingerinit/Kconfig config SLOB · line 1073
Quoted 1 line · Mike Frysinger
init/Kconfig +1
-0+ bool "Allow mmaped anonymous memory to be un-initialized"
Can you change that to be 'mmapped' and 'uninitialized'?
Other than that, it looks good.
David
PATCH v3 1/1 NOMMU: fix malloc performance by adding uninitialized flag
From: Jie Zhang <[email protected]>
The NOMMU code currently clears all anonymous mmapped memory. While this is what we want in the default case, all memory allocation from userspace under NOMMU has to go through this interface, including malloc() which is allowed to return uninitialized memory. This can easily be a significant performance penalty. So for constrained embedded systems were security is irrelevant, allow people to avoid clearing memory unnecessarily.
This also alters the ELF-FDPIC binfmt such that it obtains uninitialised memory for the brk and stack region.
---
v3
- tweak kconfig desc
Documentation/nommu-mmap.txt | 26 ++++++++++++++++++++++++++ fs/binfmt_elf_fdpic.c | 3 ++- include/asm-generic/mman-common.h | 5 +++++ init/Kconfig | 22 ++++++++++++++++++++++ mm/nommu.c | 8 +++++--- 5 files changed, 60 insertions(+), 4 deletions(-)
Documentation/nommu-mmap.txt +26 -0
@@ -119,6 +119,32 @@ FURTHER NOTES ON NO-MMU MMAP granule but will only discard the excess if appropriately configured as this has an effect on fragmentation. + (*) The memory allocated by a request for an anonymous mapping will normally + be cleared by the kernel before being returned in accordance with the + Linux man pages (ver 2.22 or later). + + In the MMU case this can be achieved with reasonable performance as + regions are backed by virtual pages, with the contents only being mapped + to cleared physical pages when a write happens on that specific page + (prior to which, the pages are effectively mapped to the global zero page + from which reads can take place). This spreads out the time it takes to + initialize the contents of a page - depending on the write-usage of the + mapping. + + In the no-MMU case, however, anonymous mappings are backed by physical + pages, and the entire map is cleared at allocation time. This can cause + significant delays during a userspace malloc() as the C library does an + anonymous mapping and the kernel then does a memset for the entire map. + + However, for memory that isn't required to be precleared - such as that + returned by malloc() - mmap() can take a MAP_UNINITIALIZED flag to + indicate to the kernel that it shouldn't bother clearing the memory before + returning it. Note that CONFIG_MMAP_ALLOW_UNINITIALIZED must be enabled + to permit this, otherwise the flag will be ignored. + + uClibc uses this to speed up malloc(), and the ELF-FDPIC binfmt uses this + to allocate the brk and stack region. + (*) A list of all the private copy and anonymous mappings on the system is visible through /proc/maps in no-MMU mode.
fs/binfmt_elf_fdpic.c +2 -1
@@ -380,7 +380,8 @@ static int load_elf_fdpic_binary(struct linux_binprm *bprm,down_write (¤t->mm->mmap_sem); current->mm->start_brk =do_mmap (NULL ,0 , stack_size,PROT_READ |PROT_WRITE |PROT_EXEC , -MAP_PRIVATE |MAP_ANONYMOUS |MAP_GROWSDOWN , +MAP_PRIVATE |MAP_ANONYMOUS | +MAP_UNINITIALIZED |MAP_GROWSDOWN ,0 );if (IS_ERR_VALUE (current->mm->start_brk)) {
include/asm-generic/mman-common.h +5 -0
@@ -19,6 +19,11 @@#define MAP_TYPE 0x0f/* Mask for type of mapping */ #define MAP_FIXED 0x10/* Interpret addr exactly */ #define MAP_ANONYMOUS 0x20/* don't use a file */ +#ifdef CONFIG_MMAP_ALLOW_UNINITIALIZED +# define MAP_UNINITIALIZED 0x4000000/* For anonymous mmap, memory could be uninitialized */ +#else +# define MAP_UNINITIALIZED 0x0/* Don't support this flag */ +#endif #define MS_ASYNC 1/* sync memory asynchronously */ #define MS_INVALIDATE 2/* invalidate the caches */
init/Kconfig +22 -0
@@ -1069,6 +1069,28 @@ config SLOB endchoice +config MMAP_ALLOW_UNINITIALIZED + bool "Allow mmapped anonymous memory to be uninitialized" + depends on EMBEDDED && !MMU + default n + help + Normally, and according to the Linux spec, anonymous memory obtained + from mmap() has it's contents cleared before it is passed to + userspace. Enabling this config option allows you to request that + mmap() skip that if it is given an MAP_UNINITIALIZED flag, thus + providing a huge performance boost. If this option is not enabled, + then the flag will be ignored. + + This is taken advantage of by uClibc's malloc(), and also by + ELF-FDPIC binfmt's brk and stack allocator. + + Because of the obvious security issues, this option should only be + enabled on embedded devices where you control what is run in + userspace. Since that isn't generally a problem on no-MMU systems, + it is normally safe to say Y here. + + See Documentation/nommu-mmap.txt for more information. + config PROFILING bool "Profiling support (EXPERIMENTAL)" help
mm/nommu.c +5 -3
@@ -1143,9 +1143,6 @@ static int do_mmap_private(struct vm_area_struct *vma,if (ret < rlen)memset (base + ret,0 , rlen - ret); - }else { -/* if it's an anonymous mapping, then just clear it */ -memset (base,0 , rlen); }return 0 ;
@@ -1343,6 +1340,11 @@ unsigned long do_mmap_pgoff(struct file *file, goto error_just_free;add_nommu_region (region); +/* clear anonymous mappings that don't ask for uninitialized data */ +if (!vma->vm_file && !(flags &MAP_UNINITIALIZED )) +memset ((void *)region->vm_start,0 , + region->vm_end - region->vm_start); +/* okay... we have a mapping; now we have to register it */ result = vma->vm_start;
Signature · 1 line
1.6.5
Re: [PATCH v3] NOMMU: fix malloc performance by adding uninitialized flag
Jivin Mike Frysinger lays it down ...
Quoted 9 lines
From: Jie Zhang <[email protected]>
The NOMMU code currently clears all anonymous mmapped memory. While this is what we want in the default case, all memory allocation from userspace under NOMMU has to go through this interface, including
malloc()which is allowed to return uninitialized memory. This can easily be a significant performance penalty. So for constrained embedded systems were security is irrelevant, allow people to avoid clearing memory unnecessarily.
Quoted 8 lines
This also alters the ELF-FDPIC binfmt such that it obtains uninitialised memory for the brk and stack region.
Signed-off-by Jie Zhang <[email protected]>Signed-off-by Robin Getz <[email protected]>Signed-off-by Mike Frysinger <[email protected]>Signed-off-by David Howells <[email protected]>Acked-by Paul Mundt <[email protected]>
Cheers,
Davidm
Quoted 141 lines
v3
- tweak kconfig descDocumentation/nommu-mmap.txt | 26 ++++++++++++++++++++++++++ fs/binfmt_elf_fdpic.c | 3 ++- include/asm-generic/mman-common.h | 5 +++++ init/Kconfig | 22 ++++++++++++++++++++++ mm/nommu.c | 8 +++++--- 5 files changed, 60 insertions(+), 4 deletions(-)Documentation/nommu-mmap.txt +26
-0@@ -119,6 +119,32 @@ FURTHER NOTES ON NO-MMU MMAP granule but will only discard the excess if appropriately configured as this has an effect on fragmentation. + (*) The memory allocated by a request for an anonymous mapping will normally + be cleared by the kernel before being returned in accordance with the + Linux man pages (ver 2.22 or later). + + In the MMU case this can be achieved with reasonable performance as + regions are backed by virtual pages, with the contents only being mapped + to cleared physical pages when a write happens on that specific page + (prior to which, the pages are effectively mapped to the global zero page + from which reads can take place). This spreads out the time it takes to + initialize the contents of a page - depending on the write-usage of the + mapping. + + In the no-MMU case, however, anonymous mappings are backed by physical + pages, and the entire map is cleared at allocation time. This can cause + significant delays during a userspace malloc() as the C library does an + anonymous mapping and the kernel then does a memset for the entire map. + + However, for memory that isn't required to be precleared - such as that + returned by malloc() - mmap() can take a MAP_UNINITIALIZED flag to + indicate to the kernel that it shouldn't bother clearing the memory before + returning it. Note that CONFIG_MMAP_ALLOW_UNINITIALIZED must be enabled + to permit this, otherwise the flag will be ignored. + + uClibc uses this to speed up malloc(), and the ELF-FDPIC binfmt uses this + to allocate the brk and stack region. + (*) A list of all the private copy and anonymous mappings on the system is visible through /proc/maps in no-MMU mode.fs/binfmt_elf_fdpic.c +2
-1@@ -380,7 +380,8 @@ static int load_elf_fdpic_binary(struct linux_binprm *bprm,down_write (¤t->mm->mmap_sem); current->mm->start_brk =do_mmap (NULL ,0 , stack_size,PROT_READ |PROT_WRITE |PROT_EXEC , -MAP_PRIVATE |MAP_ANONYMOUS |MAP_GROWSDOWN , +MAP_PRIVATE |MAP_ANONYMOUS | +MAP_UNINITIALIZED |MAP_GROWSDOWN ,0 );if (IS_ERR_VALUE (current->mm->start_brk)) {include/asm-generic/mman-common.h +5
-0@@ -19,6 +19,11 @@#define MAP_TYPE 0x0f/* Mask for type of mapping */ #define MAP_FIXED 0x10/* Interpret addr exactly */ #define MAP_ANONYMOUS 0x20/* don't use a file */ +#ifdef CONFIG_MMAP_ALLOW_UNINITIALIZED +# define MAP_UNINITIALIZED 0x4000000/* For anonymous mmap, memory could be uninitialized */ +#else +# define MAP_UNINITIALIZED 0x0/* Don't support this flag */ +#endif #define MS_ASYNC 1/* sync memory asynchronously */ #define MS_INVALIDATE 2/* invalidate the caches */ init/Kconfig +22
-0@@ -1069,6 +1069,28 @@ config SLOB endchoice +config MMAP_ALLOW_UNINITIALIZED + bool "Allow mmapped anonymous memory to be uninitialized" + depends on EMBEDDED && !MMU + default n + help + Normally, and according to the Linux spec, anonymous memory obtained + from mmap() has it's contents cleared before it is passed to + userspace. Enabling this config option allows you to request that + mmap() skip that if it is given an MAP_UNINITIALIZED flag, thus + providing a huge performance boost. If this option is not enabled, + then the flag will be ignored. + + This is taken advantage of by uClibc's malloc(), and also by + ELF-FDPIC binfmt's brk and stack allocator. + + Because of the obvious security issues, this option should only be + enabled on embedded devices where you control what is run in + userspace. Since that isn't generally a problem on no-MMU systems, + it is normally safe to say Y here. + + See Documentation/nommu-mmap.txt for more information. + config PROFILING bool "Profiling support (EXPERIMENTAL)" helpmm/nommu.c +5
-3@@ -1143,9 +1143,6 @@ static int do_mmap_private(struct vm_area_struct *vma,if (ret < rlen)memset (base + ret,0 , rlen - ret); - }else { -/* if it's an anonymous mapping, then just clear it */ -memset (base,0 , rlen); }return 0 ;@@ -1343,6 +1340,11 @@ unsigned long do_mmap_pgoff(struct file *file, goto error_just_free;add_nommu_region (region); +/* clear anonymous mappings that don't ask for uninitialized data */ +if (!vma->vm_file && !(flags &MAP_UNINITIALIZED )) +memset ((void *)region->vm_start,0 , + region->vm_end - region->vm_start); +/* okay... we have a mapping; now we have to register it */ result = vma->vm_start;Signature · 1 line
1.6.5
Signature · 2 lines
David McCullough, [email protected], Ph:+61 734352815 McAfee - SnapGear http://www.snapgear.com http://www.uCdot.org
Re: [PATCH v3] NOMMU: fix malloc performance by adding uninitialized flag
In reply to Mike FrysingerQuoted 9 lines · Mike Frysinger
From: Jie Zhang <[email protected]>
The NOMMU code currently clears all anonymous mmapped memory. While this is what we want in the default case, all memory allocation from userspace under NOMMU has to go through this interface, including
malloc()which is allowed to return uninitialized memory. This can easily be a significant performance penalty. So for constrained embedded systems were security is irrelevant, allow people to avoid clearing memory unnecessarily.
Quoted 8 lines
This also alters the ELF-FDPIC binfmt such that it obtains uninitialised memory for the brk and stack region.
Signed-off-by Jie Zhang <[email protected]>Signed-off-by Robin Getz <[email protected]>Signed-off-by Mike Frysinger <[email protected]>Signed-off-by David Howells <[email protected]>Acked-by Paul Mundt <[email protected]>
Quoted 137 lines
---
v3
- tweak kconfig descDocumentation/nommu-mmap.txt | 26 ++++++++++++++++++++++++++ fs/binfmt_elf_fdpic.c | 3 ++- include/asm-generic/mman-common.h | 5 +++++ init/Kconfig | 22 ++++++++++++++++++++++ mm/nommu.c | 8 +++++--- 5 files changed, 60 insertions(+), 4 deletions(-)Documentation/nommu-mmap.txt +26
-0@@ -119,6 +119,32 @@ FURTHER NOTES ON NO-MMU MMAP granule but will only discard the excess if appropriately configured as this has an effect on fragmentation. + (*) The memory allocated by a request for an anonymous mapping will normally + be cleared by the kernel before being returned in accordance with the + Linux man pages (ver 2.22 or later). + + In the MMU case this can be achieved with reasonable performance as + regions are backed by virtual pages, with the contents only being mapped + to cleared physical pages when a write happens on that specific page + (prior to which, the pages are effectively mapped to the global zero page + from which reads can take place). This spreads out the time it takes to + initialize the contents of a page - depending on the write-usage of the + mapping. + + In the no-MMU case, however, anonymous mappings are backed by physical + pages, and the entire map is cleared at allocation time. This can cause + significant delays during a userspace malloc() as the C library does an + anonymous mapping and the kernel then does a memset for the entire map. + + However, for memory that isn't required to be precleared - such as that + returned by malloc() - mmap() can take a MAP_UNINITIALIZED flag to + indicate to the kernel that it shouldn't bother clearing the memory before + returning it. Note that CONFIG_MMAP_ALLOW_UNINITIALIZED must be enabled + to permit this, otherwise the flag will be ignored. + + uClibc uses this to speed up malloc(), and the ELF-FDPIC binfmt uses this + to allocate the brk and stack region. + (*) A list of all the private copy and anonymous mappings on the system is visible through /proc/maps in no-MMU mode.fs/binfmt_elf_fdpic.c +2
-1@@ -380,7 +380,8 @@ static int load_elf_fdpic_binary(struct linux_binprm *bprm,down_write (¤t->mm->mmap_sem); current->mm->start_brk =do_mmap (NULL ,0 , stack_size,PROT_READ |PROT_WRITE |PROT_EXEC , -MAP_PRIVATE |MAP_ANONYMOUS |MAP_GROWSDOWN , +MAP_PRIVATE |MAP_ANONYMOUS | +MAP_UNINITIALIZED |MAP_GROWSDOWN ,0 );if (IS_ERR_VALUE (current->mm->start_brk)) {include/asm-generic/mman-common.h +5
-0@@ -19,6 +19,11 @@#define MAP_TYPE 0x0f/* Mask for type of mapping */ #define MAP_FIXED 0x10/* Interpret addr exactly */ #define MAP_ANONYMOUS 0x20/* don't use a file */ +#ifdef CONFIG_MMAP_ALLOW_UNINITIALIZED +# define MAP_UNINITIALIZED 0x4000000/* For anonymous mmap, memory could be uninitialized */ +#else +# define MAP_UNINITIALIZED 0x0/* Don't support this flag */ +#endif #define MS_ASYNC 1/* sync memory asynchronously */ #define MS_INVALIDATE 2/* invalidate the caches */ init/Kconfig +22
-0@@ -1069,6 +1069,28 @@ config SLOB endchoice +config MMAP_ALLOW_UNINITIALIZED + bool "Allow mmapped anonymous memory to be uninitialized" + depends on EMBEDDED && !MMU + default n + help + Normally, and according to the Linux spec, anonymous memory obtained + from mmap() has it's contents cleared before it is passed to + userspace. Enabling this config option allows you to request that + mmap() skip that if it is given an MAP_UNINITIALIZED flag, thus + providing a huge performance boost. If this option is not enabled, + then the flag will be ignored. + + This is taken advantage of by uClibc's malloc(), and also by + ELF-FDPIC binfmt's brk and stack allocator. + + Because of the obvious security issues, this option should only be + enabled on embedded devices where you control what is run in + userspace. Since that isn't generally a problem on no-MMU systems, + it is normally safe to say Y here. + + See Documentation/nommu-mmap.txt for more information. + config PROFILING bool "Profiling support (EXPERIMENTAL)" helpmm/nommu.c +5
-3@@ -1143,9 +1143,6 @@ static int do_mmap_private(struct vm_area_struct *vma,if (ret < rlen)memset (base + ret,0 , rlen - ret); - }else { -/* if it's an anonymous mapping, then just clear it */ -memset (base,0 , rlen); }return 0 ;@@ -1343,6 +1340,11 @@ unsigned long do_mmap_pgoff(struct file *file, goto error_just_free;add_nommu_region (region); +/* clear anonymous mappings that don't ask for uninitialized data */ +if (!vma->vm_file && !(flags &MAP_UNINITIALIZED )) +memset ((void *)region->vm_start,0 , + region->vm_end - region->vm_start); +/* okay... we have a mapping; now we have to register it */ result = vma->vm_start;
Signature · 5 lines
------------------------------------------------------------------------ Greg Ungerer -- Principal Engineer EMAIL: [email protected] SnapGear Group, McAfee PHONE: +61 7 3435 2888 825 Stanley St, FAX: +61 7 3891 3630 Woolloongabba, QLD, 4102, Australia WEB: http://www.SnapGear.com
Re: [uClinux-dev] [PATCH] NOMMU: fix malloc performance by addinguninitialized flag
In reply to Mike FrysingerOn Tue 13 Oct 2009 03:44, Mike Frysinger pondered:
Quoted 11 lines
From: Jie Zhang <[email protected]>
The no-mmu code currently clears all anonymous mmap-ed memory. While this is what we want in the default case, all memory allocation from userspace under no-mmu has to go through this interface, including
malloc()which is allowed to return uninitialized memory. This can easily be a significant performance slow down. So for constrained embedded systems were security is irrelevant, allow people to avoid unnecessarily clearing memory.Signed-off-by Jie Zhang <[email protected]>Signed-off-by Robin Getz <[email protected]>
Should be [email protected] -- [email protected] does not exist and will bounce.
Quoted 11 lines
Summary is, when I run the app "time test",
on x86:
real 0m0.066s user 0m0.008s sys 0m0.058son Blackfin:
real 3m 37.69s user 0m 0.04s sys 3m 37.58s
What would be relevant would be Blackfin with clearing and blackfin without
Quoted 5 lines
---------------------------------------------------------------
So now the question is,
Keep the same behave as MMU but with bad performance, or keep the same performance as MMU but without the same behave, Which one is more important?
Behaviour.
You can implement a /dev/pages that is a copy of /dev/zero in tiny amounts of code and use that for the specific problem cases.
Alan