Big memory, no struct page allocation
Dear all,
Hi, I've got a silly but serious question. I want to allocate a large buffer (>512MB) in kernel. Normally you use __get_free_page and handle it with page pointers. But when get to very large (say 1024MB), I will need to use 2 level of page pointer indirection to carry the page pointer array. I also find the total size of page struct is quite large when using lots of pages, what I want is to use memory pages without struct page, is this possible? By the way, can I use lots of memory in the kernel, something like 1GB of memory allocation when physically RAM available? Please give advise. Thanks.
regards,
David
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/
Quoted 9 lines · David Chow
Hi, I've got a silly but serious question. I want to allocate a large buffer (>512MB) in kernel. Normally you use
__get_free_pageand handle it with page pointers. But when get to very large (say 1024MB), I will need to use 2 level of page pointer indirection to carry the page pointer array. I also find the total size of page struct is quite large when using lots of pages, what I want is to use memory pages without struct page, is this possible? By the way, can I use lots of memory in the kernel, something like 1GB of memory allocation when physically RAM available? Please give advise. Thanks.
Try allocating it at boot-time with the bootmem allocator.
Cheers,
Bill
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/
Quoted 7 lines · William Lee Irwin III
Quoted 3 lines · David Chow
Hi, I've got a silly but serious question. I want to allocate a large buffer (>512MB) in kernel. Normally you use
__get_free_pageand handle it with page pointers. But when get to very large (say 1024MB), I will need to use 2 level of page pointer indirection to carry the page pointer array. I also find the total size of page struct is quite large when using lots of pages, what I want is to use memory pages without struct page, is this possible? By the way, can I use lots of memory in the kernel, something like 1GB of memory allocation when physically RAM available? Please give advise. Thanks.
Quoted 7 lines
Try allocating it at boot-time with the bootmem allocator.
Cheers,
Bill
Thanks for suggestions, you mean this will allow no struct page or can use memory more than 1GB? Please make clear on direction, I would love to know. Thanks.
regards,
David
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/
Quoted 1 line · William Lee Irwin III
Quoted 1 line
Try allocating it at boot-time with the bootmem allocator.
Quoted 3 lines · David Chow
Thanks for suggestions, you mean this will allow no struct page or can use memory more than 1GB? Please make clear on direction, I would love to know. Thanks.
On 32-bit machines with 3:1 process address space splits yes.
In this case you're far better off playing games with the highmem initialization in order to slice the memory out of there and kmap it.
Cheers,
Bill
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/
Quoted 14 lines · William Lee Irwin III
Quoted 3 lines · William Lee Irwin III
Quoted 3 lines
Try allocating it at boot-time with the bootmem allocator.
Quoted 3 lines · David Chow
Thanks for suggestions, you mean this will allow no struct page or can use memory more than 1GB? Please make clear on direction, I would love to know. Thanks.
Quoted 10 lines
On 32-bit machines with 3:1 process address space splits yes.
In this case you're far better off playing games with the highmem initialization in order to slice the memory out of there and kmap it.
Cheers,
Bill
In other words, even I have 2G physical memory, I cannot have benefits of using all memory for pagecache, this also means I cannot create any cache beyong a 1G size in kernel. That's a pitty for 32-bit systems, with himem, how does it work?
David
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/
Quoted 4 lines · David Chow
In other words, even I have 2G physical memory, I cannot have benefits of using all memory for pagecache, this also means I cannot create any cache beyong a 1G size in kernel. That's a pitty for 32-bit systems, with himem, how does it work?
Pagecache can use highmem just fine.
regards,
Rik
Signature · 8 lines
Bravely reimplemented by the knights who say "NIH". http://www.surriel.com/ http://distro.conectiva.com/ -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/
Quoted 4 lines · David Chow
Quoted 4 lines
In other words, even I have 2G physical memory, I cannot have benefits of using all memory for pagecache, this also means I cannot create any cache beyong a 1G size in kernel. That's a pitty for 32-bit systems, with himem, how does it work?
Quoted 3 lines · Rik van Riel
Pagecache can use highmem just fine.
regards,
Rik
Yes, pagecache doesn't care where it is, it just works with the struct pages for the memory. Things that are more internal like dcache and buffer cache need to be allocated from ZONE_NORMAL, as the kernel actually touches that memory directly.
Cheers,
Bill
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/
Quoted 19 lines · William Lee Irwin III
Quoted 6 lines · David Chow
Quoted 6 lines
In other words, even I have 2G physical memory, I cannot have benefits of using all memory for pagecache, this also means I cannot create any cache beyong a 1G size in kernel. That's a pitty for 32-bit systems, with himem, how does it work?
Quoted 5 lines · Rik van Riel
Pagecache can use highmem just fine.
regards,
Rik
Quoted 10 lines
Yes, pagecache doesn't care where it is, it just works with the struct pages for the memory. Things that are more internal like dcache and buffer cache need to be allocated from
ZONE_NORMAL, as the kernel actually touches that memory directly.Cheers,
Bill
Thanks for advice, that means allocation of slab cache never gets over 1G? Since you mention dcache, where dcache uses kmem_cache_create() calls, or it depends on the flags pass to kmem_cache_create()? What about kmalloc()?
Since before access a page, we have to do kmap(page), how does this pointer address work? I found that if my machine have less than physical 1G RAM (actually somewhere between 900-940M), I don't have to call kmap() before really accessing the page data, if more than this amount of memory, it will result an oops. It seems for system that has more than 900M memory, kernel handle page data differently (need to use kmap before accessing the page data). Is it true that kmap only translate the page into a virtual address if more than 1G RAM or leave it physical is less than 1G RAM? I am a bit confuse in this behaviour of kmap().
regards,
David
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/