Memory allocation
Hello!
I encountered with problem: one process can not allocate more then 2Gb of memory. Kernel compiled with CONFIG_HIGHMEM4G=y, CONFIG_HIGHMEM=y. Kernel is 2.4.0
As far as I know on i386 linux process has got 32 bit address space. It means that actually about 3Gb of memory should be available.
I tried to call getrlimit(). It shows only 2G available memory and there is no way to increase it.
Could you say me are there any solutions? Might be on i386 linux process can not use more than 2Gb of memory de facto? But I don't see the reason for it:
there is unsigned long type uses everywhere in kernel sources for memory allocation.
--
Regards,
Ivan Stepnikov.
Quoted 2 lines · you
I encountered with problem: one process can not allocate more then 2Gb of memory.
This is a problem that I have run into myself. I am no kernel expert, but I think I understand how this issue. Here is how the standard kernel maps the 4Gb 32-bit address space from the process's point of view:
128MB program executable mapped (twice)
128MB + program heap
1GB mmap() starts here
3GB kernel
You can see this for yourself by looking at /proc/pid/maps, where pid is the PID of the process in question.
Now glibc()'s malloc uses mmap() for 'large' allocations, so you get 2GB maximum memory. The way around this is to change the various numbers in the left-hand column.
For example, you can try the patch per-process-3.5G-IA32-no-PAE-1, at /pub/linux/kernel/people/andrea/patches/v2.4/2.4.0-test11-pre5/ on ftp.kernel.org. This will make the kernel space start at 3.5G (so that CONFIG_4G is required to use more than 384MB of physical RAM) and the mmap() space start at 224MB, giving 3G288MB of address space for mmap(). Note that only 96MB is then available for {two copies of your executable plus your program heap}.
This is more or less the most you can do, but your needs may be best suited by something in between. The above patch is quite short, so it is easy to figure out how to do that. The only hidden restriction is that the size of the kernel space must be a power of 2: 2GB, 1GB, 512MB, etc. As explained to me, this is so that the kernel can easily test a pointer to see whether it is to kernel space or user space.
Cheers,
Wayne
[Ivan Stepnikov]
Quoted 2 lines
I tried to call
getrlimit(). It shows only 2G available memory and there is no way to increase it.
Right. Architectural limit. There needs to be some room in the address space for kernel stuff, I/O, etc -- in Linux at least, having to play with your page tables every single time you enter a system call or IRQ handler would be considered a Bad Thing.
Quoted 1 line
Could you say me are there any solutions?
a) If you have that much memory, maybe you need a 64-bit CPU.
b) fork() and do IPC. It's the Unix Way.
Peter