In kvm, a single process cannot allocate more than 3.6GB ram. Why?

DISCUSSION Tanel Kokk 3 participants 10 messages Dec 11, 2009 – Dec 13, 2009 KVM

lore

Share & tools

lore

Tanel Kokk

author

2009-12-11 09:03 UTC #0 lore raw

Does anybody have a glue why process are not able to allocate more than 3.6GB memory ?

What should i do to allow process allocate more than 3.6GB memory? Is it possible at all?

I have hostserver:
linux kernel: 2.6.30.9 (arch amd64)
RAM: 64GB
CPUs: 2xQuadCore AMD Opteron 2378

Then I have several kvm guests on it, like:
linux kernel: 2.6.30.4 (arch amd64)
RAM: 6GB
CPUs: 2xQEMU Virtual CPU version 0.9.1 (vendor: AuthenticAMD) cpu flags: fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx lm pni hypervisor

VM use virtio for net and block drivers.

Tanel

Thomas Mueller

2009-12-11 10:01 UTC #1 lore raw

Am Fri, 11 Dec 2009 11:03:27 +0200 schrieb Tanel Kokk:

Quoted 6 lines

Does anybody have a glue why process are not able to allocate more than 3.6GB memory ?
What should i do to allow process allocate more than 3.6GB memory? Is it possible at all?

just some thoughts:

* is qemu/kvm 64bit?
* check "ulimit -a" on console if a process is limited on memory usage (config file /etc/security/limits.conf)

- Thomas

Tanel Kokk

author

2009-12-11 10:14 UTC #2 lore raw

Quoted 8 lines · Thomas Mueller

Am Fri, 11 Dec 2009 11:03:27 +0200 schrieb Tanel Kokk:
Quoted 3 lines
Does anybody have a glue why process are not able to allocate more than 3.6GB memory ?

Quoted 7 lines

Quoted 3 lines
What should i do to allow process allocate more than 3.6GB memory? Is it possible at all?
just some thoughts:
* is qemu/kvm 64bit?

# file /usr/bin/kvm
/usr/bin/kvm: ELF 64-bit LSB executable, x86-64, version 1 (SYSV),
dynamically linked (uses shared libs), for GNU/Linux 2.6.8, stripped

Quoted 2 lines

* check "ulimit -a" on console if a process is limited on memory usage (config file /etc/security/limits.conf)

# ulimit -a
core file size          (blocks, -c) unlimited
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 53248
max locked memory       (kbytes, -l) unlimited
max memory size         (kbytes, -m) unlimited
open files                      (-n) 99999
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) unlimited
real-time priority              (-r) 0
stack size              (kbytes, -s) unlimited
cpu time               (seconds, -t) unlimited
max user processes              (-u) 4096
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

Tanel

Avi Kivity

2009-12-12 08:55 UTC #3 lore raw

In reply to Tanel Kokk

Quoted 6 lines · Tanel Kokk

Does anybody have a glue why process are not able to allocate more than 3.6GB memory ?
What should i do to allow process allocate more than 3.6GB memory? Is it possible at all?

A guest or host process? And what happens when you try?

Please describe what you are doing step by step with more details.

Signature · 2 lines

I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Tanel Kokk

author

2009-12-12 09:23 UTC #4 lore raw

Quoted 9 lines · Avi Kivity

Quoted 6 lines · Tanel Kokk
Does anybody have a glue why process are not able to allocate more than 3.6GB memory ?
What should i do to allow process allocate more than 3.6GB memory? Is it possible at all?
A guest or host process? And what happens when you try?

guest process

We discovered that if postgres process (on guest) used to use only 3.6GB memory regardless of guest had 8GB memory in common.

Then I made simple python script, which just eat memory:
------------------------------------
#!/usr/bin/python
import time, random

i = 0
while (i < 100000000):

    a[i] = random.random()
    i = i + 1

time.sleep(600)
------------------------------------
And I get python MemoryError, when script has devoured about 3.6GB ram

Tanel

Avi Kivity

2009-12-12 09:35 UTC #5 lore raw

Quoted 11 lines · Tanel Kokk

Quoted 9 lines · Avi Kivity
Quoted 7 lines · Tanel Kokk
Does anybody have a glue why process are not able to allocate more than 3.6GB memory ?
What should i do to allow process allocate more than 3.6GB memory? Is it possible at all?

Quoted 8 lines

Quoted 2 lines
A guest or host process? And what happens when you try?
guest process
We discovered that if postgres process (on guest) used to use only 3.6GB memory regardless of guest had 8GB memory in common.

That's probably a postgres configuration error (or you're using 32-bit postgres).

Quoted 5 lines

Then I made simple python script, which just eat memory:
------------------------------------
#!/usr/bin/python
import time, random

Quoted 9 lines

i = 0
while (i< 100000000):
     a[i] = random.random()
     i = i + 1
time.sleep(600)
------------------------------------
And I get python MemoryError, when script has devoured about 3.6GB ram

Doesn't mean much - Python may allocate a lot more memory when extending the array.

What does 'cat /proc/meminfo' in the guest say?

Signature · 2 lines

I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Tanel Kokk

author

2009-12-12 10:10 UTC #6 lore raw

Quoted 17 lines · Avi Kivity

Quoted 16 lines · Tanel Kokk
Quoted 5 lines · Avi Kivity
Quoted 3 lines · Tanel Kokk
Does anybody have a glue why process are not able to allocate more than 3.6GB memory ?
Quoted 7 lines
Quoted 5 lines
What should i do to allow process allocate more than 3.6GB memory? Is it
possible at all?
A guest or host process? And what happens when you try?
guest process

Quoted 8 lines

Quoted 4 lines
We discovered that if postgres process (on guest) used to use only 3.6GB memory regardless of guest had 8GB memory in common.
That's probably a postgres configuration error (or you're using 32-bit postgres).

postgres is 64bit:
root@lu2-kvm-db1 ~]# file /usr/lib/postgresql/8.3/bin/postgres /usr/lib/postgresql/8.3/bin/postgres: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.8, not stripped

Quoted 18 lines

Quoted 7 lines
Then I made simple python script, which just eat memory:
------------------------------------
#!/usr/bin/python
import time, random
Quoted 9 lines
i = 0
while (i< 100000000):
     a[i] = random.random()
     i = i + 1
time.sleep(600)
------------------------------------
And I get python MemoryError, when script has devoured about 3.6GB ram

Quoted 5 lines

Doesn't mean much - Python may allocate a lot more memory when extending the array.
What does 'cat /proc/meminfo' in the guest say?

[root@lu2-kvm-db1 ~]# cat /proc/meminfo

MemTotal:       10267308 kB
MemFree:          228944 kB
Buffers:           19680 kB
Cached:          6069524 kB
SwapCached:            0 kB
Active:          8175200 kB
Inactive:        1374884 kB
Active(anon):    3238044 kB
Inactive(anon):   544104 kB
Active(file):    4937156 kB
Inactive(file):   830780 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:             15968 kB
Writeback:             0 kB
AnonPages:       3461108 kB
Mapped:           327816 kB
Slab:             171452 kB
SReclaimable:     121756 kB
SUnreclaim:        49696 kB
PageTables:       244268 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     8727208 kB
Committed_AS:    6056168 kB
VmallocTotal:   34359738367 kB
VmallocUsed:       38180 kB
VmallocChunk:   34359645255 kB
DirectMap4k:        4032 kB
DirectMap2M:    10481664 kB

Tanel

Avi Kivity

2009-12-13 10:04 UTC #7 lore raw

Quoted 27 lines · Tanel Kokk

[root@lu2-kvm-db1 ~]# cat /proc/meminfo

MemTotal:       10267308 kB
MemFree:          228944 kB
Buffers:           19680 kB
Cached:          6069524 kB
SwapCached:            0 kB
Active:          8175200 kB
Inactive:        1374884 kB
Active(anon):    3238044 kB
Inactive(anon):   544104 kB
Active(file):    4937156 kB
Inactive(file):   830780 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:             15968 kB
Writeback:             0 kB
AnonPages:       3461108 kB
Mapped:           327816 kB
Slab:             171452 kB
SReclaimable:     121756 kB
SUnreclaim:        49696 kB
PageTables:       244268 kB
NFS_Unstable:          0 kB
Bounce:                0 kB

Quoted 9 lines

WritebackTmp:          0 kB
CommitLimit:     8727208 kB
Committed_AS:    6056168 kB
VmallocTotal:   34359738367 kB
VmallocUsed:       38180 kB
VmallocChunk:   34359645255 kB
DirectMap4k:        4032 kB
DirectMap2M:    10481664 kB

What guest kernel is this? What's the value of the guest's /proc/sys/vm/overcommit_memory?

Signature · 1 line

error compiling committee.c: too many arguments to function

Tanel Kokk

author

2009-12-13 11:30 UTC #8 lore raw

Quoted 38 lines · Avi Kivity

Quoted 27 lines · Tanel Kokk

[root@lu2-kvm-db1 ~]# cat /proc/meminfo

MemTotal:       10267308 kB
MemFree:          228944 kB
Buffers:           19680 kB
Cached:          6069524 kB
SwapCached:            0 kB
Active:          8175200 kB
Inactive:        1374884 kB
Active(anon):    3238044 kB
Inactive(anon):   544104 kB
Active(file):    4937156 kB
Inactive(file):   830780 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:             15968 kB
Writeback:             0 kB
AnonPages:       3461108 kB
Mapped:           327816 kB
Slab:             171452 kB
SReclaimable:     121756 kB
SUnreclaim:        49696 kB
PageTables:       244268 kB
NFS_Unstable:          0 kB
Bounce:                0 kB

Quoted 9 lines

WritebackTmp:          0 kB
CommitLimit:     8727208 kB
Committed_AS:    6056168 kB
VmallocTotal:   34359738367 kB
VmallocUsed:       38180 kB
VmallocChunk:   34359645255 kB
DirectMap4k:        4032 kB
DirectMap2M:    10481664 kB

Quoted 3 lines

What guest kernel is this? What's the value of the guest's /proc/sys/vm/overcommit_memory?

[root@lu2-kvm-db1 ~]# uname -a
Linux lu2-kvm-db1 2.6.30.7 #1 SMP Mon Sep 21 17:39:41 UTC 2009 x86_64 GNU/Linux

[root@lu2-kvm-db1 ~]# cat /proc/sys/vm/overcommit_memory 2

Tanel

Avi Kivity

2009-12-13 11:32 UTC #9 lore raw

Quoted 5 lines · Tanel Kokk

Quoted 5 lines
What guest kernel is this? What's the value of the guest's /proc/sys/vm/overcommit_memory?

Quoted 9 lines

[root@lu2-kvm-db1 ~]# uname -a
Linux lu2-kvm-db1 2.6.30.7 #1 SMP Mon Sep 21 17:39:41 UTC 2009 x86_64 GNU/Linux
[root@lu2-kvm-db1 ~]# cat /proc/sys/vm/overcommit_memory 2

Ah. Set it to zero or add some swap.

Signature · 1 line

error compiling committee.c: too many arguments to function