Skip to contentSkip to conversation

mmap failed: Cannot allocate memory when init dpdk eal

DISCUSSION zhangsha (A) 3 participants 4 messages Jan 30, 2015 – Feb 5, 2015 2 messages not in the archive

lore
Share & tools
lore

zhangsha (A)

author
#0 lore raw

Hi ，all

I am suffering from the problem mmap failed as followed when init dpdk eal.

Fri Jan 30 09:03:29 2015:EAL: Setting up memory...
Fri Jan 30 09:03:34 2015:EAL: map_all_hugepages(): mmap failed: Cannot allocate memory Fri Jan 30 09:03:34 2015:EAL: Failed to mmap 2 MB hugepages
Fri Jan 30 09:03:34 2015:EAL: Cannot init memory

Before I run the demo, the free hugepages of my host is :

cat /proc/meminfo

MemTotal:       132117056 kB
MemFree:        122040292 kB
Buffers:           10984 kB
Cached:           123056 kB
SwapCached:            0 kB
Active:           120812 kB
Inactive:          85860 kB
Active(anon):      79488 kB
Inactive(anon):      364 kB
Active(file):      41324 kB
Inactive(file):    85496 kB
Unevictable:       23576 kB
Mlocked:           23576 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:              2576 kB
Writeback:             0 kB
AnonPages:         96236 kB
Mapped:            19936 kB
Shmem:               552 kB
Slab:             101344 kB
SReclaimable:      24164 kB
SUnreclaim:        77180 kB
KernelStack:        2544 kB
PageTables:         4180 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    61864224 kB
Committed_AS:     585844 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      518656 kB
VmallocChunk:   34292133264 kB
HardwareCorrupted:     0 kB
AnonHugePages:      4096 kB
HugePages_Total:    4096
HugePages_Free:     4096
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:       96256 kB
DirectMap2M:     6178816 kB
DirectMap1G:    127926272 kB

And after the demo executed, I got the hugepages like this:

cat /proc/meminfo

MemTotal:       132117056 kB
MemFree:        117325180 kB
Buffers:           33508 kB
Cached:           721912 kB
SwapCached:            0 kB
Active:          4217712 kB
Inactive:         540956 kB
Active(anon):    4019068 kB
Inactive(anon):   121136 kB
Active(file):     198644 kB
Inactive(file):   419820 kB
Unevictable:       23908 kB
Mlocked:           23908 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:              2856 kB
Writeback:             0 kB
AnonPages:       4035184 kB
Mapped:           160292 kB
Shmem:            122100 kB
Slab:             177908 kB
SReclaimable:      64808 kB
SUnreclaim:       113100 kB
KernelStack:        7560 kB
PageTables:        62128 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    61864224 kB
Committed_AS:    8789664 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      527296 kB
VmallocChunk:   34292122604 kB
HardwareCorrupted:     0 kB
AnonHugePages:    262144 kB
HugePages_Total:    4096
HugePages_Free:     2048
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      141312 kB
DirectMap2M:     9279488 kB
DirectMap1G:    124780544 kB

Only the hugepages beyond to node1 was mapped. I was told host(having 64bit OS) cannot allocate memory while node0 has 2048 free hugepages,why? Dose anyone encountered the similar problem ever?
Any response will be appreciated!
Thanks!

A message that is not in the archive (sent off-list or not yet received): [email protected]

#2 lore raw
Quoted 103 lines · zhangsha (A)

Hi ，all

I am suffering from the problem mmap failed as followed when init dpdk eal.

Fri Jan 30 09:03:29 2015:EAL: Setting up memory...
Fri Jan 30 09:03:34 2015:EAL: map_all_hugepages(): mmap failed: Cannot allocate memory Fri Jan 30 09:03:34 2015:EAL: Failed to mmap 2 MB hugepages
Fri Jan 30 09:03:34 2015:EAL: Cannot init memory

Before I run the demo, the free hugepages of my host is :

cat /proc/meminfo

MemTotal:       132117056 kB
MemFree:        122040292 kB
Buffers:           10984 kB
Cached:           123056 kB
SwapCached:            0 kB
Active:           120812 kB
Inactive:          85860 kB
Active(anon):      79488 kB
Inactive(anon):      364 kB
Active(file):      41324 kB
Inactive(file):    85496 kB
Unevictable:       23576 kB
Mlocked:           23576 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:              2576 kB
Writeback:             0 kB
AnonPages:         96236 kB
Mapped:            19936 kB
Shmem:               552 kB
Slab:             101344 kB
SReclaimable:      24164 kB
SUnreclaim:        77180 kB
KernelStack:        2544 kB
PageTables:         4180 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    61864224 kB
Committed_AS:     585844 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      518656 kB
VmallocChunk:   34292133264 kB
HardwareCorrupted:     0 kB
AnonHugePages:      4096 kB
HugePages_Total:    4096
HugePages_Free:     4096
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:       96256 kB
DirectMap2M:     6178816 kB
DirectMap1G:    127926272 kB

And after the demo executed, I got the hugepages like this:

cat /proc/meminfo

MemTotal:       132117056 kB
MemFree:        117325180 kB
Buffers:           33508 kB
Cached:           721912 kB
SwapCached:            0 kB
Active:          4217712 kB
Inactive:         540956 kB
Active(anon):    4019068 kB
Inactive(anon):   121136 kB
Active(file):     198644 kB
Inactive(file):   419820 kB
Unevictable:       23908 kB
Mlocked:           23908 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:              2856 kB
Writeback:             0 kB
AnonPages:       4035184 kB
Mapped:           160292 kB
Shmem:            122100 kB
Slab:             177908 kB
SReclaimable:      64808 kB
SUnreclaim:       113100 kB
KernelStack:        7560 kB
PageTables:        62128 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    61864224 kB
Committed_AS:    8789664 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      527296 kB
VmallocChunk:   34292122604 kB
HardwareCorrupted:     0 kB
AnonHugePages:    262144 kB
HugePages_Total:    4096
HugePages_Free:     2048
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      141312 kB
DirectMap2M:     9279488 kB
DirectMap1G:    124780544 kB
Quoted 8 lines

Only the hugepages beyond to node1 was mapped. I was told host(having 64bit OS) cannot allocate memory while node0 has 2048 free hugepages,why? Dose anyone encountered the similar problem ever?
Any response will be appreciated!
Thanks!

How do you tell kernel not to allocate memory on node0?

I guess node0 and node1 both have 2048 hugepages and you want to mmap 4096 hugepages. So you can mmap 2048 hugepages on node1.After this step you cannot mmap any hugepage files because you tell kernel not to allocate memory on node0.

Signature · 2 lines
Regards,
Haifeng

A message that is not in the archive (sent off-list or not yet received): [email protected]

author
#4 lore raw

Hi, haifeng
Dpdk will mmap all the hugepages (4096) of host in the first step. Then it sort them and unmap the hugepages unneeded. My demo got the mistake when the 1st step. Thanks!

-----邮件原件-----
发件人: Linhaifeng
发送时间: 2015年1月30日 20:24
收件人: zhangsha (A); [email protected]
主题: Re: [dpdk-dev] mmap failed: Cannot allocate memory when init dpdk eal

Quoted 103 lines · zhangsha (A)

Hi ，all

I am suffering from the problem mmap failed as followed when init dpdk eal.

Fri Jan 30 09:03:29 2015:EAL: Setting up memory...
Fri Jan 30 09:03:34 2015:EAL: map_all_hugepages(): mmap failed: Cannot allocate memory Fri Jan 30 09:03:34 2015:EAL: Failed to mmap 2 MB hugepages Fri Jan 30 09:03:34 2015:EAL: Cannot init memory

Before I run the demo, the free hugepages of my host is :

cat /proc/meminfo

MemTotal:       132117056 kB
MemFree:        122040292 kB
Buffers:           10984 kB
Cached:           123056 kB
SwapCached:            0 kB
Active:           120812 kB
Inactive:          85860 kB
Active(anon):      79488 kB
Inactive(anon):      364 kB
Active(file):      41324 kB
Inactive(file):    85496 kB
Unevictable:       23576 kB
Mlocked:           23576 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:              2576 kB
Writeback:             0 kB
AnonPages:         96236 kB
Mapped:            19936 kB
Shmem:               552 kB
Slab:             101344 kB
SReclaimable:      24164 kB
SUnreclaim:        77180 kB
KernelStack:        2544 kB
PageTables:         4180 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    61864224 kB
Committed_AS:     585844 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      518656 kB
VmallocChunk:   34292133264 kB
HardwareCorrupted:     0 kB
AnonHugePages:      4096 kB
HugePages_Total:    4096
HugePages_Free:     4096
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:       96256 kB
DirectMap2M:     6178816 kB
DirectMap1G:    127926272 kB

And after the demo executed, I got the hugepages like this:

cat /proc/meminfo

MemTotal:       132117056 kB
MemFree:        117325180 kB
Buffers:           33508 kB
Cached:           721912 kB
SwapCached:            0 kB
Active:          4217712 kB
Inactive:         540956 kB
Active(anon):    4019068 kB
Inactive(anon):   121136 kB
Active(file):     198644 kB
Inactive(file):   419820 kB
Unevictable:       23908 kB
Mlocked:           23908 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:              2856 kB
Writeback:             0 kB
AnonPages:       4035184 kB
Mapped:           160292 kB
Shmem:            122100 kB
Slab:             177908 kB
SReclaimable:      64808 kB
SUnreclaim:       113100 kB
KernelStack:        7560 kB
PageTables:        62128 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    61864224 kB
Committed_AS:    8789664 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      527296 kB
VmallocChunk:   34292122604 kB
HardwareCorrupted:     0 kB
AnonHugePages:    262144 kB
HugePages_Total:    4096
HugePages_Free:     2048
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      141312 kB
DirectMap2M:     9279488 kB
DirectMap1G:    124780544 kB
Quoted 8 lines

Only the hugepages beyond to node1 was mapped. I was told host(having 64bit OS) cannot allocate memory while node0 has 2048 free hugepages,why? Dose anyone encountered the similar problem ever?
Any response will be appreciated!
Thanks!

How do you tell kernel not to allocate memory on node0?

I guess node0 and node1 both have 2048 hugepages and you want to mmap 4096 hugepages.
So you can mmap 2048 hugepages on node1.After this step you cannot mmap any hugepage files because you tell kernel not to allocate memory on node0.

--
Regards,
Haifeng

#5 lore raw
In reply to a message not in the archive

Hi,

Please provide the environment info such as kernel version, DPDK version and the reproduce steps in detail.

Thanks!

Quoted 115 lines

-----Original Message-----
From: dev [mailto:[email protected]] On Behalf Of zhangsha (A)
Sent: Friday, January 30, 2015 7:40 PM
To: [email protected]
Subject: [dpdk-dev] mmap failed: Cannot allocate memory when init dpdk eal

Hi ，all

I am suffering from the problem mmap failed as followed when init dpdk eal.

Fri Jan 30 09:03:29 2015:EAL: Setting up memory...
Fri Jan 30 09:03:34 2015:EAL: map_all_hugepages(): mmap failed: Cannot allocate memory Fri Jan 30 09:03:34 2015:EAL: Failed to mmap 2 MB hugepages Fri Jan 30 09:03:34 2015:EAL: Cannot init memory

Before I run the demo, the free hugepages of my host is :

cat /proc/meminfo

MemTotal:       132117056 kB
MemFree:        122040292 kB
Buffers:           10984 kB
Cached:           123056 kB
SwapCached:            0 kB
Active:           120812 kB
Inactive:          85860 kB
Active(anon):      79488 kB
Inactive(anon):      364 kB
Active(file):      41324 kB
Inactive(file):    85496 kB
Unevictable:       23576 kB
Mlocked:           23576 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:              2576 kB
Writeback:             0 kB
AnonPages:         96236 kB
Mapped:            19936 kB
Shmem:               552 kB
Slab:             101344 kB
SReclaimable:      24164 kB
SUnreclaim:        77180 kB
KernelStack:        2544 kB
PageTables:         4180 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    61864224 kB
Committed_AS:     585844 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      518656 kB
VmallocChunk:   34292133264 kB
HardwareCorrupted:     0 kB
AnonHugePages:      4096 kB
HugePages_Total:    4096
HugePages_Free:     4096
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:       96256 kB
DirectMap2M:     6178816 kB
DirectMap1G:    127926272 kB

And after the demo executed, I got the hugepages like this:

cat /proc/meminfo

MemTotal:       132117056 kB
MemFree:        117325180 kB
Buffers:           33508 kB
Cached:           721912 kB
SwapCached:            0 kB
Active:          4217712 kB
Inactive:         540956 kB
Active(anon):    4019068 kB
Inactive(anon):   121136 kB
Active(file):     198644 kB
Inactive(file):   419820 kB
Unevictable:       23908 kB
Mlocked:           23908 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:              2856 kB
Writeback:             0 kB
AnonPages:       4035184 kB
Mapped:           160292 kB
Shmem:            122100 kB
Slab:             177908 kB
SReclaimable:      64808 kB
SUnreclaim:       113100 kB
KernelStack:        7560 kB
PageTables:        62128 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    61864224 kB
Committed_AS:    8789664 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      527296 kB
VmallocChunk:   34292122604 kB
HardwareCorrupted:     0 kB
AnonHugePages:    262144 kB
HugePages_Total:    4096
HugePages_Free:     2048
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      141312 kB
DirectMap2M:     9279488 kB
DirectMap1G:    124780544 kB

Only the hugepages beyond to node1 was mapped. I was told host(having 64bit OS) cannot allocate memory while node0 has 2048 free hugepages,why? Dose anyone encountered the similar problem ever?
Any response will be appreciated!
Thanks!