From: andy123 <ajs124.ajs124@gmail.com>
Subject: problems with 1G hugepages and linux 3.12-rc3
Date: Sun, 6 Oct 2013 02:47:41 +0200
Message-ID: <20131006024741.61cc8512@desk.lan>
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Cc: qemu-devel@nongnu.org
To: kvm@vger.kernel.org
Sender: kvm-owner@vger.kernel.org

Hi,

as the subject states, I have some problems with 1G hugepages with qemu(-vfio-git) on Linux 3.12-rc3.

I start qemu like this, for example:
"/usr/bin/qemu-system-x86_64 -enable-kvm -m 1024 -mem-path /dev/hugepages -drive file=/files/vm/arch.img,if=virtio,media=disk -monitor stdio"
where /dev/hugepages is "hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,mode=1770,gid=78,pagesize=1G,pagesize=1G)"
and the kernel is booted with "hugepagesz=1G hugepages=4".
This result in lots of error message in dmesg, as seen here: https://gist.github.com/ajs124/6842823 (starting at 18:04:28)

After starting and stopping multiple virtual machines, the hugepages seem to "fill up" and qemu outputs
"file_ram_alloc: can't mmap RAM pages: Cannot allocate memory", but works anyways.
With fill up, I mean that I can start qemu 2 time with "-m 2048" and 4 times with "-m 1024", before it fails to mmap.

This works without any problems in 3.11.x and also with 2M hugepages (umount /dev/hugepages && mount -t hugetlbfs -o pagesize=2048k /dev/hugepages).

I'm running this in arch linux and there has already been some discussion on the arch forums (https://bbs.archlinux.org/viewtopic.php?pid=1333469#p1333469) but I tried to add everything I wrote over there in this mail.

In case I missed something or you need more information, I'll happily supply it in order to get this resolved.