Skip to contentSkip to conversation

KVM test: Add hugepage variant

PATCH Applied Lucas Meneghel Rodrigues 3 participants 6 messages Jul 27, 2009 – Jul 29, 2009 KVM

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 KVM test: Add hugepage variant

This patch adds a small setup script to set up huge memory pages during the kvm tests execution. Also, added hugepage setup to the fc8_quick sample.

---
 client/tests/kvm/kvm_tests.cfg.sample |    6 ++
 client/tests/kvm/kvm_vm.py            |   11 +++
 client/tests/kvm/scripts/hugepage.py  |  110 +++++++++++++++++++++++++++++++++
 3 files changed, 127 insertions(+), 0 deletions(-)
 create mode 100644 client/tests/kvm/scripts/hugepage.py
client/tests/kvm/kvm_tests.cfg.sample +6 -0
index 2d75a66..4a6a174 100644
@@ -585,6 +585,11 @@ variants:
         only default
         image_format = raw
 
+variants:
+    - @kvm_smallpages:
+    - kvm_hugepages:
+        pre_command = "/usr/bin/python scripts/hugepage.py"
+
 
 variants:
     - @basic:
@@ -598,6 +603,7 @@ variants:
         only Fedora.8.32
         only install setup boot shutdown
         only rtl8139
+        only kvm_hugepages
     - @sample1:
         only qcow2
         only ide
client/tests/kvm/kvm_vm.py +11 -0
index d96b359..eba9b84 100644
@@ -397,6 +397,17 @@ class VM:
                 self.destroy()
                 return False
 
+            # Get the output so far, to see if we have any problems with
+            # hugepage setup.
+            output = self.process.get_output()
+
+            if "alloc_mem_area" in output:
+                logging.error("Could not allocate hugepage memory; "
+                              "qemu command:\n%s" % qemu_command)
+                logging.error("Output:" + kvm_utils.format_str_for_message(
+                              self.process.get_output()))
+                return False
+
             logging.debug("VM appears to be alive with PID %d",
                           self.process.get_pid())
             return True
client/tests/kvm/scripts/hugepage.py new file +110 -0

110 lines not on this page · raw message

Signature · 1 line
1.6.2.5
#1 lore raw

Yes, this looks more pythonish and actually better than my version. I'm missing only one thing, extra_params += " -mem-path /mnt/hugepage" down in configuration (see below).

This cause problem with predefined mount point, because it needs to be the same in extra_params and python script.

Dne 27.7.2009 23:10, Lucas Meneghel Rodrigues napsal(a):

Quoted 19 lines

This patch adds a small setup script to set up huge memory pages during the kvm tests execution. Also, added hugepage setup to the fc8_quick sample.

---
  client/tests/kvm/kvm_tests.cfg.sample |    6 ++
  client/tests/kvm/kvm_vm.py            |   11 +++
  client/tests/kvm/scripts/hugepage.py  |  110 +++++++++++++++++++++++++++++++++
  3 files changed, 127 insertions(+), 0 deletions(-)

create mode 100644 client/tests/kvm/scripts/hugepage.py

client/tests/kvm/kvm_tests.cfg.sample +0 -0
index 2d75a66..4a6a174 100644

client/tests/kvm/kvm_tests.cfg.sample variants: · line 591

Quoted 7 lines
client/tests/kvm/kvm_tests.cfg.sample +4 -0
@@ -585,6 +585,11 @@ variants:
          only default
          image_format = raw
 
+variants:
+    - @kvm_smallpages:
+    - kvm_hugepages:
+        pre_command = "/usr/bin/python scripts/hugepage.py"

+ extra_params += " -mem-path /mnt/hugepage"

# ^^Tells qemu to allocate guest memory as hugepage

I'd rather have this part of cfg look like this:
variants:
- @kvm_smallpages:
- kvm_hugepages:
pre_command = "/usr/bin/python scripts/hugepage.py /mnt/hugepage" extra_params += " -mem-path /mnt/hugepage"

because this way it's more clear the relation between the constants. (it doesn't changes the script itself)

Quoted 150 lines
client/tests/kvm/kvm_tests.cfg.sample +2 -0
+
 
  variants:
      - @basic:
@@ -598,6 +603,7 @@ variants:
          only Fedora.8.32
          only install setup boot shutdown
          only rtl8139
+        only kvm_hugepages
      - @sample1:
          only qcow2
          only ide
client/tests/kvm/kvm_vm.py +11 -0
index d96b359..eba9b84 100644
@@ -397,6 +397,17 @@ class VM:
                  self.destroy()
                  return False
 
+            # Get the output so far, to see if we have any problems with
+            # hugepage setup.
+            output = self.process.get_output()
+
+            if "alloc_mem_area" in output:
+                logging.error("Could not allocate hugepage memory; "
+                              "qemu command:\n%s" % qemu_command)
+                logging.error("Output:" + kvm_utils.format_str_for_message(
+                              self.process.get_output()))
+                return False
+
              logging.debug("VM appears to be alive with PID %d",
                            self.process.get_pid())
              return True
client/tests/kvm/scripts/hugepage.py new file +110 -0
new file mode 100644
index 0000000..9bc4194
@@ -0,0 +1,110 @@
+#!/usr/bin/python
+# -*- coding: utf-8 -*-
+import os, sys, time
+
+"""
+Simple script to allocate enough hugepages for KVM testing purposes.
+"""
+
+class HugePageError(Exception):
+    """
+    Simple wrapper for the builtin Exception class.
+    """
+    pass
+
+
+class HugePage:
+    def __init__(self, hugepage_path=None):
+        """
+        Gets environment variable values and calculates the target number
+        of huge memory pages.
+
+        @param hugepage_path: Path where to mount hugetlbfs path, if not
+                yet configured.
+        """
+        self.vms = len(os.environ['KVM_TEST_vms'].split())
+        self.mem = int(os.environ['KVM_TEST_mem'])
+        try:
+            self.max_vms = int(os.environ['KVM_TEST_max_vms'])
+        except KeyError:
+            self.max_vms = 0
+        if hugepage_path:
+            self.hugepage_path = hugepage_path
+        else:
+            self.hugepage_path = '/mnt/kvm_hugepage'
+        self.hugepage_size = self.get_hugepage_size()
+        self.target_hugepages = self.get_target_hugepages()
+
+
+    def get_hugepage_size(self):
+        """
+        Get the current system setting for huge memory page size.
+        """
+        meminfo = open('/proc/meminfo', 'r').readlines()
+        huge_line_list = [h for h in meminfo if h.startswith("Hugepagesize")]
+        try:
+            return int(huge_line_list[0].split()[1])
+        except ValueError, e:
+            raise HugePageError("Could not get huge page size setting from "
+                                "/proc/meminfo: %s" % e)
+
+
+    def get_target_hugepages(self):
+        """
+        Calculate the target number of hugepages for testing purposes.
+        """
+        if self.vms<  self.max_vms:
+            self.vms = self.max_vms
+        vmsm = (self.vms * self.mem) + (self.vms * 64)
+        return int(vmsm * 1024 / self.hugepage_size)
+
+
+    def set_hugepages(self):
+        """
+        Sets the hugepage limit to the target hugepage value calculated.
+        """
+        hugepage_cfg = open("/proc/sys/vm/nr_hugepages", "r+")
+        hp = hugepage_cfg.readline()
+        while int(hp)<  self.target_hugepages:
+            loop_hp = hp
+            hugepage_cfg.write(str(self.target_hugepages))
+            hugepage_cfg.flush()
+            time.sleep(5)
+            hugepage_cfg.seek(0)
+            hp = int(hugepage_cfg.readline())
+            if loop_hp == hp:
+                raise HugePageError("Cannot set the kernel hugepage setting "
+                                    "to the target value of %d hugepages." %
+                                    self.target_hugepages)
+        hugepage_cfg.close()
+
+
+    def mount_hugepage_fs(self):
+        """
+        Verify if there's a hugetlbfs mount set. If there's none, will set up
+        a hugetlbfs mount using the class attribute that defines the mount
+        point.
+        """
+        current_mounts = open("/proc/mounts", "r").readlines()
+        huge_mounts = [m for m in current_mounts if m.split()[2] == "hugetlbfs"]
+        if len(huge_mounts) == 0:
+            if not os.path.isdir(self.hugepage_path):
+                os.makedirs(self.hugepage_path)
+            cmd = "mount -t hugetlbfs none %s" % self.hugepage_path
+            if os.system(cmd):
+                raise HugePageError("Cannot mount hugetlbfs path %s" %
+                                    self.hugepage_path)
+
+
+    def setup(self):
+        self.set_hugepages()
+        self.mount_hugepage_fs()
+
+
+if __name__ == "__main__":
+    if len(sys.argv)<  2:
+        huge_page = HugePage()
+    else:
+        huge_page = HugePage(sys.argv[1])
+
+    huge_page.setup()
#2 lore raw
Quoted 3 lines · Luk?? Doktor

Yes, this looks more pythonish and actually better than my version. I'm missing only one thing, extra_params += " -mem-path /mnt/hugepage" down in configuration (see below).

Don't we also need to inspect the qemu binary to determine if it's one of the few releases that used -mempath instead of -mem-path ? Or are we ignoring those?

Quoted 211 lines

This cause problem with predefined mount point, because it needs to be the same in extra_params and python script.

Dne 27.7.2009 23:10, Lucas Meneghel Rodrigues napsal(a):

Quoted 21 lines

This patch adds a small setup script to set up huge memory pages during the kvm tests execution. Also, added hugepage setup to the fc8_quick sample.

---
 client/tests/kvm/kvm_tests.cfg.sample |    6 ++
 client/tests/kvm/kvm_vm.py            |   11 +++
 client/tests/kvm/scripts/hugepage.py  |  110 
 +++++++++++++++++++++++++++++++++
 3 files changed, 127 insertions(+), 0 deletions(-)
 create mode 100644 client/tests/kvm/scripts/hugepage.py
client/tests/kvm/kvm_tests.cfg.sample +0 -0
index 2d75a66..4a6a174 100644

client/tests/kvm/kvm_tests.cfg.sample variants: · line 591

Quoted 7 lines
client/tests/kvm/kvm_tests.cfg.sample +4 -0
@@ -585,6 +585,11 @@ variants:
         only default
         image_format = raw
 
+variants:
+    - @kvm_smallpages:
+    - kvm_hugepages:
+        pre_command = "/usr/bin/python scripts/hugepage.py"

+ extra_params += " -mem-path /mnt/hugepage"

# ^^Tells qemu to allocate guest memory as hugepage

I'd rather have this part of cfg look like this:
variants:
- @kvm_smallpages:
- kvm_hugepages:
pre_command = "/usr/bin/python scripts/hugepage.py /mnt/hugepage"
extra_params += " -mem-path /mnt/hugepage"

because this way it's more clear the relation between the constants. (it doesn't changes the script itself)

Quoted 148 lines
client/tests/kvm/kvm_tests.cfg.sample +2 -0
+
 
 variants:
     - @basic:
@@ -598,6 +603,7 @@ variants:
         only Fedora.8.32
         only install setup boot shutdown
         only rtl8139
+        only kvm_hugepages
     - @sample1:
         only qcow2
         only ide
client/tests/kvm/kvm_vm.py +8 -0
index d96b359..eba9b84 100644
@@ -397,6 +397,17 @@ class VM:
                 self.destroy()
                 return False
 
+            # Get the output so far, to see if we have any problems with
+            # hugepage setup.
+            output = self.process.get_output()
+
+            if "alloc_mem_area" in output:
+                logging.error("Could not allocate hugepage memory; "
+                              "qemu command:\n%s" % qemu_command)
+                logging.error("Output:" + 

kvm_utils.format_str_for_message(

client/tests/kvm/kvm_vm.py +3 -0
+                              self.process.get_output()))
+                return False
+
             logging.debug("VM appears to be alive with PID %d",
                           self.process.get_pid())
             return True
client/tests/kvm/scripts/hugepage.py new file +44 -0
new file mode 100644
index 0000000..9bc4194
@@ -0,0 +1,110 @@
+#!/usr/bin/python
+# -*- coding: utf-8 -*-
+import os, sys, time
+
+"""
+Simple script to allocate enough hugepages for KVM testing purposes.
+"""
+
+class HugePageError(Exception):
+    """
+    Simple wrapper for the builtin Exception class.
+    """
+    pass
+
+
+class HugePage:
+    def __init__(self, hugepage_path=None):
+        """
+        Gets environment variable values and calculates the target number
+        of huge memory pages.
+
+        @param hugepage_path: Path where to mount hugetlbfs path, if not
+                yet configured.
+        """
+        self.vms = len(os.environ['KVM_TEST_vms'].split())
+        self.mem = int(os.environ['KVM_TEST_mem'])
+        try:
+            self.max_vms = int(os.environ['KVM_TEST_max_vms'])
+        except KeyError:
+            self.max_vms = 0
+        if hugepage_path:
+            self.hugepage_path = hugepage_path
+        else:
+            self.hugepage_path = '/mnt/kvm_hugepage'
+        self.hugepage_size = self.get_hugepage_size()
+        self.target_hugepages = self.get_target_hugepages()
+
+
+    def get_hugepage_size(self):
+        """
+        Get the current system setting for huge memory page size.
+        """
+        meminfo = open('/proc/meminfo', 'r').readlines()
+        huge_line_list = [h for h in meminfo if 

h.startswith("Hugepagesize")]

client/tests/kvm/scripts/hugepage.py new file +4 -0
+        try:
+            return int(huge_line_list[0].split()[1])
+        except ValueError, e:
+            raise HugePageError("Could not get huge page size setting 

from "

client/tests/kvm/scripts/hugepage.py new file +28 -0
+                                "/proc/meminfo: %s" % e)
+
+
+    def get_target_hugepages(self):
+        """
+        Calculate the target number of hugepages for testing purposes.
+        """
+        if self.vms<  self.max_vms:
+            self.vms = self.max_vms
+        vmsm = (self.vms * self.mem) + (self.vms * 64)
+        return int(vmsm * 1024 / self.hugepage_size)
+
+
+    def set_hugepages(self):
+        """
+        Sets the hugepage limit to the target hugepage value calculated.
+        """
+        hugepage_cfg = open("/proc/sys/vm/nr_hugepages", "r+")
+        hp = hugepage_cfg.readline()
+        while int(hp)<  self.target_hugepages:
+            loop_hp = hp
+            hugepage_cfg.write(str(self.target_hugepages))
+            hugepage_cfg.flush()
+            time.sleep(5)
+            hugepage_cfg.seek(0)
+            hp = int(hugepage_cfg.readline())
+            if loop_hp == hp:
+                raise HugePageError("Cannot set the kernel hugepage 

setting "
+ "to the target value of %d
hugepages." %

client/tests/kvm/scripts/hugepage.py new file +7 -0
+                                    self.target_hugepages)
+        hugepage_cfg.close()
+
+
+    def mount_hugepage_fs(self):
+        """
+        Verify if there's a hugetlbfs mount set. If there's none, will 

set up

client/tests/kvm/scripts/hugepage.py new file +5 -0
+        a hugetlbfs mount using the class attribute that defines the mount
+        point.
+        """
+        current_mounts = open("/proc/mounts", "r").readlines()
+        huge_mounts = [m for m in current_mounts if m.split()[2] == 

"hugetlbfs"]

client/tests/kvm/scripts/hugepage.py new file +11 -0
+        if len(huge_mounts) == 0:
+            if not os.path.isdir(self.hugepage_path):
+                os.makedirs(self.hugepage_path)
+            cmd = "mount -t hugetlbfs none %s" % self.hugepage_path
+            if os.system(cmd):
+                raise HugePageError("Cannot mount hugetlbfs path %s" %
+                                    self.hugepage_path)
+
+
+    def setup(self):
+        self.set_hugepages()

client/tests/kvm/scripts/hugepage.py line 110

Quoted 10 lines
client/tests/kvm/scripts/hugepage.py new file +10 -0
+        self.mount_hugepage_fs()
+
+
+if __name__ == "__main__":
+    if len(sys.argv)<  2:
+        huge_page = HugePage()
+    else:
+        huge_page = HugePage(sys.argv[1])
+
+    huge_page.setup()

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Signature · 4 lines
Ryan Harper
Software Engineer; Linux Technology Center
IBM Corp., Austin, Tx
[email protected]
#3 lore raw
Quoted 8 lines · Ryan Harper
Quoted 3 lines · Luk?? Doktor

Yes, this looks more pythonish and actually better than my version. I'm missing only one thing, extra_params += " -mem-path /mnt/hugepage" down in configuration (see below).

Don't we also need to inspect the qemu binary to determine if it's one of the few releases that used -mempath instead of -mem-path ?  Or are we ignoring those?

I am thinking here whether it's worth to add custom logic to kvm_vm code to deal with this 'corner case'...

#4 lore raw
In reply to Ryan Harper
Quoted 8 lines · Ryan Harper
Quoted 3 lines · Luk?? Doktor

Yes, this looks more pythonish and actually better than my version. I'm missing only one thing, extra_params += " -mem-path /mnt/hugepage" down in configuration (see below).

Don't we also need to inspect the qemu binary to determine if it's one of the few releases that used -mempath instead of -mem-path ?  Or are we ignoring those?

Ok, I amended the original patch with some changes, fixing the parameter passing, and created another one, that basically keeps record of the supported qemu options and can replace known misspelling issues. IMO takes care of the problem, waiting on comments about the approach!

Thanks!

#5 lore raw
In reply to Lucas Meneghel Rodrigues
Quoted 3 lines · Lucas Meneghel Rodrigues

This patch adds a small setup script to set up huge memory pages during the kvm tests execution. Also, added hugepage setup to the fc8_quick sample.

Applied.

Quoted 182 lines

---
 client/tests/kvm/kvm_tests.cfg.sample |    6 ++ client/tests/kvm/kvm_vm.py            |   11 +++ client/tests/kvm/scripts/hugepage.py  |  110 +++++++++++++++++++++++++++++++++

 3 files changed, 127 insertions(+), 0 deletions(-)

 create mode 100644 client/tests/kvm/scripts/hugepage.py

client/tests/kvm/kvm_tests.cfg.sample +0 -0
index 2d75a66..4a6a174 100644
@@ -585,6 +585,11 @@ variants:

        only default
        image_format = raw

client/tests/kvm/kvm_tests.cfg.sample +5 -0
+variants:
+    - @kvm_smallpages:
+    - kvm_hugepages:
+        pre_command = "/usr/bin/python scripts/hugepage.py"
+

 variants:
    - @basic:

client/tests/kvm/kvm_tests.cfg.sample +0 -0
@@ -598,6 +603,7 @@ variants:

        only Fedora.8.32
        only install setup boot shutdown
        only rtl8139
+        only kvm_hugepages
    - @sample1:
        only qcow2
        only ide

client/tests/kvm/kvm_vm.py +0 -0
index d96b359..eba9b84 100644
@@ -397,6 +397,17 @@ class VM:
                 self.destroy()

                return False

+            # Get the output so far, to see if we have any problems with
+            # hugepage setup.
+            output = self.process.get_output()

client/tests/kvm/kvm_vm.py +8 -0
+
+            if "alloc_mem_area" in output:
+                logging.error("Could not allocate hugepage memory; "
+                              "qemu command:\n%s" % qemu_command)
+                logging.error("Output:" + kvm_utils.format_str_for_message(
+                              self.process.get_output()))
+                return False
+

            logging.debug("VM appears to be alive with PID %d", self.process.get_pid()) return True

client/tests/kvm/scripts/hugepage.py new file +110 -1
new file mode 100644
index 0000000..9bc4194
@@ -0,0 +1,110 @@
+#!/usr/bin/python
+# -*- coding: utf-8 -*-
+import os, sys, time
+
+"""
+Simple script to allocate enough hugepages for KVM testing purposes.
+"""
+
+class HugePageError(Exception):
+    """
+    Simple wrapper for the builtin Exception class.
+    """
+    pass
+
+
+class HugePage:
+    def __init__(self, hugepage_path=None):
+        """
+        Gets environment variable values and calculates the target number
+        of huge memory pages.
+
+        @param hugepage_path: Path where to mount hugetlbfs path, if not
+                yet configured.
+        """
+        self.vms = len(os.environ['KVM_TEST_vms'].split())
+        self.mem = int(os.environ['KVM_TEST_mem'])
+        try:
+            self.max_vms = int(os.environ['KVM_TEST_max_vms'])
+        except KeyError:
+            self.max_vms = 0
+        if hugepage_path:
+            self.hugepage_path = hugepage_path
+        else:
+            self.hugepage_path = '/mnt/kvm_hugepage'
+        self.hugepage_size = self.get_hugepage_size()
+        self.target_hugepages = self.get_target_hugepages()
+
+
+    def get_hugepage_size(self):
+        """
+        Get the current system setting for huge memory page size.
+        """
+        meminfo = open('/proc/meminfo', 'r').readlines()
+        huge_line_list = [h for h in meminfo if h.startswith("Hugepagesize")]
+        try:
+            return int(huge_line_list[0].split()[1])
+        except ValueError, e:
+            raise HugePageError("Could not get huge page size setting from "
+                                "/proc/meminfo: %s" % e)
+
+
+    def get_target_hugepages(self):
+        """
+        Calculate the target number of hugepages for testing purposes.
+        """
+        if self.vms < self.max_vms:
+            self.vms = self.max_vms
+        vmsm = (self.vms * self.mem) + (self.vms * 64)
+        return int(vmsm * 1024 / self.hugepage_size)
+
+
+    def set_hugepages(self):
+        """
+        Sets the hugepage limit to the target hugepage value calculated.
+        """
+        hugepage_cfg = open("/proc/sys/vm/nr_hugepages", "r+")
+        hp = hugepage_cfg.readline()
+        while int(hp) < self.target_hugepages:
+            loop_hp = hp
+            hugepage_cfg.write(str(self.target_hugepages))
+            hugepage_cfg.flush()
+            time.sleep(5)
+            hugepage_cfg.seek(0)
+            hp = int(hugepage_cfg.readline())
+            if loop_hp == hp:
+                raise HugePageError("Cannot set the kernel hugepage setting "
+                                    "to the target value of %d hugepages." %
+                                    self.target_hugepages)
+        hugepage_cfg.close()
+
+
+    def mount_hugepage_fs(self):
+        """
+        Verify if there's a hugetlbfs mount set. If there's none, will set up
+        a hugetlbfs mount using the class attribute that defines the mount
+        point.
+        """
+        current_mounts = open("/proc/mounts", "r").readlines()
+        huge_mounts = [m for m in current_mounts if m.split()[2] == "hugetlbfs"]
+        if len(huge_mounts) == 0:
+            if not os.path.isdir(self.hugepage_path):
+                os.makedirs(self.hugepage_path)
+            cmd = "mount -t hugetlbfs none %s" % self.hugepage_path
+            if os.system(cmd):
+                raise HugePageError("Cannot mount hugetlbfs path %s" %
+                                    self.hugepage_path)
+
+
+    def setup(self):
+        self.set_hugepages()
+        self.mount_hugepage_fs()
+
+
+if __name__ == "__main__":
+    if len(sys.argv) < 2:
+        huge_page = HugePage()
+    else:
+        huge_page = HugePage(sys.argv[1])
+
+    huge_page.setup()
--

1.6.2.5

_______________________________________________
Autotest mailing list
[email protected]
http://test.kernel.org/cgi-bin/mailman/listinfo/autotest

Signature · 1 line
Lucas Meneghel