Patchset of network related subtests
cover Patchset of network related subtests
The following series contain 11 network related subtests, welcome to give me some suggestions about correctness, design, enhancement.
Thank you so much!
---
Amos Kong (14):
KVM-test: Add a new macaddress pool algorithm
KVM Test: Add a function get_interface_name() to kvm_net_utils.py KVM Test: Add a common ping module for network related tests
KVM-test: Add a new subtest ping
KVM-test: Add a subtest jumbo
KVM-test: Add basic file transfer test
KVM-test: Add a subtest of load/unload nic driver
KVM-test: Add a subtest of nic promisc
KVM-test: Add a subtest of multicast
KVM-test: Add a subtest of pxe
KVM-test: Add a subtest of changing mac address
KVM-test: Add a subtest of netperf
KVM-test: Improve vlan subtest
KVM-test: Add subtest of testing offload by ethtool
0 files changed, 0 insertions(+), 0 deletions(-)
Signature · 1 line
Amos Kong
RFC PATCH 1/14 KVM-test: Add a new macaddress pool algorithm
Old method uses the mac address in the configuration files which could lead serious problem when multiple tests running in different hosts.
This patch adds a new macaddress pool algorithm, it generates the mac prefix based on mac address of the host which could eliminate the duplicated mac addresses between machines.
When user have set the mac_prefix in the configuration file, we should use it in stead of the dynamic generated mac prefix.
Other change:
. Fix randomly generating mac address so that it correspond to IEEE802. . Update clone function to decide clone mac address or not. . Update get_macaddr function.
. Add set_mac_address function.
New auto mac address pool algorithm:
If address_index is defined, VM will get mac from config file then record mac in to address_pool. If address_index is not defined, VM will call get_mac_from_pool to auto create mac then recored mac to address_pool in following format:
{'macpool': {'AE:9D:94:6A:9b:f9': ['20100310-165222-Wt7l:0']}}
AE:9D:94:6A:9b:f9 : mac address
20100310-165222-Wt7l : instance attribute of VM
0 : index of NIC
---
0 files changed, 0 insertions(+), 0 deletions(-)
client/tests/kvm/kvm_utils.py +99 -0
@@ -5,6 +5,7 @@ KVM test utility functions. """ import time, string, random, socket, os, signal, re, logging, commands, cPickle +import fcntl, shelve from autotest_lib.client.bin import utils from autotest_lib.client.common_lib import error, logging_config import kvm_subprocess
@@ -82,6 +83,104 @@ def get_sub_dict_names(dict, keyword): # Functions related to MAC/IP addresses +def get_mac_from_pool(root_dir, vm, nic_index, prefix='00:11:22:33:'): + """ + random generated mac address. + + 1) First try to generate macaddress based on the mac address prefix. + 2) And then try to use total random generated mac address. + + @param root_dir: Root dir for kvm + @param vm: Here we use instance of vm + @param nic_index: The index of nic. + @param prefix: Prefix of mac address. + @Return: Return mac address. + """ + + lock_filename = os.path.join(root_dir, "mac_lock") + lock_file = open(lock_filename, 'w') + fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX) + mac_filename = os.path.join(root_dir, "address_pool") + mac_shelve = shelve.open(mac_filename, writeback=False) + + mac_pool = mac_shelve.get("macpool") + + if not mac_pool: + mac_pool = {} + found = False + + val = "%s:%s" % (vm, nic_index) + for key in mac_pool.keys(): + if val in mac_pool[key]: + mac_pool[key].append(val) + found = True + mac = key + + while not found: + postfix = "%02x:%02x" % (random.randint(0x00,0xfe), + random.randint(0x00,0xfe)) + mac = prefix + postfix + mac_list = mac.split(":") + # Clear multicast bit + mac_list[0] = int(mac_list[0],16) & 0xfe + # Set local assignment bit (IEEE802) + mac_list[0] = mac_list[0] | 0x02 + mac_list[0] = "%02x" % mac_list[0] + mac = ":".join(mac_list) + if mac not in mac_pool.keys() or 0 == len(mac_pool[mac]): + mac_pool[mac] = ["%s:%s" % (vm,nic_index)] + found = True + mac_shelve["macpool"] = mac_pool + logging.debug("generating mac addr %s " % mac) + + mac_shelve.close() + fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN) + lock_file.close() + return mac + + +def put_mac_to_pool(root_dir, mac, vm): + """ + Put the macaddress back to address pool + + @param root_dir: Root dir for kvm + @param vm: Here we use instance attribute of vm + @param mac: mac address will be put. + @Return: mac address. + """ + + lock_filename = os.path.join(root_dir, "mac_lock") + lock_file = open(lock_filename, 'w') + fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX) + mac_filename = os.path.join(root_dir, "address_pool") + mac_shelve = shelve.open(mac_filename, writeback=False) + + mac_pool = mac_shelve.get("macpool") + + if not mac_pool or (not mac in mac_pool): + logging.debug("Try to free a macaddress does no in pool") + logging.debug("macaddress is %s" % mac) + logging.debug("pool is %s" % mac_pool) + else: + if len(mac_pool[mac]) <= 1: + mac_pool.pop(mac) + else: + for value in mac_pool[mac]: + if vm in value: + mac_pool[mac].remove(value) + break + if len(mac_pool[mac]) == 0: + mac_pool.pop(mac) + + mac_shelve["macpool"] = mac_pool + logging.debug("freeing mac addr %s " % mac) + + mac_shelve.close() + fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN) + lock_file.close() + return mac + + def mac_str_to_int(addr): """ Convert MAC address string to integer.
client/tests/kvm/kvm_vm.py +111 -7
206 lines not on this page · raw message
client/tests/kvm/tests_base.cfg.sample +1 -1
@@ -51,7 +51,7 @@ guest_port_remote_shell = 22 nic_mode = user #nic_mode = tap nic_script = scripts/qemu-ifup -address_index = 0 +#address_index = 0 run_tcpdump = yes # Misc
Re: [Qemu-devel] [RFC PATCH 01/14] KVM-test: Add a new macaddress pool algorithm
Quoted 23 lines · Amos Kong
Old method uses the mac address in the configuration files which could lead serious problem when multiple tests running in different hosts.
This patch adds a new macaddress pool algorithm, it generates the mac prefix based on mac address of the host which could eliminate the duplicated mac addresses between machines.
When user have set the
mac_prefixin the configuration file, we should use it in stead of the dynamic generated mac prefix.Other change:
. Fix randomly generating mac address so that it correspond to IEEE802. . Update clone function to decide clone mac address or not. . Updateget_macaddrfunction.
. Addset_mac_addressfunction.New auto mac address pool algorithm:
Ifaddress_indexis defined, VM will get mac from config file then record mac in toaddress_pool. Ifaddress_indexis not defined, VM will callget_mac_from_poolto auto create mac then recored mac toaddress_poolin following format:
{'macpool': {'AE:9D:94:6A:9b:f9': ['20100310-165222-Wt7l:0']}}
Quoted 3 lines
AE:9D:94:6A:9b:f9 : mac address
20100310-165222-Wt7l : instance attribute of VM
0 : index of NIC
Why do you use the mac address as a key, instead of the instance string
+ nic index? When the mac address is used as a key, each key has a list of values instead of just one value. This order seems unnatural. If it were the other way around (i.e. key = VM instance + nic index, value = mac address), then each key would have exactly one value, and I think this patch would be shorter and simpler.
Quoted 19 lines
Signed-off-by Jason Wang <[email protected]>Signed-off-by Feng Yang <[email protected]>Signed-off-by Amos Kong <[email protected]>---
0 files changed, 0 insertions(+), 0 deletions(-)client/tests/kvm/kvm_utils.py +1
-0@@ -5,6 +5,7 @@ KVM test utility functions. """ import time, string, random, socket, os, signal, re, logging, commands, cPickle +import fcntl, shelve from autotest_lib.client.bin import utils from autotest_lib.client.common_lib import error, logging_config import kvm_subprocess
client/tests/kvm/kvm_utils.py def get_sub_dict_names(dict, keyword): · line 86
Quoted 4 lines
client/tests/kvm/kvm_utils.py +1
-0@@ -82,6 +83,104 @@ def get_sub_dict_names(dict, keyword): # Functions related to MAC/IP addresses +def get_mac_from_pool(root_dir, vm, nic_index, prefix='00:11:22:33:'):
The name of this function is confusing because it does the exact
opposite: it puts a mac address in address_pool. Maybe the pool you're referring to in the name isn't address_pool, but still a less confusing name should probably be used.
Quoted 7 lines
client/tests/kvm/kvm_utils.py +7
-0+ """ + random generated mac address. + + 1) First try to generate macaddress based on the mac address prefix. + 2) And then try to use total random generated mac address. + + @param root_dir: Root dir for kvm
client/tests/kvm/kvm_utils.py def get_sub_dict_names(dict, keyword): · line 103
Quoted 10 lines
client/tests/kvm/kvm_utils.py +10
-0+ @param vm: Here we use instance of vm + @param nic_index: The index of nic. + @param prefix: Prefix of mac address. + @Return: Return mac address. + """ + + lock_filename = os.path.join(root_dir, "mac_lock") + lock_file = open(lock_filename, 'w') + fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX) + mac_filename = os.path.join(root_dir, "address_pool")
Maybe it makes sense to put address_pool and the lock file in /tmp, where they can be shared by more than a single autotest instance running on the same host (unlikely, but theoretically possible).
client/tests/kvm/kvm_utils.py def get_sub_dict_names(dict, keyword): · line 106
Quoted 3 lines
client/tests/kvm/kvm_utils.py +3
-0+ mac_shelve = shelve.open(mac_filename, writeback=False) + + mac_pool = mac_shelve.get("macpool")
Why is this 'macpool' needed? Why not put the keys directly in the shelve object?
client/tests/kvm/kvm_utils.py def get_sub_dict_names(dict, keyword): · line 115
Quoted 9 lines
client/tests/kvm/kvm_utils.py +9
-0+ + if not mac_pool: + mac_pool = {} + found = False + + val = "%s:%s" % (vm, nic_index) + for key in mac_pool.keys(): + if val in mac_pool[key]: + mac_pool[key].append(val)
Why append val to mac_pool[key] if val is already in mac_pool[key]?
client/tests/kvm/kvm_utils.py def get_sub_dict_names(dict, keyword): · line 128
Quoted 13 lines
client/tests/kvm/kvm_utils.py +13
-0+ found = True + mac = key + + while not found: + postfix = "%02x:%02x" % (random.randint(0x00,0xfe), + random.randint(0x00,0xfe)) + mac = prefix + postfix + mac_list = mac.split(":") + # Clear multicast bit + mac_list[0] = int(mac_list[0],16) & 0xfe + # Set local assignment bit (IEEE802) + mac_list[0] = mac_list[0] | 0x02 + mac_list[0] = "%02x" % mac_list[0]
Why is this needed? Most mac addresses begin with 00. If the mac address is generated from the address of eth0 (using the method in this patch) it begins with 00, which is fine. If the prefix is set by the user using mac_prefix, I don't think we should modify it.
Quoted 4 lines
client/tests/kvm/kvm_utils.py +4
-0+ mac = ":".join(mac_list) + if mac not in mac_pool.keys() or 0 == len(mac_pool[mac]): + mac_pool[mac] = ["%s:%s" % (vm,nic_index)] + found = True
client/tests/kvm/kvm_utils.py def get_sub_dict_names(dict, keyword): · line 142
Quoted 10 lines
client/tests/kvm/kvm_utils.py +10
-0+ mac_shelve["macpool"] = mac_pool + logging.debug("generating mac addr %s " % mac) + + mac_shelve.close() + fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN) + lock_file.close() + return mac + + +def put_mac_to_pool(root_dir, mac, vm):
This function removes a mac address from address_pool. I wonder why you chose this name.
Quoted 91 lines
client/tests/kvm/kvm_utils.py +41
-0+ """ + Put the macaddress back to address pool + + @param root_dir: Root dir for kvm + @param vm: Here we use instance attribute of vm + @param mac: mac address will be put. + @Return: mac address. + """ + + lock_filename = os.path.join(root_dir, "mac_lock") + lock_file = open(lock_filename, 'w') + fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX) + mac_filename = os.path.join(root_dir, "address_pool") + mac_shelve = shelve.open(mac_filename, writeback=False) + + mac_pool = mac_shelve.get("macpool") + + if not mac_pool or (not mac in mac_pool): + logging.debug("Try to free a macaddress does no in pool") + logging.debug("macaddress is %s" % mac) + logging.debug("pool is %s" % mac_pool) + else: + if len(mac_pool[mac]) <= 1: + mac_pool.pop(mac) + else: + for value in mac_pool[mac]: + if vm in value: + mac_pool[mac].remove(value) + break + if len(mac_pool[mac]) == 0: + mac_pool.pop(mac) + + mac_shelve["macpool"] = mac_pool + logging.debug("freeing mac addr %s " % mac) + + mac_shelve.close() + fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN) + lock_file.close() + return mac + + def mac_str_to_int(addr): """ Convert MAC address string to integer.client/tests/kvm/kvm_vm.py +11
-2@@ -5,7 +5,7 @@ Utility classes and functions to handle Virtual Machine creation using qemu. @copyright: 2008-2009 Red Hat Inc. """ -import time, socket, os, logging, fcntl, re, commands, glob +import time, socket, os, logging, fcntl, re, commands, shelve, glob import kvm_utils, kvm_subprocess, kvm_monitor, rss_file_transfer from autotest_lib.client.common_lib import error from autotest_lib.client.bin import utils@@ -117,6 +117,7 @@ class VM: self.params = params self.root_dir = root_dir self.address_cache = address_cache + self.mac_prefix = params.get('mac_prefix') self.netdev_id = [] # Find a unique identifier for this VM@@ -126,8 +127,15 @@ class VM: if not glob.glob("/tmp/*%s" % self.instance): break + if self.mac_prefix is None: + s, o = commands.getstatusoutput("ifconfig eth0") + if s == 0: + mac = re.findall("HWaddr (\S*)", o)[0] + self.mac_prefix = '00' + mac[8:] + ':' + - def clone(self, name=None, params=None, root_dir=None, address_cache=None): + def clone(self, name=None, params=None, root_dir=None, + address_cache=None, mac_clone=True): """ Return a clone of the VM object with optionally modified parameters. The clone is initially not alive and needs to be started using create().@@ -138,6 +146,7 @@ class VM: @param params: Optional new VM creation parameters @param root_dir: Optional new base directory for relative filenames @param address_cache: A dict that maps MAC addresses to IP addresses + @param mac_clone: Clone mac address or not. """ if name is None: name = self.name
client/tests/kvm/kvm_vm.py class VM: · line 162
Quoted 8 lines
client/tests/kvm/kvm_vm.py +4
-1@@ -147,9 +156,19 @@ class VM: root_dir = self.root_dir if address_cache is None: address_cache = self.address_cache - return VM(name, params, root_dir, address_cache) + vm = VM(name, params, root_dir, address_cache) + if mac_clone: + # Clone mac address by coping 'self.instance' to the new vm. + vm.instance = self.instance
Copying self.instance isn't a good idea because the monitor, serial console and testlog filenames use self.instance. self.instance should be unique. If we still want to use the same mac address for 2 VMs, for example in migration, a different solution should be found.
Quoted 27 lines
client/tests/kvm/kvm_vm.py +14
-0+ return vm + def free_mac_address(self): + nic_num = len(kvm_utils.get_sub_dict_names(self.params, "nics")) + for nic in range(nic_num): + mac = self.get_macaddr(nic_index=nic) + kvm_utils.put_mac_to_pool(self.root_dir, mac, vm=self.instance) + def make_qemu_command(self, name=None, params=None, root_dir=None): """ Generate a qemu command line. All parameters are optional. If a@@ -383,6 +402,13 @@ class VM: mac = None if "address_index" in nic_params: mac = kvm_utils.get_mac_ip_pair_from_dict(nic_params)[0] + self.set_mac_address(mac=mac, nic_index=vlan) + else: + mac = kvm_utils.get_mac_from_pool(self.root_dir, + vm=self.instance, + nic_index=vlan, + prefix=self.mac_prefix) + qemu_cmd += add_nic(help, vlan, nic_params.get("nic_model"), mac, self.netdev_id[vlan]) # Handle the '-net tap' or '-net user' part
client/tests/kvm/kvm_vm.py class VM: · line 425
Quoted 5 lines
client/tests/kvm/kvm_vm.py +1
-1@@ -396,7 +422,7 @@ class VM: if tftp: tftp = kvm_utils.get_path(root_dir, tftp) qemu_cmd += add_net(help, vlan, nic_params.get("nic_mode", "user"), - nic_params.get("nic_ifname"), + self.get_ifname(vlan),
Why is get_ifname() useful here?
Quoted 73 lines
client/tests/kvm/kvm_vm.py +21
-3script, downscript, tftp, nic_params.get("bootp"), redirs, self.netdev_id[vlan])@@ -720,7 +746,7 @@ class VM: lockfile.close() - def destroy(self, gracefully=True): + def destroy(self, gracefully=True, free_macaddr=True): """ Destroy the VM.@@ -731,6 +757,7 @@ class VM: @param gracefully: Whether an attempt will be made to end the VM using a shell command before trying to end the qemu process with a 'quit' or a kill signal. + @param free_macaddr: Whether free macaddresses when destory a vm. """ try: # Is it already dead?@@ -751,11 +778,18 @@ class VM: logging.debug("Shutdown command sent; waiting for VM " "to go down...") if kvm_utils.wait_for(self.is_dead, 60, 1, 1): - logging.debug("VM is down") + logging.debug("VM is down, free mac address.") + # free mac address + if free_macaddr: + self.free_mac_address() return finally: session.close() + # free mac address + if free_macaddr: + self.free_mac_address() + if self.monitor: # Try to destroy with a monitor command logging.debug("Trying to kill VM with monitor command...")@@ -881,10 +915,14 @@ class VM: nic_name = nics[index] nic_params = kvm_utils.get_sub_dict(self.params, nic_name) if nic_params.get("nic_mode") == "tap": - mac, ip = kvm_utils.get_mac_ip_pair_from_dict(nic_params) + mac = self.get_macaddr(index) if not mac: logging.debug("MAC address unavailable") return None + else: + mac = mac.lower() + ip = None + if not ip or nic_params.get("always_use_tcpdump") == "yes": # Get the IP address from the cache ip = self.address_cache.get(mac)@@ -897,6 +935,7 @@ class VM: for nic in nics] macs = [kvm_utils.get_mac_ip_pair_from_dict(dict)[0] for dict in nic_dicts] + macs.append(mac) if not kvm_utils.verify_ip_address_ownership(ip, macs): logging.debug("Could not verify MAC-IP address mapping: " "%s ---> %s" % (mac, ip))@@ -925,6 +964,71 @@ class VM: "redirected" % port) return self.redirs.get(port) + def get_ifname(self, nic_index=0): + """ + Return the ifname of tap device for the guest nic. + + @param nic_index: Index of the NIC
client/tests/kvm/kvm_vm.py class VM: · line 981
Quoted 10 lines
client/tests/kvm/kvm_vm.py +10
-0+ """ + + nics = kvm_utils.get_sub_dict_names(self.params, "nics") + nic_name = nics[nic_index] + nic_params = kvm_utils.get_sub_dict(self.params, nic_name) + if nic_params.get("nic_ifname"): + return nic_params.get("nic_ifname") + else: + return "%s_%s_%s" % (nic_params.get("nic_model"), + nic_index, self.vnc_port)
What's the purpose of this string?
Quoted 67 lines
client/tests/kvm/kvm_vm.py +49
-0+ def get_macaddr(self, nic_index=0): + """ + Return the macaddr of guest nic. + + @param nic_index: Index of the NIC + """ + mac_filename = os.path.join(self.root_dir, "address_pool") + mac_shelve = shelve.open(mac_filename, writeback=False) + mac_pool = mac_shelve.get("macpool") + val = "%s:%s" % (self.instance, nic_index) + for key in mac_pool.keys(): + if val in mac_pool[key]: + return key + return None + + def set_mac_address(self, mac, nic_index=0, shareable=False): + """ + Set mac address for guest. Note: It just update address pool. + + @param mac: address will set to guest + @param nic_index: Index of the NIC + @param shareable: Where VM can share mac with other VM or not. + """ + lock_filename = os.path.join(self.root_dir, "mac_lock") + lock_file = open(lock_filename, 'w') + fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX) + mac_filename = os.path.join(self.root_dir, "address_pool") + mac_shelve = shelve.open(mac_filename, writeback=False) + mac_pool = mac_shelve.get("macpool") + if not mac_pool: + mac_pool = {} + value = "%s:%s" % (self.instance, nic_index) + if mac not in mac_pool.keys(): + for key in mac_pool.keys(): + if value in mac_pool[key]: + mac_pool[key].remove(value) + if len(mac_pool[key]) == 0: + mac_pool.pop(key) + mac_pool[mac] = [value] + else: + if shareable: + mac_pool[mac].append(value) + else: + logging.error("Mac address already be used!") + return False + mac_shelve["macpool"] = mac_pool + mac_shelve.close() + fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN) + lock_file.close() def get_pid(self): """client/tests/kvm/tests_base.cfg.sample +1
-1@@ -51,7 +51,7 @@ guest_port_remote_shell = 22 nic_mode = user #nic_mode = tap nic_script = scripts/qemu-ifup -address_index = 0 +#address_index = 0 run_tcpdump = yes # Misc
Re: [Qemu-devel] [RFC PATCH 01/14] KVM-test: Add a new macaddress pool algorithm
Quoted 1 line · Michael Goldish
Michael,
Thanks for your comments. Let's simplify this method together.
Quoted 28 lines · Amos Kong
Quoted 23 lines
Old method uses the mac address in the configuration files which could lead serious problem when multiple tests running in different hosts.
This patch adds a new macaddress pool algorithm, it generates the mac prefix based on mac address of the host which could eliminate the duplicated mac addresses between machines.
When user have set the
mac_prefixin the configuration file, we should use it in stead of the dynamic generated mac prefix.Other change:
. Fix randomly generating mac address so that it correspond to IEEE802. . Update clone function to decide clone mac address or not. . Updateget_macaddrfunction.
. Addset_mac_addressfunction.New auto mac address pool algorithm:
Ifaddress_indexis defined, VM will get mac from config file then record mac in toaddress_pool. Ifaddress_indexis not defined, VM will callget_mac_from_poolto auto create mac then recored mac toaddress_poolin following format:
{'macpool': {'AE:9D:94:6A:9b:f9': ['20100310-165222-Wt7l:0']}}Quoted 3 lines
AE:9D:94:6A:9b:f9 : mac address
20100310-165222-Wt7l : instance attribute of VM
0 : index of NIC
Quoted 6 lines
Why do you use the mac address as a key, instead of the instance string
+ nic index? When the mac address is used as a key, each key has a list of values instead of just one value. This order seems unnatural. If it were the other way around (i.e. key = VM instance + nic index, value = mac address), then each key would have exactly one value, and I think this patch would be shorter and simpler.
One mac address may be used by two VMs, eg. migration.
Quoted 20 lines
Quoted 20 lines
Signed-off-by Jason Wang <[email protected]>Signed-off-by Feng Yang <[email protected]>Signed-off-by Amos Kong <[email protected]>---
0 files changed, 0 insertions(+), 0 deletions(-)client/tests/kvm/kvm_utils.py +1
-0@@ -5,6 +5,7 @@ KVM test utility functions. """ import time, string, random, socket, os, signal, re, logging, commands, cPickle +import fcntl, shelve from autotest_lib.client.bin import utils from autotest_lib.client.common_lib import error, logging_config import kvm_subprocess@@ -82,6 +83,104 @@ def get_sub_dict_names(dict, keyword):
Quoted 8 lines
client/tests/kvm/kvm_utils.py def get_sub_dict_names(dict, keyword): · line 86
Quoted 3 lines
client/tests/kvm/kvm_utils.py +1
-0# Functions related to MAC/IP addresses +def get_mac_from_pool(root_dir, vm, nic_index, prefix='00:11:22:33:'):The name of this function is confusing because it does the exact
opposite: it puts a mac address inaddress_pool. Maybe the pool you're referring to in the name isn'taddress_pool, but still a less confusing name should probably be used.
How about allocate_mac(...) ?address_pool -> address_container
Allocate mac address and record into address_container.
Quoted 13 lines
Quoted 13 lines
client/tests/kvm/kvm_utils.py +13
-0+ """ + random generated mac address. + + 1) First try to generate macaddress based on the mac address prefix. + 2) And then try to use total random generated mac address. + + @param root_dir: Root dir for kvm + @param vm: Here we use instance of vm + @param nic_index: The index of nic. + @param prefix: Prefix of mac address. + @Return: Return mac address. + """ +
Quoted 8 lines
client/tests/kvm/kvm_utils.py def get_sub_dict_names(dict, keyword): · line 103
Quoted 4 lines
client/tests/kvm/kvm_utils.py +4
-0+ lock_filename = os.path.join(root_dir, "mac_lock") + lock_file = open(lock_filename, 'w') + fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX) + mac_filename = os.path.join(root_dir, "address_pool")Maybe it makes sense to put
address_pooland the lock file in /tmp, where they can be shared by more than a single autotest instance running on the same host (unlikely, but theoretically possible).
good idea.
Quoted 6 lines
client/tests/kvm/kvm_utils.py def get_sub_dict_names(dict, keyword): · line 106
Quoted 3 lines
client/tests/kvm/kvm_utils.py +3
-0+ mac_shelve = shelve.open(mac_filename, writeback=False) + + mac_pool = mac_shelve.get("macpool")Why is this 'macpool' needed? Why not put the keys directly in the shelve object?
yes, put keys directly in the shelve object is better.
Quoted 10 lines
client/tests/kvm/kvm_utils.py def get_sub_dict_names(dict, keyword): · line 115
Quoted 8 lines
client/tests/kvm/kvm_utils.py +8
-0+ if not mac_pool: + mac_pool = {} + found = False + + val = "%s:%s" % (vm, nic_index) + for key in mac_pool.keys(): + if val in mac_pool[key]: + mac_pool[key].append(val)Why append val to mac_pool[key] if val is already in mac_pool[key]?
need drop it.
Quoted 14 lines
client/tests/kvm/kvm_utils.py def get_sub_dict_names(dict, keyword): · line 128
Quoted 13 lines
client/tests/kvm/kvm_utils.py +13
-0+ found = True + mac = key + + while not found: + postfix = "%02x:%02x" % (random.randint(0x00,0xfe), + random.randint(0x00,0xfe)) + mac = prefix + postfix + mac_list = mac.split(":") + # Clear multicast bit + mac_list[0] = int(mac_list[0],16) & 0xfe + # Set local assignment bit (IEEE802) + mac_list[0] = mac_list[0] | 0x02 + mac_list[0] = "%02x" % mac_list[0]
Quoted 4 lines
Why is this needed? Most mac addresses begin with 00. If the mac address is generated from the address of eth0 (using the method in this patch) it begins with 00, which is fine. If the prefix is set by the user using
mac_prefix, I don't think we should modify it.
linux-2.6/include/linux/etherdevice.h
/**
* random_ether_addr - Generate software assigned random Ethernet address
* @addr: Pointer to a six-byte array containing the Ethernet address *
* Generate a random Ethernet address (MAC) that is not multicast
* and has the local assigned bit set.
*/
static inline void random_ether_addr(u8 *addr)
{
get_random_bytes (addr, ETH_ALEN);
addr [0] &= 0xfe; /* clear multicast bit */
addr [0] |= 0x02; /* set local assignment bit (IEEE802) */
}Quoted 9 lines
Quoted 9 lines
client/tests/kvm/kvm_utils.py +9
-0+ mac = ":".join(mac_list) + if mac not in mac_pool.keys() or 0 == len(mac_pool[mac]): + mac_pool[mac] = ["%s:%s" % (vm,nic_index)] + found = True + mac_shelve["macpool"] = mac_pool + logging.debug("generating mac addr %s " % mac) + + mac_shelve.close() + fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN)
Quoted 8 lines
client/tests/kvm/kvm_utils.py def get_sub_dict_names(dict, keyword): · line 142
Quoted 5 lines
client/tests/kvm/kvm_utils.py +5
-0+ lock_file.close() + return mac + + +def put_mac_to_pool(root_dir, mac, vm):This function removes a mac address from
address_pool. I wonder why you chose this name.
address_pool file is a container to record macaddress. The 'pool' we get/put mac is an abstract resource pool. This is really confused ;)
Quoted 96 lines
Quoted 96 lines
client/tests/kvm/kvm_utils.py +41
-0+ """ + Put the macaddress back to address pool + + @param root_dir: Root dir for kvm + @param vm: Here we use instance attribute of vm + @param mac: mac address will be put. + @Return: mac address. + """ + + lock_filename = os.path.join(root_dir, "mac_lock") + lock_file = open(lock_filename, 'w') + fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX) + mac_filename = os.path.join(root_dir, "address_pool") + mac_shelve = shelve.open(mac_filename, writeback=False) + + mac_pool = mac_shelve.get("macpool") + + if not mac_pool or (not mac in mac_pool): + logging.debug("Try to free a macaddress does no in pool") + logging.debug("macaddress is %s" % mac) + logging.debug("pool is %s" % mac_pool) + else: + if len(mac_pool[mac]) <= 1: + mac_pool.pop(mac) + else: + for value in mac_pool[mac]: + if vm in value: + mac_pool[mac].remove(value) + break + if len(mac_pool[mac]) == 0: + mac_pool.pop(mac) + + mac_shelve["macpool"] = mac_pool + logging.debug("freeing mac addr %s " % mac) + + mac_shelve.close() + fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN) + lock_file.close() + return mac + + def mac_str_to_int(addr): """ Convert MAC address string to integer.client/tests/kvm/kvm_vm.py +12
-3@@ -5,7 +5,7 @@ Utility classes and functions to handle Virtual Machine creation using qemu. @copyright: 2008-2009 Red Hat Inc. """ -import time, socket, os, logging, fcntl, re, commands, glob +import time, socket, os, logging, fcntl, re, commands, shelve, glob import kvm_utils, kvm_subprocess, kvm_monitor, rss_file_transfer from autotest_lib.client.common_lib import error from autotest_lib.client.bin import utils@@ -117,6 +117,7 @@ class VM: self.params = params self.root_dir = root_dir self.address_cache = address_cache + self.mac_prefix = params.get('mac_prefix') self.netdev_id = [] # Find a unique identifier for this VM@@ -126,8 +127,15 @@ class VM: if not glob.glob("/tmp/*%s" % self.instance): break + if self.mac_prefix is None: + s, o = commands.getstatusoutput("ifconfig eth0") + if s == 0: + mac = re.findall("HWaddr (\S*)", o)[0] + self.mac_prefix = '00' + mac[8:] + ':' + - def clone(self, name=None, params=None, root_dir=None, address_cache=None): + def clone(self, name=None, params=None, root_dir=None, + address_cache=None, mac_clone=True): """ Return a clone of the VM object with optionally modified parameters. The clone is initially not alive and needs to be started using create().@@ -138,6 +146,7 @@ class VM: @param params: Optional new VM creation parameters @param root_dir: Optional new base directory for relative filenames @param address_cache: A dict that maps MAC addresses to IP addresses + @param mac_clone: Clone mac address or not. """ if name is None: name = self.name@@ -147,9 +156,19 @@ class VM: root_dir = self.root_dir if address_cache is None: address_cache = self.address_cache - return VM(name, params, root_dir, address_cache) + vm = VM(name, params, root_dir, address_cache)
Quoted 8 lines
client/tests/kvm/kvm_vm.py class VM: · line 162
Quoted 3 lines
client/tests/kvm/kvm_vm.py +3
-0+ if mac_clone: + # Clone mac address by coping 'self.instance' to the new vm. + vm.instance = self.instanceCopying self.instance isn't a good idea because the monitor, serial console and testlog filenames use self.instance. self.instance should be unique. If we still want to use the same mac address for 2 VMs, for example in migration, a different solution should be found.
We almost only clone mac in migration test, the src guest would be killed. If the instance of dest guest is same as src guest, the serial connection would not break, it would continually write log to original log file.
Quoted 26 lines
Quoted 26 lines
client/tests/kvm/kvm_vm.py +14
-0+ return vm + def free_mac_address(self): + nic_num = len(kvm_utils.get_sub_dict_names(self.params, "nics")) + for nic in range(nic_num): + mac = self.get_macaddr(nic_index=nic) + kvm_utils.put_mac_to_pool(self.root_dir, mac, vm=self.instance) + def make_qemu_command(self, name=None, params=None, root_dir=None): """ Generate a qemu command line. All parameters are optional. If a@@ -383,6 +402,13 @@ class VM: mac = None if "address_index" in nic_params: mac = kvm_utils.get_mac_ip_pair_from_dict(nic_params)[0] + self.set_mac_address(mac=mac, nic_index=vlan) + else: + mac = kvm_utils.get_mac_from_pool(self.root_dir, + vm=self.instance, + nic_index=vlan, + prefix=self.mac_prefix) + qemu_cmd += add_nic(help, vlan, nic_params.get("nic_model"), mac, self.netdev_id[vlan]) # Handle the '-net tap' or '-net user' part
Quoted 8 lines
client/tests/kvm/kvm_vm.py class VM: · line 425
Quoted 6 lines
client/tests/kvm/kvm_vm.py +1
-1@@ -396,7 +422,7 @@ class VM: if tftp: tftp = kvm_utils.get_path(root_dir, tftp) qemu_cmd += add_net(help, vlan, nic_params.get("nic_mode", "user"), - nic_params.get("nic_ifname"), + self.get_ifname(vlan),Why is
get_ifname()useful here?
dynamically generate the ifname is better.
Quoted 77 lines
Quoted 77 lines
client/tests/kvm/kvm_vm.py +25
-3script, downscript, tftp, nic_params.get("bootp"), redirs, self.netdev_id[vlan])@@ -720,7 +746,7 @@ class VM: lockfile.close() - def destroy(self, gracefully=True): + def destroy(self, gracefully=True, free_macaddr=True): """ Destroy the VM.@@ -731,6 +757,7 @@ class VM: @param gracefully: Whether an attempt will be made to end the VM using a shell command before trying to end the qemu process with a 'quit' or a kill signal. + @param free_macaddr: Whether free macaddresses when destory a vm. """ try: # Is it already dead?@@ -751,11 +778,18 @@ class VM: logging.debug("Shutdown command sent; waiting for VM " "to go down...") if kvm_utils.wait_for(self.is_dead, 60, 1, 1): - logging.debug("VM is down") + logging.debug("VM is down, free mac address.") + # free mac address + if free_macaddr: + self.free_mac_address() return finally: session.close() + # free mac address + if free_macaddr: + self.free_mac_address() + if self.monitor: # Try to destroy with a monitor command logging.debug("Trying to kill VM with monitor command...")@@ -881,10 +915,14 @@ class VM: nic_name = nics[index] nic_params = kvm_utils.get_sub_dict(self.params, nic_name) if nic_params.get("nic_mode") == "tap": - mac, ip = kvm_utils.get_mac_ip_pair_from_dict(nic_params) + mac = self.get_macaddr(index) if not mac: logging.debug("MAC address unavailable") return None + else: + mac = mac.lower() + ip = None + if not ip or nic_params.get("always_use_tcpdump") == "yes": # Get the IP address from the cache ip = self.address_cache.get(mac)@@ -897,6 +935,7 @@ class VM: for nic in nics] macs = [kvm_utils.get_mac_ip_pair_from_dict(dict)[0] for dict in nic_dicts] + macs.append(mac) if not kvm_utils.verify_ip_address_ownership(ip, macs): logging.debug("Could not verify MAC-IP address mapping: " "%s ---> %s" % (mac, ip))@@ -925,6 +964,71 @@ class VM: "redirected" % port) return self.redirs.get(port) + def get_ifname(self, nic_index=0): + """ + Return the ifname of tap device for the guest nic. + + @param nic_index: Index of the NIC + """ + + nics = kvm_utils.get_sub_dict_names(self.params, "nics") + nic_name = nics[nic_index]
Quoted 8 lines
client/tests/kvm/kvm_vm.py class VM: · line 981
Quoted 6 lines
client/tests/kvm/kvm_vm.py +6
-0+ nic_params = kvm_utils.get_sub_dict(self.params, nic_name) + if nic_params.get("nic_ifname"): + return nic_params.get("nic_ifname") + else: + return "%s_%s_%s" % (nic_params.get("nic_model"), + nic_index, self.vnc_port)What's the purpose of this string?
Just avoid repeated ifname. The vnc_port is unique for each VM, nic_index is unique for each nic of one VM.
Quoted 68 lines
Quoted 67 lines
client/tests/kvm/kvm_vm.py +49
-0+ def get_macaddr(self, nic_index=0): + """ + Return the macaddr of guest nic. + + @param nic_index: Index of the NIC + """ + mac_filename = os.path.join(self.root_dir, "address_pool") + mac_shelve = shelve.open(mac_filename, writeback=False) + mac_pool = mac_shelve.get("macpool") + val = "%s:%s" % (self.instance, nic_index) + for key in mac_pool.keys(): + if val in mac_pool[key]: + return key + return None + + def set_mac_address(self, mac, nic_index=0, shareable=False): + """ + Set mac address for guest. Note: It just update address pool. + + @param mac: address will set to guest + @param nic_index: Index of the NIC + @param shareable: Where VM can share mac with other VM or not. + """ + lock_filename = os.path.join(self.root_dir, "mac_lock") + lock_file = open(lock_filename, 'w') + fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX) + mac_filename = os.path.join(self.root_dir, "address_pool") + mac_shelve = shelve.open(mac_filename, writeback=False) + mac_pool = mac_shelve.get("macpool") + if not mac_pool: + mac_pool = {} + value = "%s:%s" % (self.instance, nic_index) + if mac not in mac_pool.keys(): + for key in mac_pool.keys(): + if value in mac_pool[key]: + mac_pool[key].remove(value) + if len(mac_pool[key]) == 0: + mac_pool.pop(key) + mac_pool[mac] = [value] + else: + if shareable: + mac_pool[mac].append(value) + else: + logging.error("Mac address already be used!") + return False + mac_shelve["macpool"] = mac_pool + mac_shelve.close() + fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN) + lock_file.close() def get_pid(self): """client/tests/kvm/tests_base.cfg.sample +1
-1@@ -51,7 +51,7 @@ guest_port_remote_shell = 22 nic_mode = user #nic_mode = tap nic_script = scripts/qemu-ifup -address_index = 0 +#address_index = 0 run_tcpdump = yes # Misc
Re: [Qemu-devel] [RFC PATCH 01/14] KVM-test: Add a new macaddress pool algorithm
Quoted 35 lines · Amos Kong
Quoted 1 line · Michael Goldish
Michael,
Thanks for your comments. Let's simplify this method together.
Quoted 28 lines · Amos Kong
Quoted 26 lines
Old method uses the mac address in the configuration files which could lead serious problem when multiple tests running in different hosts.
This patch adds a new macaddress pool algorithm, it generates the mac prefix based on mac address of the host which could eliminate the duplicated mac addresses between machines.
When user have set the
mac_prefixin the configuration file, we should use it in stead of the dynamic generated mac prefix.Other change:
. Fix randomly generating mac address so that it correspond to IEEE802. . Update clone function to decide clone mac address or not. . Updateget_macaddrfunction.
. Addset_mac_addressfunction.New auto mac address pool algorithm:
Ifaddress_indexis defined, VM will get mac from config file then record mac in toaddress_pool. Ifaddress_indexis not defined, VM will callget_mac_from_poolto auto create mac then recored mac toaddress_poolin following format:
{'macpool': {'AE:9D:94:6A:9b:f9': ['20100310-165222-Wt7l:0']}}AE:9D:94:6A:9b:f9 : mac address
20100310-165222-Wt7l : instance attribute of VM
0 : index of NIC
Quoted 8 lines
Quoted 6 lines
Why do you use the mac address as a key, instead of the instance string
+ nic index? When the mac address is used as a key, each key has a list of values instead of just one value. This order seems unnatural. If it were the other way around (i.e. key = VM instance + nic index, value = mac address), then each key would have exactly one value, and I think this patch would be shorter and simpler.One mac address may be used by two VMs, eg. migration.
Sure, that's why I thought the opposite direction would be better: keys = VMs (nics), values = mac addresses. That way we have one value per key, instead of a list of values per key.
To clarify, instead of using:
{'AE:9D:94:6A:9b:f9': ['20100310-165222-Wt7l:0', '20100310-165222-Wt7l:1', '20100310-165222-Wt7l:2']}
I suggest:
{'20100310-165222-Wt7l:0': 'AE:9D:94:6A:9b:f9',
'20100310-165222-Wt7l:1': 'AE:9D:94:6A:9b:f9',
'20100310-165222-Wt7l:2': 'AE:9D:94:6A:9b:f9'}
Quoted 29 lines
Quoted 20 lines
Quoted 20 lines
Signed-off-by Jason Wang <[email protected]>Signed-off-by Feng Yang <[email protected]>Signed-off-by Amos Kong <[email protected]>---
0 files changed, 0 insertions(+), 0 deletions(-)client/tests/kvm/kvm_utils.py +1
-0@@ -5,6 +5,7 @@ KVM test utility functions. """ import time, string, random, socket, os, signal, re, logging, commands, cPickle +import fcntl, shelve from autotest_lib.client.bin import utils from autotest_lib.client.common_lib import error, logging_config import kvm_subprocess@@ -82,6 +83,104 @@ def get_sub_dict_names(dict, keyword):Quoted 8 lines
client/tests/kvm/kvm_utils.py def get_sub_dict_names(dict, keyword): · line 86
Quoted 3 lines
client/tests/kvm/kvm_utils.py +1
-0# Functions related to MAC/IP addresses +def get_mac_from_pool(root_dir, vm, nic_index, prefix='00:11:22:33:'):The name of this function is confusing because it does the exact
opposite: it puts a mac address inaddress_pool. Maybe the pool you're referring to in the name isn'taddress_pool, but still a less confusing name should probably be used.
Quoted 4 lines
How about
allocate_mac(...)?address_pool->address_containerAllocate mac address and record into
address_container.
Yes, something like that, sounds less confusing.
Quoted 76 lines
Quoted 13 lines
Quoted 13 lines
client/tests/kvm/kvm_utils.py +13
-0+ """ + random generated mac address. + + 1) First try to generate macaddress based on the mac address prefix. + 2) And then try to use total random generated mac address. + + @param root_dir: Root dir for kvm + @param vm: Here we use instance of vm + @param nic_index: The index of nic. + @param prefix: Prefix of mac address. + @Return: Return mac address. + """ +Quoted 8 lines
client/tests/kvm/kvm_utils.py def get_sub_dict_names(dict, keyword): · line 103
Quoted 4 lines
client/tests/kvm/kvm_utils.py +4
-0+ lock_filename = os.path.join(root_dir, "mac_lock") + lock_file = open(lock_filename, 'w') + fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX) + mac_filename = os.path.join(root_dir, "address_pool")Maybe it makes sense to put
address_pooland the lock file in /tmp, where they can be shared by more than a single autotest instance running on the same host (unlikely, but theoretically possible).good idea.
Quoted 6 lines
client/tests/kvm/kvm_utils.py def get_sub_dict_names(dict, keyword): · line 106
Quoted 3 lines
client/tests/kvm/kvm_utils.py +3
-0+ mac_shelve = shelve.open(mac_filename, writeback=False) + + mac_pool = mac_shelve.get("macpool")Why is this 'macpool' needed? Why not put the keys directly in the shelve object?
yes, put keys directly in the shelve object is better.
Quoted 10 lines
client/tests/kvm/kvm_utils.py def get_sub_dict_names(dict, keyword): · line 115
Quoted 8 lines
client/tests/kvm/kvm_utils.py +8
-0+ if not mac_pool: + mac_pool = {} + found = False + + val = "%s:%s" % (vm, nic_index) + for key in mac_pool.keys(): + if val in mac_pool[key]: + mac_pool[key].append(val)Why append val to mac_pool[key] if val is already in mac_pool[key]?
need drop it.
Quoted 14 lines
client/tests/kvm/kvm_utils.py def get_sub_dict_names(dict, keyword): · line 128
Quoted 13 lines
client/tests/kvm/kvm_utils.py +13
-0+ found = True + mac = key + + while not found: + postfix = "%02x:%02x" % (random.randint(0x00,0xfe), + random.randint(0x00,0xfe)) + mac = prefix + postfix + mac_list = mac.split(":") + # Clear multicast bit + mac_list[0] = int(mac_list[0],16) & 0xfe + # Set local assignment bit (IEEE802) + mac_list[0] = mac_list[0] | 0x02 + mac_list[0] = "%02x" % mac_list[0]Quoted 4 lines
Why is this needed? Most mac addresses begin with 00. If the mac address is generated from the address of eth0 (using the method in this patch) it begins with 00, which is fine. If the prefix is set by the user using
mac_prefix, I don't think we should modify it.
linux-2.6/include/linux/etherdevice.h/**
*random_ether_addr- Generate software assigned random Ethernet address
* @addr: Pointer to a six-byte array containing the Ethernet address *
* Generate a random Ethernet address (MAC) that is not multicast
* and has the local assigned bit set.
*/
static inline voidrandom_ether_addr(u8 *addr)
Quoted 5 lines
{
get_random_bytes (addr, ETH_ALEN);
addr [0] &= 0xfe; /* clear multicast bit */
addr [0] |= 0x02; /* set local assignment bit (IEEE802) */
}I saw this googling, but I still don't understand why it's necessary when real devices start with 00.
Quoted 18 lines
Quoted 9 lines
Quoted 9 lines
client/tests/kvm/kvm_utils.py +9
-0+ mac = ":".join(mac_list) + if mac not in mac_pool.keys() or 0 == len(mac_pool[mac]): + mac_pool[mac] = ["%s:%s" % (vm,nic_index)] + found = True + mac_shelve["macpool"] = mac_pool + logging.debug("generating mac addr %s " % mac) + + mac_shelve.close() + fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN)Quoted 8 lines
client/tests/kvm/kvm_utils.py def get_sub_dict_names(dict, keyword): · line 142
Quoted 5 lines
client/tests/kvm/kvm_utils.py +5
-0+ lock_file.close() + return mac + + +def put_mac_to_pool(root_dir, mac, vm):This function removes a mac address from
address_pool. I wonder why you chose this name.
Quoted 3 lines
address_poolfile is a container to record macaddress. The 'pool' we get/put mac is an abstract resource pool. This is really confused ;)
Yes, I guessed it would be something like that. Still a bit confusing, I agree, but it's not a big deal to change function names.
Quoted 105 lines
Quoted 96 lines
Quoted 96 lines
client/tests/kvm/kvm_utils.py +41
-0+ """ + Put the macaddress back to address pool + + @param root_dir: Root dir for kvm + @param vm: Here we use instance attribute of vm + @param mac: mac address will be put. + @Return: mac address. + """ + + lock_filename = os.path.join(root_dir, "mac_lock") + lock_file = open(lock_filename, 'w') + fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX) + mac_filename = os.path.join(root_dir, "address_pool") + mac_shelve = shelve.open(mac_filename, writeback=False) + + mac_pool = mac_shelve.get("macpool") + + if not mac_pool or (not mac in mac_pool): + logging.debug("Try to free a macaddress does no in pool") + logging.debug("macaddress is %s" % mac) + logging.debug("pool is %s" % mac_pool) + else: + if len(mac_pool[mac]) <= 1: + mac_pool.pop(mac) + else: + for value in mac_pool[mac]: + if vm in value: + mac_pool[mac].remove(value) + break + if len(mac_pool[mac]) == 0: + mac_pool.pop(mac) + + mac_shelve["macpool"] = mac_pool + logging.debug("freeing mac addr %s " % mac) + + mac_shelve.close() + fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN) + lock_file.close() + return mac + + def mac_str_to_int(addr): """ Convert MAC address string to integer.client/tests/kvm/kvm_vm.py +12
-3@@ -5,7 +5,7 @@ Utility classes and functions to handle Virtual Machine creation using qemu. @copyright: 2008-2009 Red Hat Inc. """ -import time, socket, os, logging, fcntl, re, commands, glob +import time, socket, os, logging, fcntl, re, commands, shelve, glob import kvm_utils, kvm_subprocess, kvm_monitor, rss_file_transfer from autotest_lib.client.common_lib import error from autotest_lib.client.bin import utils@@ -117,6 +117,7 @@ class VM: self.params = params self.root_dir = root_dir self.address_cache = address_cache + self.mac_prefix = params.get('mac_prefix') self.netdev_id = [] # Find a unique identifier for this VM@@ -126,8 +127,15 @@ class VM: if not glob.glob("/tmp/*%s" % self.instance): break + if self.mac_prefix is None: + s, o = commands.getstatusoutput("ifconfig eth0") + if s == 0: + mac = re.findall("HWaddr (\S*)", o)[0] + self.mac_prefix = '00' + mac[8:] + ':' + - def clone(self, name=None, params=None, root_dir=None, address_cache=None): + def clone(self, name=None, params=None, root_dir=None, + address_cache=None, mac_clone=True): """ Return a clone of the VM object with optionally modified parameters. The clone is initially not alive and needs to be started using create().@@ -138,6 +146,7 @@ class VM: @param params: Optional new VM creation parameters @param root_dir: Optional new base directory for relative filenames @param address_cache: A dict that maps MAC addresses to IP addresses + @param mac_clone: Clone mac address or not. """ if name is None: name = self.name@@ -147,9 +156,19 @@ class VM: root_dir = self.root_dir if address_cache is None: address_cache = self.address_cache - return VM(name, params, root_dir, address_cache) + vm = VM(name, params, root_dir, address_cache)Quoted 8 lines
client/tests/kvm/kvm_vm.py class VM: · line 162
Quoted 3 lines
client/tests/kvm/kvm_vm.py +3
-0+ if mac_clone: + # Clone mac address by coping 'self.instance' to the new vm. + vm.instance = self.instanceCopying self.instance isn't a good idea because the monitor, serial console and testlog filenames use self.instance. self.instance should be unique. If we still want to use the same mac address for 2 VMs, for example in migration, a different solution should be found.
Quoted 3 lines
We almost only clone mac in migration test, the src guest would be killed. If the instance of dest guest is same as src guest, the serial connection would not break, it would continually write log to original log file.
What about the monitor(s)? The fact that the serial console connection doesn't break doesn't necessarily mean everything's OK.
Quoted 35 lines
Quoted 26 lines
Quoted 26 lines
client/tests/kvm/kvm_vm.py +14
-0+ return vm + def free_mac_address(self): + nic_num = len(kvm_utils.get_sub_dict_names(self.params, "nics")) + for nic in range(nic_num): + mac = self.get_macaddr(nic_index=nic) + kvm_utils.put_mac_to_pool(self.root_dir, mac, vm=self.instance) + def make_qemu_command(self, name=None, params=None, root_dir=None): """ Generate a qemu command line. All parameters are optional. If a@@ -383,6 +402,13 @@ class VM: mac = None if "address_index" in nic_params: mac = kvm_utils.get_mac_ip_pair_from_dict(nic_params)[0] + self.set_mac_address(mac=mac, nic_index=vlan) + else: + mac = kvm_utils.get_mac_from_pool(self.root_dir, + vm=self.instance, + nic_index=vlan, + prefix=self.mac_prefix) + qemu_cmd += add_nic(help, vlan, nic_params.get("nic_model"), mac, self.netdev_id[vlan]) # Handle the '-net tap' or '-net user' partQuoted 8 lines
client/tests/kvm/kvm_vm.py class VM: · line 425
Quoted 6 lines
client/tests/kvm/kvm_vm.py +1
-1@@ -396,7 +422,7 @@ class VM: if tftp: tftp = kvm_utils.get_path(root_dir, tftp) qemu_cmd += add_net(help, vlan, nic_params.get("nic_mode", "user"), - nic_params.get("nic_ifname"), + self.get_ifname(vlan),Why is
get_ifname()useful here?
Quoted 1 line
dynamically generate the ifname is better.
But why? Is this ifname useful for any test?
Quoted 86 lines
Quoted 77 lines
Quoted 77 lines
client/tests/kvm/kvm_vm.py +25
-3script, downscript, tftp, nic_params.get("bootp"), redirs, self.netdev_id[vlan])@@ -720,7 +746,7 @@ class VM: lockfile.close() - def destroy(self, gracefully=True): + def destroy(self, gracefully=True, free_macaddr=True): """ Destroy the VM.@@ -731,6 +757,7 @@ class VM: @param gracefully: Whether an attempt will be made to end the VM using a shell command before trying to end the qemu process with a 'quit' or a kill signal. + @param free_macaddr: Whether free macaddresses when destory a vm. """ try: # Is it already dead?@@ -751,11 +778,18 @@ class VM: logging.debug("Shutdown command sent; waiting for VM " "to go down...") if kvm_utils.wait_for(self.is_dead, 60, 1, 1): - logging.debug("VM is down") + logging.debug("VM is down, free mac address.") + # free mac address + if free_macaddr: + self.free_mac_address() return finally: session.close() + # free mac address + if free_macaddr: + self.free_mac_address() + if self.monitor: # Try to destroy with a monitor command logging.debug("Trying to kill VM with monitor command...")@@ -881,10 +915,14 @@ class VM: nic_name = nics[index] nic_params = kvm_utils.get_sub_dict(self.params, nic_name) if nic_params.get("nic_mode") == "tap": - mac, ip = kvm_utils.get_mac_ip_pair_from_dict(nic_params) + mac = self.get_macaddr(index) if not mac: logging.debug("MAC address unavailable") return None + else: + mac = mac.lower() + ip = None + if not ip or nic_params.get("always_use_tcpdump") == "yes": # Get the IP address from the cache ip = self.address_cache.get(mac)@@ -897,6 +935,7 @@ class VM: for nic in nics] macs = [kvm_utils.get_mac_ip_pair_from_dict(dict)[0] for dict in nic_dicts] + macs.append(mac) if not kvm_utils.verify_ip_address_ownership(ip, macs): logging.debug("Could not verify MAC-IP address mapping: " "%s ---> %s" % (mac, ip))@@ -925,6 +964,71 @@ class VM: "redirected" % port) return self.redirs.get(port) + def get_ifname(self, nic_index=0): + """ + Return the ifname of tap device for the guest nic. + + @param nic_index: Index of the NIC + """ + + nics = kvm_utils.get_sub_dict_names(self.params, "nics") + nic_name = nics[nic_index]Quoted 8 lines
client/tests/kvm/kvm_vm.py class VM: · line 981
Quoted 6 lines
client/tests/kvm/kvm_vm.py +6
-0+ nic_params = kvm_utils.get_sub_dict(self.params, nic_name) + if nic_params.get("nic_ifname"): + return nic_params.get("nic_ifname") + else: + return "%s_%s_%s" % (nic_params.get("nic_model"), + nic_index, self.vnc_port)What's the purpose of this string?
Quoted 1 line
Just avoid repeated ifname. The
vnc_portis unique for each VM,nic_indexis unique for each nic of one VM.
self.instance should also be unique, though it's quite long.
Quoted 69 lines
Quoted 68 lines
Quoted 67 lines
client/tests/kvm/kvm_vm.py +49
-0+ def get_macaddr(self, nic_index=0): + """ + Return the macaddr of guest nic. + + @param nic_index: Index of the NIC + """ + mac_filename = os.path.join(self.root_dir, "address_pool") + mac_shelve = shelve.open(mac_filename, writeback=False) + mac_pool = mac_shelve.get("macpool") + val = "%s:%s" % (self.instance, nic_index) + for key in mac_pool.keys(): + if val in mac_pool[key]: + return key + return None + + def set_mac_address(self, mac, nic_index=0, shareable=False): + """ + Set mac address for guest. Note: It just update address pool. + + @param mac: address will set to guest + @param nic_index: Index of the NIC + @param shareable: Where VM can share mac with other VM or not. + """ + lock_filename = os.path.join(self.root_dir, "mac_lock") + lock_file = open(lock_filename, 'w') + fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX) + mac_filename = os.path.join(self.root_dir, "address_pool") + mac_shelve = shelve.open(mac_filename, writeback=False) + mac_pool = mac_shelve.get("macpool") + if not mac_pool: + mac_pool = {} + value = "%s:%s" % (self.instance, nic_index) + if mac not in mac_pool.keys(): + for key in mac_pool.keys(): + if value in mac_pool[key]: + mac_pool[key].remove(value) + if len(mac_pool[key]) == 0: + mac_pool.pop(key) + mac_pool[mac] = [value] + else: + if shareable: + mac_pool[mac].append(value) + else: + logging.error("Mac address already be used!") + return False + mac_shelve["macpool"] = mac_pool + mac_shelve.close() + fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN) + lock_file.close() def get_pid(self): """client/tests/kvm/tests_base.cfg.sample +1
-1@@ -51,7 +51,7 @@ guest_port_remote_shell = 22 nic_mode = user #nic_mode = tap nic_script = scripts/qemu-ifup -address_index = 0 +#address_index = 0 run_tcpdump = yes # Misc
Re: [Qemu-devel] [RFC PATCH 01/14] KVM-test: Add a new macaddress pool algorithm
Quoted 7 lines · Michael Goldish
Quoted 6 lines · Amos Kong
Quoted 1 line · Michael Goldish
Michael,
Thanks for your comments. Let's simplify this method together.
I would produce a v2 later basing on our discussion.
Quoted 18 lines
Quoted 18 lines
Quoted 17 lines
Quoted 5 lines
client/tests/kvm/kvm_vm.py +5
-0+ def get_ifname(self, nic_index=0): + """ + Return the ifname of tap device for the guest nic. + + @param nic_index: Index of the NICclient/tests/kvm/kvm_vm.py class VM: · line 981
Quoted 10 lines
client/tests/kvm/kvm_vm.py +10
-0+ """ + + nics = kvm_utils.get_sub_dict_names(self.params, "nics") + nic_name = nics[nic_index] + nic_params = kvm_utils.get_sub_dict(self.params, nic_name) + if nic_params.get("nic_ifname"): + return nic_params.get("nic_ifname") + else: + return "%s_%s_%s" % (nic_params.get("nic_model"), + nic_index, self.vnc_port)What's the purpose of this string?
Quoted 3 lines
Quoted 1 line
Just avoid repeated ifname. The
vnc_portis unique for each VM,nic_indexis unique for each nic of one VM.self.instance should also be unique, though it's quite long.
qemu can only receive 15 chars as ifname, instance is too long. if the prefix of two instance are same, the actual ifnames will not be unique.
linux-user/syscall.c:
#define IFNAMSIZ 16
net/tap-linux.c:
int tap_open(...
...
if (ifname[0] != '\0')
pstrcpy(ifr.ifr_name, IFNAMSIZ, ifname);
else
pstrcpy(ifr.ifr_name, IFNAMSIZ, "tap%d");Re: [RFC PATCH 01/14] KVM-test: Add a new macaddress pool algorithm
In reply to Amos KongQuoted 9 lines · Amos Kong
Old method uses the mac address in the configuration files which could lead serious problem when multiple tests running in different hosts.
This patch adds a new macaddress pool algorithm, it generates the mac prefix based on mac address of the host which could eliminate the duplicated mac addresses between machines.
When user have set the
mac_prefixin the configuration file, we should use it in stead of the dynamic generated mac prefix.
I have some additional comments besides the good points made by Michael on his review.
Quoted 37 lines
Other change:
. Fix randomly generating mac address so that it correspond to IEEE802. . Update clone function to decide clone mac address or not. . Updateget_macaddrfunction.
. Addset_mac_addressfunction.New auto mac address pool algorithm:
Ifaddress_indexis defined, VM will get mac from config file then record mac in toaddress_pool. Ifaddress_indexis not defined, VM will callget_mac_from_poolto auto create mac then recored mac toaddress_poolin following format:
{'macpool': {'AE:9D:94:6A:9b:f9': ['20100310-165222-Wt7l:0']}}AE:9D:94:6A:9b:f9 : mac address
20100310-165222-Wt7l : instance attribute of VM
0 : index of NICSigned-off-by Jason Wang <[email protected]>Signed-off-by Feng Yang <[email protected]>Signed-off-by Amos Kong <[email protected]>---
0 files changed, 0 insertions(+), 0 deletions(-)client/tests/kvm/kvm_utils.py +1
-0@@ -5,6 +5,7 @@ KVM test utility functions. """ import time, string, random, socket, os, signal, re, logging, commands, cPickle +import fcntl, shelve from autotest_lib.client.bin import utils from autotest_lib.client.common_lib import error, logging_config import kvm_subprocess
client/tests/kvm/kvm_utils.py def get_sub_dict_names(dict, keyword): · line 88
Quoted 6 lines
client/tests/kvm/kvm_utils.py +3
-0@@ -82,6 +83,104 @@ def get_sub_dict_names(dict, keyword): # Functions related to MAC/IP addresses +def get_mac_from_pool(root_dir, vm, nic_index, prefix='00:11:22:33:'): + """ + random generated mac address.
I think I got your original idea of naming this function get_mac_from_pool (the prefix assessed is the base of the 'pool'), but as Michael pointed out, a new MAC is being generated here and added to the mac pool, so a more appropriate name should be used. A straightforward suggestion would be generate_mac_address. Also, the description of the function would be more on the lines of "Generate a MAC address and add it to the MAC pool".
Quoted 5 lines
client/tests/kvm/kvm_utils.py +5
-0+ 1) First try to generate macaddress based on the mac address prefix. + 2) And then try to use total random generated mac address. + + @param root_dir: Root dir for kvm + @param vm: Here we use instance of vm
client/tests/kvm/kvm_utils.py def get_sub_dict_names(dict, keyword): · line 104
Quoted 10 lines
client/tests/kvm/kvm_utils.py +10
-0+ @param nic_index: The index of nic. + @param prefix: Prefix of mac address. + @Return: Return mac address. + """ + + lock_filename = os.path.join(root_dir, "mac_lock") + lock_file = open(lock_filename, 'w') + fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX) + mac_filename = os.path.join(root_dir, "address_pool") + mac_shelve = shelve.open(mac_filename, writeback=False)
^ Indeed we should keep the mac pool under /tmp as Michael already pointed out.
Quoted 7 lines
client/tests/kvm/kvm_utils.py +7
-0+ mac_pool = mac_shelve.get("macpool") + + if not mac_pool: + mac_pool = {} + found = False + + val = "%s:%s" % (vm, nic_index)
client/tests/kvm/kvm_utils.py def get_sub_dict_names(dict, keyword): · line 122
Quoted 10 lines
client/tests/kvm/kvm_utils.py +10
-0+ for key in mac_pool.keys(): + if val in mac_pool[key]: + mac_pool[key].append(val) + found = True + mac = key + + while not found: + postfix = "%02x:%02x" % (random.randint(0x00,0xfe), + random.randint(0x00,0xfe)) + mac = prefix + postfix
^ It's more natural to use 'sufix' instead of 'postfix' here.
client/tests/kvm/kvm_utils.py def get_sub_dict_names(dict, keyword): · line 134
Quoted 12 lines
client/tests/kvm/kvm_utils.py +12
-0+ mac_list = mac.split(":") + # Clear multicast bit + mac_list[0] = int(mac_list[0],16) & 0xfe + # Set local assignment bit (IEEE802) + mac_list[0] = mac_list[0] | 0x02 + mac_list[0] = "%02x" % mac_list[0] + mac = ":".join(mac_list) + if mac not in mac_pool.keys() or 0 == len(mac_pool[mac]): + mac_pool[mac] = ["%s:%s" % (vm,nic_index)] + found = True + mac_shelve["macpool"] = mac_pool + logging.debug("generating mac addr %s " % mac)
^ Here the MAC was already generated, so here you could use something like "Generated MAC address %s"
client/tests/kvm/kvm_utils.py def get_sub_dict_names(dict, keyword): · line 147
Quoted 12 lines
client/tests/kvm/kvm_utils.py +12
-0+ mac_shelve.close() + fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN) + lock_file.close() + return mac + + +def put_mac_to_pool(root_dir, mac, vm): + """ + Put the macaddress back to address pool + + @param root_dir: Root dir for kvm + @param vm: Here we use instance attribute of vm
^ @param vm: Instance attribute of a vm. Also, we might want to rename this parameter to vm_instance, to make things more clear and avoid confusions (vm is referred in several places as a vm instance).
Quoted 4 lines
client/tests/kvm/kvm_utils.py +4
-0+ @param mac: mac address will be put. + @Return: mac address. + """ +
client/tests/kvm/kvm_utils.py def get_sub_dict_names(dict, keyword): · line 161
Quoted 10 lines
client/tests/kvm/kvm_utils.py +10
-0+ lock_filename = os.path.join(root_dir, "mac_lock") + lock_file = open(lock_filename, 'w') + fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX) + mac_filename = os.path.join(root_dir, "address_pool") + mac_shelve = shelve.open(mac_filename, writeback=False) + + mac_pool = mac_shelve.get("macpool") + + if not mac_pool or (not mac in mac_pool): + logging.debug("Try to free a macaddress does no in pool")
^ logging.debug("MAC not present in the MAC pool, not modifying pool")
Quoted 5 lines
client/tests/kvm/kvm_utils.py +5
-0+ logging.debug("macaddress is %s" % mac) + logging.debug("pool is %s" % mac_pool) + else: + if len(mac_pool[mac]) <= 1: + mac_pool.pop(mac)
client/tests/kvm/kvm_utils.py def get_sub_dict_names(dict, keyword): · line 176
Quoted 10 lines
client/tests/kvm/kvm_utils.py +10
-0+ else: + for value in mac_pool[mac]: + if vm in value: + mac_pool[mac].remove(value) + break + if len(mac_pool[mac]) == 0: + mac_pool.pop(mac) + + mac_shelve["macpool"] = mac_pool + logging.debug("freeing mac addr %s " % mac)
^ logging.debug("Freeing mac addr %s " % mac). Also, this should go to the beginning of the function, before the "MAC not present in the..." message.
Quoted 49 lines
client/tests/kvm/kvm_utils.py +7
-0+ + mac_shelve.close() + fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN) + lock_file.close() + return mac + + def mac_str_to_int(addr): """ Convert MAC address string to integer.client/tests/kvm/kvm_vm.py +10
-2@@ -5,7 +5,7 @@ Utility classes and functions to handle Virtual Machine creation using qemu. @copyright: 2008-2009 Red Hat Inc. """ -import time, socket, os, logging, fcntl, re, commands, glob +import time, socket, os, logging, fcntl, re, commands, shelve, glob import kvm_utils, kvm_subprocess, kvm_monitor, rss_file_transfer from autotest_lib.client.common_lib import error from autotest_lib.client.bin import utils@@ -117,6 +117,7 @@ class VM: self.params = params self.root_dir = root_dir self.address_cache = address_cache + self.mac_prefix = params.get('mac_prefix') self.netdev_id = [] # Find a unique identifier for this VM@@ -126,8 +127,15 @@ class VM: if not glob.glob("/tmp/*%s" % self.instance): break + if self.mac_prefix is None: + s, o = commands.getstatusoutput("ifconfig eth0") + if s == 0: + mac = re.findall("HWaddr (\S*)", o)[0] + self.mac_prefix = '00' + mac[8:] + ':' + - def clone(self, name=None, params=None, root_dir=None, address_cache=None): + def clone(self, name=None, params=None, root_dir=None, + address_cache=None, mac_clone=True): """ Return a clone of the VM object with optionally modified parameters. The clone is initially not alive and needs to be started using create().
client/tests/kvm/kvm_vm.py class VM: · line 150
Quoted 5 lines
client/tests/kvm/kvm_vm.py +1
-0@@ -138,6 +146,7 @@ class VM: @param params: Optional new VM creation parameters @param root_dir: Optional new base directory for relative filenames @param address_cache: A dict that maps MAC addresses to IP addresses + @param mac_clone: Clone mac address or not. """
Here if you redesign the patch as suggested by Michael, making the unique key the string 'instance' + 'nic_index', you could add a parameter like 'preserve_mac', to preserve the original mac address, for things like migration.
In such cases, first the entry of the nics present on the source vm would be removed from the MAC pool, then the new entries would be added.
Quoted 9 lines
client/tests/kvm/kvm_vm.py +2
-1if name is None: name = self.name@@ -147,9 +156,19 @@ class VM: root_dir = self.root_dir if address_cache is None: address_cache = self.address_cache - return VM(name, params, root_dir, address_cache) + vm = VM(name, params, root_dir, address_cache) + if mac_clone:
client/tests/kvm/kvm_vm.py class VM: · line 170
Quoted 10 lines
client/tests/kvm/kvm_vm.py +8
-0+ # Clone mac address by coping 'self.instance' to the new vm. + vm.instance = self.instance + return vm + def free_mac_address(self): + nic_num = len(kvm_utils.get_sub_dict_names(self.params, "nics")) + for nic in range(nic_num): + mac = self.get_macaddr(nic_index=nic) + kvm_utils.put_mac_to_pool(self.root_dir, mac, vm=self.instance)
^ Here we are removing an association between a 'instance'+'nic_index' and a MAC, so maybe this should be called 'clear_macs_from_nics'
Quoted 36 lines
client/tests/kvm/kvm_vm.py +9
-2def make_qemu_command(self, name=None, params=None, root_dir=None): """ Generate a qemu command line. All parameters are optional. If a@@ -383,6 +402,13 @@ class VM: mac = None if "address_index" in nic_params: mac = kvm_utils.get_mac_ip_pair_from_dict(nic_params)[0] + self.set_mac_address(mac=mac, nic_index=vlan) + else: + mac = kvm_utils.get_mac_from_pool(self.root_dir, + vm=self.instance, + nic_index=vlan, + prefix=self.mac_prefix) + qemu_cmd += add_nic(help, vlan, nic_params.get("nic_model"), mac, self.netdev_id[vlan]) # Handle the '-net tap' or '-net user' part@@ -396,7 +422,7 @@ class VM: if tftp: tftp = kvm_utils.get_path(root_dir, tftp) qemu_cmd += add_net(help, vlan, nic_params.get("nic_mode", "user"), - nic_params.get("nic_ifname"), + self.get_ifname(vlan), script, downscript, tftp, nic_params.get("bootp"), redirs, self.netdev_id[vlan])@@ -720,7 +746,7 @@ class VM: lockfile.close() - def destroy(self, gracefully=True): + def destroy(self, gracefully=True, free_macaddr=True): """ Destroy the VM.
client/tests/kvm/kvm_vm.py class VM: · line 760
Quoted 4 lines
client/tests/kvm/kvm_vm.py +1
-0@@ -731,6 +757,7 @@ class VM: @param gracefully: Whether an attempt will be made to end the VM using a shell command before trying to end the qemu process with a 'quit' or a kill signal. + @param free_macaddr: Whether free macaddresses when destory a vm.
^ Why would we want to conditionally remove the associations NIC-MAC? Don't we want to do it on all cases?
Quoted 133 lines
client/tests/kvm/kvm_vm.py +79
-2""" try: # Is it already dead?@@ -751,11 +778,18 @@ class VM: logging.debug("Shutdown command sent; waiting for VM " "to go down...") if kvm_utils.wait_for(self.is_dead, 60, 1, 1): - logging.debug("VM is down") + logging.debug("VM is down, free mac address.") + # free mac address + if free_macaddr: + self.free_mac_address() return finally: session.close() + # free mac address + if free_macaddr: + self.free_mac_address() + if self.monitor: # Try to destroy with a monitor command logging.debug("Trying to kill VM with monitor command...")@@ -881,10 +915,14 @@ class VM: nic_name = nics[index] nic_params = kvm_utils.get_sub_dict(self.params, nic_name) if nic_params.get("nic_mode") == "tap": - mac, ip = kvm_utils.get_mac_ip_pair_from_dict(nic_params) + mac = self.get_macaddr(index) if not mac: logging.debug("MAC address unavailable") return None + else: + mac = mac.lower() + ip = None + if not ip or nic_params.get("always_use_tcpdump") == "yes": # Get the IP address from the cache ip = self.address_cache.get(mac)@@ -897,6 +935,7 @@ class VM: for nic in nics] macs = [kvm_utils.get_mac_ip_pair_from_dict(dict)[0] for dict in nic_dicts] + macs.append(mac) if not kvm_utils.verify_ip_address_ownership(ip, macs): logging.debug("Could not verify MAC-IP address mapping: " "%s ---> %s" % (mac, ip))@@ -925,6 +964,71 @@ class VM: "redirected" % port) return self.redirs.get(port) + def get_ifname(self, nic_index=0): + """ + Return the ifname of tap device for the guest nic. + + @param nic_index: Index of the NIC + """ + + nics = kvm_utils.get_sub_dict_names(self.params, "nics") + nic_name = nics[nic_index] + nic_params = kvm_utils.get_sub_dict(self.params, nic_name) + if nic_params.get("nic_ifname"): + return nic_params.get("nic_ifname") + else: + return "%s_%s_%s" % (nic_params.get("nic_model"), + nic_index, self.vnc_port) + + def get_macaddr(self, nic_index=0): + """ + Return the macaddr of guest nic. + + @param nic_index: Index of the NIC + """ + mac_filename = os.path.join(self.root_dir, "address_pool") + mac_shelve = shelve.open(mac_filename, writeback=False) + mac_pool = mac_shelve.get("macpool") + val = "%s:%s" % (self.instance, nic_index) + for key in mac_pool.keys(): + if val in mac_pool[key]: + return key + return None + + def set_mac_address(self, mac, nic_index=0, shareable=False): + """ + Set mac address for guest. Note: It just update address pool. + + @param mac: address will set to guest + @param nic_index: Index of the NIC + @param shareable: Where VM can share mac with other VM or not. + """ + lock_filename = os.path.join(self.root_dir, "mac_lock") + lock_file = open(lock_filename, 'w') + fcntl.lockf(lock_file.fileno() ,fcntl.LOCK_EX) + mac_filename = os.path.join(self.root_dir, "address_pool") + mac_shelve = shelve.open(mac_filename, writeback=False) + mac_pool = mac_shelve.get("macpool") + if not mac_pool: + mac_pool = {} + value = "%s:%s" % (self.instance, nic_index) + if mac not in mac_pool.keys(): + for key in mac_pool.keys(): + if value in mac_pool[key]: + mac_pool[key].remove(value) + if len(mac_pool[key]) == 0: + mac_pool.pop(key) + mac_pool[mac] = [value] + else: + if shareable: + mac_pool[mac].append(value) + else: + logging.error("Mac address already be used!") + return False + mac_shelve["macpool"] = mac_pool + mac_shelve.close() + fcntl.lockf(lock_file.fileno(), fcntl.LOCK_UN) + lock_file.close() def get_pid(self): """client/tests/kvm/tests_base.cfg.sample +1
-1@@ -51,7 +51,7 @@ guest_port_remote_shell = 22 nic_mode = user #nic_mode = tap nic_script = scripts/qemu-ifup -address_index = 0 +#address_index = 0 run_tcpdump = yes # Misc
RFC PATCH 2/14 KVM Test: Add a function get_interface_name() to kvm_net_utils.py
In reply to Amos KongThe function get_interface_name is used to get the interface name of linux guest through the macaddress of specified macaddress.
---
0 files changed, 0 insertions(+), 0 deletions(-)
client/tests/kvm/kvm_net_utils.py new file +18 -0
@@ -0,0 +1,18 @@ +import re + +def get_linux_ifname(session, mac_address): + """ + Get the interface name through the mac address. + + @param session: session to the virtual machine + @mac_address: the macaddress of nic + """ + + output = session.get_command_output("ifconfig -a") + + try: + ethname = re.findall("(\w+)\s+Link.*%s" % mac_address, output, + re.IGNORECASE)[0] + return ethname + except: + return None
Re: [Qemu-devel] [RFC PATCH 02/14] KVM Test: Add a function get_interface_name() to kvm_net_utils.py
Quoted 2 lines · Amos Kong
The function
get_interface_nameis used to get the interface name of linux guest through the macaddress of specified macaddress.
I wonder if it wouldn't be overkill to have separate utility libraries on the kvm test instead of a single kvm_utils and kvm_test_utils like you are proposing. Any thoughts Michael?
Quoted 31 lines
Signed-off-by Jason Wang <[email protected]>Signed-off-by Amos Kong <[email protected]>---
0 files changed, 0 insertions(+), 0 deletions(-)client/tests/kvm/kvm_net_utils.py new file +18
-0@@ -0,0 +1,18 @@ +import re + +def get_linux_ifname(session, mac_address): + """ + Get the interface name through the mac address. + + @param session: session to the virtual machine + @mac_address: the macaddress of nic + """ + + output = session.get_command_output("ifconfig -a") + + try: + ethname = re.findall("(\w+)\s+Link.*%s" % mac_address, output, + re.IGNORECASE)[0] + return ethname + except: + return None
Re: [Qemu-devel] [RFC PATCH 02/14] KVM Test: Add a function get_interface_name() to kvm_net_utils.py
Quoted 7 lines · Lucas Meneghel Rodrigues
Quoted 2 lines · Amos Kong
The function
get_interface_nameis used to get the interface name of linux guest through the macaddress of specified macaddress.I wonder if it wouldn't be overkill to have separate utility libraries on the kvm test instead of a single
kvm_utilsandkvm_test_utilslike you are proposing. Any thoughts Michael?
Yes, looks like this could be in kvm_test_utils.py, especially if there's only a small number of functions here.
Quoted 37 lines
Quoted 21 lines
Signed-off-by Jason Wang <[email protected]>Signed-off-by Amos Kong <[email protected]>---
0 files changed, 0 insertions(+), 0 deletions(-)client/tests/kvm/kvm_net_utils.py new file +8
-0@@ -0,0 +1,18 @@ +import re + +def get_linux_ifname(session, mac_address): + """ + Get the interface name through the mac address. + + @param session: session to the virtual machine + @mac_address: the macaddress of nicclient/tests/kvm/kvm_net_utils.py line 18
Quoted 10 lines
client/tests/kvm/kvm_net_utils.py new file +10
-0+ """ + + output = session.get_command_output("ifconfig -a") + + try: + ethname = re.findall("(\w+)\s+Link.*%s" % mac_address, output, + re.IGNORECASE)[0] + return ethname + except: + return None--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Qemu-devel] [RFC PATCH 02/14] KVM Test: Add a function get_interface_name() to kvm_net_utils.py
Quoted 11 lines · Michael Goldish
Quoted 7 lines · Lucas Meneghel Rodrigues
Quoted 2 lines · Amos Kong
The function
get_interface_nameis used to get the interface name of linux guest through the macaddress of specified macaddress.I wonder if it wouldn't be overkill to have separate utility libraries on the kvm test instead of a single
kvm_utilsandkvm_test_utilslike you are proposing. Any thoughts Michael?Yes, looks like this could be in
kvm_test_utils.py, especially if there's only a small number of functions here.
Current functions of network are not too much, but we may add a lot of new utility functions in the future. It's more clear to use another file.
Quoted 38 lines
Quoted 37 lines
Quoted 21 lines
Signed-off-by Jason Wang <[email protected]>Signed-off-by Amos Kong <[email protected]>---
0 files changed, 0 insertions(+), 0 deletions(-)client/tests/kvm/kvm_net_utils.py new file +8
-0@@ -0,0 +1,18 @@ +import re + +def get_linux_ifname(session, mac_address): + """ + Get the interface name through the mac address. + + @param session: session to the virtual machine + @mac_address: the macaddress of nicclient/tests/kvm/kvm_net_utils.py line 18
Quoted 10 lines
client/tests/kvm/kvm_net_utils.py new file +10
-0+ """ + + output = session.get_command_output("ifconfig -a") + + try: + ethname = re.findall("(\w+)\s+Link.*%s" % mac_address, output, + re.IGNORECASE)[0] + return ethname + except: + return None--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
RFC PATCH 3/14 KVM Test: Add a common ping module for network related tests
In reply to Amos KongThe kvm_net_utils.py is a just a place that wraps common network related commands which is used to do the network-related tests. Use -1 as the packet ratio for loss analysis.
Use quiet mode when doing the flood ping.
---
0 files changed, 0 insertions(+), 0 deletions(-)
client/tests/kvm/kvm_net_utils.py +111 -1
@@ -1,4 +1,114 @@ -import re +import logging, re, signal +from autotest_lib.client.common_lib import error +import kvm_subprocess, kvm_utils + +def get_loss_ratio(output): + """ + Get the packet loss ratio from the output of ping + + @param output + """ + try: + return int(re.findall('(\d+)% packet loss', output)[0]) + except IndexError: + logging.debug(output) + return -1 + +def raw_ping(command, timeout, session, output_func): + """ + Low-level ping command execution. + + @param command: ping command + @param timeout: timeout of the ping command + @param session: local executon hint or session to execute the ping command + """ + if session == "localhost": + process = kvm_subprocess.run_bg(command, output_func=output_func, + timeout=timeout) + + # Send SIGINT singal to notify the timeout of running ping process, + # Because ping have the ability to catch the SIGINT signal so we can + # always get the packet loss ratio even if timeout. + if process.is_alive(): + kvm_utils.kill_process_tree(process.get_pid(), signal.SIGINT) + + status = process.get_status() + output = process.get_output() + + process.close() + return status, output + else: + session.sendline(command) + status, output = session.read_up_to_prompt(timeout=timeout, + print_func=output_func) + if status is False: + # Send ctrl+c (SIGINT) through ssh session + session.sendline("\003") + status, output2 = session.read_up_to_prompt(print_func=output_func) + output += output2 + if status is False: + # We also need to use this session to query the return value + session.sendline("\003") + + session.sendline(session.status_test_command) + s2, o2 = session.read_up_to_prompt() + if s2 is False: + status = -1 + else: + try: + status = int(re.findall("\d+", o2)[0]) + except: + status = -1 + + return status, output + +def ping(dest = "localhost", count = None, interval = None, interface = None, + packetsize = None, ttl = None, hint = None, adaptive = False, + broadcast = False, flood = False, timeout = 0, + output_func = logging.debug, session = "localhost"): + """ + Wrapper of ping. + + @param dest: destination address + @param count: count of icmp packet + @param interval: interval of two icmp echo request + @param interface: specified interface of the source address + @param packetsize: packet size of icmp + @param ttl: ip time to live + @param hint: path mtu discovery hint + @param adaptive: adaptive ping flag + @param broadcast: broadcast ping flag + @param flood: flood ping flag + @param timeout: timeout for the ping command + @param output_func: function used to log the result of ping + @param session: local executon hint or session to execute the ping command + """ + + command = "ping %s " % dest + + if count is not None: + command += " -c %s" % count + if interval is not None: + command += " -i %s" % interval + if interface is not None: + command += " -I %s" % interface + if packetsize is not None: + command += " -s %s" % packetsize + if ttl is not None: + command += " -t %s" % ttl + if hint is not None: + command += " -M %s" % hint + if adaptive is True: + command += " -A" + if broadcast is True: + command += " -b" + if flood is True: + # temporary workaround as the kvm_subprocess may not properly handle + # the timeout for the output of flood ping + command += " -f -q" + output_func = None + + return raw_ping(command, timeout, session, output_func) def get_linux_ifname(session, mac_address): """
Re: [Qemu-devel] [RFC PATCH 03/14] KVM Test: Add a common ping module for network related tests
Quoted 33 lines · Amos Kong
The
kvm_net_utils.pyis a just a place that wraps common network related commands which is used to do the network-related tests. Use -1 as the packet ratio for loss analysis.
Use quiet mode when doing the flood ping.Signed-off-by Jason Wang <[email protected]>Signed-off-by Amos Kong <[email protected]>---
0 files changed, 0 insertions(+), 0 deletions(-)client/tests/kvm/kvm_net_utils.py +16
-1@@ -1,4 +1,114 @@ -import re +import logging, re, signal +from autotest_lib.client.common_lib import error +import kvm_subprocess, kvm_utils + +def get_loss_ratio(output): + """ + Get the packet loss ratio from the output of ping + + @param output + """ + try: + return int(re.findall('(\d+)% packet loss', output)[0]) + except IndexError: + logging.debug(output) + return -1 +def raw_ping(command, timeout, session, output_func):
client/tests/kvm/kvm_net_utils.py line 27
Quoted 10 lines
client/tests/kvm/kvm_net_utils.py +10
-0+ """ + Low-level ping command execution. + + @param command: ping command + @param timeout: timeout of the ping command + @param session: local executon hint or session to execute the ping command + """ + if session == "localhost": + process = kvm_subprocess.run_bg(command, output_func=output_func, + timeout=timeout)
Do we really have to resort to kvm_subprocess here? We use subprocess on special occasions, usually when the command in question is required to live throughout the tests, which doesn't seem to be the case. kvm_subprocess is a good library, but it is a more heavyweight alternative (spawns its own shell process, for example). Maybe you are using it because you can directly send input to the process at any time, such as the ctrl+c later on this same code.
client/tests/kvm/kvm_net_utils.py line 29
Quoted 2 lines
client/tests/kvm/kvm_net_utils.py +2
-0+ + # Send SIGINT singal to notify the timeout of running ping process,
^ typo, signal
Quoted 5 lines
client/tests/kvm/kvm_net_utils.py +5
-0+ # Because ping have the ability to catch the SIGINT signal so we can + # always get the packet loss ratio even if timeout. + if process.is_alive(): + kvm_utils.kill_process_tree(process.get_pid(), signal.SIGINT) +
client/tests/kvm/kvm_net_utils.py line 44
Quoted 10 lines
client/tests/kvm/kvm_net_utils.py +10
-0+ status = process.get_status() + output = process.get_output() + + process.close() + return status, output + else: + session.sendline(command) + status, output = session.read_up_to_prompt(timeout=timeout, + print_func=output_func) + if status is False:
^ For None objects, it is better to explicitly test for is None. However the above construct seems more natural if put as
if not status:
Any particular reason you tested explicitely for False?
client/tests/kvm/kvm_net_utils.py line 55
Quoted 11 lines
client/tests/kvm/kvm_net_utils.py +11
-0+ # Send ctrl+c (SIGINT) through ssh session + session.sendline("\003") + status, output2 = session.read_up_to_prompt(print_func=output_func) + output += output2 + if status is False: + # We also need to use this session to query the return value + session.sendline("\003") + + session.sendline(session.status_test_command) + s2, o2 = session.read_up_to_prompt() + if s2 is False:
^ See comment above
client/tests/kvm/kvm_net_utils.py line 68
Quoted 13 lines
client/tests/kvm/kvm_net_utils.py +13
-0+ status = -1 + else: + try: + status = int(re.findall("\d+", o2)[0]) + except: + status = -1 + + return status, output + +def ping(dest = "localhost", count = None, interval = None, interface = None, + packetsize = None, ttl = None, hint = None, adaptive = False, + broadcast = False, flood = False, timeout = 0, + output_func = logging.debug, session = "localhost"):
^ On function headers, we pretty much follow PEP 8 and don't use spacing between argument keys, the equal sign and the default value, so this header should be rewritten
def ping(dest="localhost",...)
Quoted 31 lines
client/tests/kvm/kvm_net_utils.py +31
-0+ """ + Wrapper of ping. + + @param dest: destination address + @param count: count of icmp packet + @param interval: interval of two icmp echo request + @param interface: specified interface of the source address + @param packetsize: packet size of icmp + @param ttl: ip time to live + @param hint: path mtu discovery hint + @param adaptive: adaptive ping flag + @param broadcast: broadcast ping flag + @param flood: flood ping flag + @param timeout: timeout for the ping command + @param output_func: function used to log the result of ping + @param session: local executon hint or session to execute the ping command + """ + + command = "ping %s " % dest + + if count is not None: + command += " -c %s" % count + if interval is not None: + command += " -i %s" % interval + if interface is not None: + command += " -I %s" % interface + if packetsize is not None: + command += " -s %s" % packetsize + if ttl is not None: + command += " -t %s" % ttl + if hint is not None:
client/tests/kvm/kvm_net_utils.py line 109
Quoted 10 lines
client/tests/kvm/kvm_net_utils.py +10
-0+ command += " -M %s" % hint + if adaptive is True: + command += " -A" + if broadcast is True: + command += " -b" + if flood is True: + # temporary workaround as the kvm_subprocess may not properly handle + # the timeout for the output of flood ping + command += " -f -q" + output_func = None
^ the last 3 if clauses could be simpler if put as:
if adaptive:
if broadcast:
if flood:
Quoted 6 lines
client/tests/kvm/kvm_net_utils.py +1
-0+ return raw_ping(command, timeout, session, output_func) def get_linux_ifname(session, mac_address): """
Re: [Qemu-devel] [RFC PATCH 03/14] KVM Test: Add a common ping module for network related tests
Quoted 33 lines · Lucas Meneghel Rodrigues
Quoted 31 lines · Amos Kong
The
kvm_net_utils.pyis a just a place that wraps common network related commands which is used to do the network-related tests. Use -1 as the packet ratio for loss analysis.
Use quiet mode when doing the flood ping.Signed-off-by Jason Wang <[email protected]>Signed-off-by Amos Kong <[email protected]>---
0 files changed, 0 insertions(+), 0 deletions(-)client/tests/kvm/kvm_net_utils.py +15
-1@@ -1,4 +1,114 @@ -import re +import logging, re, signal +from autotest_lib.client.common_lib import error +import kvm_subprocess, kvm_utils + +def get_loss_ratio(output): + """ + Get the packet loss ratio from the output of ping + + @param output + """ + try: + return int(re.findall('(\d+)% packet loss', output)[0]) + except IndexError: + logging.debug(output) + return -1
Quoted 9 lines
client/tests/kvm/kvm_net_utils.py line 25
Quoted 9 lines
client/tests/kvm/kvm_net_utils.py +9
-0+def raw_ping(command, timeout, session, output_func): + """ + Low-level ping command execution. + + @param command: ping command + @param timeout: timeout of the ping command + @param session: local executon hint or session to execute the ping command + """ + if session == "localhost":
I have no problem with this, but wouldn't it be nicer to use None to indicate that the session should be local?
Quoted 35 lines
client/tests/kvm/kvm_net_utils.py line 27
Quoted 2 lines
client/tests/kvm/kvm_net_utils.py +2
-0+ process = kvm_subprocess.run_bg(command, output_func=output_func, + timeout=timeout)Do we really have to resort to
kvm_subprocesshere? We use subprocess on special occasions, usually when the command in question is required to live throughout the tests, which doesn't seem to be the case.kvm_subprocessis a good library, but it is a more heavyweight alternative (spawns its own shell process, for example). Maybe you are using it because you can directly send input to the process at any time, such as the ctrl+c later on this same code.client/tests/kvm/kvm_net_utils.py line 28
Quoted 2 lines
client/tests/kvm/kvm_net_utils.py +2
-0+ + # Send SIGINT singal to notify the timeout of running ping process,^ typo, signal
Quoted 5 lines
client/tests/kvm/kvm_net_utils.py +5
-0+ # Because ping have the ability to catch the SIGINT signal so we can + # always get the packet loss ratio even if timeout. + if process.is_alive(): + kvm_utils.kill_process_tree(process.get_pid(), signal.SIGINT) +client/tests/kvm/kvm_net_utils.py line 44
Quoted 10 lines
client/tests/kvm/kvm_net_utils.py +10
-0+ status = process.get_status() + output = process.get_output() + + process.close() + return status, output + else: + session.sendline(command) + status, output = session.read_up_to_prompt(timeout=timeout, + print_func=output_func) + if status is False:^ For None objects, it is better to explicitly test for is None. However the above construct seems more natural if put as
Quoted 3 lines
if not status:
Any particular reason you tested explicitely for False?
read_up_to_prompt() returns True/False as the first member of the tuple.
Quoted 2 lines
client/tests/kvm/kvm_net_utils.py line 45
Quoted 2 lines
client/tests/kvm/kvm_net_utils.py +2
-0+ # Send ctrl+c (SIGINT) through ssh session + session.sendline("\003")
I think you can use send() here. sendline() calls send() with a newline added to the string.
Quoted 5 lines
client/tests/kvm/kvm_net_utils.py line 51
Quoted 5 lines
client/tests/kvm/kvm_net_utils.py +5
-0+ status, output2 = session.read_up_to_prompt(print_func=output_func) + output += output2 + if status is False: + # We also need to use this session to query the return value + session.sendline("\003")
Same here.
Quoted 25 lines
client/tests/kvm/kvm_net_utils.py line 55
Quoted 4 lines
client/tests/kvm/kvm_net_utils.py +4
-0+ + session.sendline(session.status_test_command) + s2, o2 = session.read_up_to_prompt() + if s2 is False:^ See comment above
client/tests/kvm/kvm_net_utils.py line 68
Quoted 13 lines
client/tests/kvm/kvm_net_utils.py +13
-0+ status = -1 + else: + try: + status = int(re.findall("\d+", o2)[0]) + except: + status = -1 + + return status, output + +def ping(dest = "localhost", count = None, interval = None, interface = None, + packetsize = None, ttl = None, hint = None, adaptive = False, + broadcast = False, flood = False, timeout = 0, + output_func = logging.debug, session = "localhost"):^ On function headers, we pretty much follow PEP 8 and don't use spacing between argument keys, the equal sign and the default value, so this header should be rewritten
Quoted 18 lines
def
ping(dest="localhost",...)Quoted 16 lines
client/tests/kvm/kvm_net_utils.py +16
-0+ """ + Wrapper of ping. + + @param dest: destination address + @param count: count of icmp packet + @param interval: interval of two icmp echo request + @param interface: specified interface of the source address + @param packetsize: packet size of icmp + @param ttl: ip time to live + @param hint: path mtu discovery hint + @param adaptive: adaptive ping flag + @param broadcast: broadcast ping flag + @param flood: flood ping flag + @param timeout: timeout for the ping command + @param output_func: function used to log the result of ping + @param session: local executon hint or session to execute the ping command
Don't we need this for Windows as well? If we do, wouldn't it be easier to use a parameter (e.g. 'ping_options = -c 3') to directly control the ping options from the config file, instead of using an OS-specific ping wrapper?
Quoted 15 lines
Quoted 15 lines
client/tests/kvm/kvm_net_utils.py +15
-0+ """ + + command = "ping %s " % dest + + if count is not None: + command += " -c %s" % count + if interval is not None: + command += " -i %s" % interval + if interface is not None: + command += " -I %s" % interface + if packetsize is not None: + command += " -s %s" % packetsize + if ttl is not None: + command += " -t %s" % ttl + if hint is not None:
Quoted 8 lines
client/tests/kvm/kvm_net_utils.py line 107
Quoted 8 lines
client/tests/kvm/kvm_net_utils.py +8
-0+ command += " -M %s" % hint + if adaptive is True: + command += " -A" + if broadcast is True: + command += " -b" + if flood is True: + # temporary workaround as the kvm_subprocess may not properly handle + # the timeout for the output of flood ping
What do you mean by this? If there's a problem with kvm_subprocess it should be fixed. Have you tried passing internal_timeout=0 to the kvm_subprocess function you're using?
Quoted 1 line
client/tests/kvm/kvm_net_utils.py line 108
Quoted 1 line
client/tests/kvm/kvm_net_utils.py +1
-0+ command += " -f -q"
If a lot of output is generated, it may not be a bad idea to use -q here regardless of kvm_subprocess limitations.
Quoted 14 lines
client/tests/kvm/kvm_net_utils.py line 109
Quoted 1 line
client/tests/kvm/kvm_net_utils.py +1
-0+ output_func = None^ the last 3 if clauses could be simpler if put as:
if adaptive:
if broadcast:
if flood:
Quoted 4 lines
client/tests/kvm/kvm_net_utils.py +1
-0+ return raw_ping(command, timeout, session, output_func) def get_linux_ifname(session, mac_address): """
Re: [Qemu-devel] [RFC PATCH 03/14] KVM Test: Add a common ping module for network related tests
Quoted 146 lines · Michael Goldish
Quoted 33 lines · Lucas Meneghel Rodrigues
Quoted 31 lines · Amos Kong
The
kvm_net_utils.pyis a just a place that wraps common network related commands which is used to do the network-related tests. Use -1 as the packet ratio for loss analysis.
Use quiet mode when doing the flood ping.Signed-off-by Jason Wang <[email protected]>Signed-off-by Amos Kong <[email protected]>---
0 files changed, 0 insertions(+), 0 deletions(-)client/tests/kvm/kvm_net_utils.py +15
-1@@ -1,4 +1,114 @@ -import re +import logging, re, signal +from autotest_lib.client.common_lib import error +import kvm_subprocess, kvm_utils + +def get_loss_ratio(output): + """ + Get the packet loss ratio from the output of ping + + @param output + """ + try: + return int(re.findall('(\d+)% packet loss', output)[0]) + except IndexError: + logging.debug(output) + return -1Quoted 9 lines
client/tests/kvm/kvm_net_utils.py line 25
Quoted 9 lines
client/tests/kvm/kvm_net_utils.py +9
-0+def raw_ping(command, timeout, session, output_func): + """ + Low-level ping command execution. + + @param command: ping command + @param timeout: timeout of the ping command + @param session: local executon hint or session to execute the ping command + """ + if session == "localhost":I have no problem with this, but wouldn't it be nicer to use None to indicate that the session should be local?
Quoted 35 lines
client/tests/kvm/kvm_net_utils.py line 27
Quoted 2 lines
client/tests/kvm/kvm_net_utils.py +2
-0+ process = kvm_subprocess.run_bg(command, output_func=output_func, + timeout=timeout)Do we really have to resort to
kvm_subprocesshere? We use subprocess on special occasions, usually when the command in question is required to live throughout the tests, which doesn't seem to be the case.kvm_subprocessis a good library, but it is a more heavyweight alternative (spawns its own shell process, for example). Maybe you are using it because you can directly send input to the process at any time, such as the ctrl+c later on this same code.client/tests/kvm/kvm_net_utils.py line 28
Quoted 2 lines
client/tests/kvm/kvm_net_utils.py +2
-0+ + # Send SIGINT singal to notify the timeout of running ping process,^ typo, signal
Quoted 5 lines
client/tests/kvm/kvm_net_utils.py +5
-0+ # Because ping have the ability to catch the SIGINT signal so we can + # always get the packet loss ratio even if timeout. + if process.is_alive(): + kvm_utils.kill_process_tree(process.get_pid(), signal.SIGINT) +client/tests/kvm/kvm_net_utils.py line 44
Quoted 10 lines
client/tests/kvm/kvm_net_utils.py +10
-0+ status = process.get_status() + output = process.get_output() + + process.close() + return status, output + else: + session.sendline(command) + status, output = session.read_up_to_prompt(timeout=timeout, + print_func=output_func) + if status is False:^ For None objects, it is better to explicitly test for is None. However the above construct seems more natural if put as
Quoted 3 lines
if not status:
Any particular reason you tested explicitely for False?
read_up_to_prompt()returns True/False as the first member of the tuple.Quoted 2 lines
client/tests/kvm/kvm_net_utils.py line 45
Quoted 2 lines
client/tests/kvm/kvm_net_utils.py +2
-0+ # Send ctrl+c (SIGINT) through ssh session + session.sendline("\003")I think you can use
send()here.sendline()callssend()with a newline added to the string.Quoted 5 lines
client/tests/kvm/kvm_net_utils.py line 51
Quoted 5 lines
client/tests/kvm/kvm_net_utils.py +5
-0+ status, output2 = session.read_up_to_prompt(print_func=output_func) + output += output2 + if status is False: + # We also need to use this session to query the return value + session.sendline("\003")Same here.
Quoted 25 lines
client/tests/kvm/kvm_net_utils.py line 55
Quoted 4 lines
client/tests/kvm/kvm_net_utils.py +4
-0+ + session.sendline(session.status_test_command) + s2, o2 = session.read_up_to_prompt() + if s2 is False:^ See comment above
client/tests/kvm/kvm_net_utils.py line 68
Quoted 13 lines
client/tests/kvm/kvm_net_utils.py +13
-0+ status = -1 + else: + try: + status = int(re.findall("\d+", o2)[0]) + except: + status = -1 + + return status, output + +def ping(dest = "localhost", count = None, interval = None, interface = None, + packetsize = None, ttl = None, hint = None, adaptive = False, + broadcast = False, flood = False, timeout = 0, + output_func = logging.debug, session = "localhost"):^ On function headers, we pretty much follow PEP 8 and don't use spacing between argument keys, the equal sign and the default value, so this header should be rewritten
Quoted 18 lines
def
ping(dest="localhost",...)Quoted 16 lines
client/tests/kvm/kvm_net_utils.py +16
-0+ """ + Wrapper of ping. + + @param dest: destination address + @param count: count of icmp packet + @param interval: interval of two icmp echo request + @param interface: specified interface of the source address + @param packetsize: packet size of icmp + @param ttl: ip time to live + @param hint: path mtu discovery hint + @param adaptive: adaptive ping flag + @param broadcast: broadcast ping flag + @param flood: flood ping flag + @param timeout: timeout for the ping command + @param output_func: function used to log the result of ping + @param session: local executon hint or session to execute the ping command
Quoted 4 lines
Don't we need this for Windows as well? If we do, wouldn't it be easier to use a parameter (e.g. '
ping_options= -c 3') to directly control the ping options from the config file, instead of using an OS-specific ping wrapper?
Please disregard this comment. After looking at some of the other patches in the set I realized it wasn't relevant.
Quoted 47 lines
Quoted 15 lines
Quoted 15 lines
client/tests/kvm/kvm_net_utils.py +15
-0+ """ + + command = "ping %s " % dest + + if count is not None: + command += " -c %s" % count + if interval is not None: + command += " -i %s" % interval + if interface is not None: + command += " -I %s" % interface + if packetsize is not None: + command += " -s %s" % packetsize + if ttl is not None: + command += " -t %s" % ttl + if hint is not None:Quoted 8 lines
client/tests/kvm/kvm_net_utils.py line 107
Quoted 8 lines
client/tests/kvm/kvm_net_utils.py +8
-0+ command += " -M %s" % hint + if adaptive is True: + command += " -A" + if broadcast is True: + command += " -b" + if flood is True: + # temporary workaround as the kvm_subprocess may not properly handle + # the timeout for the output of flood pingWhat do you mean by this? If there's a problem with
kvm_subprocessit should be fixed. Have you tried passing internal_timeout=0 to thekvm_subprocessfunction you're using?Quoted 1 line
client/tests/kvm/kvm_net_utils.py line 108
Quoted 1 line
client/tests/kvm/kvm_net_utils.py +1
-0+ command += " -f -q"If a lot of output is generated, it may not be a bad idea to use -q here regardless of
kvm_subprocesslimitations.Quoted 14 lines
client/tests/kvm/kvm_net_utils.py line 109
Quoted 1 line
client/tests/kvm/kvm_net_utils.py +1
-0+ output_func = None^ the last 3 if clauses could be simpler if put as:
if adaptive:
if broadcast:
if flood:
Quoted 4 lines
client/tests/kvm/kvm_net_utils.py +1
-0+ return raw_ping(command, timeout, session, output_func) def get_linux_ifname(session, mac_address): """
RFC PATCH 4/14 KVM-test: Add a new subtest ping
In reply to Amos KongThis test use ping to check the virtual nics, it contains two kinds of test:
1. Packet loss ratio test, ping the guest with different size of packets.
2. Stress test, flood ping guest then use ordinary ping to test the network.
The interval and packet size could be configurated through tests_base.cfg
---
0 files changed, 0 insertions(+), 0 deletions(-)
client/tests/kvm/tests/ping.py new file +71 -0
@@ -0,0 +1,71 @@ +import logging, time, re, commands +from autotest_lib.client.common_lib import error +import kvm_subprocess, kvm_test_utils, kvm_utils, kvm_net_utils + + +def run_ping(test, params, env): + """ + Ping the guest with different size of packets. + + Packet Loss Test: + 1) Ping the guest with different size/interval of packets. + Stress Test: + 1) Flood ping the guest. + 2) Check if the network is still usable. + + @param test: Kvm test object + @param params: Dictionary with the test parameters + @param env: Dictionary with test environment. + """ + + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm")) + session = kvm_test_utils.wait_for_login(vm) + + counts = params.get("ping_counts", 100) + flood_minutes = float(params.get("flood_minutes", 10)) + nics = params.get("nics").split() + strict_check = params.get("strict_check", "no") == "yes" + + packet_size = [0, 1, 4, 48, 512, 1440, 1500, 1505, 4054, 4055, 4096, 4192, + 8878, 9000, 32767, 65507] + + try: + for i, nic in enumerate(nics): + ip = vm.get_address(i) + if not ip: + logging.error("Could not get the ip of nic index %d" % i) + continue + + for size in packet_size: + logging.info("Ping with packet size %s" % size) + status, output = kvm_net_utils.ping(ip, 10, + packetsize = size, + timeout = 20) + if strict_check: + ratio = kvm_net_utils.get_loss_ratio(output) + if ratio != 0: + raise error.TestFail(" Loss ratio is %s for packet size" + " %s" % (ratio, size)) + else: + if status != 0: + raise error.TestFail(" Ping returns non-zero value %s" % + output) + + logging.info("Flood ping test") + kvm_net_utils.ping(ip, None, flood = True, output_func= None, + timeout = flood_minutes * 60) + + logging.info("Final ping test") + status, output = kvm_net_utils.ping(ip, counts, + timeout = float(counts) * 1.5) + if strict_check: + ratio = kvm_net_utils.get_loss_ratio(output) + if ratio != 0: + raise error.TestFail("Packet loss ratio is %s after flood" + % ratio) + else: + if status != 0: + raise error.TestFail(" Ping returns non-zero value %s" % + output) + finally: + session.close()
client/tests/kvm/tests_base.cfg.sample +5 -0
@@ -349,6 +349,11 @@ variants: kill_vm_gracefully_vm2 = no address_index_vm2 = 1 + - ping: install setup unattended_install.cdrom + type = ping + counts = 100 + flood_minutes = 10 + - physical_resources_check: install setup unattended_install.cdrom type = physical_resources_check catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
Re: [RFC PATCH 04/14] KVM-test: Add a new subtest ping
Quoted 59 lines · Amos Kong
This test use ping to check the virtual nics, it contains two kinds of test:
1. Packet loss ratio test, ping the guest with different size of packets.
2. Stress test, flood ping guest then use ordinary ping to test the network.The interval and packet size could be configurated through tests_base.cfg
Signed-off-by Jason Wang <[email protected]>Signed-off-by Amos Kong <[email protected]>---
0 files changed, 0 insertions(+), 0 deletions(-)client/tests/kvm/tests/ping.py new file +42
-0@@ -0,0 +1,71 @@ +import logging, time, re, commands +from autotest_lib.client.common_lib import error +import kvm_subprocess, kvm_test_utils, kvm_utils, kvm_net_utils + + +def run_ping(test, params, env): + """ + Ping the guest with different size of packets. + + Packet Loss Test: + 1) Ping the guest with different size/interval of packets. + Stress Test: + 1) Flood ping the guest. + 2) Check if the network is still usable. + + @param test: Kvm test object + @param params: Dictionary with the test parameters + @param env: Dictionary with test environment. + """ + + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm")) + session = kvm_test_utils.wait_for_login(vm) + + counts = params.get("ping_counts", 100) + flood_minutes = float(params.get("flood_minutes", 10)) + nics = params.get("nics").split() + strict_check = params.get("strict_check", "no") == "yes" + + packet_size = [0, 1, 4, 48, 512, 1440, 1500, 1505, 4054, 4055, 4096, 4192, + 8878, 9000, 32767, 65507] + + try: + for i, nic in enumerate(nics): + ip = vm.get_address(i) + if not ip: + logging.error("Could not get the ip of nic index %d" % i) + continue + + for size in packet_size: + logging.info("Ping with packet size %s" % size) + status, output = kvm_net_utils.ping(ip, 10, + packetsize = size,
client/tests/kvm/tests/ping.py line 52
Quoted 10 lines
client/tests/kvm/tests/ping.py new file +10
-0+ timeout = 20) + if strict_check: + ratio = kvm_net_utils.get_loss_ratio(output) + if ratio != 0: + raise error.TestFail(" Loss ratio is %s for packet size" + " %s" % (ratio, size)) + else: + if status != 0: + raise error.TestFail(" Ping returns non-zero value %s" % + output)
^ "Ping failed, status: %s, output: %s" would be a better exception message.
client/tests/kvm/tests/ping.py line 56
Quoted 4 lines
client/tests/kvm/tests/ping.py new file +4
-0+ + logging.info("Flood ping test") + kvm_net_utils.ping(ip, None, flood = True, output_func= None, + timeout = flood_minutes * 60)
^ Please get rid of the spaces here
client/tests/kvm/tests/ping.py line 60
Quoted 3 lines
client/tests/kvm/tests/ping.py new file +3
-0+ logging.info("Final ping test") + status, output = kvm_net_utils.ping(ip, counts, + timeout = float(counts) * 1.5)
^ What is this last test for again?
Quoted 32 lines
client/tests/kvm/tests/ping.py new file +11
-0+ if strict_check: + ratio = kvm_net_utils.get_loss_ratio(output) + if ratio != 0: + raise error.TestFail("Packet loss ratio is %s after flood" + % ratio) + else: + if status != 0: + raise error.TestFail(" Ping returns non-zero value %s" % + output) + finally: + session.close()client/tests/kvm/tests_base.cfg.sample +5
-1@@ -349,6 +349,11 @@ variants: kill_vm_gracefully_vm2 = no address_index_vm2 = 1 + - ping: install setup unattended_install.cdrom + type = ping + counts = 100 + flood_minutes = 10 + - physical_resources_check: install setup unattended_install.cdrom type = physical_resources_check catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}' --To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCH 04/14] KVM-test: Add a new subtest ping
Quoted 79 lines · Lucas Meneghel Rodrigues
Quoted 59 lines · Amos Kong
This test use ping to check the virtual nics, it contains two kinds of test:
1. Packet loss ratio test, ping the guest with different size of packets.
2. Stress test, flood ping guest then use ordinary ping to test the network.The interval and packet size could be configurated through tests_base.cfg
Signed-off-by Jason Wang <[email protected]>Signed-off-by Amos Kong <[email protected]>---
0 files changed, 0 insertions(+), 0 deletions(-)client/tests/kvm/tests/ping.py new file +42
-0@@ -0,0 +1,71 @@ +import logging, time, re, commands +from autotest_lib.client.common_lib import error +import kvm_subprocess, kvm_test_utils, kvm_utils, kvm_net_utils + + +def run_ping(test, params, env): + """ + Ping the guest with different size of packets. + + Packet Loss Test: + 1) Ping the guest with different size/interval of packets. + Stress Test: + 1) Flood ping the guest. + 2) Check if the network is still usable. + + @param test: Kvm test object + @param params: Dictionary with the test parameters + @param env: Dictionary with test environment. + """ + + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm")) + session = kvm_test_utils.wait_for_login(vm) + + counts = params.get("ping_counts", 100) + flood_minutes = float(params.get("flood_minutes", 10)) + nics = params.get("nics").split() + strict_check = params.get("strict_check", "no") == "yes" + + packet_size = [0, 1, 4, 48, 512, 1440, 1500, 1505, 4054, 4055, 4096, 4192, + 8878, 9000, 32767, 65507] + + try: + for i, nic in enumerate(nics): + ip = vm.get_address(i) + if not ip: + logging.error("Could not get the ip of nic index %d" % i) + continue + + for size in packet_size: + logging.info("Ping with packet size %s" % size) + status, output = kvm_net_utils.ping(ip, 10, + packetsize = size,client/tests/kvm/tests/ping.py line 52
Quoted 10 lines
client/tests/kvm/tests/ping.py new file +10
-0+ timeout = 20) + if strict_check: + ratio = kvm_net_utils.get_loss_ratio(output) + if ratio != 0: + raise error.TestFail(" Loss ratio is %s for packet size" + " %s" % (ratio, size)) + else: + if status != 0: + raise error.TestFail(" Ping returns non-zero value %s" % + output)^ "Ping failed, status: %s, output: %s" would be a better exception message.
client/tests/kvm/tests/ping.py line 56
Quoted 4 lines
client/tests/kvm/tests/ping.py new file +4
-0+ + logging.info("Flood ping test") + kvm_net_utils.ping(ip, None, flood = True, output_func= None, + timeout = flood_minutes * 60)
Quoted 7 lines
^ Please get rid of the spaces here
client/tests/kvm/tests/ping.py line 60
Quoted 3 lines
client/tests/kvm/tests/ping.py new file +3
-0+ logging.info("Final ping test") + status, output = kvm_net_utils.ping(ip, counts, + timeout = float(counts) * 1.5)^ What is this last test for again?
We could not raise an error when flood ping failed(loss packet/ return non-zero), it's too strict. But we must check the ping result before/after floop-ping.
Quoted 35 lines
Quoted 33 lines
client/tests/kvm/tests/ping.py new file +11
-0+ if strict_check: + ratio = kvm_net_utils.get_loss_ratio(output) + if ratio != 0: + raise error.TestFail("Packet loss ratio is %s after flood" + % ratio) + else: + if status != 0: + raise error.TestFail(" Ping returns non-zero value %s" % + output) + finally: + session.close()client/tests/kvm/tests_base.cfg.sample +5
-1@@ -349,6 +349,11 @@ variants: kill_vm_gracefully_vm2 = no address_index_vm2 = 1 + - ping: install setup unattended_install.cdrom + type = ping + counts = 100 + flood_minutes = 10 + - physical_resources_check: install setup unattended_install.cdrom type = physical_resources_check catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}' --To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
RFC PATCH 5/14 KVM-test: Add a subtest jumbo
In reply to Amos KongAccording to different nic model set different MTU for it. And ping from guest to host, to see whether tested size can be received by host.
---
0 files changed, 0 insertions(+), 0 deletions(-)
client/tests/kvm/tests/jumbo.py new file +133 -0
@@ -0,0 +1,133 @@ +import os, re, logging, commands, time, random +from autotest_lib.client.common_lib import error +import kvm_subprocess, kvm_test_utils, kvm_utils, kvm_net_utils + +def run_jumbo(test, params, env): + """ + Test the RX jumbo frame function of vnics: + 1) boot the vm + 2) change the MTU of guest nics and host taps depends on the nic model + 3) add the static arp entry for guest nic + 4) wait for the MTU ok + 5) verify the patch mtu using ping + 6) ping the guest with large frames + 7) increament size ping + 8) flood ping the guest with large frames + 9) verify the path mtu + 10) revocer the mtu + + @param test: kvm test object + @param params: Dictionary with the test parameters + @param env: Dictionary with test environment. + """ + + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm")) + session = kvm_test_utils.wait_for_login(vm) + mtu = params.get("mtu", "1500") + flood_time = params.get("flood_time", "300") + max_icmp_pkt_size = int(mtu) - 28 + + ifname = vm.get_ifname(0) + ip = vm.get_address(0) + if ip is None: + raise error.TestError("Could not get the ip address") + + try: + # Environment preparartion + ethname = kvm_net_utils.get_linux_ifname(session, vm.get_macaddr(0)) + + logging.info("Changing the mtu of guest ...") + guest_mtu_cmd = "ifconfig %s mtu %s" % (ethname , mtu) + s, o = session.get_command_status_output(guest_mtu_cmd) + if s != 0: + logging.error(o) + raise error.TestError("Fail to set the mtu of guest nic: %s" + % ethname) + + logging.info("Chaning the mtu of host tap ...") + host_mtu_cmd = "ifconfig %s mtu %s" % (ifname, mtu) + s, o = commands.getstatusoutput(host_mtu_cmd) + if s != 0: + raise error.TestError("Fail to set the mtu of %s" % ifname) + + logging.info("Add a temporary static arp entry ...") + arp_add_cmd = "arp -s %s %s -i %s" % (ip, vm.get_macaddr(0), ifname) + s, o = commands.getstatusoutput(arp_add_cmd) + if s != 0 : + raise error.TestError("Fail to add temporary arp entry") + + def is_mtu_ok(): + s, o = kvm_net_utils.ping(ip, 1, interface = ifname, + packetsize = max_icmp_pkt_size, + hint = "do", timeout = 2) + if s != 0: + return False + else: + return True + + def verify_mtu(): + logging.info("Verify the path mtu") + s, o = kvm_net_utils.ping(ip, 10, interface = ifname, + packetsize = max_icmp_pkt_size, + hint = "do", timeout = 15) + if s != 0 : + logging.error(o) + raise error.TestFail("Path MTU is not as expected") + if kvm_net_utils.get_loss_ratio(o) != 0: + logging.error(o) + raise error.TestFail("Packet loss ratio during mtu verification" + " is not zero") + + def flood_ping(): + logging.info("Flood with large frames") + kvm_net_utils.ping(ip, interface = ifname, + packetsize = max_icmp_pkt_size, + flood = True, timeout = float(flood_time)) + + def large_frame_ping(count = 100): + logging.info("Large frame ping") + s, o = kvm_net_utils.ping(ip, count, interface = ifname, + packetsize = max_icmp_pkt_size, + timeout = float(count) * 2) + ratio = kvm_net_utils.get_loss_ratio(o) + if ratio != 0: + raise error.TestFail("Loss ratio of large frame ping is %s" \ + % ratio) + + def size_increase_ping(step = random.randrange(90, 110)): + logging.info("Size increase ping") + for size in range(0, max_icmp_pkt_size + 1, step): + logging.info("Ping %s with size %s" % (ip, size)) + s, o = kvm_net_utils.ping(ip, 1, interface = ifname, + packetsize = size, + hint = "do", timeout = 1) + if s != 0: + s, o = kvm_net_utils.ping(ip, 10, interface = ifname, + packetsize = size, + adaptive = True, hint = "do", + timeout = 20) + if kvm_net_utils.get_loss_ratio(o) > 50: + raise error.TestFail("ping loss ratio is greater " + "than 50% for size %s" % size) + + logging.info("Waiting for the MTU to be OK") + # Wait for the MTU is OK + if not kvm_utils.wait_for(is_mtu_ok, 10, 0, 1): + logging.debug(commands.getoutput("ifconfig -a")) + raise error.TestError("10 seconds elapsed while the mtu is not" + " as expected") + + # Functional Test + verify_mtu() + large_frame_ping() + size_increase_ping() + + # Stress test + flood_ping() + verify_mtu() + + finally: + # Environment clean + session.close() + logging.info("Removing the temporary arp entry") + commands.getstatusoutput("arp -d %s -i %s" % (ip, ifname))
client/tests/kvm/tests_base.cfg.sample +10 -1
@@ -354,6 +354,9 @@ variants: counts = 100 flood_minutes = 10 + - jumbo: install setup unattended_install.cdrom + type = jumbo + - physical_resources_check: install setup unattended_install.cdrom type = physical_resources_check catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
@@ -489,10 +492,16 @@ variants: variants: - @rtl8139: nic_model = rtl8139 + jumbo: + mtu = 1500 - e1000: nic_model = e1000 + jumbo: + mtu = 16110 - virtio: nic_model = virtio + jumbo: + mtu = 65520 # Guests
@@ -1024,7 +1033,7 @@ variants: # Windows section - @Windows: - no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) + no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo shutdown_command = shutdown /s /f /t 0 reboot_command = shutdown /r /f /t 0 status_test_command = echo %errorlevel%
Re: [RFC PATCH 05/14] KVM-test: Add a subtest jumbo
Quoted 14 lines · Amos Kong
According to different nic model set different MTU for it. And ping from guest to host, to see whether tested size can be received by host.
Signed-off-by Jason Wang <[email protected]>Signed-off-by Amos Kong <[email protected]>---
0 files changed, 0 insertions(+), 0 deletions(-)client/tests/kvm/tests/jumbo.py new file +0
-0
client/tests/kvm/tests/jumbo.py line 9
Quoted 9 lines
client/tests/kvm/tests/jumbo.py new file +9
-0@@ -0,0 +1,133 @@ +import os, re, logging, commands, time, random +from autotest_lib.client.common_lib import error +import kvm_subprocess, kvm_test_utils, kvm_utils, kvm_net_utils + +def run_jumbo(test, params, env): + """ + Test the RX jumbo frame function of vnics: + 1) boot the vm + 2) change the MTU of guest nics and host taps depends on the nic model
^ I think you meant 'depending on the nic model'
client/tests/kvm/tests/jumbo.py line 14
Quoted 5 lines
client/tests/kvm/tests/jumbo.py new file +5
-0+ 3) add the static arp entry for guest nic + 4) wait for the MTU ok + 5) verify the patch mtu using ping + 6) ping the guest with large frames + 7) increament size ping
^ typo, increment
client/tests/kvm/tests/jumbo.py line 17
Quoted 3 lines
client/tests/kvm/tests/jumbo.py new file +3
-0+ 8) flood ping the guest with large frames + 9) verify the path mtu + 10) revocer the mtu
^ maybe a typo, restore?
Quoted 9 lines
client/tests/kvm/tests/jumbo.py new file +9
-0+ + @param test: kvm test object + @param params: Dictionary with the test parameters + @param env: Dictionary with test environment. + """ + + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm")) + session = kvm_test_utils.wait_for_login(vm) + mtu = params.get("mtu", "1500")
client/tests/kvm/tests/jumbo.py line 36
Quoted 10 lines
client/tests/kvm/tests/jumbo.py new file +10
-0+ flood_time = params.get("flood_time", "300") + max_icmp_pkt_size = int(mtu) - 28 + + ifname = vm.get_ifname(0) + ip = vm.get_address(0) + if ip is None: + raise error.TestError("Could not get the ip address") + + try: + # Environment preparartion
^ typo, preparation
Quoted 16 lines
client/tests/kvm/tests/jumbo.py new file +16
-0+ ethname = kvm_net_utils.get_linux_ifname(session, vm.get_macaddr(0)) + + logging.info("Changing the mtu of guest ...") + guest_mtu_cmd = "ifconfig %s mtu %s" % (ethname , mtu) + s, o = session.get_command_status_output(guest_mtu_cmd) + if s != 0: + logging.error(o) + raise error.TestError("Fail to set the mtu of guest nic: %s" + % ethname) + + logging.info("Chaning the mtu of host tap ...") + host_mtu_cmd = "ifconfig %s mtu %s" % (ifname, mtu) + s, o = commands.getstatusoutput(host_mtu_cmd) + if s != 0: + raise error.TestError("Fail to set the mtu of %s" % ifname) +
client/tests/kvm/tests/jumbo.py line 62
Quoted 10 lines
client/tests/kvm/tests/jumbo.py new file +10
-0+ logging.info("Add a temporary static arp entry ...") + arp_add_cmd = "arp -s %s %s -i %s" % (ip, vm.get_macaddr(0), ifname) + s, o = commands.getstatusoutput(arp_add_cmd) + if s != 0 : + raise error.TestError("Fail to add temporary arp entry") + + def is_mtu_ok(): + s, o = kvm_net_utils.ping(ip, 1, interface = ifname, + packetsize = max_icmp_pkt_size, + hint = "do", timeout = 2)
^ please fix the default argument attribution, here and all subsequent calls
client/tests/kvm/tests/jumbo.py line 66
Quoted 4 lines
client/tests/kvm/tests/jumbo.py new file +4
-0+ if s != 0: + return False + else: + return True
^ Here you could simple use return s != 0
Quoted 19 lines
client/tests/kvm/tests/jumbo.py new file +19
-0+ + def verify_mtu(): + logging.info("Verify the path mtu") + s, o = kvm_net_utils.ping(ip, 10, interface = ifname, + packetsize = max_icmp_pkt_size, + hint = "do", timeout = 15) + if s != 0 : + logging.error(o) + raise error.TestFail("Path MTU is not as expected") + if kvm_net_utils.get_loss_ratio(o) != 0: + logging.error(o) + raise error.TestFail("Packet loss ratio during mtu verification" + " is not zero") + + def flood_ping(): + logging.info("Flood with large frames") + kvm_net_utils.ping(ip, interface = ifname, + packetsize = max_icmp_pkt_size, + flood = True, timeout = float(flood_time))
client/tests/kvm/tests/jumbo.py line 95
Quoted 10 lines
client/tests/kvm/tests/jumbo.py new file +10
-0+ + def large_frame_ping(count = 100): + logging.info("Large frame ping") + s, o = kvm_net_utils.ping(ip, count, interface = ifname, + packetsize = max_icmp_pkt_size, + timeout = float(count) * 2) + ratio = kvm_net_utils.get_loss_ratio(o) + if ratio != 0: + raise error.TestFail("Loss ratio of large frame ping is %s" \ + % ratio)
^ No need to use the backslash to end line here
Quoted 5 lines
client/tests/kvm/tests/jumbo.py new file +5
-0+ def size_increase_ping(step = random.randrange(90, 110)): + logging.info("Size increase ping") + for size in range(0, max_icmp_pkt_size + 1, step): + logging.info("Ping %s with size %s" % (ip, size)) + s, o = kvm_net_utils.ping(ip, 1, interface = ifname,
client/tests/kvm/tests/jumbo.py line 111
Quoted 10 lines
client/tests/kvm/tests/jumbo.py new file +10
-0+ packetsize = size, + hint = "do", timeout = 1) + if s != 0: + s, o = kvm_net_utils.ping(ip, 10, interface = ifname, + packetsize = size, + adaptive = True, hint = "do", + timeout = 20) + if kvm_net_utils.get_loss_ratio(o) > 50: + raise error.TestFail("ping loss ratio is greater " + "than 50% for size %s" % size)
^ How do we stablish 50% as an acceptable packet loss ratio for this test? Seems high to me, but I might be missing something
client/tests/kvm/tests/jumbo.py line 114
Quoted 3 lines
client/tests/kvm/tests/jumbo.py new file +3
-0+ + logging.info("Waiting for the MTU to be OK") + # Wait for the MTU is OK
^ Unneeded comment, as we have an info message stating the same thing
Quoted 64 lines
client/tests/kvm/tests/jumbo.py new file +19
-0+ if not kvm_utils.wait_for(is_mtu_ok, 10, 0, 1): + logging.debug(commands.getoutput("ifconfig -a")) + raise error.TestError("10 seconds elapsed while the mtu is not" + " as expected") + + # Functional Test + verify_mtu() + large_frame_ping() + size_increase_ping() + + # Stress test + flood_ping() + verify_mtu() + + finally: + # Environment clean + session.close() + logging.info("Removing the temporary arp entry") + commands.getstatusoutput("arp -d %s -i %s" % (ip, ifname))client/tests/kvm/tests_base.cfg.sample +10
-2@@ -354,6 +354,9 @@ variants: counts = 100 flood_minutes = 10 + - jumbo: install setup unattended_install.cdrom + type = jumbo + - physical_resources_check: install setup unattended_install.cdrom type = physical_resources_check catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'@@ -489,10 +492,16 @@ variants: variants: - @rtl8139: nic_model = rtl8139 + jumbo: + mtu = 1500 - e1000: nic_model = e1000 + jumbo: + mtu = 16110 - virtio: nic_model = virtio + jumbo: + mtu = 65520 # Guests@@ -1024,7 +1033,7 @@ variants: # Windows section - @Windows: - no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) + no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo shutdown_command = shutdown /s /f /t 0 reboot_command = shutdown /r /f /t 0 status_test_command = echo %errorlevel% --To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCH 05/14] KVM-test: Add a subtest jumbo
Quoted 148 lines · Lucas Meneghel Rodrigues
Quoted 14 lines · Amos Kong
According to different nic model set different MTU for it. And ping from guest to host, to see whether tested size can be received by host.
Signed-off-by Jason Wang <[email protected]>Signed-off-by Amos Kong <[email protected]>---
0 files changed, 0 insertions(+), 0 deletions(-)client/tests/kvm/tests/jumbo.py new file +0
-0client/tests/kvm/tests/jumbo.py line 9
Quoted 9 lines
client/tests/kvm/tests/jumbo.py new file +9
-0@@ -0,0 +1,133 @@ +import os, re, logging, commands, time, random +from autotest_lib.client.common_lib import error +import kvm_subprocess, kvm_test_utils, kvm_utils, kvm_net_utils + +def run_jumbo(test, params, env): + """ + Test the RX jumbo frame function of vnics: + 1) boot the vm + 2) change the MTU of guest nics and host taps depends on the nic model^ I think you meant 'depending on the nic model'
client/tests/kvm/tests/jumbo.py line 14
Quoted 5 lines
client/tests/kvm/tests/jumbo.py new file +5
-0+ 3) add the static arp entry for guest nic + 4) wait for the MTU ok + 5) verify the patch mtu using ping + 6) ping the guest with large frames + 7) increament size ping^ typo, increment
client/tests/kvm/tests/jumbo.py line 17
Quoted 3 lines
client/tests/kvm/tests/jumbo.py new file +3
-0+ 8) flood ping the guest with large frames + 9) verify the path mtu + 10) revocer the mtu^ maybe a typo, restore?
Quoted 9 lines
client/tests/kvm/tests/jumbo.py new file +9
-0+ + @param test: kvm test object + @param params: Dictionary with the test parameters + @param env: Dictionary with test environment. + """ + + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm")) + session = kvm_test_utils.wait_for_login(vm) + mtu = params.get("mtu", "1500")client/tests/kvm/tests/jumbo.py line 36
Quoted 10 lines
client/tests/kvm/tests/jumbo.py new file +10
-0+ flood_time = params.get("flood_time", "300") + max_icmp_pkt_size = int(mtu) - 28 + + ifname = vm.get_ifname(0) + ip = vm.get_address(0) + if ip is None: + raise error.TestError("Could not get the ip address") + + try: + # Environment preparartion^ typo, preparation
Quoted 16 lines
client/tests/kvm/tests/jumbo.py new file +16
-0+ ethname = kvm_net_utils.get_linux_ifname(session, vm.get_macaddr(0)) + + logging.info("Changing the mtu of guest ...") + guest_mtu_cmd = "ifconfig %s mtu %s" % (ethname , mtu) + s, o = session.get_command_status_output(guest_mtu_cmd) + if s != 0: + logging.error(o) + raise error.TestError("Fail to set the mtu of guest nic: %s" + % ethname) + + logging.info("Chaning the mtu of host tap ...") + host_mtu_cmd = "ifconfig %s mtu %s" % (ifname, mtu) + s, o = commands.getstatusoutput(host_mtu_cmd) + if s != 0: + raise error.TestError("Fail to set the mtu of %s" % ifname) +client/tests/kvm/tests/jumbo.py line 62
Quoted 10 lines
client/tests/kvm/tests/jumbo.py new file +10
-0+ logging.info("Add a temporary static arp entry ...") + arp_add_cmd = "arp -s %s %s -i %s" % (ip, vm.get_macaddr(0), ifname) + s, o = commands.getstatusoutput(arp_add_cmd) + if s != 0 : + raise error.TestError("Fail to add temporary arp entry") + + def is_mtu_ok(): + s, o = kvm_net_utils.ping(ip, 1, interface = ifname, + packetsize = max_icmp_pkt_size, + hint = "do", timeout = 2)^ please fix the default argument attribution, here and all subsequent calls
client/tests/kvm/tests/jumbo.py line 66
Quoted 4 lines
client/tests/kvm/tests/jumbo.py new file +4
-0+ if s != 0: + return False + else: + return True^ Here you could simple use return s != 0
Quoted 19 lines
client/tests/kvm/tests/jumbo.py new file +19
-0+ + def verify_mtu(): + logging.info("Verify the path mtu") + s, o = kvm_net_utils.ping(ip, 10, interface = ifname, + packetsize = max_icmp_pkt_size, + hint = "do", timeout = 15) + if s != 0 : + logging.error(o) + raise error.TestFail("Path MTU is not as expected") + if kvm_net_utils.get_loss_ratio(o) != 0: + logging.error(o) + raise error.TestFail("Packet loss ratio during mtu verification" + " is not zero") + + def flood_ping(): + logging.info("Flood with large frames") + kvm_net_utils.ping(ip, interface = ifname, + packetsize = max_icmp_pkt_size, + flood = True, timeout = float(flood_time))client/tests/kvm/tests/jumbo.py line 95
Quoted 10 lines
client/tests/kvm/tests/jumbo.py new file +10
-0+ + def large_frame_ping(count = 100): + logging.info("Large frame ping") + s, o = kvm_net_utils.ping(ip, count, interface = ifname, + packetsize = max_icmp_pkt_size, + timeout = float(count) * 2) + ratio = kvm_net_utils.get_loss_ratio(o) + if ratio != 0: + raise error.TestFail("Loss ratio of large frame ping is %s" \ + % ratio)^ No need to use the backslash to end line here
Quoted 5 lines
client/tests/kvm/tests/jumbo.py new file +5
-0+ def size_increase_ping(step = random.randrange(90, 110)): + logging.info("Size increase ping") + for size in range(0, max_icmp_pkt_size + 1, step): + logging.info("Ping %s with size %s" % (ip, size)) + s, o = kvm_net_utils.ping(ip, 1, interface = ifname,client/tests/kvm/tests/jumbo.py line 111
Quoted 10 lines
client/tests/kvm/tests/jumbo.py new file +10
-0+ packetsize = size, + hint = "do", timeout = 1) + if s != 0: + s, o = kvm_net_utils.ping(ip, 10, interface = ifname, + packetsize = size, + adaptive = True, hint = "do", + timeout = 20) + if kvm_net_utils.get_loss_ratio(o) > 50: + raise error.TestFail("ping loss ratio is greater " + "than 50% for size %s" % size)
Quoted 2 lines
^ How do we stablish 50% as an acceptable packet loss ratio for this test? Seems high to me, but I might be missing something
It's just a 'experimental' standard, we can also make it can be setup in config file.
(missing some email, reply by personal Email :)
RFC PATCH 6/14 KVM-test: Add basic file transfer test
In reply to Amos KongThis test is the basic test of transfering file between host and guest. Try to transfer a large file from host to guest, and transfer it back to host, then compare the files by diff command.
The default file size is 4000M, scp timeout is 1000s. It means if the average speed is less than 4M/s, this test will be fail.
We can extend this test by using another disk later, then we can transfer larger files without the limit of first disk size.
---
0 files changed, 0 insertions(+), 0 deletions(-)
client/tests/kvm/tests/file_transfer.py new file +54 -0
@@ -0,0 +1,54 @@ +import logging, commands +from autotest_lib.client.common_lib import error +import kvm_utils, kvm_test_utils + +def run_file_transfer(test, params, env): + """ + Test ethrnet device function by ethtool + + 1) Boot up a virtual machine + 2) Create a large file by dd on host + 3) Copy this file from host to guest + 4) Copy this file from guest to host + 5) Check if file transfers good + + @param test: Kvm test object + @param params: Dictionary with the test parameters. + @param env: Dictionary with test environment. + """ + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm")) + timeout=int(params.get("login_timeout", 360)) + logging.info("Trying to log into guest '%s' by serial", vm.name) + session = kvm_utils.wait_for(lambda: vm.serial_login(), + timeout, 0, step=2) + if not session: + raise error.TestFail("Could not log into guest '%s'" % vm.name) + + dir = test.tmpdir + scp_timeout = int(params.get("scp_timeout")) + cmd = "dd if=/dev/urandom of=%s/a.out bs=1M count=%d" % (dir, int( + params.get("filesize", 4000))) + try: + logging.info("Create file by dd command on host, cmd: %s" % cmd) + s, o = commands.getstatusoutput(cmd) + if s != 0: + raise error.TestError("Fail to create file, output:%s" % o) + + logging.info("Transfer file from host to guest") + if not vm.copy_files_to("%s/a.out" % dir, "/tmp/b.out", + timeout=scp_timeout): + raise error.TestFail("Fail to transfer file from host to guest") + + logging.info("Transfer file from guest to host") + if not vm.copy_files_from("/tmp/b.out", "%s/c.out" % dir, + timeout=scp_timeout): + raise error.TestFail("Fail to transfer file from guest to host") + + logging.debug(commands.getoutput("ls -l %s/[ac].out" % dir)) + s, o = commands.getstatusoutput("diff %s/a.out %s/c.out" % (dir, dir)) + if s != 0: + raise error.TestFail("File changed after transfer. Output:%s" % o) + finally: + session.get_command_status("rm -f /tmp/b.out") + commands.getoutput("rm -f %s/[ac].out" % dir) + session.close()
client/tests/kvm/tests_base.cfg.sample +6 -1
@@ -357,6 +357,11 @@ variants: - jumbo: install setup unattended_install.cdrom type = jumbo + - file_transfer: install setup unattended_install.cdrom + type = file_transfer + filesize = 4000 + scp_timeout = 1000 + - physical_resources_check: install setup unattended_install.cdrom type = physical_resources_check catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
@@ -1033,7 +1038,7 @@ variants: # Windows section - @Windows: - no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo + no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer shutdown_command = shutdown /s /f /t 0 reboot_command = shutdown /r /f /t 0 status_test_command = echo %errorlevel%
Re: [Qemu-devel] [RFC PATCH 06/14] KVM-test: Add basic file transfer test
Quoted 5 lines · Amos Kong
This test is the basic test of transfering file between host and guest. Try to transfer a large file from host to guest, and transfer it back to host, then compare the files by diff command.
The default file size is 4000M, scp timeout is 1000s. It means if the average speed is less than 4M/s, this test will be fail.
^ About this average 4MB/s throughput, is there some sort of agreement/standard I'm not aware of?
Quoted 26 lines
We can extend this test by using another disk later, then we can transfer larger files without the limit of first disk size.
Signed-off-by Amos Kong <[email protected]>---
0 files changed, 0 insertions(+), 0 deletions(-)client/tests/kvm/tests/file_transfer.py new file +13
-0@@ -0,0 +1,54 @@ +import logging, commands +from autotest_lib.client.common_lib import error +import kvm_utils, kvm_test_utils + +def run_file_transfer(test, params, env): + """ + Test ethrnet device function by ethtool + + 1) Boot up a virtual machine + 2) Create a large file by dd on host + 3) Copy this file from host to guest + 4) Copy this file from guest to host + 5) Check if file transfers good
client/tests/kvm/tests/file_transfer.py line 23
Quoted 10 lines
client/tests/kvm/tests/file_transfer.py new file +10
-0+ + @param test: Kvm test object + @param params: Dictionary with the test parameters. + @param env: Dictionary with test environment. + """ + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm")) + timeout=int(params.get("login_timeout", 360)) + logging.info("Trying to log into guest '%s' by serial", vm.name) + session = kvm_utils.wait_for(lambda: vm.serial_login(), + timeout, 0, step=2)
^ I assume the serial session is being used here to avoid interfering with the bandwidth required for the test to complete as little as possible, right?
client/tests/kvm/tests/file_transfer.py line 35
Quoted 12 lines
client/tests/kvm/tests/file_transfer.py new file +12
-0+ if not session: + raise error.TestFail("Could not log into guest '%s'" % vm.name) + + dir = test.tmpdir + scp_timeout = int(params.get("scp_timeout")) + cmd = "dd if=/dev/urandom of=%s/a.out bs=1M count=%d" % (dir, int( + params.get("filesize", 4000))) + try: + logging.info("Create file by dd command on host, cmd: %s" % cmd) + s, o = commands.getstatusoutput(cmd) + if s != 0: + raise error.TestError("Fail to create file, output:%s" % o)
^ I've seen this throughout the patchseries, the use of functions present on commands rather than utils.system, or utils.run. The former are preferable on autotest tests, that's important to have in mind. Would you consider changing those calls to the autotest API functions?
Quoted 4 lines
client/tests/kvm/tests/file_transfer.py new file +4
-0+ logging.info("Transfer file from host to guest") + if not vm.copy_files_to("%s/a.out" % dir, "/tmp/b.out", + timeout=scp_timeout): + raise error.TestFail("Fail to transfer file from host to guest")
client/tests/kvm/tests/file_transfer.py line 50
Quoted 10 lines
client/tests/kvm/tests/file_transfer.py new file +10
-0+ + logging.info("Transfer file from guest to host") + if not vm.copy_files_from("/tmp/b.out", "%s/c.out" % dir, + timeout=scp_timeout): + raise error.TestFail("Fail to transfer file from guest to host") + + logging.debug(commands.getoutput("ls -l %s/[ac].out" % dir)) + s, o = commands.getstatusoutput("diff %s/a.out %s/c.out" % (dir, dir)) + if s != 0: + raise error.TestFail("File changed after transfer. Output:%s" % o)
^ It seems faster to use md5 or sha1 to verify the integrity of the files rather than diff. Unless there's a good reason to use diff, I'd like you to change it to one of the programs mentioned above.
Quoted 31 lines
client/tests/kvm/tests/file_transfer.py new file +4
-0+ finally: + session.get_command_status("rm -f /tmp/b.out") + commands.getoutput("rm -f %s/[ac].out" % dir) + session.close()client/tests/kvm/tests_base.cfg.sample +6
-1@@ -357,6 +357,11 @@ variants: - jumbo: install setup unattended_install.cdrom type = jumbo + - file_transfer: install setup unattended_install.cdrom + type = file_transfer + filesize = 4000 + scp_timeout = 1000 + - physical_resources_check: install setup unattended_install.cdrom type = physical_resources_check catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'@@ -1033,7 +1038,7 @@ variants: # Windows section - @Windows: - no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo + no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer shutdown_command = shutdown /s /f /t 0 reboot_command = shutdown /r /f /t 0 status_test_command = echo %errorlevel%
Re: [Autotest] [Qemu-devel] [RFC PATCH 06/14] KVM-test: Add basic file transfer test
Quoted 9 lines · Lucas Meneghel Rodrigues
Quoted 5 lines · Amos Kong
This test is the basic test of transfering file between host and guest. Try to transfer a large file from host to guest, and transfer it back to host, then compare the files by diff command.
The default file size is 4000M, scp timeout is 1000s. It means if the average speed is less than 4M/s, this test will be fail.^ About this average 4MB/s throughput, is there some sort of agreement/standard I'm not aware of?
Also an 'experimental' standard.
The most important purpose is not check the transfer result, but catch the bug during file transfer.
If case fail of transfer timeout, we need try to augment the timeout, and compare the test result.
Quoted 32 lines
Quoted 32 lines
We can extend this test by using another disk later, then we can transfer larger files without the limit of first disk size.
Signed-off-by Amos Kong <[email protected]>---
0 files changed, 0 insertions(+), 0 deletions(-)client/tests/kvm/tests/file_transfer.py new file +19
-0@@ -0,0 +1,54 @@ +import logging, commands +from autotest_lib.client.common_lib import error +import kvm_utils, kvm_test_utils + +def run_file_transfer(test, params, env): + """ + Test ethrnet device function by ethtool + + 1) Boot up a virtual machine + 2) Create a large file by dd on host + 3) Copy this file from host to guest + 4) Copy this file from guest to host + 5) Check if file transfers good + + @param test: Kvm test object + @param params: Dictionary with the test parameters. + @param env: Dictionary with test environment. + """ + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
Quoted 8 lines
client/tests/kvm/tests/file_transfer.py line 23
Quoted 4 lines
client/tests/kvm/tests/file_transfer.py new file +4
-0+ timeout=int(params.get("login_timeout", 360)) + logging.info("Trying to log into guest '%s' by serial", vm.name) + session = kvm_utils.wait_for(lambda: vm.serial_login(), + timeout, 0, step=2)^ I assume the serial session is being used here to avoid interfering with the bandwidth required for the test to complete as little as possible, right?
Just want to split the control connection and data connection, if network is down, some clean work would not be done.
But there is only one serial console session per VM, when we use serial_login, serial console log would be disabled.
Quoted 13 lines
client/tests/kvm/tests/file_transfer.py line 35
Quoted 12 lines
client/tests/kvm/tests/file_transfer.py new file +12
-0+ if not session: + raise error.TestFail("Could not log into guest '%s'" % vm.name) + + dir = test.tmpdir + scp_timeout = int(params.get("scp_timeout")) + cmd = "dd if=/dev/urandom of=%s/a.out bs=1M count=%d" % (dir, int( + params.get("filesize", 4000))) + try: + logging.info("Create file by dd command on host, cmd: %s" % cmd) + s, o = commands.getstatusoutput(cmd) + if s != 0: + raise error.TestError("Fail to create file, output:%s" % o)
Quoted 4 lines
^ I've seen this throughout the patchseries, the use of functions present on commands rather than utils.system, or utils.run. The former are preferable on autotest tests, that's important to have in mind. Would you consider changing those calls to the autotest API functions?
ok, I will try to use autotest API.
Quoted 10 lines
Quoted 10 lines
client/tests/kvm/tests/file_transfer.py new file +10
-0+ logging.info("Transfer file from host to guest") + if not vm.copy_files_to("%s/a.out" % dir, "/tmp/b.out", + timeout=scp_timeout): + raise error.TestFail("Fail to transfer file from host to guest") + + logging.info("Transfer file from guest to host") + if not vm.copy_files_from("/tmp/b.out", "%s/c.out" % dir, + timeout=scp_timeout): + raise error.TestFail("Fail to transfer file from guest to host") +
Quoted 8 lines
client/tests/kvm/tests/file_transfer.py line 50
Quoted 4 lines
client/tests/kvm/tests/file_transfer.py new file +4
-0+ logging.debug(commands.getoutput("ls -l %s/[ac].out" % dir)) + s, o = commands.getstatusoutput("diff %s/a.out %s/c.out" % (dir, dir)) + if s != 0: + raise error.TestFail("File changed after transfer. Output:%s" % o)^ It seems faster to use md5 or sha1 to verify the integrity of the files rather than diff. Unless there's a good reason to use diff, I'd like you to change it to one of the programs mentioned above.
I would change it to md5.
Quoted 38 lines
Quoted 24 lines
client/tests/kvm/tests/file_transfer.py new file +4
-0+ finally: + session.get_command_status("rm -f /tmp/b.out") + commands.getoutput("rm -f %s/[ac].out" % dir) + session.close()client/tests/kvm/tests_base.cfg.sample +0
-0@@ -357,6 +357,11 @@ variants:- jumbo: install setup unattended_install.cdrom type = jumbo
+ -
file_transfer: install setup unattended_install.cdrom
+ type =file_transfer
+ filesize = 4000
+ scp_timeout = 1000
+
-physical_resources_check: install setup unattended_install.cdrom type =physical_resources_check
catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'client/tests/kvm/tests_base.cfg.sample +0
-0@@ -1033,7 +1038,7 @@ variants: # Windows section- @Windows:
Quoted 7 lines
- no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo + no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfershutdown_command = shutdown /s /f /t 0
reboot_command = shutdown /r /f /t 0
status_test_command = echo %errorlevel%_______________________________________________
Autotest mailing list
[email protected]
http://test.kernel.org/cgi-bin/mailman/listinfo/autotest
RFC PATCH 7/14 KVM-test: Add a subtest of load/unload nic driver
In reply to Amos KongRepeatedly load/unload nic driver, try to transfer file between guest and host by threads at the same time, and check the md5sum.
---
0 files changed, 0 insertions(+), 0 deletions(-)
client/tests/kvm/tests/nicdriver_unload.py new file +128 -0
@@ -0,0 +1,128 @@ +import logging, commands, threading, re, os +from autotest_lib.client.common_lib import error +import kvm_utils, kvm_test_utils, kvm_net_utils + +def run_nicdriver_unload(test, params, env): + """ + Test nic driver + + 1) Boot a vm + 2) Get the nic driver name + 3) Repeatedly unload/load nic driver + 4) Multi-session TCP transfer on test interface + 5) Check the test interface should still work + + @param test: KVM test object + @param params: Dictionary with the test parameters + @param env: Dictionary with test environment. + """ + timeout = int(params.get("login_timeout", 360)) + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm")) + session = kvm_test_utils.wait_for_login(vm, timeout=timeout) + logging.info("Trying to log into guest '%s' by serial", vm.name) + session2 = kvm_utils.wait_for(lambda: vm.serial_login(), + timeout, 0, step=2) + if not session2: + raise error.TestFail("Could not log into guest '%s'" % vm.name) + + ethname = kvm_net_utils.get_linux_ifname(session, vm.get_macaddr(0)) + try: + # FIXME: Try three waies to get nic driver name, because the + # modprobe.conf is dropped in latest system, and ethtool method is not + # supported by virtio_nic. + + output = session.get_command_output("cat /etc/modprobe.conf") + driver = re.findall(r'%s (\w+)' % ethname,output) + if not driver: + output = session.get_command_output("ethtool -i %s" % ethname) + driver = re.findall(r'driver: (\w+)', output) + if not driver: + output = session.get_command_output("lspci -k") + driver = re.findall("Ethernet controller.*\n.*\n.*Kernel driver" + " in use: (\w+)", output) + driver = driver[0] + except IndexError: + raise error.TestError("Could not find driver name") + + logging.info("driver is %s" % driver) + + class ThreadScp(threading.Thread): + def run(self): + remote_file = '/tmp/' + self.getName() + file_list.append(remote_file) + ret = vm.copy_files_to(file_name, remote_file, timeout=scp_timeout) + logging.debug("Copy result of %s: %s" % (remote_file, ret)) + + def compare(origin_file, receive_file): + cmd = "md5sum %s" + output1 = commands.getstatusoutput(cmd % origin_file)[1].strip() + check_sum1 = output1.split()[0] + s, output2 = session.get_command_status_output(cmd % receive_file) + if s != 0: + logging.error("Could not get md5sum of receive_file") + return False + check_sum2 = output2.strip().split()[0] + logging.debug("origin: %s, receive: %s" % (check_sum1, check_sum2)) + if check_sum1 != check_sum2: + logging.error("md5sum doesn't match") + return False + return True + + #produce sized file in host + file_size = params.get("file_size") + file_name = "/tmp/nicdriver_unload_file" + cmd = "dd if=/dev/urandom of=%s bs=%sM count=1" + s, o = commands.getstatusoutput(cmd % (file_name, file_size)) + if s != 0: + raise error.TestFail("Fail to create file by dd") + + connect_time = params.get("connect_time") + scp_timeout = int(params.get("scp_timeout")) + thread_num = int(params.get("thread_num")) + file_list = [] + + unload_load_cmd = "sleep %s && ifconfig %s down && modprobe -r %s && " + unload_load_cmd += "sleep 1 && modprobe %s && ifconfig %s up" + unload_load_cmd = unload_load_cmd % (connect_time, ethname, driver, + driver, ethname) + pid = os.fork() + if pid != 0: + logging.info("unload/load nic driver repeatedly in guest...") + while True: + logging.debug("Try to unload/load nic drive once") + if session2.get_command_status(unload_load_cmd, timeout=120) != 0: + session.get_command_output("rm -rf /tmp/Thread-*") + raise error.TestFail("Unload/load nic driver failed") + pid, s = os.waitpid(pid, os.WNOHANG) + status = os.WEXITSTATUS(s) + if (pid, status) != (0, 0): + logging.debug("Child process ending") + break + else: + logging.info("Multi-session tcp data transfer") + threads = [] + for i in range(thread_num): + t = ThreadScp() + t.start() + threads.append(t) + for t in threads: + t.join(timeout = scp_timeout) + os._exit(0) + + session2.close() + + try: + logging.info("Check md5sum for received files in multi-session") + for f in file_list: + if not compare(file_name, f): + raise error.TestFail("Fail to compare (guest)file %s" % f) + + logging.info("Test nic function after load/unload") + if not vm.copy_files_to(file_name, file_name): + raise error.TestFail("Fail to copy file from host to guest") + if not compare(file_name, file_name): + raise error.TestFail("Test nic function after load/unload fail") + + finally: + session.get_command_output("rm -rf /tmp/Thread-*") + session.close()
client/tests/kvm/tests_base.cfg.sample +9 -1
@@ -362,6 +362,14 @@ variants: filesize = 4000 scp_timeout = 1000 + - nicdriver_unload: install setup unattended_install.cdrom + type = nicdriver_unload + nic_mode = tap + file_size = 100 + connect_time = 4 + scp_timeout = 300 + thread_num = 10 + - physical_resources_check: install setup unattended_install.cdrom type = physical_resources_check catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
@@ -1038,7 +1046,7 @@ variants: # Windows section - @Windows: - no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer + no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload shutdown_command = shutdown /s /f /t 0 reboot_command = shutdown /r /f /t 0 status_test_command = echo %errorlevel%
Re: [Qemu-devel] [RFC PATCH 07/14] KVM-test: Add a subtest of load/unload nic driver
Quoted 48 lines · Amos Kong
Repeatedly load/unload nic driver, try to transfer file between guest and host by threads at the same time, and check the md5sum.
Signed-off-by Amos Kong <[email protected]>---
0 files changed, 0 insertions(+), 0 deletions(-)client/tests/kvm/tests/nicdriver_unload.py new file +35
-0@@ -0,0 +1,128 @@ +import logging, commands, threading, re, os +from autotest_lib.client.common_lib import error +import kvm_utils, kvm_test_utils, kvm_net_utils + +def run_nicdriver_unload(test, params, env): + """ + Test nic driver + + 1) Boot a vm + 2) Get the nic driver name + 3) Repeatedly unload/load nic driver + 4) Multi-session TCP transfer on test interface + 5) Check the test interface should still work + + @param test: KVM test object + @param params: Dictionary with the test parameters + @param env: Dictionary with test environment. + """ + timeout = int(params.get("login_timeout", 360)) + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm")) + session = kvm_test_utils.wait_for_login(vm, timeout=timeout) + logging.info("Trying to log into guest '%s' by serial", vm.name) + session2 = kvm_utils.wait_for(lambda: vm.serial_login(), + timeout, 0, step=2) + if not session2: + raise error.TestFail("Could not log into guest '%s'" % vm.name) + + ethname = kvm_net_utils.get_linux_ifname(session, vm.get_macaddr(0)) + try: + # FIXME: Try three waies to get nic driver name, because the + # modprobe.conf is dropped in latest system, and ethtool method is not + # supported by virtio_nic. + + output = session.get_command_output("cat /etc/modprobe.conf") + driver = re.findall(r'%s (\w+)' % ethname,output)
client/tests/kvm/tests/nicdriver_unload.py line 45
Quoted 10 lines
client/tests/kvm/tests/nicdriver_unload.py new file +10
-0+ if not driver: + output = session.get_command_output("ethtool -i %s" % ethname) + driver = re.findall(r'driver: (\w+)', output) + if not driver: + output = session.get_command_output("lspci -k") + driver = re.findall("Ethernet controller.*\n.*\n.*Kernel driver" + " in use: (\w+)", output) + driver = driver[0] + except IndexError: + raise error.TestError("Could not find driver name")
^ About this whole block where there's an attempt to discover what the driver name is. The methods listed there are not reliable (depends on ethtool being installed, lspci -k not allways will list the kernel driver in use, /etc/modprobe.conf will not be present on more recent distributions). Instead of these methods, why don't we have a variant for this test on each linux distro definition block, with the appropriate driver names? It'd be something along the lines:
- 13.64:
image_name = f13-64
cdrom_cd1 = linux/Fedora-13-x86_64-DVD.iso
md5sum = 6fbae6379cf27f36e1f2c7827ba7dc35
md5sum_1m = 68821b9de4d3b5975d6634334e7f47a6
unattended_install:
unattended_file = unattended/Fedora-13.ks
tftp = images/f13-64/tftpboot
floppy = images/f13-64/floppy.img
nicdriver_unload:
e1000:
driver = e1000c
virtio:
driver = virtio_net
rtl8139:
driver = rtl8939I believe it's safer than having to rely on the above methods.
client/tests/kvm/tests/nicdriver_unload.py line 54
Quoted 9 lines
client/tests/kvm/tests/nicdriver_unload.py new file +9
-0+ + logging.info("driver is %s" % driver) + + class ThreadScp(threading.Thread): + def run(self): + remote_file = '/tmp/' + self.getName() + file_list.append(remote_file) + ret = vm.copy_files_to(file_name, remote_file, timeout=scp_timeout) + logging.debug("Copy result of %s: %s" % (remote_file, ret))
^ Here it'd be worth to have 2 debug messages,
if ret:
logging.debug("File %s was transfered successfuly", remote_file)
else:
logging.debug("Failed to transfer file %s", remote_file)
client/tests/kvm/tests/nicdriver_unload.py line 58
Quoted 3 lines
client/tests/kvm/tests/nicdriver_unload.py new file +3
-0+ def compare(origin_file, receive_file): + cmd = "md5sum %s" + output1 = commands.getstatusoutput(cmd % origin_file)[1].strip()
^ If we are only interested on the output, getoutput() could be used. But in this case we care whether md5 succeeded or not, so better to do appropriate return code treatment, as you do below. Also, you could use utils.hash() instead of using directly md5, and that *must* yield that exact same result, being faster than resorting to subprocesses.
Also, remembering that I don't really love seeing things from the commands API instead of utils.system, utils.run and others.
client/tests/kvm/tests/nicdriver_unload.py line 65
Quoted 7 lines
client/tests/kvm/tests/nicdriver_unload.py new file +7
-0+ check_sum1 = output1.split()[0] + s, output2 = session.get_command_status_output(cmd % receive_file) + if s != 0: + logging.error("Could not get md5sum of receive_file") + return False + check_sum2 = output2.strip().split()[0] + logging.debug("origin: %s, receive: %s" % (check_sum1, check_sum2))
^ "original file md5: %s, received file md5: %s"
client/tests/kvm/tests/nicdriver_unload.py line 77
Quoted 12 lines
client/tests/kvm/tests/nicdriver_unload.py new file +12
-0+ if check_sum1 != check_sum2: + logging.error("md5sum doesn't match") + return False + return True + + #produce sized file in host + file_size = params.get("file_size") + file_name = "/tmp/nicdriver_unload_file" + cmd = "dd if=/dev/urandom of=%s bs=%sM count=1" + s, o = commands.getstatusoutput(cmd % (file_name, file_size)) + if s != 0: + raise error.TestFail("Fail to create file by dd")
Here utils.system would already raise a CmdError exception, so you could just use it and not do exception handling
client/tests/kvm/tests/nicdriver_unload.py line 87
Quoted 9 lines
client/tests/kvm/tests/nicdriver_unload.py new file +9
-0+ connect_time = params.get("connect_time") + scp_timeout = int(params.get("scp_timeout")) + thread_num = int(params.get("thread_num")) + file_list = [] + + unload_load_cmd = "sleep %s && ifconfig %s down && modprobe -r %s && " + unload_load_cmd += "sleep 1 && modprobe %s && ifconfig %s up" + unload_load_cmd = unload_load_cmd % (connect_time, ethname, driver, + driver, ethname)
^ Here it'd be better to construct the whole command on a single assignment:
unload_load_cmd = ("sleep %s && ifconfig %s down && modprobe -r %s && "
"sleep 1 && modprobe %s && ifconfig %s up" %
(connect_time, ethname, driver, driver, ethname))Quoted 71 lines
client/tests/kvm/tests/nicdriver_unload.py new file +41
-0+ pid = os.fork() + if pid != 0: + logging.info("unload/load nic driver repeatedly in guest...") + while True: + logging.debug("Try to unload/load nic drive once") + if session2.get_command_status(unload_load_cmd, timeout=120) != 0: + session.get_command_output("rm -rf /tmp/Thread-*") + raise error.TestFail("Unload/load nic driver failed") + pid, s = os.waitpid(pid, os.WNOHANG) + status = os.WEXITSTATUS(s) + if (pid, status) != (0, 0): + logging.debug("Child process ending") + break + else: + logging.info("Multi-session tcp data transfer") + threads = [] + for i in range(thread_num): + t = ThreadScp() + t.start() + threads.append(t) + for t in threads: + t.join(timeout = scp_timeout) + os._exit(0) + + session2.close() + + try: + logging.info("Check md5sum for received files in multi-session") + for f in file_list: + if not compare(file_name, f): + raise error.TestFail("Fail to compare (guest)file %s" % f) + + logging.info("Test nic function after load/unload") + if not vm.copy_files_to(file_name, file_name): + raise error.TestFail("Fail to copy file from host to guest") + if not compare(file_name, file_name): + raise error.TestFail("Test nic function after load/unload fail") + + finally: + session.get_command_output("rm -rf /tmp/Thread-*") + session.close()client/tests/kvm/tests_base.cfg.sample +9
-1@@ -362,6 +362,14 @@ variants: filesize = 4000 scp_timeout = 1000 + - nicdriver_unload: install setup unattended_install.cdrom + type = nicdriver_unload + nic_mode = tap + file_size = 100 + connect_time = 4 + scp_timeout = 300 + thread_num = 10 + - physical_resources_check: install setup unattended_install.cdrom type = physical_resources_check catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'@@ -1038,7 +1046,7 @@ variants: # Windows section - @Windows: - no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer + no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload shutdown_command = shutdown /s /f /t 0 reboot_command = shutdown /r /f /t 0 status_test_command = echo %errorlevel%
RFC PATCH 8/14 KVM-test: Add a subtest of nic promisc
In reply to Amos KongThis test mainly covers TCP sent from host to guest and from guest to host with repeatedly turn on/off NIC promiscuous mode.
---
0 files changed, 0 insertions(+), 0 deletions(-)
client/tests/kvm/tests/nic_promisc.py new file +87 -0
@@ -0,0 +1,87 @@ +import logging, commands +from autotest_lib.client.common_lib import error +import kvm_utils, kvm_test_utils, kvm_net_utils + +def run_nic_promisc(test, params, env): + """ + Test nic driver in promisc mode: + + 1) Boot up a guest + 2) Repeatedly enable/disable promiscuous mode in guest + 3) TCP data transmission from host to guest, and from guest to host, + with 1/1460/65000/100000000 bytes payloads + 4) Clean temporary files + 5) Stop enable/disable promiscuous mode change + + @param test: kvm test object + @param params: Dictionary with the test parameters + @param env: Dictionary with test environment + """ + timeout = int(params.get("login_timeout", 360)) + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm")) + session = kvm_test_utils.wait_for_login(vm, timeout=timeout) + logging.info("Trying to log into guest '%s' by serial", vm.name) + session2 = kvm_utils.wait_for(lambda: vm.serial_login(), + timeout, 0, step=2) + if not session2: + raise error.TestFail("Could not log into guest '%s'" % vm.name) + + def compare(filename): + cmd = "md5sum %s" % filename + s1, ret_host = commands.getstatusoutput(cmd) + s2, ret_guest = session.get_command_status_output(cmd) + if s1 != 0 or s2 != 0: + logging.debug("ret_host:%s, ret_guest:%s" % (ret_host, ret_guest)) + logging.error("Could not get md5, cmd:%s" % cmd) + return False + if ret_host.strip() != ret_guest.strip(): + logging.debug("ret_host :%s, ret_guest:%s" % (ret_host, ret_guest)) + logging.error("Files' md5sum mismatch" % (receiver)) + return False + return True + + ethname = kvm_net_utils.get_linux_ifname(session, vm.get_macaddr(0)) + set_promisc_cmd = "ip link set %s promisc on; sleep 0.01;" % ethname + set_promisc_cmd += "ip link set %s promisc off; sleep 0.01" % ethname + logging.info("Set promisc change repeatedly in guest") + session2.sendline("while true; do %s; done" % set_promisc_cmd) + + dd_cmd = "dd if=/dev/urandom of=%s bs=%d count=1" + filename = "/tmp/nic_promisc_file" + file_size = params.get("file_size", "1, 1460, 65000, 100000000").split(",") + try: + for size in file_size: + logging.info("Create %s bytes file on host" % size) + s, o = commands.getstatusoutput(dd_cmd % (filename, int(size))) + if s != 0: + logging.debug("Output: %s"% o) + raise error.TestFail("Create file on host failed") + + logging.info("Transfer file from host to guest") + if not vm.copy_files_to(filename, filename): + raise error.TestFail("File transfer failed") + if not compare(filename): + raise error.TestFail("Compare file failed") + + logging.info("Create %s bytes file on guest" % size) + if session.get_command_status(dd_cmd % (filename, int(size)), + timeout=100) != 0: + raise error.TestFail("Create file on guest failed") + + logging.info("Transfer file from guest to host") + if not vm.copy_files_from(filename, filename): + raise error.TestFail("File transfer failed") + if not compare(filename): + raise error.TestFail("Compare file failed") + + logging.info("Clean temporal files") + cmd = "rm -f %s" % filename + s1, o = commands.getstatusoutput(cmd) + s2 = session.get_command_status(cmd) + if s1 != 0 or s2 != 0: + raise error.TestError("Fail to clean temporal files") + finally: + logging.info("Restore the %s to the nonpromisc mode" % ethname) + session2.close() + session.get_command_status("ip link set %s promisc off" % ethname) + session.close()
client/tests/kvm/tests_base.cfg.sample +5 -1
@@ -370,6 +370,10 @@ variants: scp_timeout = 300 thread_num = 10 + - nic_promisc: install setup unattended_install.cdrom + type = nic_promisc + file_size = 1, 1460, 65000, 100000000 + - physical_resources_check: install setup unattended_install.cdrom type = physical_resources_check catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
@@ -1046,7 +1050,7 @@ variants: # Windows section - @Windows: - no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload + no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc shutdown_command = shutdown /s /f /t 0 reboot_command = shutdown /r /f /t 0 status_test_command = echo %errorlevel%
Re: [RFC PATCH 08/14] KVM-test: Add a subtest of nic promisc
Quoted 43 lines · Amos Kong
This test mainly covers TCP sent from host to guest and from guest to host with repeatedly turn on/off NIC promiscuous mode.
Signed-off-by Amos Kong <[email protected]>---
0 files changed, 0 insertions(+), 0 deletions(-)client/tests/kvm/tests/nic_promisc.py new file +30
-0@@ -0,0 +1,87 @@ +import logging, commands +from autotest_lib.client.common_lib import error +import kvm_utils, kvm_test_utils, kvm_net_utils + +def run_nic_promisc(test, params, env): + """ + Test nic driver in promisc mode: + + 1) Boot up a guest + 2) Repeatedly enable/disable promiscuous mode in guest + 3) TCP data transmission from host to guest, and from guest to host, + with 1/1460/65000/100000000 bytes payloads + 4) Clean temporary files + 5) Stop enable/disable promiscuous mode change + + @param test: kvm test object + @param params: Dictionary with the test parameters + @param env: Dictionary with test environment + """ + timeout = int(params.get("login_timeout", 360)) + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm")) + session = kvm_test_utils.wait_for_login(vm, timeout=timeout) + logging.info("Trying to log into guest '%s' by serial", vm.name) + session2 = kvm_utils.wait_for(lambda: vm.serial_login(), + timeout, 0, step=2) + if not session2: + raise error.TestFail("Could not log into guest '%s'" % vm.name) + + def compare(filename): + cmd = "md5sum %s" % filename
client/tests/kvm/tests/nic_promisc.py line 40
Quoted 10 lines
client/tests/kvm/tests/nic_promisc.py new file +10
-0+ s1, ret_host = commands.getstatusoutput(cmd) + s2, ret_guest = session.get_command_status_output(cmd) + if s1 != 0 or s2 != 0: + logging.debug("ret_host:%s, ret_guest:%s" % (ret_host, ret_guest)) + logging.error("Could not get md5, cmd:%s" % cmd) + return False + if ret_host.strip() != ret_guest.strip(): + logging.debug("ret_host :%s, ret_guest:%s" % (ret_host, ret_guest)) + logging.error("Files' md5sum mismatch" % (receiver)) + return False
^ The above debug messages will be confusing when looked by someone who is not familiar with the test code, so we should make their lives easier:
def compare(filename):
cmd = "md5sum %s" % filename
rc_host, md5_host = commands.getstatusoutput(cmd)
rc_guest, md5_guest = session.get_command_status_output(cmd)
if rc_host:
logging.debug('Could not get MD5 hash for file %s on host, output: %s', filename, md5_host) return False
if rc_guest:
logging.debug('Could not get MD5 hash for file %s on guest, output: %s', filename, md5_guest) return False
md5host = md5host.strip()
md5guest = md5guest.strip()
if md5host != md5guest:
logging.error('MD5 hash mismatch between file %s present on guest and on host', filename) logging.error('MD5 hash for file on guest: %s, MD5 hash for file on host: %s', md5_host, md5_guest) return False
return True
client/tests/kvm/tests/nic_promisc.py line 45
Quoted 5 lines
client/tests/kvm/tests/nic_promisc.py new file +5
-0+ return True + + ethname = kvm_net_utils.get_linux_ifname(session, vm.get_macaddr(0)) + set_promisc_cmd = "ip link set %s promisc on; sleep 0.01;" % ethname + set_promisc_cmd += "ip link set %s promisc off; sleep 0.01" % ethname
^ You could do the above on a single attribution, see comment on patch 7 of the patchseries.
Quoted 9 lines
client/tests/kvm/tests/nic_promisc.py new file +9
-0+ logging.info("Set promisc change repeatedly in guest") + session2.sendline("while true; do %s; done" % set_promisc_cmd) + + dd_cmd = "dd if=/dev/urandom of=%s bs=%d count=1" + filename = "/tmp/nic_promisc_file" + file_size = params.get("file_size", "1, 1460, 65000, 100000000").split(",") + try: + for size in file_size: + logging.info("Create %s bytes file on host" % size)
client/tests/kvm/tests/nic_promisc.py line 64
Quoted 10 lines
client/tests/kvm/tests/nic_promisc.py new file +10
-0+ s, o = commands.getstatusoutput(dd_cmd % (filename, int(size))) + if s != 0: + logging.debug("Output: %s"% o) + raise error.TestFail("Create file on host failed") + + logging.info("Transfer file from host to guest") + if not vm.copy_files_to(filename, filename): + raise error.TestFail("File transfer failed") + if not compare(filename): + raise error.TestFail("Compare file failed")
^ It'd be better if we don't abruptly fail the whole test if we get a failure for a single size, what about having a global failure counter, and increment it if we have failures, making sure we log errors appropriately?
client/tests/kvm/tests/nic_promisc.py line 75
Quoted 10 lines
client/tests/kvm/tests/nic_promisc.py new file +10
-0+ logging.info("Create %s bytes file on guest" % size) + if session.get_command_status(dd_cmd % (filename, int(size)), + timeout=100) != 0: + raise error.TestFail("Create file on guest failed") + + logging.info("Transfer file from guest to host") + if not vm.copy_files_from(filename, filename): + raise error.TestFail("File transfer failed") + if not compare(filename): + raise error.TestFail("Compare file failed")
^ Same comment as above.
client/tests/kvm/tests/nic_promisc.py line 77
Quoted 1 line
client/tests/kvm/tests/nic_promisc.py new file +1
-0+ logging.info("Clean temporal files")
^ Typo, temporary
client/tests/kvm/tests/nic_promisc.py line 82
Quoted 5 lines
client/tests/kvm/tests/nic_promisc.py new file +5
-0+ cmd = "rm -f %s" % filename + s1, o = commands.getstatusoutput(cmd) + s2 = session.get_command_status(cmd) + if s1 != 0 or s2 != 0: + raise error.TestError("Fail to clean temporal files")
^ Same as above, with the exception that maybe a failure during cleanup is not something so bad we'd need to fail the entire test because of it.
Quoted 34 lines
client/tests/kvm/tests/nic_promisc.py new file +5
-0+ finally: + logging.info("Restore the %s to the nonpromisc mode" % ethname) + session2.close() + session.get_command_status("ip link set %s promisc off" % ethname) + session.close()client/tests/kvm/tests_base.cfg.sample +5
-2@@ -370,6 +370,10 @@ variants: scp_timeout = 300 thread_num = 10 + - nic_promisc: install setup unattended_install.cdrom + type = nic_promisc + file_size = 1, 1460, 65000, 100000000 + - physical_resources_check: install setup unattended_install.cdrom type = physical_resources_check catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'@@ -1046,7 +1050,7 @@ variants: # Windows section - @Windows: - no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload + no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc shutdown_command = shutdown /s /f /t 0 reboot_command = shutdown /r /f /t 0 status_test_command = echo %errorlevel% --To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Autotest] [RFC PATCH 08/14] KVM-test: Add a subtest of nic promisc
Quoted 103 lines · Lucas Meneghel Rodrigues
Quoted 43 lines · Amos Kong
This test mainly covers TCP sent from host to guest and from guest to host with repeatedly turn on/off NIC promiscuous mode.
Signed-off-by Amos Kong <[email protected]>---
0 files changed, 0 insertions(+), 0 deletions(-)client/tests/kvm/tests/nic_promisc.py new file +30
-0@@ -0,0 +1,87 @@ +import logging, commands +from autotest_lib.client.common_lib import error +import kvm_utils, kvm_test_utils, kvm_net_utils + +def run_nic_promisc(test, params, env): + """ + Test nic driver in promisc mode: + + 1) Boot up a guest + 2) Repeatedly enable/disable promiscuous mode in guest + 3) TCP data transmission from host to guest, and from guest to host, + with 1/1460/65000/100000000 bytes payloads + 4) Clean temporary files + 5) Stop enable/disable promiscuous mode change + + @param test: kvm test object + @param params: Dictionary with the test parameters + @param env: Dictionary with test environment + """ + timeout = int(params.get("login_timeout", 360)) + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm")) + session = kvm_test_utils.wait_for_login(vm, timeout=timeout) + logging.info("Trying to log into guest '%s' by serial", vm.name) + session2 = kvm_utils.wait_for(lambda: vm.serial_login(), + timeout, 0, step=2) + if not session2: + raise error.TestFail("Could not log into guest '%s'" % vm.name) + + def compare(filename): + cmd = "md5sum %s" % filenameclient/tests/kvm/tests/nic_promisc.py line 40
Quoted 10 lines
client/tests/kvm/tests/nic_promisc.py new file +10
-0+ s1, ret_host = commands.getstatusoutput(cmd) + s2, ret_guest = session.get_command_status_output(cmd) + if s1 != 0 or s2 != 0: + logging.debug("ret_host:%s, ret_guest:%s" % (ret_host, ret_guest)) + logging.error("Could not get md5, cmd:%s" % cmd) + return False + if ret_host.strip() != ret_guest.strip(): + logging.debug("ret_host :%s, ret_guest:%s" % (ret_host, ret_guest)) + logging.error("Files' md5sum mismatch" % (receiver)) + return False^ The above debug messages will be confusing when looked by someone who is not familiar with the test code, so we should make their lives easier:
def compare(filename):
cmd = "md5sum %s" % filename
rc_host,md5_host=commands.getstatusoutput(cmd)
rc_guest,md5_guest=session.get_command_status_output(cmd)
ifrc_host:
logging.debug('Could not get MD5 hash for file %s on host, output: %s', filename,md5_host) return False
ifrc_guest:
logging.debug('Could not get MD5 hash for file %s on guest, output: %s', filename,md5_guest) return False
md5host =md5host.strip()
md5guest =md5guest.strip()
if md5host != md5guest:
logging.error('MD5 hash mismatch between file %s present on guest and on host', filename)logging.error('MD5 hash for file on guest: %s, MD5 hash for file on host: %s', md5_host, md5_guest)return False
return Trueclient/tests/kvm/tests/nic_promisc.py line 45
Quoted 5 lines
client/tests/kvm/tests/nic_promisc.py new file +5
-0+ return True + + ethname = kvm_net_utils.get_linux_ifname(session, vm.get_macaddr(0)) + set_promisc_cmd = "ip link set %s promisc on; sleep 0.01;" % ethname + set_promisc_cmd += "ip link set %s promisc off; sleep 0.01" % ethname^ You could do the above on a single attribution, see comment on patch 7 of the patchseries.
Quoted 16 lines
client/tests/kvm/tests/nic_promisc.py new file +16
-0+ logging.info("Set promisc change repeatedly in guest") + session2.sendline("while true; do %s; done" % set_promisc_cmd) + + dd_cmd = "dd if=/dev/urandom of=%s bs=%d count=1" + filename = "/tmp/nic_promisc_file" + file_size = params.get("file_size", "1, 1460, 65000, 100000000").split(",") + try: + for size in file_size: + logging.info("Create %s bytes file on host" % size) + s, o = commands.getstatusoutput(dd_cmd % (filename, int(size))) + if s != 0: + logging.debug("Output: %s"% o) + raise error.TestFail("Create file on host failed") + + logging.info("Transfer file from host to guest") + if not vm.copy_files_to(filename, filename):
Quoted 8 lines
client/tests/kvm/tests/nic_promisc.py line 64
Quoted 3 lines
client/tests/kvm/tests/nic_promisc.py new file +3
-0+ raise error.TestFail("File transfer failed") + if not compare(filename): + raise error.TestFail("Compare file failed")^ It'd be better if we don't abruptly fail the whole test if we get a failure for a single size, what about having a global failure counter, and increment it if we have failures, making sure we log errors appropriately?
How about add a success counter ?
only success_counter+1 after successfully compare.
If the end success_counter doesn't equals to len(file_size), raise and fail exception.
Quoted 67 lines
client/tests/kvm/tests/nic_promisc.py line 75
Quoted 10 lines
client/tests/kvm/tests/nic_promisc.py new file +10
-0+ logging.info("Create %s bytes file on guest" % size) + if session.get_command_status(dd_cmd % (filename, int(size)), + timeout=100) != 0: + raise error.TestFail("Create file on guest failed") + + logging.info("Transfer file from guest to host") + if not vm.copy_files_from(filename, filename): + raise error.TestFail("File transfer failed") + if not compare(filename): + raise error.TestFail("Compare file failed")^ Same comment as above.
client/tests/kvm/tests/nic_promisc.py line 77
Quoted 1 line
client/tests/kvm/tests/nic_promisc.py new file +1
-0+ logging.info("Clean temporal files")^ Typo, temporary
client/tests/kvm/tests/nic_promisc.py line 82
Quoted 5 lines
client/tests/kvm/tests/nic_promisc.py new file +5
-0+ cmd = "rm -f %s" % filename + s1, o = commands.getstatusoutput(cmd) + s2 = session.get_command_status(cmd) + if s1 != 0 or s2 != 0: + raise error.TestError("Fail to clean temporal files")^ Same as above, with the exception that maybe a failure during cleanup is not something so bad we'd need to fail the entire test because of it.
Quoted 26 lines
client/tests/kvm/tests/nic_promisc.py new file +5
-0+ finally: + logging.info("Restore the %s to the nonpromisc mode" % ethname) + session2.close() + session.get_command_status("ip link set %s promisc off" % ethname) + session.close()client/tests/kvm/tests_base.cfg.sample +0
-0@@ -370,6 +370,10 @@ variants:scp_timeout = 300
thread_num = 10+ -
nic_promisc: install setup unattended_install.cdrom
+ type =nic_promisc
+ file_size = 1, 1460, 65000, 100000000
+
-physical_resources_check: install setup unattended_install.cdrom type =physical_resources_check
catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'client/tests/kvm/tests_base.cfg.sample +0
-0@@ -1046,7 +1050,7 @@ variants: # Windows section- @Windows:
- no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload + no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promiscQuoted 8 lines
shutdown_command = shutdown /s /f /t 0
reboot_command = shutdown /r /f /t 0
status_test_command = echo %errorlevel%--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html_______________________________________________
Autotest mailing list
[email protected]
http://test.kernel.org/cgi-bin/mailman/listinfo/autotest
RFC PATCH 9/14 KVM-test: Add a subtest of multicast
In reply to Amos KongUse 'ping' to test send/recive multicat packets. Flood ping test is also added. Limit guest network as 'bridge' mode, because multicast packets could not be transmitted to guest when using 'user' network.
Add join_mcast.py for joining machine into multicast groups.
---
0 files changed, 0 insertions(+), 0 deletions(-)
client/tests/kvm/scripts/join_mcast.py new file +29 -0
@@ -0,0 +1,29 @@ +import socket, struct, os, signal, sys +# this script is used to join machine into multicast groups +# author: Amos Kong <[email protected]> + +if len(sys.argv) < 4: + print """%s [mgroup_count] [prefix] [suffix] + mgroup_count: count of multicast addresses + prefix: multicast address prefix + suffix: multicast address suffix""" % sys.argv[0] + sys.exit() + +mgroup_count = int(sys.argv[1]) +prefix = sys.argv[2] +suffix = int(sys.argv[3]) + +s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) +for i in range(mgroup_count): + mcast = prefix + "." + str(suffix + i) + try: + mreq = struct.pack("4sl", socket.inet_aton(mcast), socket.INADDR_ANY) + s.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, mreq) + except: + s.close() + print "Could not join multicast: %s" % mcast + raise + +print "join_mcast_pid:%s" % os.getpid() +os.kill(os.getpid(), signal.SIGSTOP) +s.close()
client/tests/kvm/tests/multicast.py new file +67 -0
67 lines not on this page · raw message
client/tests/kvm/tests_base.cfg.sample +8 -1
@@ -374,6 +374,13 @@ variants: type = nic_promisc file_size = 1, 1460, 65000, 100000000 + - multicast: install setup unattended_install.cdrom + type = multicast + nic_mode = tap + mcast = 225.0.0.1 + mgroup_count = 20 + flood_minutes = 1 + - physical_resources_check: install setup unattended_install.cdrom type = physical_resources_check catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
@@ -1050,7 +1057,7 @@ variants: # Windows section - @Windows: - no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc + no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast shutdown_command = shutdown /s /f /t 0 reboot_command = shutdown /r /f /t 0 status_test_command = echo %errorlevel%
Re: [Qemu-devel] [RFC PATCH 09/14] KVM-test: Add a subtest of multicast
Quoted 65 lines · Amos Kong
Use 'ping' to test send/recive multicat packets. Flood ping test is also added. Limit guest network as 'bridge' mode, because multicast packets could not be transmitted to guest when using 'user' network.
Addjoin_mcast.pyfor joining machine into multicast groups.Signed-off-by Amos Kong <[email protected]>---
0 files changed, 0 insertions(+), 0 deletions(-)client/tests/kvm/scripts/join_mcast.py new file +29
-0@@ -0,0 +1,29 @@ +import socket, struct, os, signal, sys +# this script is used to join machine into multicast groups +# author: Amos Kong <[email protected]> + +if len(sys.argv) < 4: + print """%s [mgroup_count] [prefix] [suffix] + mgroup_count: count of multicast addresses + prefix: multicast address prefix + suffix: multicast address suffix""" % sys.argv[0] + sys.exit() + +mgroup_count = int(sys.argv[1]) +prefix = sys.argv[2] +suffix = int(sys.argv[3]) + +s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) +for i in range(mgroup_count): + mcast = prefix + "." + str(suffix + i) + try: + mreq = struct.pack("4sl", socket.inet_aton(mcast), socket.INADDR_ANY) + s.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, mreq) + except: + s.close() + print "Could not join multicast: %s" % mcast + raise + +print "join_mcast_pid:%s" % os.getpid() +os.kill(os.getpid(), signal.SIGSTOP) +s.close()client/tests/kvm/tests/multicast.py new file +15
-0@@ -0,0 +1,67 @@ +import logging, commands, os, re +from autotest_lib.client.common_lib import error +import kvm_test_utils, kvm_net_utils + + +def run_multicast(test, params, env): + """ + Test multicast function of nic (rtl8139/e1000/virtio) + + 1) Create a VM + 2) Join guest into multicast groups + 3) Ping multicast addresses on host + 4) Flood ping test with different size of packets + 5) Final ping test and check if lose packet +
client/tests/kvm/tests/multicast.py line 25
Quoted 10 lines
client/tests/kvm/tests/multicast.py new file +10
-0+ @param test: Kvm test object + @param params: Dictionary with the test parameters. + @param env: Dictionary with test environment. + """ + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm")) + session = kvm_test_utils.wait_for_login(vm, + timeout=int(params.get("login_timeout", 360))) + + # stop iptable/selinux on guest/host + cmd = "/etc/init.d/iptables stop && echo 0 > /selinux/enforce"
^ Different linux distros handle init scripts differently, so it's better to just flush the firewall rules with iptables -F instead of resorting to the init script.
Also, not all distros use selinux, so it'd be better to test for the presence of selinux on the system (for example, by checking if there's a /selinux/enforce file before trying to write to it). Also, need to do proper return code checking/treatment.
Quoted 7 lines
client/tests/kvm/tests/multicast.py new file +7
-0+ session.get_command_status(cmd) + commands.getoutput(cmd) + # make sure guest replies to broadcasts + session.get_command_status("echo 0 > /proc/sys/net/ipv4/icmp_echo_ignore" + "_broadcasts && echo 0 > /proc/sys/net/ipv4/icmp_echo_ignore_all") + + # base multicast address
client/tests/kvm/tests/multicast.py line 42
Quoted 10 lines
client/tests/kvm/tests/multicast.py new file +10
-0+ mcast = params.get("mcast", "225.0.0.1") + # count of multicast addresses, less than 20 + mgroup_count = int(params.get("mgroup_count", 5)) + flood_minutes = float(params.get("flood_minutes", 10)) + ifname = vm.get_ifname() + prefix = re.findall("\d+.\d+.\d+", mcast)[0] + suffix = int(re.findall("\d+", mcast)[-1]) + # copy python script to guest for joining guest to multicast groups + mcast_path = os.path.join(test.bindir, "scripts/join_mcast.py") + vm.copy_files_to(mcast_path, "/tmp")
^ What if copy_files_to fails? Need to do proper return handling here
Quoted 12 lines
client/tests/kvm/tests/multicast.py new file +12
-0+ output = session.get_command_output("python /tmp/join_mcast.py %d %s %d" % + (mgroup_count, prefix, suffix)) + # if success to join multicast the process will be paused, and return pid. + if not re.findall("join_mcast_pid:(\d+)", output): + raise error.TestFail("Can't join multicast groups,output:%s" % output) + pid = output.split()[0] + + try: + for i in range(mgroup_count): + new_suffix = suffix + i + mcast = "%s.%d" % (prefix, new_suffix) + logging.info("Initial ping test, mcast: %s", mcast)
client/tests/kvm/tests/multicast.py line 64
Quoted 10 lines
client/tests/kvm/tests/multicast.py new file +10
-0+ s, o = kvm_net_utils.ping(mcast, 10, interface=ifname, timeout=20) + if s != 0: + raise error.TestFail(" Ping return non-zero value %s" % o) + logging.info("Flood ping test, mcast: %s", mcast) + kvm_net_utils.ping(mcast, None, interface=ifname, flood=True, + output_func=None, timeout=flood_minutes*60) + logging.info("Final ping test, mcast: %s", mcast) + s, o = kvm_net_utils.ping(mcast, 10, interface=ifname, timeout=20) + if s != 0: + raise error.TestFail(" Ping return non-zero value %s" % o)
^ "Ping failed, status: %s, output: %s" % (s, o) would be a better exception
Quoted 36 lines
client/tests/kvm/tests/multicast.py new file +3
-0+ finally: + session.get_command_output("kill -s SIGCONT %s" % pid) + session.close()client/tests/kvm/tests_base.cfg.sample +8
-2@@ -374,6 +374,13 @@ variants: type = nic_promisc file_size = 1, 1460, 65000, 100000000 + - multicast: install setup unattended_install.cdrom + type = multicast + nic_mode = tap + mcast = 225.0.0.1 + mgroup_count = 20 + flood_minutes = 1 + - physical_resources_check: install setup unattended_install.cdrom type = physical_resources_check catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'@@ -1050,7 +1057,7 @@ variants: # Windows section - @Windows: - no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc + no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast shutdown_command = shutdown /s /f /t 0 reboot_command = shutdown /r /f /t 0 status_test_command = echo %errorlevel% --To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
RFC PATCH 10/14 KVM-test: Add a subtest of pxe
In reply to Amos KongThis case just snoop tftp packet through tcpdump, it depends on public dhcp server, better to test it through dnsmasq.
---
0 files changed, 0 insertions(+), 0 deletions(-)
client/tests/kvm/tests/pxe.py new file +30 -0
@@ -0,0 +1,30 @@ +import logging +from autotest_lib.client.common_lib import error +import kvm_subprocess, kvm_test_utils, kvm_utils + + +def run_pxe(test, params, env): + """ + PXE test: + + 1) Snoop the tftp packet in the tap device + 2) Wait for some seconds + 3) Check whether capture tftp packets + + @param test: kvm test object + @param params: Dictionary with the test parameters + @param env: Dictionary with test environment. + """ + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm")) + timeout = int(params.get("pxe_timeout", 60)) + + logging.info("Try to boot from pxe") + status, output = kvm_subprocess.run_fg("tcpdump -nli %s" % vm.get_ifname(), + logging.debug, + "(pxe) ", + timeout) + + logging.info("Analysing the tcpdump result...") + if not "tftp" in output: + raise error.TestFail("Couldn't find tftp packet in %s seconds" % timeout) + logging.info("Found tftp packet")
client/tests/kvm/tests_base.cfg.sample +13 -0
@@ -381,6 +381,19 @@ variants: mgroup_count = 20 flood_minutes = 1 + - pxe: + type = pxe + images = pxe + image_name_pxe = pxe-test + image_size_pxe = 1G + force_create_image_pxe = yes + remove_image_pxe = yes + extra_params += ' -boot n' + kill_vm_on_error = yes + network = bridge + restart_vm = yes + pxe_timeout = 60 + - physical_resources_check: install setup unattended_install.cdrom type = physical_resources_check catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
Re: [Qemu-devel] [RFC PATCH 10/14] KVM-test: Add a subtest of pxe
Quoted 2 lines · Amos Kong
This case just snoop tftp packet through tcpdump, it depends on public dhcp server, better to test it through dnsmasq.
It would be a good idea to have an alternate implementation using dnsmasq, but not urgent.
Quoted 26 lines
Signed-off-by Jason Wang <[email protected]>Signed-off-by Amos Kong <[email protected]>---
0 files changed, 0 insertions(+), 0 deletions(-)client/tests/kvm/tests/pxe.py new file +15
-0@@ -0,0 +1,30 @@ +import logging +from autotest_lib.client.common_lib import error +import kvm_subprocess, kvm_test_utils, kvm_utils + + +def run_pxe(test, params, env): + """ + PXE test: + + 1) Snoop the tftp packet in the tap device + 2) Wait for some seconds + 3) Check whether capture tftp packets + + @param test: kvm test object + @param params: Dictionary with the test parameters
client/tests/kvm/tests/pxe.py line 25
Quoted 10 lines
client/tests/kvm/tests/pxe.py new file +10
-0+ @param env: Dictionary with test environment. + """ + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm")) + timeout = int(params.get("pxe_timeout", 60)) + + logging.info("Try to boot from pxe") + status, output = kvm_subprocess.run_fg("tcpdump -nli %s" % vm.get_ifname(), + logging.debug, + "(pxe) ", + timeout)
^ The only complaint I could make here is that since this command doesn't need to live throughout tests, utils.run would do just fine. Other than that, looks fine to me.
client/tests/kvm/tests/pxe.py line 27
Quoted 1 line
client/tests/kvm/tests/pxe.py new file +1
-0+ logging.info("Analysing the tcpdump result...")
^ typo, analyzing
Quoted 29 lines
client/tests/kvm/tests/pxe.py new file +3
-0+ if not "tftp" in output: + raise error.TestFail("Couldn't find tftp packet in %s seconds" % timeout) + logging.info("Found tftp packet")client/tests/kvm/tests_base.cfg.sample +13
-0@@ -381,6 +381,19 @@ variants: mgroup_count = 20 flood_minutes = 1 + - pxe: + type = pxe + images = pxe + image_name_pxe = pxe-test + image_size_pxe = 1G + force_create_image_pxe = yes + remove_image_pxe = yes + extra_params += ' -boot n' + kill_vm_on_error = yes + network = bridge + restart_vm = yes + pxe_timeout = 60 + - physical_resources_check: install setup unattended_install.cdrom type = physical_resources_check catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
Re: [Qemu-devel] [RFC PATCH 10/14] KVM-test: Add a subtest of pxe
Quoted 6 lines · Lucas Meneghel Rodrigues
Quoted 2 lines · Amos Kong
This case just snoop tftp packet through tcpdump, it depends on public dhcp server, better to test it through dnsmasq.
It would be a good idea to have an alternate implementation using dnsmasq, but not urgent.
I changed unattended_install to use dnsmasq, but it was blocked by a pxe bug. Also need completed test, we can add this later.
Quoted 32 lines
Quoted 32 lines
Signed-off-by Jason Wang <[email protected]>Signed-off-by Amos Kong <[email protected]>---
0 files changed, 0 insertions(+), 0 deletions(-)client/tests/kvm/tests/pxe.py new file +21
-0@@ -0,0 +1,30 @@ +import logging +from autotest_lib.client.common_lib import error +import kvm_subprocess, kvm_test_utils, kvm_utils + + +def run_pxe(test, params, env): + """ + PXE test: + + 1) Snoop the tftp packet in the tap device + 2) Wait for some seconds + 3) Check whether capture tftp packets + + @param test: kvm test object + @param params: Dictionary with the test parameters + @param env: Dictionary with test environment. + """ + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm")) + timeout = int(params.get("pxe_timeout", 60)) + + logging.info("Try to boot from pxe")
Quoted 8 lines
client/tests/kvm/tests/pxe.py line 25
Quoted 4 lines
client/tests/kvm/tests/pxe.py new file +4
-0+ status, output = kvm_subprocess.run_fg("tcpdump -nli %s" % vm.get_ifname(), + logging.debug, + "(pxe) ", + timeout)^ The only complaint I could make here is that since this command doesn't need to live throughout tests, utils.run would do just fine. Other than that, looks fine to me.
utils.run() desen't support timeout, tcpdump would not stop by itself. also could not add output prefix "(pxe) " ?
Quoted 35 lines
client/tests/kvm/tests/pxe.py line 27
Quoted 1 line
client/tests/kvm/tests/pxe.py new file +1
-0+ logging.info("Analysing the tcpdump result...")^ typo, analyzing
Quoted 29 lines
client/tests/kvm/tests/pxe.py new file +3
-0+ if not "tftp" in output: + raise error.TestFail("Couldn't find tftp packet in %s seconds" % timeout) + logging.info("Found tftp packet")client/tests/kvm/tests_base.cfg.sample +13
-0@@ -381,6 +381,19 @@ variants: mgroup_count = 20 flood_minutes = 1 + - pxe: + type = pxe + images = pxe + image_name_pxe = pxe-test + image_size_pxe = 1G + force_create_image_pxe = yes + remove_image_pxe = yes + extra_params += ' -boot n' + kill_vm_on_error = yes + network = bridge + restart_vm = yes + pxe_timeout = 60 + - physical_resources_check: install setup unattended_install.cdrom type = physical_resources_check catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
RFC PATCH 11/14 KVM-test: Add a subtest of changing mac address
In reply to Amos KongMainly test steps:
1. get a new mac from pool, and the old mac addr of guest.
2. execute the mac_change.sh in guest.
3. relogin to guest and query the interfaces info by ifconfig
---
0 files changed, 0 insertions(+), 0 deletions(-)
client/tests/kvm/tests/mac_change.py new file +66 -0
@@ -0,0 +1,66 @@ +import logging +from autotest_lib.client.common_lib import error +import kvm_utils, kvm_test_utils, kvm_net_utils + + +def run_mac_change(test, params, env): + """ + Change MAC Address of Guest. + + 1. get a new mac from pool, and the old mac addr of guest. + 2. set new mac in guest and regain new IP. + 3. re-log into guest with new mac + + @param test: kvm test object + @param params: Dictionary with the test parameters + @param env: Dictionary with test environment. + """ + timeout = int(params.get("login_timeout", 360)) + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm")) + logging.info("Trying to log into guest '%s' by serial", vm.name) + session = kvm_utils.wait_for(lambda: vm.serial_login(), + timeout, 0, step=2) + if not session: + raise error.TestFail("Could not log into guest '%s'" % vm.name) + + old_mac = vm.get_macaddr(0) + kvm_utils.put_mac_to_pool(vm.root_dir, old_mac, vm.instance) + new_mac = kvm_utils.get_mac_from_pool(vm.root_dir, + vm=vm.instance, + nic_index=0, + prefix=vm.mac_prefix) + logging.info("The initial MAC address is %s" % old_mac) + interface = kvm_net_utils.get_linux_ifname(session, old_mac) + + # Start change mac address + logging.info("Changing mac address to %s" % new_mac) + change_cmd = "ifconfig %s down && ifconfig %s hw ether %s && ifconfig %s up"\ + % (interface, interface, new_mac, interface) + if session.get_command_status(change_cmd) != 0: + raise error.TestFail("Fail to send mac_change command") + + # Verify whether mac address is changed to new one + logging.info("Verifying the new mac address") + if session.get_command_status("ifconfig | grep -i %s" % new_mac) != 0: + raise error.TestFail("Fail to change mac address") + + # Restart `dhclient' to regain IP for new mac address + logging.info("Re-start the network to gain new ip") + dhclient_cmd = "dhclient -r && dhclient %s" % interface + session.sendline(dhclient_cmd) + + # Re-log into the guest after changing mac address + if kvm_utils.wait_for(session.is_responsive, 120, 20, 3): + # Just warning when failed to see the session become dead, + # because there is a little chance the ip does not change. + logging.warn("The session is still responsive, settings may fail.") + session.close() + + # Re-log into guest and check if session is responsive + logging.info("Re-log into the guest") + session = kvm_test_utils.wait_for_login(vm, + timeout=int(params.get("login_timeout", 360))) + if not session.is_responsive(): + raise error.TestFail("The new session is not responsive.") + + session.close()
client/tests/kvm/tests_base.cfg.sample +5 -1
@@ -394,6 +394,10 @@ variants: restart_vm = yes pxe_timeout = 60 + - mac_change: install setup unattended_install.cdrom + type = mac_change + kill_vm = yes + - physical_resources_check: install setup unattended_install.cdrom type = physical_resources_check catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
@@ -1070,7 +1074,7 @@ variants: # Windows section - @Windows: - no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast + no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast mac_change shutdown_command = shutdown /s /f /t 0 reboot_command = shutdown /r /f /t 0 status_test_command = echo %errorlevel%
Re: [Qemu-devel] [RFC PATCH 11/14] KVM-test: Add a subtest of changing mac address
Quoted 28 lines · Amos Kong
Mainly test steps:
1. get a new mac from pool, and the old mac addr of guest.
2. execute themac_change.shin guest.
3. relogin to guest and query the interfaces info byifconfigSigned-off-by Cao, Chen <[email protected]>Signed-off-by Amos Kong <[email protected]>---
0 files changed, 0 insertions(+), 0 deletions(-)client/tests/kvm/tests/mac_change.py new file +12
-0@@ -0,0 +1,66 @@ +import logging +from autotest_lib.client.common_lib import error +import kvm_utils, kvm_test_utils, kvm_net_utils + + +def run_mac_change(test, params, env): + """ + Change MAC Address of Guest. + + 1. get a new mac from pool, and the old mac addr of guest. + 2. set new mac in guest and regain new IP. + 3. re-log into guest with new mac
client/tests/kvm/tests/mac_change.py line 22
Quoted 10 lines
client/tests/kvm/tests/mac_change.py new file +10
-0+ + @param test: kvm test object + @param params: Dictionary with the test parameters + @param env: Dictionary with test environment. + """ + timeout = int(params.get("login_timeout", 360)) + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm")) + logging.info("Trying to log into guest '%s' by serial", vm.name) + session = kvm_utils.wait_for(lambda: vm.serial_login(), + timeout, 0, step=2)
^ One thing that I forgot to comment on previous patches: For more clarity, it'd be good to name the session variable in a way that lets people refer easily that is a serial session
session_serial = ...
Quoted 24 lines
client/tests/kvm/tests/mac_change.py new file +24
-0+ if not session: + raise error.TestFail("Could not log into guest '%s'" % vm.name) + + old_mac = vm.get_macaddr(0) + kvm_utils.put_mac_to_pool(vm.root_dir, old_mac, vm.instance) + new_mac = kvm_utils.get_mac_from_pool(vm.root_dir, + vm=vm.instance, + nic_index=0, + prefix=vm.mac_prefix) + logging.info("The initial MAC address is %s" % old_mac) + interface = kvm_net_utils.get_linux_ifname(session, old_mac) + + # Start change mac address + logging.info("Changing mac address to %s" % new_mac) + change_cmd = "ifconfig %s down && ifconfig %s hw ether %s && ifconfig %s up"\ + % (interface, interface, new_mac, interface) + if session.get_command_status(change_cmd) != 0: + raise error.TestFail("Fail to send mac_change command") + + # Verify whether mac address is changed to new one + logging.info("Verifying the new mac address") + if session.get_command_status("ifconfig | grep -i %s" % new_mac) != 0: + raise error.TestFail("Fail to change mac address") +
client/tests/kvm/tests/mac_change.py line 56
Quoted 10 lines
client/tests/kvm/tests/mac_change.py new file +10
-0+ # Restart `dhclient' to regain IP for new mac address + logging.info("Re-start the network to gain new ip") + dhclient_cmd = "dhclient -r && dhclient %s" % interface + session.sendline(dhclient_cmd) + + # Re-log into the guest after changing mac address + if kvm_utils.wait_for(session.is_responsive, 120, 20, 3): + # Just warning when failed to see the session become dead, + # because there is a little chance the ip does not change. + logging.warn("The session is still responsive, settings may fail.")
^ Isn't this a serial session? Then why the IP of guest changing would make this session un-responsive? I think the best idea here is to:
1) Release the IP through dhclient -r [interface]
2) Make sure we can't stablish a ssh based session to the guest by making a try/except block with kvm_test_utils.wait_for_login() with the appropriate timeouts and other parameters, if succeeds, fail the test, if it doesn't, proceed with the test.
3) Get a new IP with dhclient [interface]
4) Try to stablish a new, ssh based session to the guest and see if that works.
client/tests/kvm/tests/mac_change.py line 64
Quoted 8 lines
client/tests/kvm/tests/mac_change.py new file +8
-0+ session.close() + + # Re-log into guest and check if session is responsive + logging.info("Re-log into the guest") + session = kvm_test_utils.wait_for_login(vm, + timeout=int(params.get("login_timeout", 360))) + if not session.is_responsive(): + raise error.TestFail("The new session is not responsive.")
^ Is it possible that right after you stablish the session it becomes non-responsive? It seems like a redundant verification step to me.
Quoted 31 lines
client/tests/kvm/tests/mac_change.py new file +1
-0+ session.close()client/tests/kvm/tests_base.cfg.sample +5
-2@@ -394,6 +394,10 @@ variants: restart_vm = yes pxe_timeout = 60 + - mac_change: install setup unattended_install.cdrom + type = mac_change + kill_vm = yes + - physical_resources_check: install setup unattended_install.cdrom type = physical_resources_check catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'@@ -1070,7 +1074,7 @@ variants: # Windows section - @Windows: - no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast + no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast mac_change shutdown_command = shutdown /s /f /t 0 reboot_command = shutdown /r /f /t 0 status_test_command = echo %errorlevel% --To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
RFC PATCH 12/14 KVM-test: Add a subtest of netperf
In reply to Amos KongAdd network load by netperf, server is launched on guest, execute netperf client with different protocols on host. if all clients execute successfully, case will be pass. Test result will be record into result.txt. Now this case only tests with "TCP_RR TCP_CRR UDP_RR TCP_STREAM TCP_MAERTS TCP_SENDFILE UDP_STREAM". DLPI only supported by Unix, unix domain test is not necessary, so drop test of DLPI and unix domain.
---
0 files changed, 0 insertions(+), 0 deletions(-)
client/tests/kvm/tests/netperf.py new file +56 -0
@@ -0,0 +1,56 @@ +import logging, commands, os +from autotest_lib.client.common_lib import error +import kvm_subprocess, kvm_test_utils, kvm_utils + +def run_netperf(test, params, env): + """ + Network stress test with netperf + + 1) Boot up a virtual machine + 2) Launch netserver on guest + 3) Execute netperf client on host with different protocols + 4) Outout the test result + + @param test: Kvm test object + @param params: Dictionary with the test parameters. + @param env: Dictionary with test environment. + """ + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm")) + session = kvm_test_utils.wait_for_login(vm, + timeout=int(params.get("login_timeout", 360))) + netperf_dir = os.path.join(os.environ['AUTODIR'], "tests/netperf2") + setup_cmd = params.get("setup_cmd") + guest_ip = vm.get_address() + result_file = os.path.join(test.debugdir, "result.txt") + + session.get_command_output("service iptables stop") + for i in params.get("netperf_files").split(): + if not vm.copy_files_to(os.path.join(netperf_dir, i), "/tmp"): + raise error.TestError("Could not copy files to guest") + if session.get_command_status(setup_cmd % "/tmp", timeout=100) != 0: + raise error.TestFail("Fail to setup netperf on guest") + if session.get_command_status(params.get("netserver_cmd") % "/tmp") != 0: + raise error.TestFail("Fail to start netperf server on guest") + + try: + logging.info("Setup and run netperf client on host") + s, o = commands.getstatusoutput(setup_cmd % netperf_dir) + if s != 0: + raise error.TestFail("Fail to setup netperf on host, o: %s" % o) + success = True + file(result_file, "w").write("Netperf Test Result\n") + for i in params.get("protocols").split(): + cmd = params.get("netperf_cmd") % (netperf_dir, i, guest_ip) + logging.debug("Execute netperf client test: %s" % cmd) + s, o = commands.getstatusoutput(cmd) + if s != 0: + logging.error("Fail to execute netperf test, protocol:%s" % i) + success = False + else: + logging.info(o) + file(result_file, "a+").write("%s\n" % o) + if not success: + raise error.TestFail("Not all the test passed") + finally: + session.get_command_output("killall netserver") + session.close()
client/tests/kvm/tests_base.cfg.sample +10 -0
@@ -398,6 +398,16 @@ variants: type = mac_change kill_vm = yes + - netperf: install setup unattended_install.cdrom + type = netperf + nic_mode = tap + netperf_files = netperf-2.4.5.tar.bz2 wait_before_data.patch + setup_cmd = "cd %s && tar xvfj netperf-2.4.5.tar.bz2 && cd netperf-2.4.5 && patch -p0 < ../wait_before_data.patch && ./configure && make" + netserver_cmd = %s/netperf-2.4.5/src/netserver + # test time is 60 seconds, set the buffer size to 1 for more hardware interrupt + netperf_cmd = %s/netperf-2.4.5/src/netperf -t %s -H %s -l 60 -- -m 1 + protocols = "TCP_STREAM TCP_MAERTS TCP_RR TCP_CRR UDP_RR TCP_SENDFILE UDP_STREAM" + - physical_resources_check: install setup unattended_install.cdrom type = physical_resources_check catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
Re: [Qemu-devel] [RFC PATCH 12/14] KVM-test: Add a subtest of netperf
Quoted 6 lines · Amos Kong
Add network load by netperf, server is launched on guest, execute netperf client with different protocols on host. if all clients execute successfully, case will be pass. Test result will be record into result.txt. Now this case only tests with "
TCP_RRTCP_CRRUDP_RRTCP_STREAMTCP_MAERTSTCP_SENDFILEUDP_STREAM". DLPI only supported by Unix, unix domain test is not necessary, so drop test of DLPI and unix domain.
Some general thoughts about this test:
* netperf2 has a fairly good wrapper code, that can do postprocessing
of the results. It'd be good to make use of it. What I was thinking:
- Import the netperf2 wrapper class on host and call run_once with
appropriate parameters on host
- Create a client control file and run it on guest usingkvm_test_utils.run_autotest()
- As this approach wouldn't work on windows, we could make anetperf_windows test that would be very much like this test.
Of course, we could do that on a later time, I don't think it is something urgent anyway.
Quoted 24 lines
Signed-off-by Amos Kong <[email protected]>---
0 files changed, 0 insertions(+), 0 deletions(-)client/tests/kvm/tests/netperf.py new file +14
-0@@ -0,0 +1,56 @@ +import logging, commands, os +from autotest_lib.client.common_lib import error +import kvm_subprocess, kvm_test_utils, kvm_utils + +def run_netperf(test, params, env): + """ + Network stress test with netperf + + 1) Boot up a virtual machine + 2) Launch netserver on guest + 3) Execute netperf client on host with different protocols + 4) Outout the test result + + @param test: Kvm test object
client/tests/kvm/tests/netperf.py line 24
Quoted 10 lines
client/tests/kvm/tests/netperf.py new file +10
-0+ @param params: Dictionary with the test parameters. + @param env: Dictionary with test environment. + """ + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm")) + session = kvm_test_utils.wait_for_login(vm, + timeout=int(params.get("login_timeout", 360))) + netperf_dir = os.path.join(os.environ['AUTODIR'], "tests/netperf2") + setup_cmd = params.get("setup_cmd") + guest_ip = vm.get_address() + result_file = os.path.join(test.debugdir, "result.txt")
^ test.resultsdir would be better. Also, it's better if we take into account the fact that we can have more than one iteration of the test, so we'd have something like:
self.results_path = os.path.join(self.resultsdir, 'raw_output_%s' %self.iteration)
Which is what we are doing currently for other benchmarks.
client/tests/kvm/tests/netperf.py line 26
Quoted 1 line
client/tests/kvm/tests/netperf.py new file +1
-0+ session.get_command_output("service iptables stop")
^ Again I think it'd be better to simply flush the firewall rules with iptables -F because that's more likely to work on different linux guests.
Also, this linux specific command makes the entire test linux specific, so we have to put a rule telling the kvm autotest to not run it on windows guests.
Quoted 17 lines
client/tests/kvm/tests/netperf.py new file +17
-0+ for i in params.get("netperf_files").split(): + if not vm.copy_files_to(os.path.join(netperf_dir, i), "/tmp"): + raise error.TestError("Could not copy files to guest") + if session.get_command_status(setup_cmd % "/tmp", timeout=100) != 0: + raise error.TestFail("Fail to setup netperf on guest") + if session.get_command_status(params.get("netserver_cmd") % "/tmp") != 0: + raise error.TestFail("Fail to start netperf server on guest") + + try: + logging.info("Setup and run netperf client on host") + s, o = commands.getstatusoutput(setup_cmd % netperf_dir) + if s != 0: + raise error.TestFail("Fail to setup netperf on host, o: %s" % o) + success = True + file(result_file, "w").write("Netperf Test Result\n") + for i in params.get("protocols").split(): + cmd = params.get("netperf_cmd") % (netperf_dir, i, guest_ip)
client/tests/kvm/tests/netperf.py line 53
Quoted 10 lines
client/tests/kvm/tests/netperf.py new file +10
-0+ logging.debug("Execute netperf client test: %s" % cmd) + s, o = commands.getstatusoutput(cmd) + if s != 0: + logging.error("Fail to execute netperf test, protocol:%s" % i) + success = False + else: + logging.info(o) + file(result_file, "a+").write("%s\n" % o) + if not success: + raise error.TestFail("Not all the test passed")
^ "Some of the netperf tests failed"
Quoted 26 lines
client/tests/kvm/tests/netperf.py new file +3
-0+ finally: + session.get_command_output("killall netserver") + session.close()client/tests/kvm/tests_base.cfg.sample +10
-0@@ -398,6 +398,16 @@ variants: type = mac_change kill_vm = yes + - netperf: install setup unattended_install.cdrom + type = netperf + nic_mode = tap + netperf_files = netperf-2.4.5.tar.bz2 wait_before_data.patch + setup_cmd = "cd %s && tar xvfj netperf-2.4.5.tar.bz2 && cd netperf-2.4.5 && patch -p0 < ../wait_before_data.patch && ./configure && make" + netserver_cmd = %s/netperf-2.4.5/src/netserver + # test time is 60 seconds, set the buffer size to 1 for more hardware interrupt + netperf_cmd = %s/netperf-2.4.5/src/netperf -t %s -H %s -l 60 -- -m 1 + protocols = "TCP_STREAM TCP_MAERTS TCP_RR TCP_CRR UDP_RR TCP_SENDFILE UDP_STREAM" + - physical_resources_check: install setup unattended_install.cdrom type = physical_resources_check catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
RFC PATCH 13/14 KVM-test: Improve vlan subtest
In reply to Amos KongThis is an enhancement of existed vlan test. Rename the vlan_tag.py to vlan.py, it is more reasonable.
. Setup arp from "/proc/sys/net/ipv4/conf/all/arp_ignore"
. Multiple vlans exist simultaneously
. Test ping between same and different vlans
. Test by TCP data transfer, floop ping between same vlan
. Maximal plumb/unplumb vlans
---
0 files changed, 0 insertions(+), 0 deletions(-)
client/tests/kvm/tests/vlan.py new file +181 -0
@@ -0,0 +1,181 @@ +import logging, time, re +from autotest_lib.client.common_lib import error +import kvm_subprocess, kvm_test_utils, kvm_utils, kvm_net_utils + +def run_vlan(test, params, env): + """ + Test 802.1Q vlan of NIC, config it by vconfig command. + + 1) Create two VMs + 2) Setup guests in 10 different vlans by vconfig and using hard-coded + ip address + 3) Test by ping between same and different vlans of two VMs + 4) Test by TCP data transfer, floop ping between same vlan of two VMs + 5) Test maximal plumb/unplumb vlans + 6) Recover the vlan config + + @param test: KVM test object. + @param params: Dictionary with the test parameters. + @param env: Dictionary with test environment. + """ + + vm = [] + session = [] + vm_ip = [] + digest_origin = [] + vlan_ip = ['', ''] + ip_unit = ['1', '2'] + subnet = params.get("subnet") + vlan_num = int(params.get("vlan_num")) + maximal = int(params.get("maximal")) + file_size = params.get("file_size") + + vm.append(kvm_test_utils.get_living_vm(env, params.get("main_vm"))) + vm.append(kvm_test_utils.get_living_vm(env, "vm2")) + + def add_vlan(session, id, iface="eth0"): + if session.get_command_status("vconfig add %s %s" % (iface, id)) != 0: + raise error.TestError("Fail to add %s.%s" % (iface, id)) + + def set_ip_vlan(session, id, ip, iface="eth0"): + iface = "%s.%s" % (iface, id) + if session.get_command_status("ifconfig %s %s" % (iface, ip)) != 0: + raise error.TestError("Fail to configure ip for %s" % iface) + + def set_arp_ignore(session, iface="eth0"): + ignore_cmd = "echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore" + if session.get_command_status(ignore_cmd) != 0: + raise error.TestError("Fail to set arp_ignore of %s" % session) + + def rem_vlan(session, id, iface="eth0"): + rem_vlan_cmd = "if [[ -e /proc/net/vlan/%s ]];then vconfig rem %s;fi" + iface = "%s.%s" % (iface, id) + s = session.get_command_status(rem_vlan_cmd % (iface, iface)) + return s + + def nc_transfer(src, dst): + nc_port = kvm_utils.find_free_port(1025, 5334, vm_ip[dst]) + listen_cmd = params.get("listen_cmd") + send_cmd = params.get("send_cmd") + + #listen in dst + listen_cmd = listen_cmd % (nc_port, "receive") + session[dst].sendline(listen_cmd) + time.sleep(2) + #send file from src to dst + send_cmd = send_cmd % (vlan_ip[dst], str(nc_port), "file") + if session[src].get_command_status(send_cmd, timeout = 60) != 0: + raise error.TestFail ("Fail to send file" + " from vm%s to vm%s" % (src+1, dst+1)) + s, o = session[dst].read_up_to_prompt(timeout=60) + if s != True: + raise error.TestFail ("Fail to receive file" + " from vm%s to vm%s" % (src+1, dst+1)) + #check MD5 message digest of receive file in dst + output = session[dst].get_command_output("md5sum receive").strip() + digest_receive = re.findall(r'(\w+)', output)[0] + if digest_receive == digest_origin[src]: + logging.info("file succeed received in vm %s" % vlan_ip[dst]) + else: + logging.info("digest_origin is %s" % digest_origin[src]) + logging.info("digest_receive is %s" % digest_receive) + raise error.TestFail("File transfered differ from origin") + session[dst].get_command_status("rm -f receive") + + for i in range(2): + session.append(kvm_test_utils.wait_for_login(vm[i], + timeout=int(params.get("login_timeout", 360)))) + if not session[i] : + raise error.TestError("Could not log into guest(vm%d)" % i) + logging.info("Logged in") + + #get guest ip + vm_ip.append(vm[i].get_address()) + + #produce sized file in vm + dd_cmd = "dd if=/dev/urandom of=file bs=1024k count=%s" + if session[i].get_command_status(dd_cmd % file_size) != 0: + raise error.TestFail("File producing failed") + #record MD5 message digest of file + s, output =session[i].get_command_status_output("md5sum file", + timeout=60) + if s != 0: + raise error.TestFail("File MD5_checking failed" ) + digest_origin.append(re.findall(r'(\w+)', output)[0]) + + #stop firewall in vm + session[i].get_command_status("/etc/init.d/iptables stop") + + #load 8021q module for vconfig + load_8021q_cmd = "modprobe 8021q" + if session[i].get_command_status(load_8021q_cmd) != 0: + raise error.TestError("Fail to load 8021q module on VM%s" % i) + + try: + for i in range(2): + for vlan_i in range(1, vlan_num+1): + add_vlan(session[i], vlan_i) + set_ip_vlan(session[i], vlan_i, "%s.%s.%s" % \ + (subnet, vlan_i, ip_unit[i])) + set_arp_ignore(session[i]) + + for vlan in range(1, vlan_num+1): + logging.info("Test for vlan %s" % vlan) + + logging.info("Ping between vlans") + interface = 'eth0.' + str(vlan) + for vlan2 in range(1, vlan_num+1): + for i in range(2): + dest = subnet +'.'+ str(vlan2)+ '.' + ip_unit[(i+1)%2] + s, o = kvm_net_utils.ping(dest, count=2, + interface=interface, + session=session[i], timeout=30) + if ((vlan == vlan2) ^ (s == 0)): + raise error.TestFail ("%s ping %s unexpected" % + (interface, dest)) + + vlan_ip[0] = subnet + '.' + str(vlan) + '.' + ip_unit[0] + vlan_ip[1] = subnet + '.' + str(vlan) + '.' + ip_unit[1] + + logging.info("Flood ping") + def flood_ping(src, dst): + # we must use a dedicated session becuase the kvm_subprocess + # does not have the other method to interrupt the process in + # the guest rather than close the session. + session_flood = kvm_test_utils.wait_for_login(vm[src], + timeout = 60) + kvm_net_utils.ping(vlan_ip[dst], flood=True, + interface=interface, + session=session_flood, timeout=10) + session_flood.close() + + flood_ping(0,1) + flood_ping(1,0) + + logging.info("Transfering data through nc") + nc_transfer(0, 1) + nc_transfer(1, 0) + + finally: + for vlan in range(1, vlan_num+1): + rem_vlan(session[0], vlan) + rem_vlan(session[1], vlan) + logging.info("rem eth0.%s" % vlan) + + #Plumb/unplumb maximal unber of vlan interfaces + i = 1 + s = 0 + try: + logging.info("Testing the plumb of vlan interface") + for i in range (1, maximal+1): + add_vlan(session[0], i) + finally: + for j in range (1, i+1): + s = s or rem_vlan(session[0], j) + if s == 0: + logging.info("maximal interface plumb test done") + else: + logging.error("maximal interface plumb test failed") + + session[0].close() + session[1].close()
client/tests/kvm/tests/vlan_tag.py deleted +0 -68
68 lines not on this page · raw message
client/tests/kvm/tests_base.cfg.sample +10 -6
@@ -337,17 +337,21 @@ variants: - fmt_raw: image_format_stg = raw - - vlan_tag: install setup unattended_install.cdrom - type = vlan_tag + - vlan: install setup unattended_install.cdrom + type = vlan # subnet should not be used by host - subnet = 192.168.123 - vlans = "10 20" + subnet = "192.168" + vlan_num = 5 + file_size = 10 + maximal = 4094 + listen_cmd = "nc -l %s > %s" + send_cmd = "nc %s %s < %s" nic_mode = tap vms += " vm2" extra_params_vm1 += " -snapshot" extra_params_vm2 += " -snapshot" + kill_vm_vm2 = yes kill_vm_gracefully_vm2 = no - address_index_vm2 = 1 - ping: install setup unattended_install.cdrom type = ping
@@ -1084,7 +1088,7 @@ variants: # Windows section - @Windows: - no autotest linux_s3 vlan_tag ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast mac_change + no autotest linux_s3 vlan ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast mac_change shutdown_command = shutdown /s /f /t 0 reboot_command = shutdown /r /f /t 0 status_test_command = echo %errorlevel%
RFC PATCH 14/14 KVM-test: Add subtest of testing offload by ethtool
In reply to Amos KongThe latest case contains TX/RX/SG/TSO/GSO/GRO/LRO test. RTL8139 NIC doesn't support TSO, LRO, it's too old, so drop offload test from rtl8139. LRO, GRO are only supported by latest kernel, virtio nic doesn't support receive offloading function.
Initialize the callbacks first and execute all the sub tests one by one, all the result will be check at the end.
When execute this test, vhost should be enabled, then most of new feature can be used. Vhost doestn't support VIRTIO_NET_F_MRG_RXBUF, so do not check large packets in received offload test.
Transfer files by scp between host and guest, match new opened TCP port by netstat. Capture the packages info by tcpdump, it contains package length.
---
0 files changed, 0 insertions(+), 0 deletions(-)
client/tests/kvm/tests/ethtool.py new file +205 -0
@@ -0,0 +1,205 @@ +import time, os, logging, commands, re +from autotest_lib.client.common_lib import error +from autotest_lib.client.bin import utils +import kvm_test_utils, kvm_utils, kvm_net_utils + +def run_ethtool(test, params, env): + """ + Test offload functions of ethernet device by ethtool + + 1) Log into a guest + 2) Initialize the callback of sub functions + 3) Enable/disable sub function of NIC + 4) Execute callback function + 5) Check the return value + 6) Restore original configuration + + @param test: Kvm test object + @param params: Dictionary with the test parameters. + @param env: Dictionary with test environment. + """ + def ethtool_get(type): + feature_pattern = { + 'tx': 'tx.*checksumming', + 'rx': 'rx.*checksumming', + 'sg': 'scatter.*gather', + 'tso': 'tcp.*segmentation.*offload', + 'gso': 'generic.*segmentation.*offload', + 'gro': 'generic.*receive.*offload', + 'lro': 'large.*receive.*offload', + } + s, o = session.get_command_status_output("ethtool -k %s" % ethname) + try: + return re.findall("%s: (.*)" % feature_pattern.get(type), o)[0] + except IndexError: + logging.debug("Could not get %s status" % type) + + def ethtool_set(type, status): + """ + Set ethernet device offload status + + @param type: Offload type name + @param status: New status will be changed to + """ + logging.info("Try to set %s %s" % (type, status)) + if status not in ["off", "on"]: + return False + cmd = "ethtool -K %s %s %s" % (ethname, type, status) + if ethtool_get(type) != status: + return session.get_command_status(cmd) == 0 + if ethtool_get(type) != status: + logging.error("Fail to set %s %s" % (type, status)) + return False + return True + + def ethtool_save_params(): + logging.info("Save ethtool configuration") + for i in supported_features: + feature_status[i] = ethtool_get(i) + + def ethtool_restore_params(): + logging.info("Restore ethtool configuration") + for i in supported_features: + ethtool_set(i, feature_status[i]) + + def compare_md5sum(name): + logging.info("Compare md5sum of the files on guest and host") + host_result = utils.hash_file(name, method="md5") + try: + o = session.get_command_output("md5sum %s" % name) + guest_result = re.findall("\w+", o)[0] + except IndexError: + logging.error("Could not get file md5sum in guest") + return False + logging.debug("md5sum: guest(%s), host(%s)" % (guest_result, + host_result)) + return guest_result == host_result + + def transfer_file(src="guest"): + """ + Transfer file by scp, use tcpdump to capture packets, then check the + return string. + + @param src: Source host of transfer file + @return: Tuple (status, error msg/tcpdump result) + """ + session2.get_command_status("rm -rf %s" % filename) + dd_cmd = "dd if=/dev/urandom of=%s bs=1M count=%s" % (filename, + params.get("filesize")) + logging.info("Creat file in source host, cmd: %s" % dd_cmd) + tcpdump_cmd = "tcpdump -lep -s 0 tcp -vv port ssh" + if src == "guest": + s = session.get_command_status(dd_cmd, timeout=360) + tcpdump_cmd += " and src %s" % guest_ip + copy_files_fun = vm.copy_files_from + else: + s, o = commands.getstatusoutput(dd_cmd) + tcpdump_cmd += " and dst %s" % guest_ip + copy_files_fun = vm.copy_files_to + if s != 0: + return (False, "Fail to create file by dd, cmd: %s" % dd_cmd) + + # only capture the new tcp port after offload setup + original_tcp_ports = re.findall("tcp.*:(\d+).*%s" % guest_ip, + commands.getoutput("/bin/netstat -nap")) + for i in original_tcp_ports: + tcpdump_cmd += " and not port %s" % i + logging.debug("Listen by command: %s" % tcpdump_cmd) + session2.sendline(tcpdump_cmd) + if not kvm_utils.wait_for(lambda: session.get_command_status( + "pgrep tcpdump") == 0, 30): + return (False, "Tcpdump process wasn't launched") + + logging.info("Start to transfer file") + if not copy_files_fun(filename, filename): + return (False, "Child process transfer file failed") + logging.info("Transfer file completed") + if session.get_command_status("killall tcpdump") != 0: + return (False, "Could not kill all tcpdump process") + s, tcpdump_string = session2.read_up_to_prompt(timeout=60) + if not s: + return (False, "Fail to read tcpdump's output") + + if not compare_md5sum(filename): + return (False, "Files' md5sum mismatched") + return (True, tcpdump_string) + + def tx_callback(status="on"): + s, o = transfer_file(src="guest") + if not s: + logging.error(o) + return False + return True + + def rx_callback(status="on"): + s, o = transfer_file(src="host") + if not s: + logging.error(o) + return False + return True + + def so_callback(status="on"): + s, o = transfer_file(src="guest") + if not s: + logging.error(o) + return False + logging.info("Check if contained large frame") + # mtu: default IPv4 MTU is 1500 Bytes, ethernet header is 14 Bytes + return (status == "on") ^ (len([i for i in re.findall( + "length (\d*):", o) if int(i) > mtu]) == 0) + + def ro_callback(status="on"): + s, o = transfer_file(src="host") + if not s: + logging.error(o) + return False + return True + + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm")) + session = kvm_test_utils.wait_for_login(vm, + timeout=int(params.get("login_timeout", 360))) + session2 = kvm_test_utils.wait_for_login(vm, + timeout=int(params.get("login_timeout", 360))) + mtu = 1514 + feature_status = {} + filename = "/tmp/ethtool.dd" + guest_ip = vm.get_address() + ethname = kvm_net_utils.get_linux_ifname(session, vm.get_macaddr(0)) + supported_features = params.get("supported_features").split() + test_matrix = { + # type:(callback, (dependence), (exclude) + "tx": (tx_callback, (), ()), + "rx": (rx_callback, (), ()), + "sg": (tx_callback, ("tx",), ()), + "tso": (so_callback, ("tx", "sg",), ("gso",)), + "gso": (so_callback, (), ("tso",)), + "gro": (ro_callback, ("rx",), ("lro",)), + "lro": (rx_callback, (), ("gro",)), + } + ethtool_save_params() + success = True + try: + for type in supported_features: + callback = test_matrix[type][0] + for i in test_matrix[type][2]: + if not ethtool_set(i, "off"): + logging.error("Fail to disable %s" % i) + success = False + for i in [f for f in test_matrix[type][1]] + [type]: + if not ethtool_set(i, "on"): + logging.error("Fail to enable %s" % i) + success = False + if not callback(): + raise error.TestFail("Test failed, %s: on" % type) + + if not ethtool_set(type, "off"): + logging.error("Fail to disable %s" % type) + success = False + if not callback(status="off"): + raise error.TestFail("Test failed, %s: off" % type) + if not success: + raise error.TestError("Enable/disable offload function fail") + finally: + ethtool_restore_params() + session.close() + session2.close()
client/tests/kvm/tests_base.cfg.sample +14 -1
@@ -412,6 +412,19 @@ variants: netperf_cmd = %s/netperf-2.4.5/src/netperf -t %s -H %s -l 60 -- -m 1 protocols = "TCP_STREAM TCP_MAERTS TCP_RR TCP_CRR UDP_RR TCP_SENDFILE UDP_STREAM" + - ethtool: install setup unattended_install.cdrom + type = ethtool + filesize = 512 + nic_mode = tap + variants: + # gso gro lro is only supported by latest kernel + - nic_virtio: + pci_model = virtio + supported_features = "tx sg tso gso" + - nic_e1000: + pci_model = e1000 + supported_features = "tx rx sg tso gso gro lro" + - physical_resources_check: install setup unattended_install.cdrom type = physical_resources_check catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'
@@ -1088,7 +1101,7 @@ variants: # Windows section - @Windows: - no autotest linux_s3 vlan ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast mac_change + no autotest linux_s3 vlan ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast mac_change ethtool shutdown_command = shutdown /s /f /t 0 reboot_command = shutdown /r /f /t 0 status_test_command = echo %errorlevel%
Re: [RFC PATCH 14/14] KVM-test: Add subtest of testing offload by ethtool
Quoted 11 lines · Amos Kong
The latest case contains TX/RX/SG/TSO/GSO/GRO/LRO test. RTL8139 NIC doesn't support TSO, LRO, it's too old, so drop offload test from rtl8139. LRO, GRO are only supported by latest kernel, virtio nic doesn't support receive offloading function.
Initialize the callbacks first and execute all the sub tests one by one, all the result will be check at the end.
When execute this test, vhost should be enabled, then most of new feature can be used. Vhost doestn't supportVIRTIO_NET_F_MRG_RXBUF, so do not check large packets in received offload test.
Transfer files by scp between host and guest, match new opened TCP port by netstat. Capture the packages info by tcpdump, it contains package length.
This test is heavily dependent on ethtool, so we need to make sure the package is going to be installed on linux guests. The default package selection for Fedora 13 does not include it for example. So, we need to modify linux guest kickstarts/XMLs to add ethtool to the default package selection.
Quoted 168 lines
Signed-off-by Amos Kong <[email protected]>---
0 files changed, 0 insertions(+), 0 deletions(-)client/tests/kvm/tests/ethtool.py new file +158
-0@@ -0,0 +1,205 @@ +import time, os, logging, commands, re +from autotest_lib.client.common_lib import error +from autotest_lib.client.bin import utils +import kvm_test_utils, kvm_utils, kvm_net_utils + +def run_ethtool(test, params, env): + """ + Test offload functions of ethernet device by ethtool + + 1) Log into a guest + 2) Initialize the callback of sub functions + 3) Enable/disable sub function of NIC + 4) Execute callback function + 5) Check the return value + 6) Restore original configuration + + @param test: Kvm test object + @param params: Dictionary with the test parameters. + @param env: Dictionary with test environment. + """ + def ethtool_get(type): + feature_pattern = { + 'tx': 'tx.*checksumming', + 'rx': 'rx.*checksumming', + 'sg': 'scatter.*gather', + 'tso': 'tcp.*segmentation.*offload', + 'gso': 'generic.*segmentation.*offload', + 'gro': 'generic.*receive.*offload', + 'lro': 'large.*receive.*offload', + } + s, o = session.get_command_status_output("ethtool -k %s" % ethname) + try: + return re.findall("%s: (.*)" % feature_pattern.get(type), o)[0] + except IndexError: + logging.debug("Could not get %s status" % type) + + def ethtool_set(type, status): + """ + Set ethernet device offload status + + @param type: Offload type name + @param status: New status will be changed to + """ + logging.info("Try to set %s %s" % (type, status)) + if status not in ["off", "on"]: + return False + cmd = "ethtool -K %s %s %s" % (ethname, type, status) + if ethtool_get(type) != status: + return session.get_command_status(cmd) == 0 + if ethtool_get(type) != status: + logging.error("Fail to set %s %s" % (type, status)) + return False + return True + + def ethtool_save_params(): + logging.info("Save ethtool configuration") + for i in supported_features: + feature_status[i] = ethtool_get(i) + + def ethtool_restore_params(): + logging.info("Restore ethtool configuration") + for i in supported_features: + ethtool_set(i, feature_status[i]) + + def compare_md5sum(name): + logging.info("Compare md5sum of the files on guest and host") + host_result = utils.hash_file(name, method="md5") + try: + o = session.get_command_output("md5sum %s" % name) + guest_result = re.findall("\w+", o)[0] + except IndexError: + logging.error("Could not get file md5sum in guest") + return False + logging.debug("md5sum: guest(%s), host(%s)" % (guest_result, + host_result)) + return guest_result == host_result + + def transfer_file(src="guest"): + """ + Transfer file by scp, use tcpdump to capture packets, then check the + return string. + + @param src: Source host of transfer file + @return: Tuple (status, error msg/tcpdump result) + """ + session2.get_command_status("rm -rf %s" % filename) + dd_cmd = "dd if=/dev/urandom of=%s bs=1M count=%s" % (filename, + params.get("filesize")) + logging.info("Creat file in source host, cmd: %s" % dd_cmd) + tcpdump_cmd = "tcpdump -lep -s 0 tcp -vv port ssh" + if src == "guest": + s = session.get_command_status(dd_cmd, timeout=360) + tcpdump_cmd += " and src %s" % guest_ip + copy_files_fun = vm.copy_files_from + else: + s, o = commands.getstatusoutput(dd_cmd) + tcpdump_cmd += " and dst %s" % guest_ip + copy_files_fun = vm.copy_files_to + if s != 0: + return (False, "Fail to create file by dd, cmd: %s" % dd_cmd) + + # only capture the new tcp port after offload setup + original_tcp_ports = re.findall("tcp.*:(\d+).*%s" % guest_ip, + commands.getoutput("/bin/netstat -nap")) + for i in original_tcp_ports: + tcpdump_cmd += " and not port %s" % i + logging.debug("Listen by command: %s" % tcpdump_cmd) + session2.sendline(tcpdump_cmd) + if not kvm_utils.wait_for(lambda: session.get_command_status( + "pgrep tcpdump") == 0, 30): + return (False, "Tcpdump process wasn't launched") + + logging.info("Start to transfer file") + if not copy_files_fun(filename, filename): + return (False, "Child process transfer file failed") + logging.info("Transfer file completed") + if session.get_command_status("killall tcpdump") != 0: + return (False, "Could not kill all tcpdump process") + s, tcpdump_string = session2.read_up_to_prompt(timeout=60) + if not s: + return (False, "Fail to read tcpdump's output") + + if not compare_md5sum(filename): + return (False, "Files' md5sum mismatched") + return (True, tcpdump_string) + + def tx_callback(status="on"): + s, o = transfer_file(src="guest") + if not s: + logging.error(o) + return False + return True + + def rx_callback(status="on"): + s, o = transfer_file(src="host") + if not s: + logging.error(o) + return False + return True + + def so_callback(status="on"): + s, o = transfer_file(src="guest") + if not s: + logging.error(o) + return False + logging.info("Check if contained large frame") + # mtu: default IPv4 MTU is 1500 Bytes, ethernet header is 14 Bytes + return (status == "on") ^ (len([i for i in re.findall( + "length (\d*):", o) if int(i) > mtu]) == 0) + + def ro_callback(status="on"): + s, o = transfer_file(src="host") + if not s: + logging.error(o) + return False + return True + + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm"))
client/tests/kvm/tests/ethtool.py line 168
Quoted 10 lines
client/tests/kvm/tests/ethtool.py new file +10
-0+ session = kvm_test_utils.wait_for_login(vm, + timeout=int(params.get("login_timeout", 360))) + session2 = kvm_test_utils.wait_for_login(vm, + timeout=int(params.get("login_timeout", 360))) + mtu = 1514 + feature_status = {} + filename = "/tmp/ethtool.dd" + guest_ip = vm.get_address() + ethname = kvm_net_utils.get_linux_ifname(session, vm.get_macaddr(0)) + supported_features = params.get("supported_features").split()
^ We could use ethtool to query the machine for the supported features, and if it's not possible to do that (virtio_net), we resort to supported_feature being set on the config file, what do you think?
Quoted 71 lines
client/tests/kvm/tests/ethtool.py new file +37
-0+ test_matrix = { + # type:(callback, (dependence), (exclude) + "tx": (tx_callback, (), ()), + "rx": (rx_callback, (), ()), + "sg": (tx_callback, ("tx",), ()), + "tso": (so_callback, ("tx", "sg",), ("gso",)), + "gso": (so_callback, (), ("tso",)), + "gro": (ro_callback, ("rx",), ("lro",)), + "lro": (rx_callback, (), ("gro",)), + } + ethtool_save_params() + success = True + try: + for type in supported_features: + callback = test_matrix[type][0] + for i in test_matrix[type][2]: + if not ethtool_set(i, "off"): + logging.error("Fail to disable %s" % i) + success = False + for i in [f for f in test_matrix[type][1]] + [type]: + if not ethtool_set(i, "on"): + logging.error("Fail to enable %s" % i) + success = False + if not callback(): + raise error.TestFail("Test failed, %s: on" % type) + + if not ethtool_set(type, "off"): + logging.error("Fail to disable %s" % type) + success = False + if not callback(status="off"): + raise error.TestFail("Test failed, %s: off" % type) + if not success: + raise error.TestError("Enable/disable offload function fail") + finally: + ethtool_restore_params() + session.close() + session2.close()client/tests/kvm/tests_base.cfg.sample +14
-1@@ -412,6 +412,19 @@ variants: netperf_cmd = %s/netperf-2.4.5/src/netperf -t %s -H %s -l 60 -- -m 1 protocols = "TCP_STREAM TCP_MAERTS TCP_RR TCP_CRR UDP_RR TCP_SENDFILE UDP_STREAM" + - ethtool: install setup unattended_install.cdrom + type = ethtool + filesize = 512 + nic_mode = tap + variants: + # gso gro lro is only supported by latest kernel + - nic_virtio: + pci_model = virtio + supported_features = "tx sg tso gso" + - nic_e1000: + pci_model = e1000 + supported_features = "tx rx sg tso gso gro lro" + - physical_resources_check: install setup unattended_install.cdrom type = physical_resources_check catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'@@ -1088,7 +1101,7 @@ variants: # Windows section - @Windows: - no autotest linux_s3 vlan ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast mac_change + no autotest linux_s3 vlan ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast mac_change ethtool shutdown_command = shutdown /s /f /t 0 reboot_command = shutdown /r /f /t 0 status_test_command = echo %errorlevel%
Re: [RFC PATCH 14/14] KVM-test: Add subtest of testing offload by ethtool
Quoted 13 lines · Lucas Meneghel Rodrigues
Quoted 11 lines · Amos Kong
The latest case contains TX/RX/SG/TSO/GSO/GRO/LRO test. RTL8139 NIC doesn't support TSO, LRO, it's too old, so drop offload test from rtl8139. LRO, GRO are only supported by latest kernel, virtio nic doesn't support receive offloading function.
Initialize the callbacks first and execute all the sub tests one by one, all the result will be check at the end.
When execute this test, vhost should be enabled, then most of new feature can be used. Vhost doestn't supportVIRTIO_NET_F_MRG_RXBUF, so do not check large packets in received offload test.
Transfer files by scp between host and guest, match new opened TCP port by netstat. Capture the packages info by tcpdump, it contains package length.
Quoted 5 lines
This test is heavily dependent on ethtool, so we need to make sure the package is going to be installed on linux guests. The default package selection for Fedora 13 does not include it for example. So, we need to modify linux guest kickstarts/XMLs to add ethtool to the default package selection.
OK.
Quoted 174 lines
Quoted 174 lines
Signed-off-by Amos Kong <[email protected]>---
0 files changed, 0 insertions(+), 0 deletions(-)client/tests/kvm/tests/ethtool.py new file +164
-0@@ -0,0 +1,205 @@ +import time, os, logging, commands, re +from autotest_lib.client.common_lib import error +from autotest_lib.client.bin import utils +import kvm_test_utils, kvm_utils, kvm_net_utils + +def run_ethtool(test, params, env): + """ + Test offload functions of ethernet device by ethtool + + 1) Log into a guest + 2) Initialize the callback of sub functions + 3) Enable/disable sub function of NIC + 4) Execute callback function + 5) Check the return value + 6) Restore original configuration + + @param test: Kvm test object + @param params: Dictionary with the test parameters. + @param env: Dictionary with test environment. + """ + def ethtool_get(type): + feature_pattern = { + 'tx': 'tx.*checksumming', + 'rx': 'rx.*checksumming', + 'sg': 'scatter.*gather', + 'tso': 'tcp.*segmentation.*offload', + 'gso': 'generic.*segmentation.*offload', + 'gro': 'generic.*receive.*offload', + 'lro': 'large.*receive.*offload', + } + s, o = session.get_command_status_output("ethtool -k %s" % ethname) + try: + return re.findall("%s: (.*)" % feature_pattern.get(type), o)[0] + except IndexError: + logging.debug("Could not get %s status" % type) + + def ethtool_set(type, status): + """ + Set ethernet device offload status + + @param type: Offload type name + @param status: New status will be changed to + """ + logging.info("Try to set %s %s" % (type, status)) + if status not in ["off", "on"]: + return False + cmd = "ethtool -K %s %s %s" % (ethname, type, status) + if ethtool_get(type) != status: + return session.get_command_status(cmd) == 0 + if ethtool_get(type) != status: + logging.error("Fail to set %s %s" % (type, status)) + return False + return True + + def ethtool_save_params(): + logging.info("Save ethtool configuration") + for i in supported_features: + feature_status[i] = ethtool_get(i) + + def ethtool_restore_params(): + logging.info("Restore ethtool configuration") + for i in supported_features: + ethtool_set(i, feature_status[i]) + + def compare_md5sum(name): + logging.info("Compare md5sum of the files on guest and host") + host_result = utils.hash_file(name, method="md5") + try: + o = session.get_command_output("md5sum %s" % name) + guest_result = re.findall("\w+", o)[0] + except IndexError: + logging.error("Could not get file md5sum in guest") + return False + logging.debug("md5sum: guest(%s), host(%s)" % (guest_result, + host_result)) + return guest_result == host_result + + def transfer_file(src="guest"): + """ + Transfer file by scp, use tcpdump to capture packets, then check the + return string. + + @param src: Source host of transfer file + @return: Tuple (status, error msg/tcpdump result) + """ + session2.get_command_status("rm -rf %s" % filename) + dd_cmd = "dd if=/dev/urandom of=%s bs=1M count=%s" % (filename, + params.get("filesize")) + logging.info("Creat file in source host, cmd: %s" % dd_cmd) + tcpdump_cmd = "tcpdump -lep -s 0 tcp -vv port ssh" + if src == "guest": + s = session.get_command_status(dd_cmd, timeout=360) + tcpdump_cmd += " and src %s" % guest_ip + copy_files_fun = vm.copy_files_from + else: + s, o = commands.getstatusoutput(dd_cmd) + tcpdump_cmd += " and dst %s" % guest_ip + copy_files_fun = vm.copy_files_to + if s != 0: + return (False, "Fail to create file by dd, cmd: %s" % dd_cmd) + + # only capture the new tcp port after offload setup + original_tcp_ports = re.findall("tcp.*:(\d+).*%s" % guest_ip, + commands.getoutput("/bin/netstat -nap")) + for i in original_tcp_ports: + tcpdump_cmd += " and not port %s" % i + logging.debug("Listen by command: %s" % tcpdump_cmd) + session2.sendline(tcpdump_cmd) + if not kvm_utils.wait_for(lambda: session.get_command_status( + "pgrep tcpdump") == 0, 30): + return (False, "Tcpdump process wasn't launched") + + logging.info("Start to transfer file") + if not copy_files_fun(filename, filename): + return (False, "Child process transfer file failed") + logging.info("Transfer file completed") + if session.get_command_status("killall tcpdump") != 0: + return (False, "Could not kill all tcpdump process") + s, tcpdump_string = session2.read_up_to_prompt(timeout=60) + if not s: + return (False, "Fail to read tcpdump's output") + + if not compare_md5sum(filename): + return (False, "Files' md5sum mismatched") + return (True, tcpdump_string) + + def tx_callback(status="on"): + s, o = transfer_file(src="guest") + if not s: + logging.error(o) + return False + return True + + def rx_callback(status="on"): + s, o = transfer_file(src="host") + if not s: + logging.error(o) + return False + return True + + def so_callback(status="on"): + s, o = transfer_file(src="guest") + if not s: + logging.error(o) + return False + logging.info("Check if contained large frame") + # mtu: default IPv4 MTU is 1500 Bytes, ethernet header is 14 Bytes + return (status == "on") ^ (len([i for i in re.findall( + "length (\d*):", o) if int(i) > mtu]) == 0) + + def ro_callback(status="on"): + s, o = transfer_file(src="host") + if not s: + logging.error(o) + return False + return True + + vm = kvm_test_utils.get_living_vm(env, params.get("main_vm")) + session = kvm_test_utils.wait_for_login(vm, + timeout=int(params.get("login_timeout", 360))) + session2 = kvm_test_utils.wait_for_login(vm, + timeout=int(params.get("login_timeout", 360))) + mtu = 1514 + feature_status = {}
Quoted 8 lines
client/tests/kvm/tests/ethtool.py line 168
Quoted 4 lines
client/tests/kvm/tests/ethtool.py new file +4
-0+ filename = "/tmp/ethtool.dd" + guest_ip = vm.get_address() + ethname = kvm_net_utils.get_linux_ifname(session, vm.get_macaddr(0)) + supported_features = params.get("supported_features").split()^ We could use ethtool to query the machine for the supported features, and if it's not possible to do that (
virtio_net), we resort tosupported_featurebeing set on the config file, what do you think?
How to query which offload features are supported by nic dev by ethtool ? '# ethtool -k eth0' can only list the function status.
I try to enable all the function, by this order 'tx rx gs tso ufo gso gro lro', if some of them failed, it should not be supported ? This method is not dependable in test environment.
Quoted 70 lines
Quoted 70 lines
client/tests/kvm/tests/ethtool.py new file +37
-0+ test_matrix = { + # type:(callback, (dependence), (exclude) + "tx": (tx_callback, (), ()), + "rx": (rx_callback, (), ()), + "sg": (tx_callback, ("tx",), ()), + "tso": (so_callback, ("tx", "sg",), ("gso",)), + "gso": (so_callback, (), ("tso",)), + "gro": (ro_callback, ("rx",), ("lro",)), + "lro": (rx_callback, (), ("gro",)), + } + ethtool_save_params() + success = True + try: + for type in supported_features: + callback = test_matrix[type][0] + for i in test_matrix[type][2]: + if not ethtool_set(i, "off"): + logging.error("Fail to disable %s" % i) + success = False + for i in [f for f in test_matrix[type][1]] + [type]: + if not ethtool_set(i, "on"): + logging.error("Fail to enable %s" % i) + success = False + if not callback(): + raise error.TestFail("Test failed, %s: on" % type) + + if not ethtool_set(type, "off"): + logging.error("Fail to disable %s" % type) + success = False + if not callback(status="off"): + raise error.TestFail("Test failed, %s: off" % type) + if not success: + raise error.TestError("Enable/disable offload function fail") + finally: + ethtool_restore_params() + session.close() + session2.close()client/tests/kvm/tests_base.cfg.sample +0
-0@@ -412,6 +412,19 @@ variants:netperf_cmd = %s/netperf-2.4.5/src/netperf -t %s -H %s -l 60 -- -m 1 protocols = "
TCP_STREAMTCP_MAERTSTCP_RRTCP_CRRUDP_RRTCP_SENDFILEUDP_STREAM"+ - ethtool: install setup unattended_install.cdrom
+ type = ethtool
+ filesize = 512
+ nic_mode = tap
+ variants:
+ # gso gro lro is only supported by latest kernel
+ -nic_virtio:
+ pci_model = virtio
+ supported_features = "tx sg tso gso"
+ -nic_e1000:
+ pci_model = e1000
+ supported_features = "tx rx sg tso gso gro lro" +
-physical_resources_check: install setup unattended_install.cdrom type =physical_resources_check
catch_uuid_cmd = dmidecode | awk -F: '/UUID/ {print $2}'client/tests/kvm/tests_base.cfg.sample +0
-0@@ -1088,7 +1101,7 @@ variants: # Windows section- @Windows:
- no autotest linux_s3 vlan ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast mac_change + no autotest linux_s3 vlan ioquit unattended_install.(url|nfs|remote_ks) jumbo file_transfer nicdriver_unload nic_promisc multicast mac_change ethtoolshutdown_command = shutdown /s /f /t 0
reboot_command = shutdown /r /f /t 0
status_test_command = echo %errorlevel%
Quoted 2 lines · Amos Kong
The following series contain 11 network related subtests, welcome to give me some suggestions about correctness, design, enhancement.
Awesome work, will start to review them today. Thanks!
Quoted 23 lines
Thank you so much!
---
Amos Kong (14):
KVM-test: Add a new macaddress pool algorithm
KVM Test: Add a functionget_interface_name()tokvm_net_utils.pyKVM Test: Add a common ping module for network related tests
KVM-test: Add a new subtest ping
KVM-test: Add a subtest jumbo
KVM-test: Add basic file transfer test
KVM-test: Add a subtest of load/unload nic driver
KVM-test: Add a subtest of nic promisc
KVM-test: Add a subtest of multicast
KVM-test: Add a subtest of pxe
KVM-test: Add a subtest of changing mac address
KVM-test: Add a subtest of netperf
KVM-test: Improve vlan subtest
KVM-test: Add subtest of testing offload by ethtool0 files changed, 0 insertions(+), 0 deletions(-)
Quoted 4 lines · Amos Kong
The following series contain 11 network related subtests, welcome to give me some suggestions about correctness, design, enhancement.
Thank you so much!
Ok Amos, now that I made the first review of this patchset, I'll wait for a v2 and supersed v1.
Thanks!
Quoted 21 lines
---
Amos Kong (14):
KVM-test: Add a new macaddress pool algorithm
KVM Test: Add a functionget_interface_name()tokvm_net_utils.pyKVM Test: Add a common ping module for network related tests
KVM-test: Add a new subtest ping
KVM-test: Add a subtest jumbo
KVM-test: Add basic file transfer test
KVM-test: Add a subtest of load/unload nic driver
KVM-test: Add a subtest of nic promisc
KVM-test: Add a subtest of multicast
KVM-test: Add a subtest of pxe
KVM-test: Add a subtest of changing mac address
KVM-test: Add a subtest of netperf
KVM-test: Improve vlan subtest
KVM-test: Add subtest of testing offload by ethtool0 files changed, 0 insertions(+), 0 deletions(-)