Skip to contentSkip to conversation

Added functionality to the preprocessor to run scripts

PATCH In review Michael Goldish 4 participants 5 messages May 22, 2009 – May 31, 2009 1 message not in the archive KVM

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): 1222268607.226581242993156722.JavaMail.root@zmail05.collab.prod.int.phx2.redhat.com

#1 lore raw

Looks good to me. See some comments below.

Quoted 6 lines · ----- "David Huff

This patch will run pre and post scripts defined in config file with the parameter pre_command and post_command post_command.

Also exports all the prameters in preprocess for passing arguments to the script.

Why not do this for post_command as well?

Quoted 11 lines
---
 client/tests/kvm_runtest_2/kvm_preprocessing.py |   31
+++++++++++++++++++++-
 1 files changed, 29 insertions(+), 2 deletions(-)
client/tests/kvm_runtest_2/kvm_preprocessing.py +0 -0
index c9eb35d..02df615 100644

client/tests/kvm_runtest_2/kvm_preprocessing.py def postprocess_vm(test, params, env, name): · line 139

Quoted 7 lines
client/tests/kvm_runtest_2/kvm_preprocessing.py +1 -2
@@ -135,8 +135,7 @@ def postprocess_vm(test, params, env, name):
                 "Waiting for VM to kill itself..."):
             kvm_log.debug("'kill_vm' specified; killing VM...")
         vm.destroy(gracefully = params.get("kill_vm_gracefully") ==
 "yes")
-
-
+        

I hate to be petty, but we usually keep two blank lines between top level functions.

Also, you have some trailing whitespace there...

Quoted 21 lines
client/tests/kvm_runtest_2/kvm_preprocessing.py +0 -0
 def process(test, params, env, image_func, vm_func):
     """Pre- or post-process VMs and images according to the

instructions in params.

client/tests/kvm_runtest_2/kvm_preprocessing.py +1 -0
@@ -169,6 +168,7 @@ def preprocess(test, params, env):
     params -- a dict containing all VM and image parameters
     env -- the environment (a dict-like object)
 
+    Also, runs any setup command defined in the parameter

pre_command
Also, collect some host information, such as the KVM version. """
# Verify the identities of all living VMs

client/tests/kvm_runtest_2/kvm_preprocessing.py +4 -0
@@ -192,6 +192,22 @@ def preprocess(test, params, env):
             vm.destroy()
             del env[key]
 
+   #execute any pre_commands
+    pre_command = params.get("pre_command")
+    if pre_command:
+        # export environment vars

client/tests/kvm_runtest_2/kvm_preprocessing.py def preprocess(test, params, env): · line 208

Quoted 10 lines
client/tests/kvm_runtest_2/kvm_preprocessing.py +8 -0
+        for k in params.keys():
+            kvm_log.info("Adding KVM_TEST_%s to Environment" % (k))
+            os.putenv("KVM_TEST_%s" % (k), str(params[k]))
+
+        # execute command
+        kvm_log.info("Executing command '%s'..." % pre_command)
+        (status, pid, output) = kvm_utils.run_bg("cd %s; %s" %
 (test.bindir, pre_command),
+                                                 None, kvm_log.debug,
 "(pre_command) ", timeout=600)

It wouldn't hurt to make this timeout user-configurable with a default value of 600 or so:

timeout = int(params.get("pre_commmand_timeout", "600")) (status, pid, output) = kvm_utils.run_bg(..., timeout=timeout)

We can also do that in a separate patch.

Quoted 34 lines
client/tests/kvm_runtest_2/kvm_preprocessing.py +3 -0
+        if status != 0:
+            kvm_utils.safe_kill(pid, signal.SIGTERM)
+            raise error.TestError, "Custom processing pre_command

failed"

client/tests/kvm_runtest_2/kvm_preprocessing.py +9 -0
+
     # Preprocess all VMs and images
     process(test, params, env, preprocess_image, preprocess_vm)
@@ -232,6 +248,8 @@ def postprocess(test, params, env):
     test -- an Autotest test object
     params -- a dict containing all VM and image parameters
     env -- the environment (a dict-like object)
+    
+    Also, runs any command defined in the parameter post_command
     """
     process(test, params, env, postprocess_image, postprocess_vm)
 
@@ -241,6 +259,15 @@ def postprocess(test, params, env):
         kvm_log.debug("'keep_ppm_files' not specified; removing all
 PPM files from results dir...")
         kvm_utils.run_bg("rm -vf %s" % os.path.join(test.debugdir,
 "*.ppm"), None, kvm_log.debug, "(rm) ", timeout=5.0)
 
+    #execute any post_commands
+    post_command = params.get("post_command")
+    if post_command:
+        kvm_log.info("Executing command '%s'..." % post_command)
+        (status, pid, output) = kvm_utils.run_bg("cd %s; %s" %
 (test.bindir, post_command),
+                                                 None, kvm_log.debug,

"(post_command) ", timeout=600)

client/tests/kvm_runtest_2/kvm_preprocessing.py +3 -0
+        if status != 0:
+            kvm_utils.safe_kill(pid, signal.SIGTERM)
+            raise error.TestError, "Custom processing command
Quoted 1 line

failed"

Same for post_command.

Quoted 3 lines

def postprocess_on_error(test, params, env):
"""Perform postprocessing operations required only if the test
failed.

Thanks,
Michael

#2 lore raw
Quoted 4 lines · Michael Goldish

Looks good to me. See some comments below.

----- "David Huff" <[email protected]> wrote:

Quoted 8 lines
Quoted 6 lines

This patch will run pre and post scripts defined in config file with the parameter pre_command and post_command post_command.

Also exports all the prameters in preprocess for passing arguments to the script.

Why not do this for post_command as well?

I didn't do post_command b/c I figured that they would already be exported in the pre_command, however I guess that there can be the case where there is no pre and only post and I guess exporting twice will not hurt anything.... I will add this to the patch.

Quoted 23 lines
Quoted 12 lines
---
 client/tests/kvm_runtest_2/kvm_preprocessing.py |   31
+++++++++++++++++++++-
 1 files changed, 29 insertions(+), 2 deletions(-)
client/tests/kvm_runtest_2/kvm_preprocessing.py +0 -0
index c9eb35d..02df615 100644

client/tests/kvm_runtest_2/kvm_preprocessing.py def postprocess_vm(test, params, env, name): · line 139

Quoted 7 lines
client/tests/kvm_runtest_2/kvm_preprocessing.py +1 -2
@@ -135,8 +135,7 @@ def postprocess_vm(test, params, env, name):
                 "Waiting for VM to kill itself..."):
             kvm_log.debug("'kill_vm' specified; killing VM...")
         vm.destroy(gracefully = params.get("kill_vm_gracefully") ==
 "yes")
-
-
+        

I hate to be petty, but we usually keep two blank lines between top level functions.

Quoted 1 line

Also, you have some trailing whitespace there...

good catch, I will take care of this

Quoted 8 lines

It wouldn't hurt to make this timeout user-configurable with a default value of 600 or so:

timeout = int(params.get("pre_commmand_timeout", "600")) (status, pid, output) = kvm_utils.run_bg(..., timeout=timeout)

We can also do that in a separate patch.

I'll go ahead and add this while I rework the patch...

-D

#3 lore raw

PATCH 1/1 Added functionality to the preprocessor to run scripts

This patch will run pre and post scripts defined in config file with the parameter pre_command and post_command post_command.

Also exports all the prameters in preprocess for passing arguments to the script.

---
 client/tests/kvm_runtest_2/kvm_preprocessing.py |   34 +++++++++++++++++++++++
 1 files changed, 34 insertions(+), 0 deletions(-)
client/tests/kvm_runtest_2/kvm_preprocessing.py +34 -0
index c9eb35d..07bfdeb 100644
@@ -169,6 +169,7 @@ def preprocess(test, params, env):
     params -- a dict containing all VM and image parameters
     env -- the environment (a dict-like object)
 
+    Also, runs any setup command defined in the parameter pre_command
     Also, collect some host information, such as the KVM version.
     """
     # Verify the identities of all living VMs
@@ -192,6 +193,22 @@ def preprocess(test, params, env):
             vm.destroy()
             del env[key]
 
+   #execute any pre_commands
+    pre_command = params.get("pre_command")
+    if pre_command:
+        # export environment vars
+        for k in params.keys():
+            kvm_log.info("Adding KVM_TEST_%s to Environment" % (k))
+            os.putenv("KVM_TEST_%s" % (k), str(params[k]))
+        # execute command
+        kvm_log.info("Executing command '%s'..." % pre_command)
+        timeout = int(params.get("pre_commmand_timeout", "600"))
+        (status, pid, output) = kvm_utils.run_bg("cd %s; %s" % (test.bindir, pre_command),
+                                                 None, kvm_log.debug, "(pre_command) ", timeout=timeout)
+        if status != 0:
+            kvm_utils.safe_kill(pid, signal.SIGTERM)
+            raise error.TestError, "Custom processing pre_command failed"
+
     # Preprocess all VMs and images
     process(test, params, env, preprocess_image, preprocess_vm)
 
@@ -232,6 +249,8 @@ def postprocess(test, params, env):
     test -- an Autotest test object
     params -- a dict containing all VM and image parameters
     env -- the environment (a dict-like object)
+
+    Also, runs any command defined in the parameter post_command
     """
     process(test, params, env, postprocess_image, postprocess_vm)
 
@@ -241,6 +260,21 @@ def postprocess(test, params, env):
         kvm_log.debug("'keep_ppm_files' not specified; removing all PPM files from results dir...")
         kvm_utils.run_bg("rm -vf %s" % os.path.join(test.debugdir, "*.ppm"), None, kvm_log.debug, "(rm) ", timeout=5.0)
 
+    #execute any post_commands
+    post_command = params.get("post_command")
+    if post_command:
+        # export environment vars
+        for k in params.keys():
+            kvm_log.info("Adding KVM_TEST_%s to Environment" % (k))
+            os.putenv("KVM_TEST_%s" % (k), str(params[k]))
+        #execute command
+        kvm_log.info("Executing command '%s'..." % post_command)
+        timeout = int(params.get("post_commmand_timeout", "600"))
+        (status, pid, output) = kvm_utils.run_bg("cd %s; %s" % (test.bindir, post_command),
+                                                 None, kvm_log.debug, "(pre_command) ", timeout=timeout)
+        if status != 0:
+            kvm_utils.safe_kill(pid, signal.SIGTERM)
+            raise error.TestError, "Custom processing command failed"
 
 def postprocess_on_error(test, params, env):
     """Perform postprocessing operations required only if the test failed.
Signature · 1 line
1.6.0.6
#4 lore raw
Quoted 13 lines · David Huff

This patch will run pre and post scripts defined in config file with the parameter pre_command and post_command post_command.

Also exports all the prameters in preprocess for passing arguments to the script.

client/tests/kvm_runtest_2/kvm_preprocessing.py +5 -0
+   #execute any pre_commands
+    pre_command = params.get("pre_command")
+    if pre_command:
+        # export environment vars
+        for k in params.keys():

client/tests/kvm_runtest_2/kvm_preprocessing.py def preprocess(test, params, env): · line 210

Quoted 10 lines
client/tests/kvm_runtest_2/kvm_preprocessing.py +10 -0
+            kvm_log.info("Adding KVM_TEST_%s to Environment" % (k))
+            os.putenv("KVM_TEST_%s" % (k), str(params[k]))
+        # execute command
+        kvm_log.info("Executing command '%s'..." % pre_command)
+        timeout = int(params.get("pre_commmand_timeout", "600"))
+        (status, pid, output) = kvm_utils.run_bg("cd %s; %s" % (test.bindir, pre_command),
+                                                 None, kvm_log.debug, "(pre_command) ", timeout=timeout)
+        if status != 0:
+            kvm_utils.safe_kill(pid, signal.SIGTERM)
+            raise error.TestError, "Custom processing pre_command failed"

kvm_utils.run_bg should throw an exception instead of returning status.

But if status != 0, will there actually be a pid to kill?

Signature · 1 line
error compiling committee.c: too many arguments to function
#5 lore raw
In reply to David Huff
Quoted 49 lines · David Huff
Quoted 4 lines · Michael Goldish

Looks good to me. See some comments below.

----- "David Huff" <[email protected]> wrote:

Quoted 8 lines
Quoted 6 lines

This patch will run pre and post scripts defined in config file with the parameter pre_command and post_command post_command.

Also exports all the prameters in preprocess for passing arguments to the script.

Why not do this for post_command as well?

I didn't do post_command b/c I figured that they would already be exported in the pre_command, however I guess that there can be the case where there is no pre and only post and I guess exporting twice will not hurt anything....  I will add this to the patch.

Quoted 23 lines
Quoted 16 lines

---
 client/tests/kvm_runtest_2/kvm_preprocessing.py |   31
+++++++++++++++++++++-

 1 files changed, 29 insertions(+), 2 deletions(-)
client/tests/kvm_runtest_2/kvm_preprocessing.py +0 -0
index c9eb35d..02df615 100644
@@ -135,8 +135,7 @@ def postprocess_vm(test, params, env, name):

                 "Waiting for VM to kill itself..."):
             kvm_log.debug("'kill_vm' specified; killing VM...") vm.destroy(gracefully = params.get("kill_vm_gracefully") == "yes")

client/tests/kvm_runtest_2/kvm_preprocessing.py

Quoted 3 lines
client/tests/kvm_runtest_2/kvm_preprocessing.py +1 -2
-
-
+

I hate to be petty, but we usually keep two blank lines between top level functions.

Quoted 1 line

Also, you have some trailing whitespace there...

good catch, I will take care of this

Quoted 3 lines

It wouldn't hurt to make this timeout user-configurable with a default value of 600 or so:

Quoted 7 lines
Quoted 5 lines

timeout = int(params.get("pre_commmand_timeout", "600")) (status, pid, output) = kvm_utils.run_bg(..., timeout=timeout)

We can also do that in a separate patch.

I'll go ahead and add this while I rework the patch...

Please ensure that we have a number of timeout categories now.(remote_login timeout, test execution timeout, migration timeout etc) If we wana make timeout a variable let us do it for all the timeouts.

Quoted 7 lines

-D
--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Signature · 1 line
Sudhir Kumar