Add common QEMU control functionality to qemu-iotests

This creates some common functions for bash language qemu-iotests to control, and communicate with, a running QEMU process.

4 functions are introduced:

1. _launch_qemu()
This launches the QEMU process(es), and sets up the file descriptors and fifos for communication. You can choose to launch each QEMU process listening for either QMP or HMP monitor. You can call this function multiple times, and save the handle returned from each. The returned handle is in $QEMU_HANDLE. You must copy this value.

Commands 2 and 3 use the handle received from _launch_qemu(), to talk to the appropriate process.

2. _send_qemu_cmd()
Sends a command string, specified by $2, to QEMU. If $2 is non-NULL, _send_qemu_cmd() will wait to receive $2 as a

required result string from QEMU. Failure to receive $3 will cause the test to fail. The command can optionally be retried $qemu_cmd_repeat number of times.

3. _timed_wait_for()
Waits for a response, for up to a default of 10 seconds. If $2 is not seen in that time (anywhere in the response), then the test fails. Primarily used by _send_qemu_cmd, but could be useful standalone, as well. To prevent automatic exit (and therefore test failure), set $qemu_wait_no_error to a non-NULL value. If $silent is a non-NULL value, then output to stdout will be suppressed.

4. _cleanup_qemu()
Kills the running QEMU processes, and removes the fifos.

Signed-off-by Jeff Cody <[email protected]>

---
 tests/qemu-iotests/common.qemu | 195 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 195 insertions(+)
 create mode 100644 tests/qemu-iotests/common.qemu

tests/qemu-iotests/common.qemu new file +145 -0

new file mode 100644
index 0000000..12c42f1
@@ -0,0 +1,195 @@
+#!/bin/bash
+#
+# This allows for launching of multiple QEMU instances, with independent
+# communication possible to each instance.
+#
+# Each instance can choose, at launch, to use either the QMP or the
+# HMP (monitor) interface.
+#
+# All instances are cleaned up via _cleanup_qemu, including killing the
+# running qemu instance.
+#
+# Copyright (C) 2014 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+QEMU_COMM_TIMEOUT=10
+
+QEMU_FIFO_IN="${TEST_DIR}/qmp-in-$$"
+QEMU_FIFO_OUT="${TEST_DIR}/qmp-out-$$"
+
+QEMU_PID=
+_QEMU_HANDLE=0
+QEMU_HANDLE=0
+
+# If bash version is >= 4.1, these will be overwritten and dynamic
+# file descriptor values assigned.
+_out_fd=3
+_in_fd=4
+
+# Wait for expected QMP response from QEMU.  Will time out
+# after 10 seconds, which counts as failure.
+#
+# Override QEMU_COMM_TIMEOUT for a timeout different than the
+# default 10 seconds
+#
+# $1: The handle to use
+# $2+ All remaining arguments comprise the string to search for
+#    in the response.
+#
+# If $silent is set to anything but an empty string, then
+# response is not echoed out.
+function _timed_wait_for()
+{
+    local h=${1}
+    shift
+
+    QEMU_STATUS[$h]=0
+    while read -t ${QEMU_COMM_TIMEOUT} resp <&${QEMU_OUT[$h]}
+    do
+        if [ -z "${silent}" ]; then
+            echo "${resp}" | _filter_testdir | _filter_qemu \
+                           | _filter_qemu_io | _filter_qmp
+        fi
+        grep -q "${*}" < <(echo ${resp})
+        if [ $? -eq 0 ]; then
+            return
+        fi
+    done
+    QEMU_STATUS[$h]=-1
+    if [ -z "${qemu_wait_no_error}" ]; then
+        echo "Timeout waiting for ${*} on handle ${h}"
+        exit 1  # Timeout means the test failed
+    fi
+}
+
+
+# Sends QMP or HMP command to QEMU, and waits for the expected response
+#
+# $1:       QEMU handle to use
+# $2:       String of the QMP command to send
+# ${@: -1}  (Last string passed)
+#             String that the QEMU response should contain. If it is a null
+#             string, do not wait for a response
+#
+# Set qemu_cmd_repeat to the number of times to repeat the cmd
+# until either timeout, or a response.  If it is not set, or <=0,
+# then the command is only sent once.
+#
+function _send_qemu_cmd()
+{
+    local h=${1}
+    local count=1
+    local cmd=
+    local use_error=
+    shift
+
+    if [ ${qemu_cmd_repeat} -gt 0 ] 2>/dev/null; then
+        count=${qemu_cmd_repeat}
+        use_error="no"
+    fi
+    # This array element extraction is done to accomodate pathnames with spaces
+    cmd=${@: 1:${#@}-1}
+    shift $(($# - 1))
+
+    while [ ${count} -gt 0 ]
+    do
+        echo "${cmd}" >&${QEMU_IN[${h}]}
+        if [ -n "${1}" ]; then
+            qemu_wait_no_error=${use_error} _timed_wait_for ${h} "${1}"
+            if [ ${QEMU_STATUS[$h]} -eq 0 ]; then
+                return
+            fi
+        fi
+        let count--;
+    done
+    if [ ${QEMU_STATUS[$h]} -ne 0 ]; then
+        echo "Timeout waiting for ${1} on handle ${h}"
+        exit 1 #Timeout means the test failed
+    fi
+}
+
+
+# Launch a QEMU process.
+#
+# Input parameters:
+# $qemu_comm_method: set this variable to 'monitor' (case insensitive)
+#                    to use the QEMU HMP monitor for communication.
+#                    Otherwise, the default of QMP is used.
+# Returns:
+# $QEMU_HANDLE: set to a handle value to communicate with this QEMU instance.
+#
+function _launch_qemu()
+{
+    local comm=
+    local fifo_out=
+    local fifo_in=
+
+    if (shopt -s nocasematch; [[ "${qemu_comm_method}" == "monitor" ]])
+    then
+        comm="-monitor stdio"
+    else
+        local qemu_comm_method="qmp"
+        comm="-monitor none -qmp stdio"

tests/qemu-iotests/common.qemu new file +40 -0

+    QEMU_PID[${_QEMU_HANDLE}]=$!
+
+    if [ "${BASH_VERSINFO[0]}" -ge "4" ] && [ "${BASH_VERSINFO[1]}" -ge "1" ]
+    then
+        # bash >= 4.1 required for automatic fd
+        exec {_out_fd}<"${fifo_out}"
+        exec {_in_fd}>"${fifo_in}"
+    else
+        let _out_fd++
+        let _in_fd++
+        eval "exec ${_out_fd}<'${fifo_out}'"
+        eval "exec ${_in_fd}>'${fifo_in}'"
+    fi
+
+    QEMU_OUT[${_QEMU_HANDLE}]=${_out_fd}
+    QEMU_IN[${_QEMU_HANDLE}]=${_in_fd}
+    QEMU_STATUS[${_QEMU_HANDLE}]=0
+
+    if [ "${qemu_comm_method}" == "qmp" ]
+    then
+        # Don't print response, since it has version information in it
+        silent=yes _timed_wait_for ${_QEMU_HANDLE} "capabilities"
+    fi
+    QEMU_HANDLE=${_QEMU_HANDLE}
+    let _QEMU_HANDLE++
+}
+
+
+# Silenty kills the QEMU process
+function _cleanup_qemu()
+{
+    # QEMU_PID[], QEMU_IN[], QEMU_OUT[] all use same indices
+    for i in "${!QEMU_OUT[@]}"
+    do
+        kill -KILL ${QEMU_PID[$i]}
+        wait ${QEMU_PID[$i]} 2>/dev/null # silent kill
+        rm -f "${QEMU_FIFO_IN}_${i}" "${QEMU_FIFO_OUT}_${i}"
+    done
+}
+

Signature · 1 line

1.8.3.1

This is an initial, simple live migration test from one running VM to another, using monitor commands.

This is also an example on using the new common.qemu functions for controlling multiple running qemu instances, for tests that need a live qemu vm.

Signed-off-by Jeff Cody <[email protected]>

---
 tests/qemu-iotests/089     | 97 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/089.out | 20 ++++++++++

 tests/qemu-iotests/group   |  1 +
 3 files changed, 118 insertions(+)
 create mode 100755 tests/qemu-iotests/089
 create mode 100644 tests/qemu-iotests/089.out

tests/qemu-iotests/089 new file +97 -0

new file mode 100755
index 0000000..22a7cf1
@@ -0,0 +1,97 @@
+#!/bin/bash
+#
+# Live migration test
+#
+# Performs a migration from one VM to another via monitor commands
+#
+# Copyright (C) 2014 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+# creator
[email protected]
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+status=1	# failure is the default!
+
+MIG_FIFO="${TEST_DIR}/migrate"
+
+_cleanup()
+{
+    rm -f "${MIG_FIFO}"
+    _cleanup_qemu
+	_cleanup_test_img
+
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+. ./common.qemu
+
+_supported_fmt qcow2
+_supported_proto file
+_supported_os Linux
+
+size=1G
+
+IMGOPTS="cluster_size=512" _make_test_img $size
+
+mkfifo "${MIG_FIFO}"
+
+echo
+echo === Starting QEMU VM1 ===
+echo
+
+qemu_comm_method="monitor"
+_launch_qemu -drive file="${TEST_IMG}",cache=none,id=disk
+h1=$QEMU_HANDLE
+
+echo
+echo === Starting QEMU VM2 ===
+echo
+_launch_qemu -drive file="${TEST_IMG}",cache=none,id=disk \
+             -incoming "exec: cat '${MIG_FIFO}'"
+h2=$QEMU_HANDLE
+
+echo
+echo === VM 1: Migrate from VM1 to VM2  ===
+echo
+
+silent=yes
+_send_qemu_cmd $h1 'qemu-io disk "write 0 4M"' "(qemu)"
+echo "vm1: qemu-io disk write complete"
+_send_qemu_cmd $h1 "migrate \"exec: cat > '${MIG_FIFO}'\"" "(qemu)"
+echo "vm1: live migration started"
+qemu_cmd_repeat=20 _send_qemu_cmd $h1 "info migrate" "completed"
+echo "vm1: live migration completed"
+
+echo
+echo === VM 2: Post-migration, write to disk, verify running ===
+echo
+
+_send_qemu_cmd $h2 'qemu-io disk "write 4M 1M"' "(qemu)"
+echo "vm2: qemu-io disk write complete"
+qemu_cmd_repeat=20 _send_qemu_cmd $h2 "info status" "running"
+echo "vm2: qemu process running successfully"
+
+
+echo "*** done"
+rm -f $seq.full
+status=0

tests/qemu-iotests/089.out new file +20 -0

new file mode 100644
index 0000000..4e9e6c9
@@ -0,0 +1,20 @@
+QA output created by 089
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
+
+=== Starting QEMU VM1 ===
+
+
+=== Starting QEMU VM2 ===
+
+
+=== VM 1: Migrate from VM1 to VM2 ===
+
+vm1: qemu-io disk write complete
+vm1: live migration started
+vm1: live migration completed
+
+=== VM 2: Post-migration, write to disk, verify running ===
+
+vm2: qemu-io disk write complete
+vm2: qemu process running successfully
+*** done

tests/qemu-iotests/group +1 -0

index 864643d..73e6b5d 100644
@@ -95,3 +95,4 @@
 086 rw auto quick
 087 rw auto
 088 rw auto
+089 rw auto

Signature · 1 line

1.8.3.1

The new functionality of common.qemu implements the QEMU control and communication functionality that was originally in test 085.

This removes that now-duplicate functionality, and uses the common.qemu functions.

Both tests 019 and 086 need proper quotations to work with pathnames that contain spaces.

Signed-off-by Jeff Cody <[email protected]>

---
 tests/qemu-iotests/019 | 2 +-
 tests/qemu-iotests/086 | 8 ++++----
 2 files changed, 5 insertions(+), 5 deletions(-)

tests/qemu-iotests/019 +1 -1

index e67445c..f5ecbf5 100755
@@ -96,7 +96,7 @@ mv "$TEST_IMG" "$TEST_IMG.orig"
 for backing_option in "-B " "-o backing_file="; do
 
     echo
-    echo Testing conversion with $backing_option$TEST_IMG.base | _filter_testdir | _filter_imgfmt
+    echo Testing conversion with $backing_option"$TEST_IMG.base" | _filter_testdir | _filter_imgfmt
     echo
     $QEMU_IMG convert -O $IMGFMT $backing_option"$TEST_IMG.base" "$TEST_IMG.orig" "$TEST_IMG"
 

tests/qemu-iotests/086 +4 -4

index 48fe85b..d9a80cf 100755
@@ -51,10 +51,10 @@ function run_qemu_img()
 size=128M
 
 _make_test_img $size
-$QEMU_IO -c 'write 0 1M' $TEST_IMG | _filter_qemu_io
-$QEMU_IO -c 'write 2M 1M' $TEST_IMG | _filter_qemu_io
-$QEMU_IO -c 'write 4M 1M' $TEST_IMG | _filter_qemu_io
-$QEMU_IO -c 'write 32M 1M' $TEST_IMG | _filter_qemu_io
+$QEMU_IO -c 'write 0 1M' "$TEST_IMG" | _filter_qemu_io
+$QEMU_IO -c 'write 2M 1M' "$TEST_IMG" | _filter_qemu_io
+$QEMU_IO -c 'write 4M 1M' "$TEST_IMG" | _filter_qemu_io
+$QEMU_IO -c 'write 32M 1M' "$TEST_IMG" | _filter_qemu_io
 
 $QEMU_IMG convert -p -O $IMGFMT -f $IMGFMT "$TEST_IMG" "$TEST_IMG".base  2>&1 |\
     _filter_testdir | sed -e 's/\r/\n/g'

Signature · 1 line

1.8.3.1

The _rm_test_img() function in common.rc did not quote the image file, which left droppings in the scratch directory (and performed a potentially unsafe rm -f).

This adds the necessary quotes.

Signed-off-by Jeff Cody <[email protected]>

---
 tests/qemu-iotests/common.rc | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

tests/qemu-iotests/common.rc +0 -0

index 7f00883..195c564 100644

tests/qemu-iotests/common.rc +2 -2

     if [ "$IMGFMT" = "vmdk" ]; then
         # Remove all the extents for vmdk
-        $QEMU_IMG info $img 2>/dev/null | grep 'filename:' | cut -f 2 -d: \
+        "$QEMU_IMG" info "$img" 2>/dev/null | grep 'filename:' | cut -f 2 -d: \
             | xargs -I {} rm -f "{}"
     fi
-    rm -f $img
+    rm -f "$img"
 }
 
 _cleanup_test_img()

Signature · 1 line

1.8.3.1

Quoted 7 lines · Jeff Cody

This is an initial, simple live migration test from one running VM to another, using monitor commands.

This is also an example on using the new common.qemu functions for controlling multiple running qemu instances, for tests that need a live qemu vm.

Quoted 4 lines

Signed-off-by Jeff Cody <[email protected]>

---
 tests/qemu-iotests/089     | 97 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/089.out | 20 ++++++++++

I used 089 in my last image fleecing series, (originally 083 but already used). So one of us need to shift the case number.

Quoted 148 lines

 tests/qemu-iotests/group   |  1 +
 3 files changed, 118 insertions(+)
 create mode 100755 tests/qemu-iotests/089
 create mode 100644 tests/qemu-iotests/089.out

tests/qemu-iotests/089 new file +97 -0

new file mode 100755
index 0000000..22a7cf1
@@ -0,0 +1,97 @@
+#!/bin/bash
+#
+# Live migration test
+#
+# Performs a migration from one VM to another via monitor commands
+#
+# Copyright (C) 2014 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+# creator
[email protected]
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+status=1	# failure is the default!
+
+MIG_FIFO="${TEST_DIR}/migrate"
+
+_cleanup()
+{
+    rm -f "${MIG_FIFO}"
+    _cleanup_qemu
+	_cleanup_test_img
+
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+. ./common.qemu
+
+_supported_fmt qcow2
+_supported_proto file
+_supported_os Linux
+
+size=1G
+
+IMGOPTS="cluster_size=512" _make_test_img $size
+
+mkfifo "${MIG_FIFO}"
+
+echo
+echo === Starting QEMU VM1 ===
+echo
+
+qemu_comm_method="monitor"
+_launch_qemu -drive file="${TEST_IMG}",cache=none,id=disk
+h1=$QEMU_HANDLE
+
+echo
+echo === Starting QEMU VM2 ===
+echo
+_launch_qemu -drive file="${TEST_IMG}",cache=none,id=disk \
+             -incoming "exec: cat '${MIG_FIFO}'"
+h2=$QEMU_HANDLE
+
+echo
+echo === VM 1: Migrate from VM1 to VM2  ===
+echo
+
+silent=yes
+_send_qemu_cmd $h1 'qemu-io disk "write 0 4M"' "(qemu)"
+echo "vm1: qemu-io disk write complete"
+_send_qemu_cmd $h1 "migrate \"exec: cat > '${MIG_FIFO}'\"" "(qemu)"
+echo "vm1: live migration started"
+qemu_cmd_repeat=20 _send_qemu_cmd $h1 "info migrate" "completed"
+echo "vm1: live migration completed"
+
+echo
+echo === VM 2: Post-migration, write to disk, verify running ===
+echo
+
+_send_qemu_cmd $h2 'qemu-io disk "write 4M 1M"' "(qemu)"
+echo "vm2: qemu-io disk write complete"
+qemu_cmd_repeat=20 _send_qemu_cmd $h2 "info status" "running"
+echo "vm2: qemu process running successfully"
+
+
+echo "*** done"
+rm -f $seq.full
+status=0

tests/qemu-iotests/089.out new file +20 -0

new file mode 100644
index 0000000..4e9e6c9
@@ -0,0 +1,20 @@
+QA output created by 089
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 
+
+=== Starting QEMU VM1 ===
+
+
+=== Starting QEMU VM2 ===
+
+
+=== VM 1: Migrate from VM1 to VM2 ===
+
+vm1: qemu-io disk write complete
+vm1: live migration started
+vm1: live migration completed
+
+=== VM 2: Post-migration, write to disk, verify running ===
+
+vm2: qemu-io disk write complete
+vm2: qemu process running successfully
+*** done

tests/qemu-iotests/group +1 -0

index 864643d..73e6b5d 100644
@@ -95,3 +95,4 @@
 086 rw auto quick
 087 rw auto
 088 rw auto
+089 rw auto

Signature · 1 line

1.8.3.1

Quoted 11 lines · Jeff Cody

The _rm_test_img() function in common.rc did not quote the image file, which left droppings in the scratch directory (and performed a potentially unsafe rm -f).

This adds the necessary quotes.

Signed-off-by Jeff Cody <[email protected]>

---
 tests/qemu-iotests/common.rc | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

tests/qemu-iotests/common.rc _rm_test_img() · line 178

Quoted 6 lines

tests/qemu-iotests/common.rc +0 -0

index 7f00883..195c564 100644
@@ -178,10 +178,10 @@ _rm_test_img()
     local img=$1

Since we are quoting $img, should we quote $1 as well?

Quoted 16 lines

tests/qemu-iotests/common.rc +2 -2

     if [ "$IMGFMT" = "vmdk" ]; then
         # Remove all the extents for vmdk
-        $QEMU_IMG info $img 2>/dev/null | grep 'filename:' | cut -f 2 -d: \
+        "$QEMU_IMG" info "$img" 2>/dev/null | grep 'filename:' | cut -f 2 -d: \
             | xargs -I {} rm -f "{}"
     fi
-    rm -f $img
+    rm -f "$img"
 }
 
 _cleanup_test_img()

Signature · 1 line

1.8.3.1

Quoted 6 lines

tests/qemu-iotests/common.rc _rm_test_img() · line 178

Quoted 3 lines

+++ b/tests/qemu-iotests/common.rc

tests/qemu-iotests/common.rc +0 -0

@@ -178,10 +178,10 @@ _rm_test_img()
     local img=$1

Since we are quoting $img, should we quote $1 as well?

I believe not, because variable assignment won't undergo all the shell expansions. Notably, in variable assignment word splitting is not performed on the parameter expansion on the argument immediately to the right of the '='. Quote removal, however, will still be performed. So img=$1 and img="$1" are identical once processed.

Quoted 6 lines · Jeff Cody

Quoted 6 lines

tests/qemu-iotests/common.rc _rm_test_img() · line 178

Quoted 3 lines

+++ b/tests/qemu-iotests/common.rc

tests/qemu-iotests/common.rc +0 -0

@@ -178,10 +178,10 @@ _rm_test_img()
     local img=$1

Since we are quoting $img, should we quote $1 as well?

http://austingroupbugs.net/view.php?id=351

But even with the notion of an assignment-context argument added to a future version of POSIX, the reality is that given the present standard, it's safer to either use "" to ensure no word splitting:

Quoted 10 lines · Eric Blake

Quoted 6 lines · Jeff Cody

Quoted 6 lines

tests/qemu-iotests/common.rc _rm_test_img() · line 178

Quoted 3 lines

+++ b/tests/qemu-iotests/common.rc

tests/qemu-iotests/common.rc +0 -0

@@ -178,10 +178,10 @@ _rm_test_img()
     local img=$1

Since we are quoting $img, should we quote $1 as well?

Quoted 5 lines

http://austingroupbugs.net/view.php?id=351

But even with the notion of an assignment-context argument added to a future version of POSIX, the reality is that given the present standard, it's safer to either use "" to ensure no word splitting:

Well, if you were trying to be portable to multiple shells, then it would matter. But as this script is explicitly being run under /bin/bash, and as bash already has support for declaration utilities where local is one such utility, your script as written is safe without "" in the arguments to local. So I'm fine whether you choose to change it in a respin or to leave it as written in this version.

Quoted 13 lines · Jeff Cody

This creates some common functions for bash language qemu-iotests to control, and communicate with, a running QEMU process.

4 functions are introduced:

1. _launch_qemu()
This launches the QEMU process(es), and sets up the file descriptors and fifos for communication. You can choose to launch each QEMU process listening for either QMP or HMP monitor. You can call this function multiple times, and save the handle returned from each. The returned handle is in $QEMU_HANDLE. You must copy this value.

Quoted 6 lines

Commands 2 and 3 use the handle received from _launch_qemu(), to talk to the appropriate process.

2. _send_qemu_cmd()
Sends a command string, specified by $2, to QEMU. If $2 is non-NULL, _send_qemu_cmd() will wait to receive $2 as a

Do you mean $3 in this sentence?

Quoted 177 lines

required result string from QEMU. Failure to receive $3 will cause the test to fail. The command can optionally be retried $qemu_cmd_repeat number of times.

3. _timed_wait_for()
Waits for a response, for up to a default of 10 seconds. If $2 is not seen in that time (anywhere in the response), then the test fails. Primarily used by _send_qemu_cmd, but could be useful standalone, as well. To prevent automatic exit (and therefore test failure), set $qemu_wait_no_error to a non-NULL value. If $silent is a non-NULL value, then output to stdout will be suppressed.

4. _cleanup_qemu()
Kills the running QEMU processes, and removes the fifos.

Signed-off-by Jeff Cody <[email protected]>

---
 tests/qemu-iotests/common.qemu | 195 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 195 insertions(+)
 create mode 100644 tests/qemu-iotests/common.qemu

tests/qemu-iotests/common.qemu new file +149 -0

new file mode 100644
index 0000000..12c42f1
@@ -0,0 +1,195 @@
+#!/bin/bash
+#
+# This allows for launching of multiple QEMU instances, with independent
+# communication possible to each instance.
+#
+# Each instance can choose, at launch, to use either the QMP or the
+# HMP (monitor) interface.
+#
+# All instances are cleaned up via _cleanup_qemu, including killing the
+# running qemu instance.
+#
+# Copyright (C) 2014 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+QEMU_COMM_TIMEOUT=10
+
+QEMU_FIFO_IN="${TEST_DIR}/qmp-in-$$"
+QEMU_FIFO_OUT="${TEST_DIR}/qmp-out-$$"
+
+QEMU_PID=
+_QEMU_HANDLE=0
+QEMU_HANDLE=0
+
+# If bash version is >= 4.1, these will be overwritten and dynamic
+# file descriptor values assigned.
+_out_fd=3
+_in_fd=4
+
+# Wait for expected QMP response from QEMU.  Will time out
+# after 10 seconds, which counts as failure.
+#
+# Override QEMU_COMM_TIMEOUT for a timeout different than the
+# default 10 seconds
+#
+# $1: The handle to use
+# $2+ All remaining arguments comprise the string to search for
+#    in the response.
+#
+# If $silent is set to anything but an empty string, then
+# response is not echoed out.
+function _timed_wait_for()
+{
+    local h=${1}
+    shift
+
+    QEMU_STATUS[$h]=0
+    while read -t ${QEMU_COMM_TIMEOUT} resp <&${QEMU_OUT[$h]}
+    do
+        if [ -z "${silent}" ]; then
+            echo "${resp}" | _filter_testdir | _filter_qemu \
+                           | _filter_qemu_io | _filter_qmp
+        fi
+        grep -q "${*}" < <(echo ${resp})
+        if [ $? -eq 0 ]; then
+            return
+        fi
+    done
+    QEMU_STATUS[$h]=-1
+    if [ -z "${qemu_wait_no_error}" ]; then
+        echo "Timeout waiting for ${*} on handle ${h}"
+        exit 1  # Timeout means the test failed
+    fi
+}
+
+
+# Sends QMP or HMP command to QEMU, and waits for the expected response
+#
+# $1:       QEMU handle to use
+# $2:       String of the QMP command to send
+# ${@: -1}  (Last string passed)
+#             String that the QEMU response should contain. If it is a null
+#             string, do not wait for a response
+#
+# Set qemu_cmd_repeat to the number of times to repeat the cmd
+# until either timeout, or a response.  If it is not set, or <=0,
+# then the command is only sent once.
+#
+function _send_qemu_cmd()
+{
+    local h=${1}
+    local count=1
+    local cmd=
+    local use_error=
+    shift
+
+    if [ ${qemu_cmd_repeat} -gt 0 ] 2>/dev/null; then
+        count=${qemu_cmd_repeat}
+        use_error="no"
+    fi
+    # This array element extraction is done to accomodate pathnames with spaces
+    cmd=${@: 1:${#@}-1}
+    shift $(($# - 1))
+
+    while [ ${count} -gt 0 ]
+    do
+        echo "${cmd}" >&${QEMU_IN[${h}]}
+        if [ -n "${1}" ]; then
+            qemu_wait_no_error=${use_error} _timed_wait_for ${h} "${1}"
+            if [ ${QEMU_STATUS[$h]} -eq 0 ]; then
+                return
+            fi
+        fi
+        let count--;
+    done
+    if [ ${QEMU_STATUS[$h]} -ne 0 ]; then
+        echo "Timeout waiting for ${1} on handle ${h}"
+        exit 1 #Timeout means the test failed
+    fi
+}
+
+
+# Launch a QEMU process.
+#
+# Input parameters:
+# $qemu_comm_method: set this variable to 'monitor' (case insensitive)
+#                    to use the QEMU HMP monitor for communication.
+#                    Otherwise, the default of QMP is used.
+# Returns:
+# $QEMU_HANDLE: set to a handle value to communicate with this QEMU instance.
+#
+function _launch_qemu()
+{
+    local comm=
+    local fifo_out=
+    local fifo_in=
+
+    if (shopt -s nocasematch; [[ "${qemu_comm_method}" == "monitor" ]])
+    then
+        comm="-monitor stdio"
+    else
+        local qemu_comm_method="qmp"
+        comm="-monitor none -qmp stdio"
+    fi
+
+    fifo_out=${QEMU_FIFO_OUT}_${_QEMU_HANDLE}
+    fifo_in=${QEMU_FIFO_IN}_${_QEMU_HANDLE}

tests/qemu-iotests/common.qemu line 155

Quoted 6 lines

tests/qemu-iotests/common.qemu new file +6 -0

+    mkfifo "${fifo_out}"
+    mkfifo "${fifo_in}"
+
+    "${QEMU}" -nographic -serial none ${comm} "${@}" 2>&1 \
+                                                     >"${fifo_out}" \
+                                                     <"${fifo_in}" &

Maybe add "-machine accel=qtest"?

Quoted 45 lines

tests/qemu-iotests/common.qemu new file +40 -0

+    QEMU_PID[${_QEMU_HANDLE}]=$!
+
+    if [ "${BASH_VERSINFO[0]}" -ge "4" ] && [ "${BASH_VERSINFO[1]}" -ge "1" ]
+    then
+        # bash >= 4.1 required for automatic fd
+        exec {_out_fd}<"${fifo_out}"
+        exec {_in_fd}>"${fifo_in}"
+    else
+        let _out_fd++
+        let _in_fd++
+        eval "exec ${_out_fd}<'${fifo_out}'"
+        eval "exec ${_in_fd}>'${fifo_in}'"
+    fi
+
+    QEMU_OUT[${_QEMU_HANDLE}]=${_out_fd}
+    QEMU_IN[${_QEMU_HANDLE}]=${_in_fd}
+    QEMU_STATUS[${_QEMU_HANDLE}]=0
+
+    if [ "${qemu_comm_method}" == "qmp" ]
+    then
+        # Don't print response, since it has version information in it
+        silent=yes _timed_wait_for ${_QEMU_HANDLE} "capabilities"
+    fi
+    QEMU_HANDLE=${_QEMU_HANDLE}
+    let _QEMU_HANDLE++
+}
+
+
+# Silenty kills the QEMU process
+function _cleanup_qemu()
+{
+    # QEMU_PID[], QEMU_IN[], QEMU_OUT[] all use same indices
+    for i in "${!QEMU_OUT[@]}"
+    do
+        kill -KILL ${QEMU_PID[$i]}
+        wait ${QEMU_PID[$i]} 2>/dev/null # silent kill
+        rm -f "${QEMU_FIFO_IN}_${i}" "${QEMU_FIFO_OUT}_${i}"
+    done
+}
+

Signature · 1 line

1.8.3.1

Quoted 3 lines · Jeff Cody

The new functionality of common.qemu implements the QEMU control and communication functionality that was originally in test 085.

Quoted 3 lines

This removes that now-duplicate functionality, and uses the common.qemu functions.

Just a note.

A quick grep shows 067, 071, 081 and 087 are also bash cases with QEMU process. Not necessarily in this series but they are also candidates too convert.

Quoted 17 lines · Eric Blake

Quoted 15 lines · Eric Blake

Quoted 6 lines · Jeff Cody

Quoted 6 lines

tests/qemu-iotests/common.rc _rm_test_img() · line 178

Quoted 3 lines

+++ b/tests/qemu-iotests/common.rc

tests/qemu-iotests/common.rc +0 -0

@@ -178,10 +178,10 @@ _rm_test_img()
     local img=$1

Since we are quoting $img, should we quote $1 as well?

http://austingroupbugs.net/view.php?id=351

But even with the notion of an assignment-context argument added to a future version of POSIX, the reality is that given the present standard, it's safer to either use "" to ensure no word splitting:

Quoted 6 lines

Well, if you were trying to be portable to multiple shells, then it would matter. But as this script is explicitly being run under /bin/bash, and as bash already has support for declaration utilities where local is one such utility, your script as written is safe without "" in the arguments to local. So I'm fine whether you choose to change it in a respin or to leave it as written in this version.

Hi Eric,

Thanks - I consulted specifically with just the bash documentation, so you are right, this script (and likely most of qemu-iotests) is bash-only.

That particular line is context as well, and not an actual change - so while it may be a good idea to quote it to make the scripts closer to posix-only, my guess is there are quite a few similar lines throughout all the qemu-iotests scripts.

Given that, if we address that it would probably make sense to do that in a bash->posix conversion series for all the scripts (likely a low priority, however).