GlusterFS support in QEMU - v10

Hi,

This is v10 of the patchset to support GlusterFS backend from QEMU.

Changes in v10
--------------
- Included another fix to qemu_aio_wait() from Paolo in the series.
- A few more fixes to the volume and image parsing code.
- Removed unnecessary unescaping of URI query parameters.
- Fail if server or port are specified for unix type of transport.
- Save errno in case glfs_fini() fails in qemu_gluster_init().
- s/qemu_gluster_send_pipe/qemu_write_full
- Remove unnecessary clearing of O_ACCMODE from open_flags in qemu_gluster_open.

Previous versions can be found here:

v9 - http://lists.nongnu.org/archive/html/qemu-devel/2012-09/msg04016.html
v8 - http://lists.gnu.org/archive/html/qemu-devel/2012-09/msg04007.html
v7 - http://lists.nongnu.org/archive/html/qemu-devel/2012-09/msg02532.html
v6 - http://lists.nongnu.org/archive/html/qemu-devel/2012-08/msg01536.html
v5 - http://lists.nongnu.org/archive/html/qemu-devel/2012-08/msg01023.html
v4 - http://lists.nongnu.org/archive/html/qemu-devel/2012-08/msg00147.html
v3 - http://lists.nongnu.org/archive/html/qemu-devel/2012-07/msg03322.html
v2 - http://lists.nongnu.org/archive/html/qemu-devel/2012-07/msg02718.html
v1 - http://lists.nongnu.org/archive/html/qemu-devel/2012-06/msg01745.html

Regards,
Bharata.

aio: Fix qemu_aio_wait() to maintain correct walking_handlers count

From: Paolo Bonzini <[email protected]>

Fix qemu_aio_wait() to ensure that registered aio handlers don't get deleted when they are still active. This is ensured by maintaning the right count of walking_handlers.

---

 aio.c |    8 ++++----
 1 files changed, 4 insertions(+), 4 deletions(-)

index 0a9eb10..99b8b72 100644
@@ -119,7 +119,7 @@ bool qemu_aio_wait(void)
         return true;
     }
 
-    walking_handlers = 1;
+    walking_handlers++;
 
     FD_ZERO(&rdfds);
     FD_ZERO(&wrfds);

@@ -147,7 +147,7 @@ bool qemu_aio_wait(void)
         }
     }
 
-    walking_handlers = 0;
+    walking_handlers--;
 
     /* No AIO operations?  Get us out of here */
     if (!busy) {

@@ -159,7 +159,7 @@ bool qemu_aio_wait(void)
 
     /* if we have any readable fds, dispatch event */
     if (ret > 0) {
-        walking_handlers = 1;
+        walking_handlers++;
 
         /* we have to walk very carefully in case
          * qemu_aio_set_fd_handler is called while we're walking */

@@ -187,7 +187,7 @@ bool qemu_aio_wait(void)
             }
         }
 
-        walking_handlers = 0;
+        walking_handlers--;
     }
 
     return true;

aio: Another fix to the walking_handlers logic

From: Paolo Bonzini <[email protected]>

The AIO dispatch loop will call QLIST_REMOVE and g_free even if there are other pending calls to qemu_aio_wait outside the current one.

---

 aio.c |   10 +++++-----
 1 files changed, 5 insertions(+), 5 deletions(-)

index 99b8b72..c738a4e 100644
@@ -159,14 +159,14 @@ bool qemu_aio_wait(void)
 
     /* if we have any readable fds, dispatch event */
     if (ret > 0) {
-        walking_handlers++;
-
         /* we have to walk very carefully in case
          * qemu_aio_set_fd_handler is called while we're walking */
         node = QLIST_FIRST(&aio_handlers);
         while (node) {
             AioHandler *tmp;
 
+            walking_handlers++;
+
             if (!node->deleted &&
                 FD_ISSET(node->fd, &rdfds) &&
                 node->io_read) {

@@ -181,13 +181,13 @@ bool qemu_aio_wait(void)
             tmp = node;
             node = QLIST_NEXT(node, node);
 
-            if (tmp->deleted) {
+            walking_handlers--;
+
+            if (!walking_handlers && tmp->deleted) {
                 QLIST_REMOVE(tmp, node);
                 g_free(tmp);
             }
         }
-
-        walking_handlers--;
     }
 
     return true;

qemu: URI parsing library

From: Paolo Bonzini <[email protected]>

Add a new URI parsing library to QEMU. The code has been borrowed from libxml2 and libvirt.

---

 Makefile.objs |    2 
 uri.c         | 2249 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 uri.h         |  113 +++
 3 files changed, 2363 insertions(+), 1 deletions(-)
 create mode 100644 uri.c
 create mode 100644 uri.h

index 4412757..7c1c682 100644
@@ -42,7 +42,7 @@ coroutine-obj-$(CONFIG_WIN32) += coroutine-win32.o
 # block-obj-y is code used by both qemu system emulation and qemu-img
 
 block-obj-y = cutils.o iov.o cache-utils.o qemu-option.o module.o async.o
-block-obj-y += nbd.o block.o aio.o aes.o qemu-config.o qemu-progress.o qemu-sockets.o
+block-obj-y += nbd.o block.o aio.o aes.o qemu-config.o qemu-progress.o qemu-sockets.o uri.o
 block-obj-y += $(coroutine-obj-y) $(qobject-obj-y) $(version-obj-y)
 block-obj-$(CONFIG_POSIX) += posix-aio-compat.o
 block-obj-$(CONFIG_LINUX_AIO) += linux-aio.o

IMHO, you should also be importing the URI code test suite from libvirt to verify that the way you merged/changed the codebases did not break anything.

Daniel

|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Il 27/09/2012 16:36, Daniel P. Berrange ha scritto:

Yes, can be done separately though.

Paolo

In this case I disagree. The URI parsing code here is complex enough that I don't think any reviewer can credibly claim to spot flaws that might have been introduced when combing the libvirt + libxml2 URI code parts. A test suite is the only way to validate this kind of complex code IMHO and so should be included in this patch.

Daniel

|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Il 28/09/2012 10:39, Daniel P. Berrange ha scritto:

The libvirt testsuite has ~10 testcases for URI, one of them for IPv6-specific behavior that is not part of uri.c, and 13 for parsing query parameters. At least for the URI part, this would be false safety.

FWIW, this is what I had "tested" the code with:

void test(const char *x)

{
    URI *uri = uri_parse(x);
    QueryParams *qp;
    char *out;

    if (!uri) {
        printf ("INVALID: %s\n", x);
        return;
    }

    /* Escape the parameters.  */
    qp = query_params_parse(uri->query);
    g_free(uri->query);
    uri->query = query_params_to_string(qp);
    query_params_free(qp);

    out = uri_to_string(uri);
    uri_free(uri);
    printf("VALID: %s\n", out);
    g_free(out);
}

int main()

{
    test("gluster+unix:///b?c=d");
    test("gluster+tcp://user:pwd@foo:80/b?c=d");
    test("gluster+tcp://user:pwd@foo:8b0/b?c=d");
    test("http://a/b?c=d");
    test("http://a/b?c=/d/e");
    test("http://a/b?c=/d/e&f");
    test("http://a/b?c=/d/e&f=&&ggg");
    return 0;
}

Paolo

configure: Add a config option for GlusterFS as block backend

From: Bharata B Rao <[email protected]>

GlusterFS support in QEMU depends on libgfapi, libgfrpc and libgfxdr provided by GlusterFS.

---

 configure |   35 +++++++++++++++++++++++++++++++++++
 1 files changed, 35 insertions(+), 0 deletions(-)

index 1b86517..eb1af9f 100755
@@ -219,6 +219,7 @@ want_tools="yes"
 libiscsi=""
 coroutine=""
 seccomp=""
+glusterfs=""
 
 # parse CC options first
 for opt do

@@ -856,6 +857,10 @@ for opt do
   ;;
   --disable-seccomp) seccomp="no"
   ;;
+  --disable-glusterfs) glusterfs="no"
+  ;;
+  --enable-glusterfs) glusterfs="yes"
+  ;;
   *) echo "ERROR: unknown option $opt"; show_help="yes"
   ;;
   esac

@@ -1128,6 +1133,8 @@ echo "  --disable-seccomp        disable seccomp support"
 echo "  --enable-seccomp         enables seccomp support"
 echo "  --with-coroutine=BACKEND coroutine backend. Supported options:"
 echo "                           gthread, ucontext, sigaltstack, windows"
+echo "  --enable-glusterfs       enable GlusterFS backend"
+echo "  --disable-glusterfs      disable GlusterFS backend"
 echo ""
 echo "NOTE: The object files are built at the place where configure is launched"
 exit 1

@@ -2307,6 +2314,29 @@ EOF
   fi
 fi
 
+##########################################
+# glusterfs probe
+if test "$glusterfs" != "no" ; then
+  cat > $TMPC <<EOF
+#include <glusterfs/api/glfs.h>
+int main(void) {
+    (void) glfs_new("volume");
+    return 0;
+}
+EOF
+  glusterfs_libs="-lgfapi -lgfrpc -lgfxdr"
+  if compile_prog "" "$glusterfs_libs" ; then
+    glusterfs=yes
+    libs_tools="$glusterfs_libs $libs_tools"
+    libs_softmmu="$glusterfs_libs $libs_softmmu"
+  else
+    if test "$glusterfs" = "yes" ; then
+      feature_not_found "GlusterFS backend support"
+    fi
+    glusterfs=no
+  fi
+fi
+
 #
 # Check for xxxat() functions when we are building linux-user
 # emulator.  This is done because older glibc versions don't

@@ -3174,6 +3204,7 @@ echo "libiscsi support  $libiscsi"
 echo "build guest agent $guest_agent"
 echo "seccomp support   $seccomp"
 echo "coroutine backend $coroutine_backend"
+echo "GlusterFS support $glusterfs"
 
 if test "$sdl_too_old" = "yes"; then
 echo "-> Your SDL version is too old - please upgrade to have SDL support"

@@ -3520,6 +3551,10 @@ if test "$has_environ" = "yes" ; then
   echo "CONFIG_HAS_ENVIRON=y" >> $config_host_mak
 fi
 
+if test "$glusterfs" = "yes" ; then
+  echo "CONFIG_GLUSTERFS=y" >> $config_host_mak
+fi
+
 # USB host support
 case "$usb" in
 linux)

block: Support GlusterFS as a QEMU block backend.

From: Bharata B Rao <[email protected]>

This patch adds gluster as the new block backend in QEMU. This gives QEMU the ability to boot VM images from gluster volumes. Its already possible to boot from VM images on gluster volumes using FUSE mount, but this patchset provides the ability to boot VM images from gluster volumes by by-passing the FUSE layer in gluster. This is made possible by using libgfapi routines to perform IO on gluster volumes directly.

VM Image on gluster volume is specified like this:

file=gluster[+transport]://[server[:port]]/volname/image[?socket=...]

'gluster' is the protocol.

'transport' specifies the transport type used to connect to gluster management daemon (glusterd). Valid transport types are tcp, unix and rdma. If a transport type isn't specified, then tcp type is assumed.

'server' specifies the server where the volume file specification for the given volume resides. This can be either hostname, ipv4 address or ipv6 address. ipv6 address needs to be within square brackets [ ]. If transport type is 'unix', then 'server' field should not be specifed. The 'socket' field needs to be populated with the path to unix domain socket.

'port' is the port number on which glusterd is listening. This is optional and if not specified, QEMU will send 0 which will make gluster to use the default port. If the transport type is unix, then 'port' should not be specified.

'volname' is the name of the gluster volume which contains the VM image.

'image' is the path to the actual VM image that resides on gluster volume.

Examples:

file=gluster://1.2.3.4/testvol/a.img file=gluster+tcp://1.2.3.4/testvol/a.img file=gluster+tcp://1.2.3.4:24007/testvol/dir/a.img file=gluster+tcp://[1:2:3:4:5:6:7:8]/testvol/dir/a.img file=gluster+tcp://[1:2:3:4:5:6:7:8]:24007/testvol/dir/a.img file=gluster+tcp://server.domain.com:24007/testvol/dir/a.img file=gluster+unix:///testvol/dir/a.img?socket=/tmp/glusterd.socket file=gluster+rdma://1.2.3.4:24007/testvol/a.img

---

 block/Makefile.objs |    1 
 block/gluster.c     |  624 +++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 625 insertions(+), 0 deletions(-)
 create mode 100644 block/gluster.c

index b5754d3..a1ae67f 100644
@@ -9,3 +9,4 @@ block-obj-$(CONFIG_POSIX) += raw-posix.o
 block-obj-$(CONFIG_LIBISCSI) += iscsi.o
 block-obj-$(CONFIG_CURL) += curl.o
 block-obj-$(CONFIG_RBD) += rbd.o
+block-obj-$(CONFIG_GLUSTERFS) += gluster.o

Am 27.09.2012 15:55, schrieb Bharata B Rao:

Thanks, applied all to the block branch.

Kevin