Skip to contentSkip to conversation

Make qemu-img/qemu-nbd/qemu-io CLI more flexible

PATCH Superseded Daniel P. Berrange 3 participants 29 messages Dec 22, 2015 – Dec 24, 2015

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover Make qemu-img/qemu-nbd/qemu-io CLI more flexible

This series of patches expands the syntax of the qemu-img, qemu-nbd and qemu-io commands to make them more flexible.

First all three gain a --object parameter, which allows instantiation of user creatable object types. The immediate use case is to allow for creation of the 'secret' object type to pass passwords for curl, iscsi and rbd drivers. For qemu-nbd this will also be needed to create TLS certificates for encryption support.

Then all three gain a '--source' parameter which exposes the full set of block backend parameters in a single arg. This obsoletes the current approach where the filename is specified as a positional arg and then custom CLI args need to be created for each block parameter, eg obsoletes need for things like '-f format'. Again the immediate use case is to allow the user to specify the ID of the 'secret' object then just created.

These patches were previousl sent as part of a larger series here:

http://lists.gnu.org/archive/html/qemu-devel/2015-10/msg04365.html

I split them off, since some parts of that series are merged, and this set of patches is fairly independant of the rest, so doesn't need to gate on them

Daniel P. Berrange (7):
qom: add user_creatable_add & user_creatable_del methods
qemu-img: add support for --object command line arg
qemu-nbd: add support for --object command line arg
qemu-io: add support for --object command line arg
qemu-io: allow specifying image as a set of options args
qemu-nbd: allow specifying image as a set of options args
qemu-img: allow specifying image as a set of options args

 hmp.c                           |  11 +-
 include/monitor/monitor.h       |   3 -
 include/qemu/option.h           |   1 +
 include/qom/object_interfaces.h |  31 ++
 qemu-img-cmds.hx                |  44 +--
 qemu-img.c                      | 772 +++++++++++++++++++++++++++++++++++-----
 qemu-img.texi                   |   8 +
 qemu-io.c                       | 124 ++++++-
 qemu-nbd.c                      | 142 +++++++-
 qemu-nbd.texi                   |   7 +
 qmp.c                           |  75 +---
 qom/object_interfaces.c         |  76 ++++
 util/qemu-option.c              |   6 +
 vl.c                            |   8 +-
 14 files changed, 1120 insertions(+), 188 deletions(-)
Signature · 1 line
2.5.0
#1 lore raw

PATCH 1/7 qom: add user_creatable_add & user_creatable_del methods

The QMP monitor code has two helper methods object_add and qmp_object_del that are called from several places in the code (QMP, HMP and main emulator startup).

We soon need to use this code from qemu-img, qemu-io and qemu-nbd too, but don't want those to depend on the monitor.

To avoid this, move object_add to user_creatable_add an qmp_object_del to user_creatable_del, to the object_interfaces.c file where they can be easily shared with all users of QOM

---
 hmp.c                           | 11 ++++--
 include/monitor/monitor.h       |  3 --
 include/qom/object_interfaces.h | 31 +++++++++++++++++
 qmp.c                           | 75 ++++------------------------------------
 qom/object_interfaces.c         | 76 +++++++++++++++++++++++++++++++++++++++++
 vl.c                            |  8 +++--
 6 files changed, 127 insertions(+), 77 deletions(-)
hmp.c +8 -3
index c2b2c16..57fa3f5 100644
@@ -29,6 +29,7 @@
 #include "qapi/string-output-visitor.h"
 #include "qapi/util.h"
 #include "qapi-visit.h"
+#include "qom/object_interfaces.h"
 #include "ui/console.h"
 #include "block/qapi.h"
 #include "qemu-io.h"
@@ -1670,6 +1671,7 @@ void hmp_object_add(Monitor *mon, const QDict *qdict)
     void *dummy = NULL;
     OptsVisitor *ov;
     QDict *pdict;
+    Object *obj = NULL;
 
     opts = qemu_opts_from_qdict(qemu_find_opts("object"), qdict, &err);
     if (err) {
@@ -1696,12 +1698,12 @@ void hmp_object_add(Monitor *mon, const QDict *qdict)
         goto out_end;
     }
 
-    object_add(type, id, pdict, opts_get_visitor(ov), &err);
+    obj = user_creatable_add(type, id, pdict, opts_get_visitor(ov), &err);
 
 out_end:
     visit_end_struct(opts_get_visitor(ov), &err_end);
     if (!err && err_end) {
-        qmp_object_del(id, NULL);
+        user_creatable_del(id, NULL);
     }
     error_propagate(&err, err_end);
 out_clean:
@@ -1712,6 +1714,9 @@ out_clean:
     g_free(id);
     g_free(type);
     g_free(dummy);
+    if (obj) {
+        object_unref(obj);
+    }
 
 out:
     hmp_handle_error(mon, &err);
@@ -1944,7 +1949,7 @@ void hmp_object_del(Monitor *mon, const QDict *qdict)
     const char *id = qdict_get_str(qdict, "id");
     Error *err = NULL;
 
-    qmp_object_del(id, &err);
+    user_creatable_del(id, &err);
     hmp_handle_error(mon, &err);
 }
 
include/monitor/monitor.h +0 -3
index 91b95ae..aa0f373 100644
@@ -43,9 +43,6 @@ void monitor_read_command(Monitor *mon, int show_prompt);
 int monitor_read_password(Monitor *mon, ReadLineFunc *readline_func,
                           void *opaque);
 
-void object_add(const char *type, const char *id, const QDict *qdict,
-                Visitor *v, Error **errp);
-
 AddfdInfo *monitor_fdset_add_fd(int fd, bool has_fdset_id, int64_t fdset_id,
                                 bool has_opaque, const char *opaque,
                                 Error **errp);
include/qom/object_interfaces.h +31 -0
index 283ae0d..3e2afeb 100644
@@ -2,6 +2,8 @@
 #define OBJECT_INTERFACES_H
 
 #include "qom/object.h"
+#include "qapi/qmp/qdict.h"
+#include "qapi/visitor.h"
 
 #define TYPE_USER_CREATABLE "user-creatable"
 
@@ -72,4 +74,33 @@ void user_creatable_complete(Object *obj, Error **errp);
  * from implements USER_CREATABLE interface.
  */
 bool user_creatable_can_be_deleted(UserCreatable *uc, Error **errp);
+
+/**
+ * user_creatable_add:
+ * @type: the object type name
+ * @id: the unique ID for the object
+ * @qdict: the object parameters
+ * @v: the visitor
+ * @errp: if an error occurs, a pointer to an area to store the error
+ *
+ * Create an instance of the user creatable object @type, placing
+ * it in the object composition tree with name @id, initializing
+ * it with properties from @qdict
+ *
+ * Returns: the newly created object or NULL on error
+ */
+Object *user_creatable_add(const char *type, const char *id,
+                           const QDict *qdict,
+                           Visitor *v, Error **errp);
+
+/**
+ * user_creatable_del:
+ * @id: the unique ID for the object
+ * @errp: if an error occurs, a pointer to an area to store the error
+ *
+ * Delete an instance of the user creatable object identified
+ * by @id.
+ */
+void user_creatable_del(const char *id, Error **errp);
+
 #endif
qmp.c +6 -69

98 lines not on this page · raw message

qom/object_interfaces.c +76 -0

82 lines not on this page · raw message

vl.c +6 -2
index 5aaea77..812e039 100644
@@ -2834,6 +2834,7 @@ static int object_create(void *opaque, QemuOpts *opts, Error **errp)
     OptsVisitor *ov;
     QDict *pdict;
     bool (*type_predicate)(const char *) = opaque;
+    Object *obj = NULL;
 
     ov = opts_visitor_new(opts);
     pdict = qemu_opts_to_qdict(opts, NULL);
@@ -2858,13 +2859,13 @@ static int object_create(void *opaque, QemuOpts *opts, Error **errp)
         goto out;
     }
 
-    object_add(type, id, pdict, opts_get_visitor(ov), &err);
+    obj = user_creatable_add(type, id, pdict, opts_get_visitor(ov), &err);
     if (err) {
         goto out;
     }
     visit_end_struct(opts_get_visitor(ov), &err);
     if (err) {
-        qmp_object_del(id, NULL);
+        user_creatable_del(id, NULL);
     }
 
 out:
@@ -2874,6 +2875,9 @@ out:
     g_free(id);
     g_free(type);
     g_free(dummy);
+    if (obj) {
+        object_unref(obj);
+    }
     if (err) {
         error_report_err(err);
         return -1;
Signature · 1 line
2.5.0
Reviewed-by on v1 1/7
#2 lore raw

Re: [Qemu-devel] [PATCH 1/7] qom: add user_creatable_add & user_creatable_del methods

Quoted 14 lines · Daniel P. Berrange

The QMP monitor code has two helper methods object_add and qmp_object_del that are called from several places in the code (QMP, HMP and main emulator startup).

We soon need to use this code from qemu-img, qemu-io and qemu-nbd too, but don't want those to depend on the monitor.

To avoid this, move object_add to user_creatable_add an qmp_object_del to user_creatable_del, to the object_interfaces.c file where they can be easily shared with all users of QOM

Quoted 8 lines
---
 hmp.c                           | 11 ++++--
 include/monitor/monitor.h       |  3 --
 include/qom/object_interfaces.h | 31 +++++++++++++++++
 qmp.c                           | 75 ++++------------------------------------
 qom/object_interfaces.c         | 76 +++++++++++++++++++++++++++++++++++++++++
 vl.c                            |  8 +++--
 6 files changed, 127 insertions(+), 77 deletions(-)

Looks like fairly straightforward code motion.

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#3 lore raw

PATCH 2/7 qemu-img: add support for --object command line arg

In reply to Daniel P. Berrange

Allow creation of user creatable object types with qemu-img via a --object command line arg. This will be used to supply passwords and/or encryption keys to the various block driver backends via the recently added 'secret' object type.

 # echo -n letmein > mypasswd.txt
 # qemu-img info --object secret,id=sec0,file=mypasswd.txt \
      ...other info args...
---
 qemu-img-cmds.hx |  44 ++++----
 qemu-img.c       | 300 +++++++++++++++++++++++++++++++++++++++++++++++++++++--
 qemu-img.texi    |   8 ++
 3 files changed, 322 insertions(+), 30 deletions(-)
qemu-img-cmds.hx +22 -22
index 9567774..5bb1de7 100644
@@ -10,68 +10,68 @@ STEXI
 ETEXI
 
 DEF("check", img_check,
-    "check [-q] [-f fmt] [--output=ofmt] [-r [leaks | all]] [-T src_cache] filename")
+    "check [-q] [--object objectdef] [-f fmt] [--output=ofmt] [-r [leaks | all]] [-T src_cache] filename")
 STEXI
-@item check [-q] [-f @var{fmt}] [--output=@var{ofmt}] [-r [leaks | all]] [-T @var{src_cache}] @var{filename}
+@item check [--object objectdef] [-q] [-f @var{fmt}] [--output=@var{ofmt}] [-r [leaks | all]] [-T @var{src_cache}] @var{filename}
 ETEXI
 
 DEF("create", img_create,
-    "create [-q] [-f fmt] [-o options] filename [size]")
+    "create [-q] [--object objectdef] [-f fmt] [-o options] filename [size]")
 STEXI
-@item create [-q] [-f @var{fmt}] [-o @var{options}] @var{filename} [@var{size}]
+@item create [--object objectdef] [-q] [-f @var{fmt}] [-o @var{options}] @var{filename} [@var{size}]
 ETEXI
 
 DEF("commit", img_commit,
-    "commit [-q] [-f fmt] [-t cache] [-b base] [-d] [-p] filename")
+    "commit [-q] [--object objectdef] [-f fmt] [-t cache] [-b base] [-d] [-p] filename")
 STEXI
-@item commit [-q] [-f @var{fmt}] [-t @var{cache}] [-b @var{base}] [-d] [-p] @var{filename}
+@item commit [--object objectdef] [-q] [-f @var{fmt}] [-t @var{cache}] [-b @var{base}] [-d] [-p] @var{filename}
 ETEXI
 
 DEF("compare", img_compare,
-    "compare [-f fmt] [-F fmt] [-T src_cache] [-p] [-q] [-s] filename1 filename2")
+    "compare [--object objectdef] [-f fmt] [-F fmt] [-T src_cache] [-p] [-q] [-s] filename1 filename2")
 STEXI
-@item compare [-f @var{fmt}] [-F @var{fmt}] [-T @var{src_cache}] [-p] [-q] [-s] @var{filename1} @var{filename2}
+@item compare [--object objectdef] [-f @var{fmt}] [-F @var{fmt}] [-T @var{src_cache}] [-p] [-q] [-s] @var{filename1} @var{filename2}
 ETEXI
 
 DEF("convert", img_convert,
-    "convert [-c] [-p] [-q] [-n] [-f fmt] [-t cache] [-T src_cache] [-O output_fmt] [-o options] [-s snapshot_id_or_name] [-l snapshot_param] [-S sparse_size] filename [filename2 [...]] output_filename")
+    "convert [--object objectdef] [-c] [-p] [-q] [-n] [-f fmt] [-t cache] [-T src_cache] [-O output_fmt] [-o options] [-s snapshot_id_or_name] [-l snapshot_param] [-S sparse_size] filename [filename2 [...]] output_filename")
 STEXI
-@item convert [-c] [-p] [-q] [-n] [-f @var{fmt}] [-t @var{cache}] [-T @var{src_cache}] [-O @var{output_fmt}] [-o @var{options}] [-s @var{snapshot_id_or_name}] [-l @var{snapshot_param}] [-S @var{sparse_size}] @var{filename} [@var{filename2} [...]] @var{output_filename}
+@item convert [--object objectdef] [-c] [-p] [-q] [-n] [-f @var{fmt}] [-t @var{cache}] [-T @var{src_cache}] [-O @var{output_fmt}] [-o @var{options}] [-s @var{snapshot_id_or_name}] [-l @var{snapshot_param}] [-S @var{sparse_size}] @var{filename} [@var{filename2} [...]] @var{output_filename}
 ETEXI
 
 DEF("info", img_info,
-    "info [-f fmt] [--output=ofmt] [--backing-chain] filename")
+    "info [--object objectdef] [-f fmt] [--output=ofmt] [--backing-chain] filename")
 STEXI
-@item info [-f @var{fmt}] [--output=@var{ofmt}] [--backing-chain] @var{filename}
+@item info [--object objectdef] [-f @var{fmt}] [--output=@var{ofmt}] [--backing-chain] @var{filename}
 ETEXI
 
 DEF("map", img_map,
-    "map [-f fmt] [--output=ofmt] filename")
+    "map [--object objectdef] [-f fmt] [--output=ofmt] filename")
 STEXI
-@item map [-f @var{fmt}] [--output=@var{ofmt}] @var{filename}
+@item map [--object objectdef] [-f @var{fmt}] [--output=@var{ofmt}] @var{filename}
 ETEXI
 
 DEF("snapshot", img_snapshot,
-    "snapshot [-q] [-l | -a snapshot | -c snapshot | -d snapshot] filename")
+    "snapshot [--object objectdef] [-q] [-l | -a snapshot | -c snapshot | -d snapshot] filename")
 STEXI
-@item snapshot [-q] [-l | -a @var{snapshot} | -c @var{snapshot} | -d @var{snapshot}] @var{filename}
+@item snapshot [--object objectdef] [-q] [-l | -a @var{snapshot} | -c @var{snapshot} | -d @var{snapshot}] @var{filename}
 ETEXI
 
 DEF("rebase", img_rebase,
-    "rebase [-q] [-f fmt] [-t cache] [-T src_cache] [-p] [-u] -b backing_file [-F backing_fmt] filename")
+    "rebase [--object objectdef] [-q] [-f fmt] [-t cache] [-T src_cache] [-p] [-u] -b backing_file [-F backing_fmt] filename")
 STEXI
-@item rebase [-q] [-f @var{fmt}] [-t @var{cache}] [-T @var{src_cache}] [-p] [-u] -b @var{backing_file} [-F @var{backing_fmt}] @var{filename}
+@item rebase [--object objectdef] [-q] [-f @var{fmt}] [-t @var{cache}] [-T @var{src_cache}] [-p] [-u] -b @var{backing_file} [-F @var{backing_fmt}] @var{filename}
 ETEXI
 
 DEF("resize", img_resize,
-    "resize [-q] filename [+ | -]size")
+    "resize [--object objectdef] [-q] filename [+ | -]size")
 STEXI
-@item resize [-q] @var{filename} [+ | -]@var{size}
+@item resize [--object objectdef] [-q] @var{filename} [+ | -]@var{size}
 ETEXI
 
 DEF("amend", img_amend,
-    "amend [-p] [-q] [-f fmt] [-t cache] -o options filename")
+    "amend [--object objectdef] [-p] [-q] [-f fmt] [-t cache] -o options filename")
 STEXI
-@item amend [-p] [-q] [-f @var{fmt}] [-t @var{cache}] -o @var{options} @var{filename}
+@item amend [--object objectdef] [-p] [-q] [-f @var{fmt}] [-t @var{cache}] -o @var{options} @var{filename}
 @end table
 ETEXI
qemu-img.c +292 -8

575 lines not on this page · raw message

qemu-img.texi +8 -0
index 55c6be3..da93272 100644
@@ -24,6 +24,14 @@ Command parameters:
 @table @var
 @item filename
  is a disk image filename
+
+@item --object @var{objectdef}
+
+is a QEMU user creatable object definition. See the @code{qemu(1)} manual
+page for a description of the object properties. The only object type that
+it makes sense to define is the @code{secret} object, which is used to
+supply passwords and/or encryption keys.
+
 @item fmt
 is the disk image format. It is guessed automatically in most cases. See below
 for a description of the supported disk formats.
Signature · 1 line
2.5.0
#4 lore raw

Re: [Qemu-devel] [PATCH 2/7] qemu-img: add support for --object command line arg

Quoted 2 lines · Daniel P. Berrange

Allow creation of user creatable object types with qemu-img via a --object command line arg. This will be used to supply

Does this read better as "a dash-dash-object", or "an object", in which case you may have an article mismatch? You can skirt the issue by adding an adjective: "a new --object" works with either pronunciation of "--object" :)

Quoted 4 lines

passwords and/or encryption keys to the various block driver backends via the recently added 'secret' object type.

 # echo -n letmein > mypasswd.txt

'echo -n' is not portable; although it doesn't matter here, I tend to favor 'printf letmein' for both its portability and for less typing.

Quoted 9 lines
 # qemu-img info --object secret,id=sec0,file=mypasswd.txt \
      ...other info args...
---
 qemu-img-cmds.hx |  44 ++++----
 qemu-img.c       | 300 +++++++++++++++++++++++++++++++++++++++++++++++++++++--
 qemu-img.texi    |   8 ++
 3 files changed, 322 insertions(+), 30 deletions(-)

How will libvirt discover whether qemu-img is new enough to support this syntax? Then again, qemu-img isn't used quite as heavily as qemu, and the speedups we gain by using QMP instead of -help scraping on qemu don't matter quite as much as what we can attempt with -help scraping on qemu-img.

qemu-img.c static void QEMU_NORETURN help(void) · line 101

Quoted 5 lines
qemu-img.c +1 -0
@@ -94,6 +98,11 @@ static void QEMU_NORETURN help(void)
            "\n"
            "Command parameters:\n"
            "  'filename' is a disk image filename\n"
+           "  'objectdef' is a QEMU user creatable object definition. See \n"

Trailing whitespace on the user's terminal.

qemu-img.c static void QEMU_NORETURN help(void) · line 108

Quoted 7 lines
qemu-img.c +4 -0
+           "    the @code{qemu(1)} manual page for a description of the object\n"
+           "    properties. The only object type that it makes sense to define\n"
+           "    is the @code{secret} object, which is used to supply passwords\n"
+           "    and/or encryption keys.\n"
            "  'fmt' is the disk image format. It is guessed automatically in most cases\n"
            "  'cache' is the cache mode used to write the output disk image, the valid\n"
            "    options are: 'none', 'writeback' (default, except for convert), 'writethrough',\n"

You wrapped the text to fit in 80 source columns, but the lines below wrapped to keep it at 80 user display columns (at the expense of longer source text). I'd actually lean towards the longer lines in this case, even if we have to manually ignore checkpatch.pl.

[GNU coreutils does it like:

    printf("\

long line starting in column 0\n\
etc.");

so that you can fit much closer to 80 output characters while still staying within 80 source columns; but I don't think we need the churn of taking on that style]

qemu-img.c static void QEMU_NORETURN help(void) · line 180

Quoted 6 lines
qemu-img.c +6 -0
+static int object_create(void *opaque, QemuOpts *opts, Error **errp)
+{
+    Error *err = NULL;
+    char *type = NULL;
+    char *id = NULL;
+    void *dummy = NULL;

Drop this.

qemu-img.c static void QEMU_NORETURN help(void) · line 182

Quoted 2 lines
qemu-img.c +2 -0
+    OptsVisitor *ov;
+    QDict *pdict;

Add a Visitor *v; helper variable.

qemu-img.c static void QEMU_NORETURN help(void) · line 187

Quoted 5 lines
qemu-img.c +5 -0
+
+    ov = opts_visitor_new(opts);
+    pdict = qemu_opts_to_qdict(opts, NULL);
+
+    visit_start_struct(opts_get_visitor(ov), &dummy, NULL, NULL, 0, &err);

This conflicts with my pending patches:
https://lists.gnu.org/archive/html/qemu-devel/2015-12/msg03863.html

If mine go in first, you'll want this to be:

visit_start_struct(v, NULL, NULL, 0, &err);

And even if yours goes in first, you should make it look more like this, so I don't have to fix it up after you:
https://lists.gnu.org/archive/html/qemu-devel/2015-12/msg03862.html

(since it looks like you copied from there anyways :)

qemu-img.c static void QEMU_NORETURN help(void) · line 208

Quoted 6 lines
qemu-img.c +6 -0
+
+    user_creatable_add(type, id, pdict, opts_get_visitor(ov), &err);
+    if (err) {
+        goto out;
+    }
+    visit_end_struct(opts_get_visitor(ov), &err);

visit_end_struct() needs to be called unconditionally if visit_start_struct() succeeded. Again, if my series goes in first, rebase it to look like my changes to vl.c; if yours goes in first, I'll have to touch up your additions to match what I do elsewhere in my series.

qemu-img.c static int img_create(int argc, char **argv) · line 404

Quoted 9 lines
qemu-img.c +5 -0
@@ -319,6 +397,13 @@ static int img_create(int argc, char **argv)
         case 'q':
             quiet = true;
             break;
+        case OPTION_OBJECT:
+            opts = qemu_opts_parse_noisily(qemu_find_opts("object"),
+                                           optarg, true);
+            if (!opts) {
+                exit(1);

Not for this patch, but maybe someday we should switch to exit(EXIT_FAILURE) throughout the file.

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#5 lore raw

Re: [Qemu-devel] [PATCH 2/7] qemu-img: add support for --object command line arg

Quoted 8 lines · Eric Blake
Quoted 2 lines · Daniel P. Berrange

Allow creation of user creatable object types with qemu-img via a --object command line arg. This will be used to supply

Does this read better as "a dash-dash-object", or "an object", in which case you may have an article mismatch? You can skirt the issue by adding an adjective: "a new --object" works with either pronunciation of "--object" :)

Heh, ok

Quoted 7 lines
Quoted 4 lines

passwords and/or encryption keys to the various block driver backends via the recently added 'secret' object type.

 # echo -n letmein > mypasswd.txt

'echo -n' is not portable; although it doesn't matter here, I tend to favor 'printf letmein' for both its portability and for less typing.

Yep, I fixed my other patches to use printf previously too.

Quoted 10 lines
Quoted 4 lines
 qemu-img-cmds.hx |  44 ++++----
 qemu-img.c       | 300 +++++++++++++++++++++++++++++++++++++++++++++++++++++--
 qemu-img.texi    |   8 ++
 3 files changed, 322 insertions(+), 30 deletions(-)

How will libvirt discover whether qemu-img is new enough to support this syntax? Then again, qemu-img isn't used quite as heavily as qemu, and the speedups we gain by using QMP instead of -help scraping on qemu don't matter quite as much as what we can attempt with -help scraping on qemu-img.

Yeah, qemu-img feature detection is an outstanding unsolved problem that I don't really have an answer for. In general though, I think libvirt will just take the approach of blindly try to use it, if and only if, we actually need the new feature. That should not create us any back-compat problems, and get us moderately acceptable error reporting if we try to use new feature with old qemu-img.

Quoted 8 lines

qemu-img.c static void QEMU_NORETURN help(void) · line 108

Quoted 7 lines
qemu-img.c +4 -0
+           "    the @code{qemu(1)} manual page for a description of the object\n"
+           "    properties. The only object type that it makes sense to define\n"
+           "    is the @code{secret} object, which is used to supply passwords\n"
+           "    and/or encryption keys.\n"
            "  'fmt' is the disk image format. It is guessed automatically in most cases\n"
            "  'cache' is the cache mode used to write the output disk image, the valid\n"
            "    options are: 'none', 'writeback' (default, except for convert), 'writethrough',\n"
Quoted 4 lines

You wrapped the text to fit in 80 source columns, but the lines below wrapped to keep it at 80 user display columns (at the expense of longer source text). I'd actually lean towards the longer lines in this case, even if we have to manually ignore checkpatch.pl.

Yep, good point, I missed that distinction.

Quoted 36 lines

[GNU coreutils does it like:

    printf("\

long line starting in column 0\n\
etc.");

so that you can fit much closer to 80 output characters while still staying within 80 source columns; but I don't think we need the churn of taking on that style]

qemu-img.c static void QEMU_NORETURN help(void) · line 180

Quoted 6 lines
qemu-img.c +6 -0
+static int object_create(void *opaque, QemuOpts *opts, Error **errp)
+{
+    Error *err = NULL;
+    char *type = NULL;
+    char *id = NULL;
+    void *dummy = NULL;

Drop this.

qemu-img.c static void QEMU_NORETURN help(void) · line 182

Quoted 2 lines
qemu-img.c +2 -0
+    OptsVisitor *ov;
+    QDict *pdict;

Add a Visitor *v; helper variable.

qemu-img.c static void QEMU_NORETURN help(void) · line 187

Quoted 5 lines
qemu-img.c +5 -0
+
+    ov = opts_visitor_new(opts);
+    pdict = qemu_opts_to_qdict(opts, NULL);
+
+    visit_start_struct(opts_get_visitor(ov), &dummy, NULL, NULL, 0, &err);

This conflicts with my pending patches:
https://lists.gnu.org/archive/html/qemu-devel/2015-12/msg03863.html

If mine go in first, you'll want this to be:

Quoted 7 lines

visit_start_struct(v, NULL, NULL, 0, &err);

And even if yours goes in first, you should make it look more like this, so I don't have to fix it up after you:
https://lists.gnu.org/archive/html/qemu-devel/2015-12/msg03862.html

(since it looks like you copied from there anyways :)

Yep, ok.

Quoted 14 lines

qemu-img.c static void QEMU_NORETURN help(void) · line 208

Quoted 8 lines
qemu-img.c +6 -0
+
+    user_creatable_add(type, id, pdict, opts_get_visitor(ov), &err);
+    if (err) {
+        goto out;
+    }
+    visit_end_struct(opts_get_visitor(ov), &err);

visit_end_struct() needs to be called unconditionally if visit_start_struct() succeeded. Again, if my series goes in first, rebase it to look like my changes to vl.c; if yours goes in first, I'll have to touch up your additions to match what I do elsewhere in my series.

Quoted 12 lines

qemu-img.c static int img_create(int argc, char **argv) · line 404

Quoted 9 lines
qemu-img.c +5 -0
@@ -319,6 +397,13 @@ static int img_create(int argc, char **argv)
         case 'q':
             quiet = true;
             break;
+        case OPTION_OBJECT:
+            opts = qemu_opts_parse_noisily(qemu_find_opts("object"),
+                                           optarg, true);
+            if (!opts) {
+                exit(1);

Not for this patch, but maybe someday we should switch to exit(EXIT_FAILURE) throughout the file.

Yeah, that came to mind when I was working on these patches. A cleanup for another day though, lest my number of pending patches exceed 100 !

Regards,
Daniel

Signature · 4 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|
#6 lore raw

PATCH 3/7 qemu-nbd: add support for --object command line arg

In reply to Daniel P. Berrange

Allow creation of user creatable object types with qemu-nbd via a --object command line arg. This will be used to supply passwords and/or encryption keys to the various block driver backends via the recently added 'secret' object type.

 # echo -n letmein > mypasswd.txt
 # qemu-nbd --object secret,id=sec0,file=mypasswd.txt \
      ...other nbd args...
---
 qemu-nbd.c    | 85 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 qemu-nbd.texi |  7 +++++
 2 files changed, 92 insertions(+)
qemu-nbd.c +85 -0
index 65dc30c..b4b6681 100644
@@ -23,9 +23,12 @@
 #include "qemu/main-loop.h"
 #include "qemu/sockets.h"
 #include "qemu/error-report.h"
+#include "qemu/config-file.h"
 #include "block/snapshot.h"
 #include "qapi/util.h"
 #include "qapi/qmp/qstring.h"
+#include "qapi/opts-visitor.h"
+#include "qom/object_interfaces.h"
 
 #include <stdarg.h>
 #include <stdio.h>
@@ -45,6 +48,7 @@
 #define QEMU_NBD_OPT_AIO           2
 #define QEMU_NBD_OPT_DISCARD       3
 #define QEMU_NBD_OPT_DETECT_ZEROES 4
+#define QEMU_NBD_OPT_OBJECT        5
 
 static NBDExport *exp;
 static int verbose;
@@ -78,6 +82,9 @@ static void usage(const char *name)
 "  -o, --offset=OFFSET       offset into the image\n"
 "  -P, --partition=NUM       only expose partition NUM\n"
 "\n"
+"General purpose options:\n"
+"  --object type,id=ID,...   define an object such as 'secret' for providing\n"
+"                            passwords and/or encryption keys\n"
 #ifdef __linux__
 "Kernel NBD client support:\n"
 "  -c, --connect=DEV         connect FILE to the local NBD device DEV\n"
@@ -380,6 +387,67 @@ static SocketAddress *nbd_build_socket_address(const char *sockpath,
 }
 
 
+static QemuOptsList qemu_object_opts = {
+    .name = "object",
+    .implied_opt_name = "qom-type",
+    .head = QTAILQ_HEAD_INITIALIZER(qemu_object_opts.head),
+    .desc = {
+        { }
+    },
+};
+
+static int object_create(void *opaque, QemuOpts *opts, Error **errp)
+{
+    Error *err = NULL;
+    char *type = NULL;
+    char *id = NULL;
+    void *dummy = NULL;
+    OptsVisitor *ov;
+    QDict *pdict;
+
+    ov = opts_visitor_new(opts);
+    pdict = qemu_opts_to_qdict(opts, NULL);
+
+    visit_start_struct(opts_get_visitor(ov), &dummy, NULL, NULL, 0, &err);
+    if (err) {
+        goto out;
+    }
+
+    qdict_del(pdict, "qom-type");
+    visit_type_str(opts_get_visitor(ov), &type, "qom-type", &err);
+    if (err) {
+        goto out;
+    }
+
+    qdict_del(pdict, "id");
+    visit_type_str(opts_get_visitor(ov), &id, "id", &err);
+    if (err) {
+        goto out;
+    }
+
+    user_creatable_add(type, id, pdict, opts_get_visitor(ov), &err);
+    if (err) {
+        goto out;
+    }
+    visit_end_struct(opts_get_visitor(ov), &err);
+    if (err) {
+        user_creatable_del(id, NULL);
+    }
+
+out:
+    opts_visitor_cleanup(ov);
+
+    QDECREF(pdict);
+    g_free(id);
+    g_free(type);
+    g_free(dummy);
+    if (err) {
+        error_report_err(err);
+        return -1;
+    }
+    return 0;
+}
+
 int main(int argc, char **argv)
 {
     BlockBackend *blk;
@@ -417,6 +485,7 @@ int main(int argc, char **argv)
         { "format", 1, NULL, 'f' },
         { "persistent", 0, NULL, 't' },
         { "verbose", 0, NULL, 'v' },
+        { "object", 1, NULL, QEMU_NBD_OPT_OBJECT },
         { NULL, 0, NULL, 0 }
     };
     int ch;
@@ -434,6 +503,7 @@ int main(int argc, char **argv)
     Error *local_err = NULL;
     BlockdevDetectZeroesOptions detect_zeroes = BLOCKDEV_DETECT_ZEROES_OPTIONS_OFF;
     QDict *options = NULL;
+    QemuOpts *opts;
 
     /* The client thread uses SIGTERM to interrupt the server.  A signal
      * handler ensures that "qemu-nbd -v -c" exits with a nice status code.
@@ -442,6 +512,8 @@ int main(int argc, char **argv)
     memset(&sa_sigterm, 0, sizeof(sa_sigterm));
     sa_sigterm.sa_handler = termsig_handler;
     sigaction(SIGTERM, &sa_sigterm, NULL);
+    module_call_init(MODULE_INIT_QOM);
+    qemu_add_opts(&qemu_object_opts);
     qemu_init_exec_dir(argv[0]);
 
     while ((ch = getopt_long(argc, argv, sopt, lopt, &opt_ind)) != -1) {
@@ -578,6 +650,13 @@ int main(int argc, char **argv)
             usage(argv[0]);
             exit(0);
             break;
+        case QEMU_NBD_OPT_OBJECT:
+            opts = qemu_opts_parse_noisily(qemu_find_opts("object"),
+                                           optarg, true);
+            if (!opts) {
+                exit(1);
+            }
+            break;
         case '?':
             errx(EXIT_FAILURE, "Try `%s --help' for more information.",
                  argv[0]);
@@ -590,6 +669,12 @@ int main(int argc, char **argv)
              argv[0]);
     }
 
+    if (qemu_opts_foreach(qemu_find_opts("object"),
+                          object_create,
+                          NULL, NULL)) {
+        exit(1);
+    }
+
     if (disconnect) {
         fd = open(argv[optind], O_RDWR);
         if (fd < 0) {
qemu-nbd.texi +7 -0
index 46fd483..3b2004e 100644
@@ -14,6 +14,13 @@ Export QEMU disk image using NBD protocol.
 @table @option
 @item @var{filename}
  is a disk image filename
+@item --object type,id=@var{id},...props...
+  define a new instance of the @var{type} object class
+  identified by @var{id}. See the @code{qemu(1)} manual
+  page for full details of the properties supported.
+  The only object type that it makes sense to define
+  is the @code{secret} object, which is used to supply
+  passwords and/or encryption keys.
 @item -p, --port=@var{port}
   port to listen on (default @samp{10809})
 @item -o, --offset=@var{offset}
Signature · 1 line
2.5.0
#7 lore raw

Re: [Qemu-devel] [PATCH 3/7] qemu-nbd: add support for --object command line arg

Quoted 8 lines · Daniel P. Berrange

Allow creation of user creatable object types with qemu-nbd via a --object command line arg. This will be used to supply passwords and/or encryption keys to the various block driver backends via the recently added 'secret' object type.

 # echo -n letmein > mypasswd.txt
 # qemu-nbd --object secret,id=sec0,file=mypasswd.txt \
      ...other nbd args...

Same comments as on 2/7.

qemu-nbd.c line 51

Quoted 5 lines
qemu-nbd.c +1 -0
@@ -45,6 +48,7 @@
 #define QEMU_NBD_OPT_AIO           2
 #define QEMU_NBD_OPT_DISCARD       3
 #define QEMU_NBD_OPT_DETECT_ZEROES 4
+#define QEMU_NBD_OPT_OBJECT        5

Pre-existing, but these are unsafe; they conflict with actual byte values. As long as you are touching this, you should fix them to start at 256 (a separate patch wouldn't hurt).

qemu-nbd.c static SocketAddress *nbd_build_socket_address(const char *sockpath, · line 411

Quoted 13 lines
qemu-nbd.c +13 -0
+static int object_create(void *opaque, QemuOpts *opts, Error **errp)
+{
+    Error *err = NULL;
+    char *type = NULL;
+    char *id = NULL;
+    void *dummy = NULL;
+    OptsVisitor *ov;
+    QDict *pdict;
+
+    ov = opts_visitor_new(opts);
+    pdict = qemu_opts_to_qdict(opts, NULL);
+
+    visit_start_struct(opts_get_visitor(ov), &dummy, NULL, NULL, 0, &err);

Same comments as on 2/7.

qemu-nbd.c int main(int argc, char **argv) · line 489

Quoted 6 lines
qemu-nbd.c +1 -0
@@ -417,6 +485,7 @@ int main(int argc, char **argv)
         { "format", 1, NULL, 'f' },
         { "persistent", 0, NULL, 't' },
         { "verbose", 0, NULL, 'v' },
+        { "object", 1, NULL, QEMU_NBD_OPT_OBJECT },
         { NULL, 0, NULL, 0 }

Pre-existing, but 0 and 1 are magic numbers; I prefer the symbolic names no_argument and required_argument (and optional_argument for 2).

qemu-nbd.texi Export QEMU disk image using NBD protocol. · line 23

Quoted 12 lines
qemu-nbd.texi +8 -0
+++ b/qemu-nbd.texi
@@ -14,6 +14,13 @@ Export QEMU disk image using NBD protocol.
 @table @option
 @item @var{filename}
  is a disk image filename
+@item --object type,id=@var{id},...props...
+  define a new instance of the @var{type} object class
+  identified by @var{id}. See the @code{qemu(1)} manual
+  page for full details of the properties supported.
+  The only object type that it makes sense to define
+  is the @code{secret} object, which is used to supply
+  passwords and/or encryption keys.

Awfully short line-wrapping; although it doesn't matter in the final generated docs.

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#8 lore raw

PATCH 4/7 qemu-io: add support for --object command line arg

In reply to Daniel P. Berrange

Allow creation of user creatable object types with qemu-io via a --object command line arg. This will be used to supply passwords and/or encryption keys to the various block driver backends via the recently added 'secret' object type.

 # echo -n letmein > mypasswd.txt
 # qemu-io --object secret,id=sec0,file=mypasswd.txt \
      ...other args...
---
 qemu-io.c | 87 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 87 insertions(+)
qemu-io.c +87 -0
index 269f17c..cf1dac6 100644
@@ -21,6 +21,8 @@
 #include "qemu/config-file.h"
 #include "qemu/readline.h"
 #include "qapi/qmp/qstring.h"
+#include "qapi/opts-visitor.h"
+#include "qom/object_interfaces.h"
 #include "sysemu/block-backend.h"
 #include "block/block_int.h"
 #include "trace/control.h"
@@ -205,6 +207,9 @@ static void usage(const char *name)
 "Usage: %s [-h] [-V] [-rsnm] [-f FMT] [-c STRING] ... [file]\n"
 "QEMU Disk exerciser\n"
 "\n"
+"  --object OBJECTDEF   define a object such as 'secret' for\n"
+"                       providing passwords and/or encryption\n"
+"                       keys\n"
 "  -c, --cmd STRING     execute command with its arguments\n"
 "                       from the given string\n"
 "  -f, --format FMT     specifies the block driver to use\n"
@@ -366,6 +371,71 @@ static void reenable_tty_echo(void)
     qemu_set_tty_echo(STDIN_FILENO, true);
 }
 
+enum {
+    OPTION_OBJECT = 258,
+};
+
+static QemuOptsList qemu_object_opts = {
+    .name = "object",
+    .implied_opt_name = "qom-type",
+    .head = QTAILQ_HEAD_INITIALIZER(qemu_object_opts.head),
+    .desc = {
+        { }
+    },
+};
+
+static int object_create(void *opaque, QemuOpts *opts, Error **errp)
+{
+    Error *err = NULL;
+    char *type = NULL;
+    char *id = NULL;
+    void *dummy = NULL;
+    OptsVisitor *ov;
+    QDict *pdict;
+
+    ov = opts_visitor_new(opts);
+    pdict = qemu_opts_to_qdict(opts, NULL);
+
+    visit_start_struct(opts_get_visitor(ov), &dummy, NULL, NULL, 0, &err);
+    if (err) {
+        goto out;
+    }
+
+    qdict_del(pdict, "qom-type");
+    visit_type_str(opts_get_visitor(ov), &type, "qom-type", &err);
+    if (err) {
+        goto out;
+    }
+
+    qdict_del(pdict, "id");
+    visit_type_str(opts_get_visitor(ov), &id, "id", &err);
+    if (err) {
+        goto out;
+    }
+
+    user_creatable_add(type, id, pdict, opts_get_visitor(ov), &err);
+    if (err) {
+        goto out;
+    }
+    visit_end_struct(opts_get_visitor(ov), &err);
+    if (err) {
+        user_creatable_del(id, NULL);
+    }
+
+out:
+    opts_visitor_cleanup(ov);
+
+    QDECREF(pdict);
+    g_free(id);
+    g_free(type);
+    g_free(dummy);
+    if (err) {
+        error_report_err(err);
+        return -1;
+    }
+    return 0;
+}
+
 int main(int argc, char **argv)
 {
     int readonly = 0;
@@ -384,6 +454,7 @@ int main(int argc, char **argv)
         { "discard", 1, NULL, 'd' },
         { "cache", 1, NULL, 't' },
         { "trace", 1, NULL, 'T' },
+        { "object", 1, NULL, OPTION_OBJECT },
         { NULL, 0, NULL, 0 }
     };
     int c;
@@ -391,6 +462,7 @@ int main(int argc, char **argv)
     int flags = BDRV_O_UNMAP;
     Error *local_error = NULL;
     QDict *opts = NULL;
+    QemuOpts *qopts = NULL;
 
 #ifdef CONFIG_POSIX
     signal(SIGPIPE, SIG_IGN);
@@ -399,6 +471,8 @@ int main(int argc, char **argv)
     progname = basename(argv[0]);
     qemu_init_exec_dir(argv[0]);
 
+    module_call_init(MODULE_INIT_QOM);
+    qemu_add_opts(&qemu_object_opts);
     bdrv_init();
 
     while ((c = getopt_long(argc, argv, sopt, lopt, &opt_index)) != -1) {
@@ -450,6 +524,13 @@ int main(int argc, char **argv)
         case 'h':
             usage(progname);
             exit(0);
+        case OPTION_OBJECT:
+            qopts = qemu_opts_parse_noisily(qemu_find_opts("object"),
+                                            optarg, true);
+            if (!qopts) {
+                exit(1);
+            }
+            break;
         default:
             usage(progname);
             exit(1);
@@ -466,6 +547,12 @@ int main(int argc, char **argv)
         exit(1);
     }
 
+    if (qemu_opts_foreach(qemu_find_opts("object"),
+                          object_create,
+                          NULL, NULL)) {
+        exit(1);
+    }
+
     /* initialize commands */
     qemuio_add_command(&quit_cmd);
     qemuio_add_command(&open_cmd);
Signature · 1 line
2.5.0
#9 lore raw

Re: [Qemu-devel] [PATCH 4/7] qemu-io: add support for --object command line arg

Quoted 8 lines · Daniel P. Berrange

Allow creation of user creatable object types with qemu-io via a --object command line arg. This will be used to supply passwords and/or encryption keys to the various block driver backends via the recently added 'secret' object type.

 # echo -n letmein > mypasswd.txt
 # qemu-io --object secret,id=sec0,file=mypasswd.txt \
      ...other args...

Same comments as on 3/7.

qemu-io.c static void usage(const char *name) · line 210

Quoted 11 lines
---
 qemu-io.c | 87 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 87 insertions(+)
qemu-io.c +1 -0
@@ -205,6 +207,9 @@ static void usage(const char *name)
 "Usage: %s [-h] [-V] [-rsnm] [-f FMT] [-c STRING] ... [file]\n"
 "QEMU Disk exerciser\n"
 "\n"
+"  --object OBJECTDEF   define a object such as 'secret' for\n"

s/a object/an object/

qemu-io.c static void usage(const char *name) · line 212

Quoted 2 lines
qemu-io.c +2 -0
+"                       providing passwords and/or encryption\n"
+"                       keys\n"

3 lines feels long; you got away with only 2 lines in 3/7 by using longer line wrapping, while still fitting in the user's 80 column output:

+" --object type,id=ID,... define an object such as 'secret' for
providing\n"
+" passwords and/or encryption keys\n"

qemu-io.c static void reenable_tty_echo(void) · line 376

Quoted 4 lines
qemu-io.c +3 -0
+enum {
+    OPTION_OBJECT = 258,
+};

256 would work. But 258 doesn't hurt.

qemu-io.c static void reenable_tty_echo(void) · line 399

Quoted 13 lines
qemu-io.c +13 -0
+static int object_create(void *opaque, QemuOpts *opts, Error **errp)
+{
+    Error *err = NULL;
+    char *type = NULL;
+    char *id = NULL;
+    void *dummy = NULL;
+    OptsVisitor *ov;
+    QDict *pdict;
+
+    ov = opts_visitor_new(opts);
+    pdict = qemu_opts_to_qdict(opts, NULL);
+
+    visit_start_struct(opts_get_visitor(ov), &dummy, NULL, NULL, 0, &err);

Same comments as on 3/7.

We now have 5 very similar functions (hmp.c, vl.c, and your three additions); should this be factored into a common reusable function rather than open-coding it into each client?

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#10 lore raw

Re: [Qemu-devel] [PATCH 4/7] qemu-io: add support for --object command line arg

Quoted 61 lines · Eric Blake
Quoted 8 lines · Daniel P. Berrange

Allow creation of user creatable object types with qemu-io via a --object command line arg. This will be used to supply passwords and/or encryption keys to the various block driver backends via the recently added 'secret' object type.

 # echo -n letmein > mypasswd.txt
 # qemu-io --object secret,id=sec0,file=mypasswd.txt \
      ...other args...

Same comments as on 3/7.

qemu-io.c static void usage(const char *name) · line 210

Quoted 11 lines
---
 qemu-io.c | 87 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 87 insertions(+)
qemu-io.c +1 -0
@@ -205,6 +207,9 @@ static void usage(const char *name)
 "Usage: %s [-h] [-V] [-rsnm] [-f FMT] [-c STRING] ... [file]\n"
 "QEMU Disk exerciser\n"
 "\n"
+"  --object OBJECTDEF   define a object such as 'secret' for\n"

s/a object/an object/

qemu-io.c static void usage(const char *name) · line 212

Quoted 2 lines
qemu-io.c +2 -0
+"                       providing passwords and/or encryption\n"
+"                       keys\n"

3 lines feels long; you got away with only 2 lines in 3/7 by using longer line wrapping, while still fitting in the user's 80 column output:

+" --object type,id=ID,... define an object such as 'secret' for
providing\n"
+" passwords and/or encryption keys\n"

qemu-io.c static void reenable_tty_echo(void) · line 376

Quoted 4 lines
qemu-io.c +3 -0
+enum {
+    OPTION_OBJECT = 258,
+};

256 would work. But 258 doesn't hurt.

qemu-io.c static void reenable_tty_echo(void) · line 399

Quoted 13 lines
qemu-io.c +13 -0
+static int object_create(void *opaque, QemuOpts *opts, Error **errp)
+{
+    Error *err = NULL;
+    char *type = NULL;
+    char *id = NULL;
+    void *dummy = NULL;
+    OptsVisitor *ov;
+    QDict *pdict;
+
+    ov = opts_visitor_new(opts);
+    pdict = qemu_opts_to_qdict(opts, NULL);
+
+    visit_start_struct(opts_get_visitor(ov), &dummy, NULL, NULL, 0, &err);

Same comments as on 3/7.

Quoted 3 lines

We now have 5 very similar functions (hmp.c, vl.c, and your three additions); should this be factored into a common reusable function rather than open-coding it into each client?

Yeah, it is a bit questionable to have this duplication. I already saved a fair bit with the first patch introducing the shared user_createable_new.

I was a little reluctant to move this 'object_create' method into the qom/ code though, since I hate the idea of the legacy 'QemuOpts' data anywhere near those nice new APIs. I guess I could perhaps just keep the qemu_opts_to_qdict() call in the caller.

Regards,
Daniel

Signature · 4 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|
#11 lore raw

Re: [Qemu-devel] [PATCH 4/7] qemu-io: add support for --object command line arg

Quoted 4 lines · Daniel P. Berrange

I was a little reluctant to move this 'object_create' method into the qom/ code though, since I hate the idea of the legacy 'QemuOpts' data anywhere near those nice new APIs. I guess I could perhaps just keep the qemu_opts_to_qdict() call in the caller.

I disagree with calling QemuOpts legacy, but I agree that this shouldn't be in qom/object_interfaces.c. However, it could be moved to a separate file in qom/.

Paolo

#12 lore raw

Re: [Qemu-devel] [PATCH 4/7] qemu-io: add support for --object command line arg

Quoted 11 lines · Paolo Bonzini
Quoted 4 lines · Daniel P. Berrange

I was a little reluctant to move this 'object_create' method into the qom/ code though, since I hate the idea of the legacy 'QemuOpts' data anywhere near those nice new APIs. I guess I could perhaps just keep the qemu_opts_to_qdict() call in the caller.

I disagree with calling QemuOpts legacy, but I agree that this shouldn't be in qom/object_interfaces.c. However, it could be moved to a separate file in qom/.

Ok, yeah, I shouldn't have called it legacy really. This was my shorthand for saying that internal infrastructure APIs should strive to be qapi/qom based, and QemuOpts should only be used at the very top most level (ie cli ARGV) and converted to qapi/qom as soon as possible. I think that my v2 series achieves that goal fairly well.

Regards,
Daniel

Signature · 4 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|
#13 lore raw

PATCH 5/7 qemu-io: allow specifying image as a set of options args

In reply to Daniel P. Berrange

Currently qemu-io allows an image filename to be passed on the command line, but does not have a way to set any options except the format eg

qemu-io https://127.0.0.1/images/centos7.iso
qemu-io /home/berrange/demo.qcow2

This adds a --source arg (that is mutually exclusive with a positional filename arg and -f arg) that accepts a full option string, as well as the original syntax eg

qemu-io --source driver=http,url=https://127.0.0.1/images,sslverify=off qemu-io --source https://127.0.0.1/images/centos7.iso
qemu-io --source file=/home/berrange/demo.qcow2
qemu-io --source /home/berrange/demo.qcow2

---
 qemu-io.c | 37 ++++++++++++++++++++++++++++++++++++-
 1 file changed, 36 insertions(+), 1 deletion(-)
qemu-io.c +36 -1
index cf1dac6..fc7f81b 100644
@@ -373,6 +373,7 @@ static void reenable_tty_echo(void)
 
 enum {
     OPTION_OBJECT = 258,
+    OPTION_SOURCE = 259,
 };
 
 static QemuOptsList qemu_object_opts = {
@@ -436,6 +437,16 @@ out:
     return 0;
 }
 
+static QemuOptsList file_opts = {
+    .name = "file",
+    .implied_opt_name = "file",
+    .head = QTAILQ_HEAD_INITIALIZER(file_opts.head),
+    .desc = {
+        /* no elements => accept any params */
+        { /* end of list */ }
+    },
+};
+
 int main(int argc, char **argv)
 {
     int readonly = 0;
@@ -455,6 +466,7 @@ int main(int argc, char **argv)
         { "cache", 1, NULL, 't' },
         { "trace", 1, NULL, 'T' },
         { "object", 1, NULL, OPTION_OBJECT },
+        { "source", 1, NULL, OPTION_SOURCE },
         { NULL, 0, NULL, 0 }
     };
     int c;
@@ -531,6 +543,12 @@ int main(int argc, char **argv)
                 exit(1);
             }
             break;
+        case OPTION_SOURCE:
+            if (!qemu_opts_parse_noisily(&file_opts, optarg, false)) {
+                qemu_opts_reset(&file_opts);
+                return 0;
+            }
+            break;
         default:
             usage(progname);
             exit(1);
@@ -572,7 +590,24 @@ int main(int argc, char **argv)
         flags |= BDRV_O_RDWR;
     }
 
-    if ((argc - optind) == 1) {
+    qopts = qemu_opts_find(&file_opts, NULL);
+    if (qopts) {
+        char *file;
+        if (opts) {
+            error_report("--source and -f are mutually exclusive");
+            exit(1);
+        }
+        if ((argc - optind) == 1) {
+            error_report("--source and filename are mutually exclusive");
+            exit(1);
+        }
+        file = g_strdup(qemu_opt_get(qopts, "file"));
+        qemu_opt_unset(qopts, "file");
+        opts = qemu_opts_to_qdict(qopts, NULL);
+        qemu_opts_reset(&file_opts);
+        openfile(file, flags, opts);
+        g_free(file);
+    } else if ((argc - optind) == 1) {
         openfile(argv[optind], flags, opts);
     }
     command_loop();
Signature · 1 line
2.5.0
#14 lore raw

Re: [Qemu-devel] [PATCH 5/7] qemu-io: allow specifying image as a set of options args

Quoted 6 lines · Daniel P. Berrange

Currently qemu-io allows an image filename to be passed on the command line, but does not have a way to set any options except the format eg

qemu-io https://127.0.0.1/images/centos7.iso
qemu-io /home/berrange/demo.qcow2

Well, you CAN set the options by specifying the file as:
qemu-io json:{...}

but that's a mouthful to type, so I don't mind adding the new syntax.

Quoted 19 lines

This adds a --source arg (that is mutually exclusive with a positional filename arg and -f arg) that accepts a full option string, as well as the original syntax eg

qemu-io --source driver=http,url=https://127.0.0.1/images,sslverify=off qemu-io --source https://127.0.0.1/images/centos7.iso
qemu-io --source file=/home/berrange/demo.qcow2
qemu-io --source /home/berrange/demo.qcow2

---
 qemu-io.c | 37 ++++++++++++++++++++++++++++++++++++-
 1 file changed, 36 insertions(+), 1 deletion(-)
qemu-io.c +0 -0
@@ -572,7 +590,24 @@ int main(int argc, char **argv)
         flags |= BDRV_O_RDWR;
     }

qemu-io.c int main(int argc, char **argv) · line 600

Quoted 10 lines
qemu-io.c +8 -1
 
-    if ((argc - optind) == 1) {
+    qopts = qemu_opts_find(&file_opts, NULL);
+    if (qopts) {
+        char *file;
+        if (opts) {
+            error_report("--source and -f are mutually exclusive");
+            exit(1);
+        }
+        if ((argc - optind) == 1) {

What if I specify more than one option? Shouldn't this be >=, rather than ==? Also, the inner () are redundant.

qemu-io.c int main(int argc, char **argv) · line 610

Quoted 10 lines
qemu-io.c +10 -0
+            error_report("--source and filename are mutually exclusive");
+            exit(1);
+        }
+        file = g_strdup(qemu_opt_get(qopts, "file"));
+        qemu_opt_unset(qopts, "file");
+        opts = qemu_opts_to_qdict(qopts, NULL);
+        qemu_opts_reset(&file_opts);
+        openfile(file, flags, opts);
+        g_free(file);
+    } else if ((argc - optind) == 1) {

Pre-existing, but the inner () are redundant.

Quoted 4 lines
qemu-io.c +0 -0
         openfile(argv[optind], flags, opts);
     }
     command_loop();
Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#15 lore raw

Re: [Qemu-devel] [PATCH 5/7] qemu-io: allow specifying image as a set of options args

Quoted 5 lines · Eric Blake
Quoted 4 lines · Daniel P. Berrange

Currently qemu-io allows an image filename to be passed on the command line, but does not have a way to set any options except the format eg

Quoted 7 lines
Quoted 2 lines

qemu-io https://127.0.0.1/images/centos7.iso
qemu-io /home/berrange/demo.qcow2

Well, you CAN set the options by specifying the file as:
qemu-io json:{...}

but that's a mouthful to type, so I don't mind adding the new syntax.

Yep, but as a user, good luck finding out the syntax you have to provide for that - AFAIK its not documented in any of our user facing docs. What everyone knows is the syntax accepted by the -drive command, so I figure we should allow it everywhere.

I didn't even know about the ability to use json until Kevin mentioned it to me last time I asked about this :-)

Regards,
Daniel

Signature · 4 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|
#16 lore raw

PATCH 6/7 qemu-nbd: allow specifying image as a set of options args

In reply to Daniel P. Berrange

Currently qemu-nbd allows an image filename to be passed on the command line, but does not have a way to set any options except the format eg

qemu-nbd https://127.0.0.1/images/centos7.iso
qemu-nbd /home/berrange/demo.qcow2

This adds a --source arg (that is mutually exclusive with a positional filename arg and -f arg) that accepts a full option string, as well as the original syntax eg

   qemu-nbd --source driver=http,url=https://127.0.0.1/images,sslverify=off
   qemu-nbd --source https://127.0.0.1/images/centos7.iso
   qemu-nbd --source file=/home/berrange/demo.qcow2
   qemu-nbd --source /home/berrange/demo.qcow2
---
 qemu-nbd.c | 57 ++++++++++++++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 50 insertions(+), 7 deletions(-)
qemu-nbd.c +50 -7
index b4b6681..0b9f99e 100644
@@ -49,6 +49,7 @@
 #define QEMU_NBD_OPT_DISCARD       3
 #define QEMU_NBD_OPT_DETECT_ZEROES 4
 #define QEMU_NBD_OPT_OBJECT        5
+#define QEMU_NBD_OPT_SOURCE        6
 
 static NBDExport *exp;
 static int verbose;
@@ -387,6 +388,16 @@ static SocketAddress *nbd_build_socket_address(const char *sockpath,
 }
 
 
+static QemuOptsList file_opts = {
+    .name = "file",
+    .implied_opt_name = "file",
+    .head = QTAILQ_HEAD_INITIALIZER(file_opts.head),
+    .desc = {
+        /* no elements => accept any params */
+        { /* end of list */ }
+    },
+};
+
 static QemuOptsList qemu_object_opts = {
     .name = "object",
     .implied_opt_name = "qom-type",
@@ -486,6 +497,7 @@ int main(int argc, char **argv)
         { "persistent", 0, NULL, 't' },
         { "verbose", 0, NULL, 'v' },
         { "object", 1, NULL, QEMU_NBD_OPT_OBJECT },
+        { "source", 1, NULL, QEMU_NBD_OPT_SOURCE },
         { NULL, 0, NULL, 0 }
     };
     int ch;
@@ -657,13 +669,23 @@ int main(int argc, char **argv)
                 exit(1);
             }
             break;
+        case QEMU_NBD_OPT_SOURCE:
+            if (srcpath) {
+                errx(EXIT_FAILURE, "--source can only be used once");
+            }
+            if (!qemu_opts_parse_noisily(&file_opts, optarg, true)) {
+                qemu_opts_reset(&file_opts);
+                exit(EXIT_FAILURE);
+            }
+            srcpath = optarg;
+            break;
         case '?':
             errx(EXIT_FAILURE, "Try `%s --help' for more information.",
                  argv[0]);
         }
     }
 
-    if ((argc - optind) != 1) {
+    if ((argc - optind) > 1) {
         errx(EXIT_FAILURE, "Invalid number of argument.\n"
              "Try `%s --help' for more information.",
              argv[0]);
@@ -757,15 +779,36 @@ int main(int argc, char **argv)
     bdrv_init();
     atexit(bdrv_close_all);
 
-    if (fmt) {
-        options = qdict_new();
-        qdict_put(options, "driver", qstring_from_str(fmt));
+    if (srcpath) {
+        char *file = NULL;
+        opts = qemu_opts_find(&file_opts, NULL);
+        if (fmt) {
+            errx(EXIT_FAILURE, "--source and -f are mutually exclusive");
+        }
+        if ((argc - optind) > 1) {
+            errx(EXIT_FAILURE, "--source and filename are mutually exclusive");
+        }
+        file = g_strdup(qemu_opt_get(opts, "file"));
+        qemu_opt_unset(opts, "file");
+        options = qemu_opts_to_qdict(opts, NULL);
+        qemu_opts_reset(&file_opts);
+        blk = blk_new_open("hda", file, NULL, options, flags, &local_err);
+        g_free(file);
+    } else {
+        if (fmt) {
+            options = qdict_new();
+            qdict_put(options, "driver", qstring_from_str(fmt));
+        }
+        if ((argc - optind) != 1) {
+            errx(EXIT_FAILURE, "one of --source or filename are expected");
+        }
+
+        srcpath = argv[optind];
+        blk = blk_new_open("hda", srcpath, NULL, options, flags, &local_err);
     }
 
-    srcpath = argv[optind];
-    blk = blk_new_open("hda", srcpath, NULL, options, flags, &local_err);
     if (!blk) {
-        errx(EXIT_FAILURE, "Failed to blk_new_open '%s': %s", argv[optind],
+        errx(EXIT_FAILURE, "Failed to blk_new_open '%s': %s", srcpath,
              error_get_pretty(local_err));
     }
     bs = blk_bs(blk);
Signature · 1 line
2.5.0
#17 lore raw

Re: [Qemu-devel] [PATCH 6/7] qemu-nbd: allow specifying image as a set of options args

Quoted 6 lines · Daniel P. Berrange

Currently qemu-nbd allows an image filename to be passed on the command line, but does not have a way to set any options except the format eg

   qemu-nbd https://127.0.0.1/images/centos7.iso
   qemu-nbd /home/berrange/demo.qcow2

Well, technically we have:
qemu-nbd json:{...}

but I'm fine with adding the nicer syntax.

Quoted 20 lines

This adds a --source arg (that is mutually exclusive with a positional filename arg and -f arg) that accepts a full option string, as well as the original syntax eg

   qemu-nbd --source driver=http,url=https://127.0.0.1/images,sslverify=off
   qemu-nbd --source https://127.0.0.1/images/centos7.iso
   qemu-nbd --source file=/home/berrange/demo.qcow2
   qemu-nbd --source /home/berrange/demo.qcow2
---
 qemu-nbd.c | 57 ++++++++++++++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 50 insertions(+), 7 deletions(-)
qemu-nbd.c +0 -0
index b4b6681..0b9f99e 100644

qemu-nbd.c line 52

Quoted 4 lines
qemu-nbd.c +1 -0
@@ -49,6 +49,7 @@
 #define QEMU_NBD_OPT_DISCARD       3
 #define QEMU_NBD_OPT_DETECT_ZEROES 4
 #define QEMU_NBD_OPT_OBJECT        5
+#define QEMU_NBD_OPT_SOURCE        6

Same comment as earlier about this being safer if it is > 255.

qemu-nbd.c int main(int argc, char **argv) · line 500

Quoted 5 lines
qemu-nbd.c +1 -0
@@ -486,6 +497,7 @@ int main(int argc, char **argv)
         { "persistent", 0, NULL, 't' },
         { "verbose", 0, NULL, 'v' },
         { "object", 1, NULL, QEMU_NBD_OPT_OBJECT },
+        { "source", 1, NULL, QEMU_NBD_OPT_SOURCE },

Same comment as earlier on spelling this as required_argument.

qemu-nbd.c int main(int argc, char **argv) · line 674

Quoted 10 lines
qemu-nbd.c +3 -0
         { NULL, 0, NULL, 0 }
     };
     int ch;
@@ -657,13 +669,23 @@ int main(int argc, char **argv)
                 exit(1);
             }
             break;
+        case QEMU_NBD_OPT_SOURCE:
+            if (srcpath) {
+                errx(EXIT_FAILURE, "--source can only be used once");

Markus has a series getting rid of errx() usage; you'll want to follow suit:
https://lists.gnu.org/archive/html/qemu-devel/2015-12/msg03492.html

Quoted 5 lines
qemu-nbd.c +5 -0
+            }
+            if (!qemu_opts_parse_noisily(&file_opts, optarg, true)) {
+                qemu_opts_reset(&file_opts);
+                exit(EXIT_FAILURE);
+            }

qemu-nbd.c int main(int argc, char **argv) · line 688

Quoted 10 lines
qemu-nbd.c +3 -1
+            srcpath = optarg;
+            break;
         case '?':
             errx(EXIT_FAILURE, "Try `%s --help' for more information.",
                  argv[0]);
         }
     }
 
-    if ((argc - optind) != 1) {
+    if ((argc - optind) > 1) {

Inner () are redundant.

Quoted 7 lines
qemu-nbd.c +0 -0
         errx(EXIT_FAILURE, "Invalid number of argument.\n"
              "Try `%s --help' for more information.",
              argv[0]);
@@ -757,15 +779,36 @@ int main(int argc, char **argv)
     bdrv_init();
     atexit(bdrv_close_all);
 

qemu-nbd.c int main(int argc, char **argv) · line 788

Quoted 10 lines
qemu-nbd.c +7 -3
-    if (fmt) {
-        options = qdict_new();
-        qdict_put(options, "driver", qstring_from_str(fmt));
+    if (srcpath) {
+        char *file = NULL;
+        opts = qemu_opts_find(&file_opts, NULL);
+        if (fmt) {
+            errx(EXIT_FAILURE, "--source and -f are mutually exclusive");
+        }
+        if ((argc - optind) > 1) {

and again.

qemu-nbd.c int main(int argc, char **argv) · line 789

Quoted 1 line
qemu-nbd.c +1 -0
+            errx(EXIT_FAILURE, "--source and filename are mutually exclusive");

More errx() to avoid.

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#18 lore raw

PATCH 7/7 qemu-img: allow specifying image as a set of options args

In reply to Daniel P. Berrange

Currently qemu-img allows an image filename to be passed on the command line, but does not have a way to set any options except the format eg

qemu-img info https://127.0.0.1/images/centos7.iso

This adds a --source arg (that is mutually exclusive with a positional filename arg and -f arg) that accepts a full option string, as well as the original syntax eg

   qemu-img info --source driver=http,url=https://127.0.0.1/images,sslverify=off
---
 include/qemu/option.h |   1 +
 qemu-img.c            | 474 ++++++++++++++++++++++++++++++++++++++++++--------
 util/qemu-option.c    |   6 +
 3 files changed, 407 insertions(+), 74 deletions(-)
include/qemu/option.h +1 -0
index 71f5f27..caf5a3b 100644
@@ -104,6 +104,7 @@ int qemu_opt_foreach(QemuOpts *opts, qemu_opt_loopfunc func, void *opaque,
                      Error **errp);
 
 QemuOpts *qemu_opts_find(QemuOptsList *list, const char *id);
+QemuOpts *qemu_opts_next(QemuOpts *opts);
 QemuOpts *qemu_opts_create(QemuOptsList *list, const char *id,
                            int fail_if_exists, Error **errp);
 void qemu_opts_reset(QemuOptsList *list);
qemu-img.c +400 -74

841 lines not on this page · raw message

util/qemu-option.c +6 -0
index a50ecea..48e1cc5 100644
@@ -637,6 +637,12 @@ QemuOpts *qemu_opts_find(QemuOptsList *list, const char *id)
     return NULL;
 }
 
+
+QemuOpts *qemu_opts_next(QemuOpts *opts)
+{
+    return QTAILQ_NEXT(opts, next);
+}
+
 QemuOpts *qemu_opts_create(QemuOptsList *list, const char *id,
                            int fail_if_exists, Error **errp)
 {
Signature · 1 line
2.5.0
#19 lore raw

Re: [Qemu-devel] [PATCH 7/7] qemu-img: allow specifying image as a set of options args

Quoted 11 lines · Daniel P. Berrange

Currently qemu-img allows an image filename to be passed on the command line, but does not have a way to set any options except the format eg

   qemu-img info https://127.0.0.1/images/centos7.iso

This adds a --source arg (that is mutually exclusive with a positional filename arg and -f arg) that accepts a full option string, as well as the original syntax eg

   qemu-img info --source driver=http,url=https://127.0.0.1/images,sslverify=off

Don't we also need this for destinations and their matching options, for 'compare'(-F), 'convert'(-O) [oh yikes: convert takes more than one input file - how will we support that?], and 'rebase'(-b/-F)?

The idea is nice, but I don't think we've fully covered the design space.

/me reads ahead

Oh, you DO add a --target, but didn't spell it out in the commit message...[1]

Quoted 5 lines
---
 include/qemu/option.h |   1 +
 qemu-img.c            | 474 ++++++++++++++++++++++++++++++++++++++++++--------
 util/qemu-option.c    |   6 +
 3 files changed, 407 insertions(+), 74 deletions(-)

And where's the documentation patches, for the man page?

include/qemu/option.h int qemu_opt_foreach(QemuOpts *opts, qemu_opt_loopfunc func, void *opaque, · line 107

Quoted 6 lines

+++ b/include/qemu/option.h

include/qemu/option.h +1 -0
@@ -104,6 +104,7 @@ int qemu_opt_foreach(QemuOpts *opts, qemu_opt_loopfunc func, void *opaque,
                      Error **errp);
 
 QemuOpts *qemu_opts_find(QemuOptsList *list, const char *id);
+QemuOpts *qemu_opts_next(QemuOpts *opts);

Should exposing this be a separate patch?

include/qemu/option.h int qemu_opt_foreach(QemuOpts *opts, qemu_opt_loopfunc func, void *opaque, · line 109

Quoted 2 lines
include/qemu/option.h +0 -0
 QemuOpts *qemu_opts_create(QemuOptsList *list, const char *id,
                            int fail_if_exists, Error **errp);

While we're touching QemuOpts, should we switch fail_if_exists to bool (separate patch)?

qemu-img.c line 41

Quoted 10 lines
include/qemu/option.h +0 -0
 void qemu_opts_reset(QemuOptsList *list);
qemu-img.c +1 -0
index 47f0006..7a6ce81 100644
@@ -38,6 +38,7 @@
 #include "block/blockjob.h"
 #include "block/qapi.h"
 #include <getopt.h>
+#include <err.h>

No. Markus is trying to get rid of this.
https://lists.gnu.org/archive/html/qemu-devel/2015-12/msg03492.html
because it does not give consistent error messages (things like timestamps, for example).

qemu-img.c enum { · line 56

Quoted 9 lines
qemu-img.c +2 -0
 #define QEMU_IMG_VERSION "qemu-img version " QEMU_VERSION QEMU_PKGVERSION \
                           ", Copyright (c) 2004-2008 Fabrice Bellard\n"
@@ -51,6 +52,8 @@ enum {
     OPTION_OUTPUT = 256,
     OPTION_BACKING_CHAIN = 257,
     OPTION_OBJECT = 258,
+    OPTION_SOURCE = 259,
+    OPTION_TARGET = 260,

[1]...not mentioned in the commit message.

Quoted 4 lines
qemu-img.c +4 -0
+static BlockBackend *img_open_opts(const char *id,
+                                   QemuOpts *opts, int flags)
+{
+    QDict *options;

qemu-img.c static int print_block_option_help(const char *filename, const char *fmt) · line 295

Quoted 10 lines
qemu-img.c +10 -0
+    Error *local_err = NULL;
+    char *file = NULL;
+    BlockBackend *blk;
+    file = g_strdup(qemu_opt_get(opts, "file"));
+    qemu_opt_unset(opts, "file");
+    options = qemu_opts_to_qdict(opts, NULL);
+    blk = blk_new_open(id, file, NULL, options, flags, &local_err);
+    if (!blk) {
+        error_report("Could not open '%s': %s", file ? file : "",
+                     error_get_pretty(local_err));

Markus' series adds the ability to prefix your message rather than calling error_get_pretty(). Particularly nice if blk_new_open() starts appending hints to the error. It might be worth waiting for his series to land and then rebase yours on top of his.

Quoted 9 lines
qemu-img.c +1 -0
@@ -1111,6 +1199,7 @@ static int img_compare(int argc, char **argv)
         static const struct option long_options[] = {
             {"help", no_argument, 0, 'h'},
             {"object", required_argument, 0, OPTION_OBJECT},
+            {"source", required_argument, 0, OPTION_SOURCE},
             {0, 0, 0, 0}
         };
         c = getopt_long(argc, argv, "hf:F:T:pqs",

qemu-img.c static int img_compare(int argc, char **argv) · line 1242

Quoted 6 lines
qemu-img.c +3 -0
@@ -1148,6 +1237,20 @@ static int img_compare(int argc, char **argv)
                 exit(1);
             }
             break;
+        case OPTION_SOURCE:
+            if (filename2) {
+                errx(EXIT_FAILURE, "--source can only be specified twice");

Weird. Yes, we're reading two files, but --source/--target might be easier to explain/enforce than --source/--source...

Quoted 25 lines
qemu-img.c +15 -3
+            } else if (filename1) {
+                filename2 = optarg;
+            } else {
+                filename1 = optarg;
+            }
+            opts = qemu_opts_parse_noisily(qemu_find_opts("source"),
+                                           optarg, true);
+            if (!opts) {
+                exit(1);
+            }
+            break;
         }
     }
 
@@ -1156,12 +1259,20 @@ static int img_compare(int argc, char **argv)
         progress = false;
     }
 
-
-    if (optind != argc - 2) {
-        error_exit("Expecting two image file names");
+    if (filename1) {
+        if (optind != argc) {
+            error_exit("--source and filenames are mutually exclusive");
+        }

qemu-img.c static int img_compare(int argc, char **argv) · line 1275

Quoted 10 lines
qemu-img.c +9 -0
+        if (!filename2) {
+            error_exit("Expecting two --source arguments");
+        }
+    } else {
+        if (optind != argc - 2) {
+            error_exit("Expecting two image file names");
+        }
+        filename1 = argv[optind++];
+        filename2 = argv[optind++];
     }

...I think you managed to enforce that there are either two --source or two positional arguments, and nothing else is valid, but it's not typical.

qemu-img.c static int img_compare(int argc, char **argv) · line 1297

Quoted 6 lines
qemu-img.c +4 -2
-    }
-    bs1 = blk_bs(blk1);
+    opts = qemu_opts_find(&qemu_source_opts, NULL);
+    if (opts) {
+        if (fmt1 || fmt2) {
+            error_report("--source and -f or -F are mutuall exclusive");

s/mutuall/mutually/

Quoted 10 lines
qemu-img.c +3 -3
@@ -1808,20 +1949,33 @@ static int img_convert(int argc, char **argv)
     }
     qemu_progress_init(progress, 1.0);
 
-
-    bs_n = argc - optind - 1;
-    out_filename = bs_n >= 1 ? argv[argc - 1] : NULL;
+    if (!bs_n) {
+        out_filename = (argc - optind - 1) >= 1 ? argv[argc - 1] : NULL;
+    }

qemu-img.c static int img_convert(int argc, char **argv) · line 1962

Quoted 10 lines
qemu-img.c +2 -2
 
     if (options && has_help_option(options)) {
         ret = print_block_option_help(out_filename, out_fmt);
         goto out;
     }
 
-    if (bs_n < 1) {
-        error_exit("Must specify image file name");
+    if (bs_n) {
+        if (argc > (optind + 1)) {

Redundant inner ().

Overall, I'm left wondering whether requiring '--source FOO' vs. positional 'FOO', and manually enforcing mutual exclusion between the two, is necessary, or if we could stick with positional. But I guess the main argument is backwards-compatibility: previously, using 'driver=file,file=/path/to/file' as a filename would try to look in a relative directory 'driver=file,file=', whereas your proposal of always using the new '--source' option would make it obvious that we are expecting to parse a QemuOpts string rather than defaulting to a literal file name.

On the other hand, the existing positional parameters have allowed 'file:file:with_weird_name' to explicitly specify that we want to use './file:with_weird_name' as a relative file in the current directory (that is, the first 'file:' prefix is sufficient to avoid any back-compat issues with any other possible change in interpretation to a prefix), so on that grounds, I'd argue that adding --source is not necessary, and we can just require users to write 'file:$string_that_might_now_be_QemuOpts' anywhere they used to use '$string_that_might_now_be_QemuOpts'.

Maybe other block developers have an opinion to offer on whether the last three patches in this series should be adding a new --source option as mutually exclusive with positional args, vs. just adding a new interpretation of the existing mandatory positional arguments?

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#20 lore raw

Re: [Qemu-devel] [PATCH 7/7] qemu-img: allow specifying image as a set of options args

Quoted 13 lines · Eric Blake
Quoted 11 lines · Daniel P. Berrange

Currently qemu-img allows an image filename to be passed on the command line, but does not have a way to set any options except the format eg

   qemu-img info https://127.0.0.1/images/centos7.iso

This adds a --source arg (that is mutually exclusive with a positional filename arg and -f arg) that accepts a full option string, as well as the original syntax eg

   qemu-img info --source driver=http,url=https://127.0.0.1/images,sslverify=off

[snip]

Quoted 21 lines

Overall, I'm left wondering whether requiring '--source FOO' vs. positional 'FOO', and manually enforcing mutual exclusion between the two, is necessary, or if we could stick with positional. But I guess the main argument is backwards-compatibility: previously, using 'driver=file,file=/path/to/file' as a filename would try to look in a relative directory 'driver=file,file=', whereas your proposal of always using the new '--source' option would make it obvious that we are expecting to parse a QemuOpts string rather than defaulting to a literal file name.

On the other hand, the existing positional parameters have allowed 'file:file:with_weird_name' to explicitly specify that we want to use './file:with_weird_name' as a relative file in the current directory (that is, the first 'file:' prefix is sufficient to avoid any back-compat issues with any other possible change in interpretation to a prefix), so on that grounds, I'd argue that adding --source is not necessary, and we can just require users to write 'file:$string_that_might_now_be_QemuOpts' anywhere they used to use '$string_that_might_now_be_QemuOpts'.

Quoted 4 lines

Maybe other block developers have an opinion to offer on whether the last three patches in this series should be adding a new --source option as mutually exclusive with positional args, vs. just adding a new interpretation of the existing mandatory positional arguments?

Yep, back compatibility to avoid breaking any existing possible filenames was my main motivation for adding '--source'. I agree it would be nice if we decided that the risk was acceptable based on what you say above, and thus avoid --source, and just extend existing positional args.

If block maintainers OK that approach, I'd happily rewrite the last 3 patches in this series in that manner.

Regards,
Daniel

Signature · 4 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|
#21 lore raw

Re: [Qemu-devel] [PATCH 7/7] qemu-img: allow specifying image as a set of options args

Quoted 10 lines · Daniel P. Berrange
Quoted 10 lines

Overall, I'm left wondering whether requiring '--source FOO' vs. positional 'FOO', and manually enforcing mutual exclusion between the two, is necessary, or if we could stick with positional. But I guess the main argument is backwards-compatibility: previously, using 'driver=file,file=/path/to/file' as a filename would try to look in a relative directory 'driver=file,file=', whereas your proposal of always using the new '--source' option would make it obvious that we are expecting to parse a QemuOpts string rather than defaulting to a literal file name.

Quoted 9 lines
Quoted 9 lines

On the other hand, the existing positional parameters have allowed 'file:file:with_weird_name' to explicitly specify that we want to use './file:with_weird_name' as a relative file in the current directory (that is, the first 'file:' prefix is sufficient to avoid any back-compat issues with any other possible change in interpretation to a prefix), so on that grounds, I'd argue that adding --source is not necessary, and we can just require users to write 'file:$string_that_might_now_be_QemuOpts' anywhere they used to use '$string_that_might_now_be_QemuOpts'.

I guess there's also the issue of literal commas.

Right now, we have:

$ echo hi > a,b
$ qemu-img info a,b
image: a,b
file format: raw
virtual size: 512 (512 bytes)
disk size: 4.0K
$ qemu-img info file:a,b
image: a,b
file format: raw
virtual size: 512 (512 bytes)
disk size: 4.0K

If we magically change things to interpret the positionals as QemuOpts strings, we'd have a change that:

$ qemu-img info a,b
$ qemu-img info file:a,b

would error out ('a' and 'file:a' are not a known options, and we are expecting =), but at the same time:

$ qemu-img info a,,b
$ qemu-img info file:a,,b

would start working (because the use of ',,' is the QemuOpts way to escape a literal ',' that is not separating options packed into the single argument).

Quoted 6 lines
Quoted 5 lines

Maybe other block developers have an opinion to offer on whether the last three patches in this series should be adding a new --source option as mutually exclusive with positional args, vs. just adding a new interpretation of the existing mandatory positional arguments?

Quoted 8 lines

Yep, back compatibility to avoid breaking any existing possible filenames was my main motivation for adding '--source'. I agree it would be nice if we decided that the risk was acceptable based on what you say above, and thus avoid --source, and just extend existing positional args.

If block maintainers OK that approach, I'd happily rewrite the last 3 patches in this series in that manner.

It may be mid-January before the decision is made, but we've got time before 2.6 soft freeze. I can wait :)

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#22 lore raw

Re: [Qemu-devel] [PATCH 7/7] qemu-img: allow specifying image as a set of options args

Quoted 46 lines · Eric Blake
Quoted 10 lines · Daniel P. Berrange
Quoted 10 lines

Overall, I'm left wondering whether requiring '--source FOO' vs. positional 'FOO', and manually enforcing mutual exclusion between the two, is necessary, or if we could stick with positional. But I guess the main argument is backwards-compatibility: previously, using 'driver=file,file=/path/to/file' as a filename would try to look in a relative directory 'driver=file,file=', whereas your proposal of always using the new '--source' option would make it obvious that we are expecting to parse a QemuOpts string rather than defaulting to a literal file name.

Quoted 9 lines
Quoted 9 lines

On the other hand, the existing positional parameters have allowed 'file:file:with_weird_name' to explicitly specify that we want to use './file:with_weird_name' as a relative file in the current directory (that is, the first 'file:' prefix is sufficient to avoid any back-compat issues with any other possible change in interpretation to a prefix), so on that grounds, I'd argue that adding --source is not necessary, and we can just require users to write 'file:$string_that_might_now_be_QemuOpts' anywhere they used to use '$string_that_might_now_be_QemuOpts'.

I guess there's also the issue of literal commas.

Right now, we have:

$ echo hi > a,b
$ qemu-img info a,b
image: a,b
file format: raw
virtual size: 512 (512 bytes)
disk size: 4.0K
$ qemu-img info file:a,b
image: a,b
file format: raw
virtual size: 512 (512 bytes)
disk size: 4.0K

If we magically change things to interpret the positionals as QemuOpts strings, we'd have a change that:

$ qemu-img info a,b
$ qemu-img info file:a,b

would error out ('a' and 'file:a' are not a known options, and we are expecting =), but at the same time:

Quoted 6 lines
$ qemu-img info a,,b
$ qemu-img info file:a,,b

would start working (because the use of ',,' is the QemuOpts way to escape a literal ',' that is not separating options packed into the single argument).

A third option would be to keep using positional arguments, but add a '--source-opts' *boolean* flag to indicate how to interpret the positional arguments. ie without --source-opts we use the historic syntax, but with --source-opts, we assume the full QemuOpts syntax.

Regards,
Daniel

Signature · 4 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|
#23 lore raw

Re: [Qemu-devel] [PATCH 7/7] qemu-img: allow specifying image as a set of options args

Quoted 5 lines · Daniel P. Berrange

A third option would be to keep using positional arguments, but add a '--source-opts' *boolean* flag to indicate how to interpret the positional arguments. ie without --source-opts we use the historic syntax, but with --source-opts, we assume the full QemuOpts syntax.

Oh, nice compromise. It's relatively discoverable (grep --help output), preserves back-compat of old scripts, and offers the full power for clients that want the full power.

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#24 lore raw

Re: [Qemu-devel] [PATCH 7/7] qemu-img: allow specifying image as a set of options args

Quoted 11 lines · Eric Blake
Quoted 5 lines · Daniel P. Berrange

A third option would be to keep using positional arguments, but add a '--source-opts' *boolean* flag to indicate how to interpret the positional arguments. ie without --source-opts we use the historic syntax, but with --source-opts, we assume the full QemuOpts syntax.

Oh, nice compromise. It's relatively discoverable (grep --help output), preserves back-compat of old scripts, and offers the full power for clients that want the full power.

I've implemented this now and it makes the patches soooo much simpler too, so an added win.

Regards,
Daniel

Signature · 4 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|
#25 lore raw

Re: [Qemu-devel] [PATCH 7/7] qemu-img: allow specifying image as a set of options args

Quoted 11 lines · Daniel P. Berrange
Quoted 9 lines
Quoted 9 lines
Quoted 5 lines
Quoted 5 lines

A third option would be to keep using positional arguments, but add a '--source-opts' *boolean* flag to indicate how to interpret the positional arguments. ie without --source-opts we use the historic syntax, but with --source-opts, we assume the full QemuOpts syntax.

Oh, nice compromise. It's relatively discoverable (grep --help output), preserves back-compat of old scripts, and offers the full power for clients that want the full power.

I've implemented this now and it makes the patches soooo much simpler too, so an added win.

Hmm, looks like I'm a bit late, but here's another possibility: making --source-options (aka -o) take an argument, and if you specify both --source-options and the positional argument, the latter is added as a "file" key.

This way you can do "qemu-img info --object secret... -o secret=foo iscsi://foo/".

Paolo

#26 lore raw

Re: [Qemu-devel] [PATCH 7/7] qemu-img: allow specifying image as a set of options args

Quoted 15 lines · Paolo Bonzini
Quoted 11 lines · Daniel P. Berrange
Quoted 9 lines
Quoted 9 lines
Quoted 5 lines
Quoted 5 lines

A third option would be to keep using positional arguments, but add a '--source-opts' *boolean* flag to indicate how to interpret the positional arguments. ie without --source-opts we use the historic syntax, but with --source-opts, we assume the full QemuOpts syntax.

Oh, nice compromise. It's relatively discoverable (grep --help output), preserves back-compat of old scripts, and offers the full power for clients that want the full power.

I've implemented this now and it makes the patches soooo much simpler too, so an added win.

Quoted 7 lines

Hmm, looks like I'm a bit late, but here's another possibility: making --source-options (aka -o) take an argument, and if you specify both --source-options and the positional argument, the latter is added as a "file" key.

This way you can do "qemu-img info --object secret... -o secret=foo iscsi://foo/".

This gets somewhat messy for the commands which needs to accept more than one filename, eg 'compare' (2 files) and 'convert' (any number of files)'. eg if you have 3 positional file names and want to set an option on the 2nd file, '-o foo=bar' is not really going to say which positional file it applies to. The style I just implemented with the boolean '--image-opts' flag avoids this messiness nicely.

Regards,
Daniel

Signature · 4 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|
#27 lore raw

Re: [Qemu-devel] [PATCH 7/7] qemu-img: allow specifying image as a set of options args

----- Original Message -----

Quoted 30 lines

From: "Daniel P. Berrange" <[email protected]>
To: "Paolo Bonzini" <[email protected]>
Cc: "Eric Blake" <[email protected]>, "Kevin Wolf" <[email protected]>, [email protected], [email protected], "Markus Armbruster" <[email protected]>, "Andreas Färber" <[email protected]>
Sent: Wednesday, December 23, 2015 8:23:13 PM
Subject: Re: [Qemu-devel] [PATCH 7/7] qemu-img: allow specifying image as a set of options args

Quoted 22 lines · Paolo Bonzini
Quoted 7 lines · Daniel P. Berrange
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
Quoted 6 lines

A third option would be to keep using positional arguments, but add a '--source-opts' *boolean* flag to indicate how to interpret the positional arguments. ie without --source-opts we use the historic syntax, but with --source-opts, we assume the full QemuOpts
syntax.

Quoted 6 lines
Quoted 4 lines
Quoted 4 lines

Oh, nice compromise. It's relatively discoverable (grep --help output),
preserves back-compat of old scripts, and offers the full power for clients that want the full power.

I've implemented this now and it makes the patches soooo much simpler too, so an added win.

Hmm, looks like I'm a bit late, but here's another possibility: making --source-options (aka -o) take an argument, and if you specify both --source-options and the positional argument, the latter is added as a "file" key.

Quoted 8 lines
Quoted 2 lines

This way you can do "qemu-img info --object secret... -o secret=foo iscsi://foo/".

This gets somewhat messy for the commands which needs to accept more than one filename, eg 'compare' (2 files) and 'convert' (any number of files)'. eg if you have 3 positional file names and want to set an option on the 2nd file, '-o foo=bar' is not really going to say which positional file it applies to.

That would be -o/-O, like -f/-F.

Paolo

#28 lore raw

Re: [Qemu-devel] [PATCH 7/7] qemu-img: allow specifying image as a set of options args

Quoted 36 lines · Paolo Bonzini

----- Original Message -----

Quoted 33 lines

From: "Daniel P. Berrange" <[email protected]>
To: "Paolo Bonzini" <[email protected]>
Cc: "Eric Blake" <[email protected]>, "Kevin Wolf" <[email protected]>, [email protected], [email protected], "Markus Armbruster" <[email protected]>, "Andreas Färber" <[email protected]>
Sent: Wednesday, December 23, 2015 8:23:13 PM
Subject: Re: [Qemu-devel] [PATCH 7/7] qemu-img: allow specifying image as a set of options args

Quoted 24 lines · Paolo Bonzini
Quoted 7 lines · Daniel P. Berrange
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
Quoted 6 lines

A third option would be to keep using positional arguments, but add a '--source-opts' *boolean* flag to indicate how to interpret the positional arguments. ie without --source-opts we use the historic syntax, but with --source-opts, we assume the full QemuOpts
syntax.

Quoted 6 lines
Quoted 4 lines
Quoted 4 lines

Oh, nice compromise. It's relatively discoverable (grep --help output),
preserves back-compat of old scripts, and offers the full power for clients that want the full power.

I've implemented this now and it makes the patches soooo much simpler too, so an added win.

Hmm, looks like I'm a bit late, but here's another possibility: making --source-options (aka -o) take an argument, and if you specify both --source-options and the positional argument, the latter is added as a "file" key.

This way you can do "qemu-img info --object secret... -o secret=foo iscsi://foo/".

Quoted 7 lines
Quoted 5 lines

This gets somewhat messy for the commands which needs to accept more than one filename, eg 'compare' (2 files) and 'convert' (any number of files)'. eg if you have 3 positional file names and want to set an option on the 2nd file, '-o foo=bar' is not really going to say which positional file it applies to.

That would be -o/-O, like -f/-F.

That only works for two files, unless you keep inventing yet more characters for the 3rd, 4th, ... file - 'convert' takes an arbitrary number of files.

Regards,
Daniel

Signature · 4 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|