Skip to contentSkip to conversation

libxl: Enable save/restore/migration of a restricted QEMU + libxl__ev_qmp_*

PATCH New Anthony PERARD 1 participant 2 messages Jan 11, 2019

lore
Share & tools
lore
#0 lore raw

v9 cover libxl: Enable save/restore/migration of a restricted QEMU + libxl__ev_qmp_*

Patch series available in this git branch:
https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git br.libxl-ev-qmp-v9

Changes in v9:
fix on assert in "libxl_qmp: Implementation of libxl__ev_qmp_*". all patch acked.

Cheers,

Anthony PERARD (17):
libxl: Enhance libxl__sendmsg_fds to deal with EINTR and EWOULDBLOCK
libxl_qmp: Separate QMP message generation from qmp_send_prepare
libxl_qmp: Change qmp_qemu_check_version to compare version
libxl: Add wrapper around libxl__json_object_to_json JSON
libxl: Design of an async API to issue QMP commands to QEMU
libxl_qmp: Implementation of libxl__ev_qmp_*
libxl_exec: Add libxl__spawn_initiate_failure
libxl: Add init/dispose of for libxl__domain_build_state
libxl_dm: Pre-open QMP socket for QEMU
libxl: Add dmss_init/dispose for libxl__dm_spawn_state
libxl: QEMU startup sync based on QMP
libxl_qmp: Store advertised QEMU version in libxl__ev_qmp
libxl: Change libxl__domain_suspend_device_model() to be async
libxl: Re-implement domain_suspend_device_model using libxl__ev_qmp
libxl: Remove unused arg from libxl__sendmsg_fds
libxl_json: Remove libxl__json_object_append_to from header
libxl: Add comments to libxl__json_*get* functions

 tools/libxl/libxl_aoutils.c     |   2 +-
 tools/libxl/libxl_create.c      |  54 +-
 tools/libxl/libxl_dm.c          | 143 ++++-
 tools/libxl/libxl_dom_suspend.c |  37 +-
 tools/libxl/libxl_exec.c        |  11 +-
 tools/libxl/libxl_internal.h    | 197 ++++++-
 tools/libxl/libxl_json.c        |   9 +-
 tools/libxl/libxl_qmp.c         | 991 ++++++++++++++++++++++++++++++--
 tools/libxl/libxl_types.idl     |   7 +
 tools/libxl/libxl_utils.c       |  28 +-
 10 files changed, 1356 insertions(+), 123 deletions(-)
Signature · 7 lines
Anthony PERARD


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#1 lore raw

PATCH v9 6/17 libxl_qmp: Implementation of libxl__ev_qmp_*

This patch implement the API libxl__ev_qmp documented in the previous patch, "libxl: Design of an async API to issue QMP commands to QEMU".

Since this API is to interact with QEMU via the QMP protocol, it also implement a QMP client. The specification for the QEMU Machine Protocol (QMP) can be found in the QEMU repository at:
https://git.qemu.org/?p=qemu.git;a=blob_plain;f=docs/interop/qmp-spec.txt

---

Notes:
v9:
- assert(r>0) of the write return value
- acked

v8:
- using STATE_AO_GC everywhere
- call qmp_ev_ensure_reading_writing from qmp_ev_set_state instead of other places. But add a call to qmp_ev_ensure_reading_writing when the tx_buf is updated (data been transmitted to QEMU)
- assert return value of write().

- fix implementation doc of rx_buf (can be free when used)
- fix doc of qmp_ev_ensure_reading_writing

- reorder qmp_ev_parse_error_messages and qmp_ev_handle_message

v7:
- Make use of the new ao field filled by the caller
(no more free(tx_buf))
- Have reworked the state table to be more accurate, easier to read and with better desc of the waiting_reply state with sub-states as well as better description of the state of the ev_fd efd.
- There is state transition changes, now we have cap.neg -> waiting_reply (before it was cap.neg -> connected -> waiting_reply), that makes the internal connected state the same as the external Connected state.
- there are now 3 different id variable instead of a single one, the currently sent one id, the next one next_id, and the one associated with the caller's command msg_id, id will always be the for the next expected message from QEMU.
- qmp_ev_fd_callback now check for POLLERR, and read async error with getsockopt(SO_ERROR).
- qmp_ev_callback_readable takes care once again to also parse messages. But it now first attempt to read messages that would be in the rx_buf before reading from the socket.
- Some functions state changes have been updated, qmp_ev_ensure_reading_writing, qmp_ev_set_state, qmp_ev_callback_writable, qmp_ev_callback_readable
- Have cleaned up qmp_ev_ensure_reading_writing
- Have cleaned up qmp_error_class_to_libxl_error_code, and log unknown error classes.
- In qmp_ev_callback_readable, the size increase of the buffer have been reworked to update both rx_buf_size and rx_buf at the same time, and remove the use of max()
- Return value of qmp_prepare_cmd calles is checked and logged on error.
- comments in RHS of struct
- New qmp_ev_tx_buf_clear func
- Add a link to the QMP spec.
- update callers of qmp_prepare_cmd, which doesn't provide the string lenght anymore also remove ev->msg_len
- rename internal fields qmp_cfd, qmp_efd, qmp_state to cfd, efd and state.
- change qmp_ev_connect to only allow disconnect on entry
- squash qmp_ev_prepare_cmd into libxl__ev_qmp_send
- reduce max rx buffer size to 1M
query-vcpus with 71 cpus active yield 14484 bytes.
- use JSON to print libxl__json_object

v6.2:
Add definition of the internal broken state updated comments about states
v6.1:
Adding some comment about possible internal state changes

v6:
This is a squash of 7 commits on the previous version:
- libxl_qmp: Connect to QMP socket
- libxl_qmp: Implement fd callback and read data
- libxl_qmp: Parse JSON input from QMP
- libxl_qmp: Prepare the command to be sent
- libxl_qmp: Handle write to QMP socket
- libxl_qmp: Handle messages from QEMU
- libxl_qmp: Respond to QMP greeting

General rework of the implementation.
Added more comment, with a description of allowed internal states. Check for EINPROGRESS after connect().
Read until EWOULDBLOCK.
Handle EWOULDBLOCK on write and sendmsg.
Using memmem instead of strstr.
Using memmove instead of having an offset in rx_buf. Rework buffer allocation
Don't feed \r into json parser anymore
Add a check for a maximum RX buffer size
Added more error messages
New error code ERROR_PROTOCOL_ERROR_QMP
Rewrite conversion of QMP ErrorClass to libxl_error code Added helpers: qmp_ev_ensure_reading_writing, qmp_ev_set_state Split some functions, squash others
Added ev->msg* to store generated user command as tx_buf is used during
connection (for qmp_capabilities)
Remove qmp_state_greeting
Added qmp_state_waiting_reply

v5:
nits
use a define instead of a static int for QMP_CAPABILITY_NEGOCIATION_MSGID use a switch in qmp_ev_callback_writable to check qmp_state Add a description of the different value of libxl__qmp_state enum. some cleanup
remove read loop that only handled EINTR, simply return initialize len and s at declaration time
remove old comment
rename buf_fd to send_fd
Adding default:abort() in qmp_ev_handle_message.

v4:
remove use of a linked list of receive buffer, and use realloc instead. simplification of the patch due to use of a single allocated space for the receive buffer.

 tools/libxl/libxl_internal.h |  34 ++
 tools/libxl/libxl_qmp.c      | 739 +++++++++++++++++++++++++++++++++++
 tools/libxl/libxl_types.idl  |   6 +
 3 files changed, 779 insertions(+)
tools/libxl/libxl_internal.h +34 -0
index fadafa5f11..be18d4f341 100644
@@ -418,6 +418,19 @@ _hidden int libxl__ev_qmp_send(libxl__gc *gc, libxl__ev_qmp *ev,
                                const char *cmd, libxl__json_object *args);
 _hidden void libxl__ev_qmp_dispose(libxl__gc *gc, libxl__ev_qmp *ev);
 
+typedef enum {
+    /* initial state */
+    qmp_state_disconnected = 1,
+    /* connected to QMP socket, waiting for greeting message */
+    qmp_state_connecting,
+    /* qmp_capabilities command sent, waiting for reply */
+    qmp_state_capability_negotiation,
+    /* sending user's cmd and waiting for reply */
+    qmp_state_waiting_reply,
+    /* ready to send commands */
+    qmp_state_connected,
+} libxl__qmp_state;
+
 struct libxl__ev_qmp {
     /* caller should include this in their own struct */
     /* caller must fill these in, and they must all remain valid */
@@ -425,6 +438,27 @@ struct libxl__ev_qmp {
     libxl_domid domid;
     libxl__ev_qmp_callback *callback;
     int payload_fd; /* set to send a fd with the command, -1 otherwise */
+
+    /*
+     * remaining fields are private to libxl_ev_qmp_*
+     */
+
+    libxl__carefd *cfd;
+    libxl__ev_fd efd;
+    libxl__qmp_state state;
+    int id;
+    int next_id;        /* next id to use */
+    /* receive buffer */
+    char *rx_buf;
+    size_t rx_buf_size; /* current allocated size */
+    size_t rx_buf_used; /* actual data in the buffer */
+    /* sending buffer */
+    char *tx_buf;
+    size_t tx_buf_len;  /* tx_buf size */
+    size_t tx_buf_off;  /* already sent */
+    /* The message to send when ready */
+    char *msg;
+    int msg_id;
 };
 
 
tools/libxl/libxl_qmp.c +739 -0

756 lines not on this page · raw message

tools/libxl/libxl_types.idl +6 -0
index 141c46e42a..212b00a677 100644
@@ -69,6 +69,12 @@ libxl_error = Enumeration("error", [
     (-23, "NOTFOUND"),
     (-24, "DOMAIN_DESTROYED"), # Target domain ceased to exist during op
     (-25, "FEATURE_REMOVED"), # For functionality that has been removed
+    (-26, "PROTOCOL_ERROR_QMP"),
+    (-27, "UNKNOWN_QMP_ERROR"),
+    (-28, "QMP_GENERIC_ERROR"), # unspecified qmp error
+    (-29, "QMP_COMMAND_NOT_FOUND"), # the requested command has not been found
+    (-30, "QMP_DEVICE_NOT_ACTIVE"), # a device has failed to be become active
+    (-31, "QMP_DEVICE_NOT_FOUND"), # the requested device has not been found
     ], value_namespace = "")
 
 libxl_domain_type = Enumeration("domain_type", [
Signature · 7 lines
Anthony PERARD


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel