RFC: few random hacks to improve eventfd fallback path
cover RFC: few random hacks to improve eventfd fallback path
These few patches are the result of a random hacking I did to make the qemu cope with eventfd allocation failure, when using an iothread, as it happened in bz #1897550.
I am not 100% sure which patches in this series are worth to merge, or if this can be fixed in a better way.
After this patch series applied, qemu still hangs while running reproducer for this BZ due to ABBA lock inversion which needs some heavy rework to get rid of. I explained all the (gory) details in the bugzilla.
This patch series was (lightly) tested with make check, iotests and with the reproducer.
Best regards,
Maxim Levitsky
Maxim Levitsky (3):
scsi: virtio-scsi: don't process IO on fenced dataplane
virtio-scsi: don't uninitialize queues that we didn't initialize
event_notifier: handle initialization failure better
hw/scsi/virtio-scsi-dataplane.c | 26 +++++++++++++++++++------- include/qemu/event_notifier.h | 1 + util/event_notifier-posix.c | 16 ++++++++++++++++ 3 files changed, 36 insertions(+), 7 deletions(-)
Signature · 1 line
2.26.2
PATCH 1/3 scsi: virtio-scsi: don't process IO on fenced dataplane
If virtio_scsi_dataplane_start fails, there is a small window when it drops the aio lock (in aio_wait_bh_oneshot) and the dataplane's AIO handler can still run during that window.
This is done after the dataplane was marked as fenced, thus we use this flag to avoid it doing any IO.
--- hw/scsi/virtio-scsi-dataplane.c | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-)
hw/scsi/virtio-scsi-dataplane.c +12 -6
@@ -56,8 +56,10 @@ static bool virtio_scsi_data_plane_handle_cmd(VirtIODevice *vdev,VirtIOSCSI *s =VIRTIO_SCSI (vdev);virtio_scsi_acquire (s); -assert (s->ctx && s->dataplane_started); - progress =virtio_scsi_handle_cmd_vq (s, vq); +if (!s->dataplane_fenced) { +assert (s->ctx && s->dataplane_started); + progress =virtio_scsi_handle_cmd_vq (s, vq); + }virtio_scsi_release (s);return progress; }
@@ -69,8 +71,10 @@ static bool virtio_scsi_data_plane_handle_ctrl(VirtIODevice *vdev,VirtIOSCSI *s =VIRTIO_SCSI (vdev);virtio_scsi_acquire (s); -assert (s->ctx && s->dataplane_started); - progress =virtio_scsi_handle_ctrl_vq (s, vq); +if (!s->dataplane_fenced) { +assert (s->ctx && s->dataplane_started); + progress =virtio_scsi_handle_ctrl_vq (s, vq); + }virtio_scsi_release (s);return progress; }
@@ -82,8 +86,10 @@ static bool virtio_scsi_data_plane_handle_event(VirtIODevice *vdev,VirtIOSCSI *s =VIRTIO_SCSI (vdev);virtio_scsi_acquire (s); -assert (s->ctx && s->dataplane_started); - progress =virtio_scsi_handle_event_vq (s, vq); +if (!s->dataplane_fenced) { +assert (s->ctx && s->dataplane_started); + progress =virtio_scsi_handle_event_vq (s, vq); + }virtio_scsi_release (s);return progress; }
Signature · 1 line
2.26.2
PATCH 2/3 virtio-scsi: don't uninitialize queues that we didn't initialize
In reply to Maxim LevitskyCount number of queues that we initialized and only deinitialize these that we initialized successfully.
--- hw/scsi/virtio-scsi-dataplane.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
hw/scsi/virtio-scsi-dataplane.c +7 -1
@@ -132,6 +132,7 @@ int virtio_scsi_dataplane_start(VirtIODevice *vdev) {int i;int rc; +int vq_init_count =0 ;BusState *qbus =qdev_get_parent_bus (DEVICE (vdev));VirtioBusClass *k =VIRTIO_BUS_GET_CLASS (qbus);VirtIOSCSICommon *vs =VIRTIO_SCSI_COMMON (vdev);
@@ -159,17 +160,22 @@ int virtio_scsi_dataplane_start(VirtIODevice *vdev)if (rc) { goto fail_vrings; } + + vq_init_count++; rc =virtio_scsi_vring_init (s, vs->event_vq,1 , virtio_scsi_data_plane_handle_event);if (rc) { goto fail_vrings; } + + vq_init_count++;for (i =0 ; i < vs->conf.num_queues; i++) { rc =virtio_scsi_vring_init (s, vs->cmd_vqs[i], i +2 , virtio_scsi_data_plane_handle_cmd);if (rc) { goto fail_vrings; } + vq_init_count++; } s->dataplane_starting = false;
@@ -180,7 +186,7 @@ int virtio_scsi_dataplane_start(VirtIODevice *vdev) fail_vrings:aio_wait_bh_oneshot (s->ctx, virtio_scsi_dataplane_stop_bh, s);aio_context_release (s->ctx); -for (i =0 ; i < vs->conf.num_queues +2 ; i++) { +for (i =0 ; i < vq_init_count; i++) {virtio_bus_set_host_notifier (VIRTIO_BUS (qbus), i, false);virtio_bus_cleanup_host_notifier (VIRTIO_BUS (qbus), i); }
Signature · 1 line
2.26.2
PATCH 3/3 event_notifier: handle initialization failure better
In reply to Maxim LevitskyAdd 'initialized' field and use it to avoid touching event notifiers which are either not initialized or if their initialization failed.
This is somewhat a hack, but it seems the less intrusive way to make virtio code deal with event notifiers that failed initialization.
--- include/qemu/event_notifier.h | 1 + util/event_notifier-posix.c | 16 ++++++++++++++++ 2 files changed, 17 insertions(+)
include/qemu/event_notifier.h +1 -0
@@ -24,6 +24,7 @@ struct EventNotifier {#else int rfd;int wfd; +bool initialized;#endif };
util/event_notifier-posix.c +16 -0
@@ -29,6 +29,7 @@ void event_notifier_init_fd(EventNotifier *e, int fd) { e->rfd = fd; e->wfd = fd; + e->initialized = true; }#endif
@@ -68,6 +69,7 @@ int event_notifier_init(EventNotifier *e, int active)if (active) {event_notifier_set (e); } + e->initialized = true;return 0 ; fail:
@@ -78,12 +80,18 @@ fail:void event_notifier_cleanup (EventNotifier *e) { +if (!e->initialized) { +return ; + } +if (e->rfd != e->wfd) {close (e->rfd); } + e->rfd =-1 ;close (e->wfd); e->wfd =-1 ; + e->initialized = false; }int event_notifier_get_fd (const EventNotifier *e)
@@ -96,6 +104,10 @@ int event_notifier_set(EventNotifier *e)static const uint64_t value =1 ;ssize_t ret; +if (!e->initialized) { +return -1 ; + } +do { ret =write (e->wfd, &value,sizeof (value)); }while (ret <0 && errno ==EINTR );
@@ -113,6 +125,10 @@ int event_notifier_test_and_clear(EventNotifier *e)ssize_t len;char buffer[512 ]; +if (!e->initialized) { +return 0 ; + } +/* Drain the notify pipe. For eventfd, only 8 bytes will be read. */ value =0 ; do {
Signature · 1 line
2.26.2
Quoted 22 lines · Maxim Levitsky
These few patches are the result of a random hacking I did to make the qemu cope with eventfd allocation failure, when using an iothread, as it happened in bz #1897550.
I am not 100% sure which patches in this series are worth to merge, or if this can be fixed in a better way.
After this patch series applied, qemu still hangs while running reproducer for this BZ due to ABBA lock inversion which needs some heavy rework to get rid of. I explained all the (gory) details in the bugzilla.
This patch series was (lightly) tested with make check, iotests and with the reproducer.
Best regards,
Maxim LevitskyMaxim Levitsky (3):
scsi: virtio-scsi: don't process IO on fenced dataplane
virtio-scsi: don't uninitialize queues that we didn't initialize
event_notifier: handle initialization failure better
Quoted 9 lines
hw/scsi/virtio-scsi-dataplane.c | 26 +++++++++++++++++++------- include/qemu/event_notifier.h | 1 + util/event_notifier-posix.c | 16 ++++++++++++++++ 3 files changed, 36 insertions(+), 7 deletions(-)Signature · 1 line
2.26.2
Any update on this?
Best regards,
Maxim Levitsky
Paolo Bonzini
AppliedQuoted 22 lines · Maxim Levitsky
These few patches are the result of a random hacking I did to make the qemu cope with eventfd allocation failure, when using an iothread, as it happened in bz #1897550.
I am not 100% sure which patches in this series are worth to merge, or if this can be fixed in a better way.
After this patch series applied, qemu still hangs while running reproducer for this BZ due to ABBA lock inversion which needs some heavy rework to get rid of. I explained all the (gory) details in the bugzilla.
This patch series was (lightly) tested with make check, iotests and with the reproducer.
Best regards,
Maxim LevitskyMaxim Levitsky (3):
scsi: virtio-scsi: don't process IO on fenced dataplane
virtio-scsi: don't uninitialize queues that we didn't initialize
event_notifier: handle initialization failure better
Quoted 5 lines
hw/scsi/virtio-scsi-dataplane.c | 26 +++++++++++++++++++------- include/qemu/event_notifier.h | 1 + util/event_notifier-posix.c | 16 ++++++++++++++++ 3 files changed, 36 insertions(+), 7 deletions(-)
Queued, thanks.
Paolo