Skip to contentSkip to conversation

add watermark reporting for block devices

PATCH Superseded Francesco Romani 4 participants 9 messages Jul 8, 2014 – Aug 8, 2014

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 add watermark reporting for block devices

Hello everyone

I'm one of the oVirt developers (http://www.ovirt.org);
oVirt is a virtualization management application built around qemu/kvm, so it is nice to get in touch :)

We have begun a big scalability improvement effort, aiming to support without problems hundreds of VMs per host, with plans to support thousands in a not so distant future.
In doing so, we are reviewing our usage flows.

One of them is thin-provisioned storage, which is used quite extensively, with block devices (ISCSI for example) and COW images.
When using thin provisioning, oVirt tries hard to hide this fact from the guest OS, and to do so watches closely the usage of the device, and resize it when its usage exceeds a configured threshold (the "high water mark"), in order to avoid the guest OS to get paused for space exhausted.

To do the watching, we poll he devices using libvirt (virDomainGetBlockInfo), which in turn uses query-blockstats. This is suboptimal with just one VM, but with hundereds of them, let alone thousands, it doesn't scale and it is quite a resource hog.

Would be great to have this watermark concept supported into qemu, with a new event to be raised when the limit is crossed.

To track this RFE I filed https://bugs.launchpad.net/qemu/+bug/1338957

Moreover, I had the chance to take a look at the QEMU sources and come up with this tentative patch which I'd also like to submit.

Comments and thoughts very welcome!

Thanks and best regards,

Francesco Romani (1):
block: add watermark event

 block.c                   | 56 +++++++++++++++++++++++++++++++++++++++++++++++
 blockdev.c                | 21 ++++++++++++++++++
 include/block/block.h     |  2 ++
 include/block/block_int.h |  3 +++
 qapi/block-core.json      | 33 ++++++++++++++++++++++++++++
 qmp-commands.hx           | 24 ++++++++++++++++++++
 6 files changed, 139 insertions(+)
Signature · 1 line
1.9.3
#1 lore raw

PATCH 1/1 block: add watermark event

Managing applications, like oVirt (http://www.ovirt.org), make extensive use of thin-provisioned disk images.
In order to let the guest run flawlessly and be not unnecessarily paused, oVirt sets a watermark based on the percentage occupation of the device against the advertised size, and automatically resizes the image once the watermark is reached or exceeded.

In order to detect the mark crossing, the managing application has no choice than aggressively polling the QEMU monitor using the query-blockstats command. This lead to unnecessary system load, and is made even worse under scale: scenarios
with hundreds of VM are becoming not unusual.

To fix this, this patch adds:
* A new monitor command to set a mark for a given block device.
* A new event to report if a block device usage exceeds the threshold.

This will allow the managing application to drop the polling alltogether and just wait for a watermark crossing event.

---
 block.c                   | 56 +++++++++++++++++++++++++++++++++++++++++++++++
 blockdev.c                | 21 ++++++++++++++++++
 include/block/block.h     |  2 ++
 include/block/block_int.h |  3 +++
 qapi/block-core.json      | 33 ++++++++++++++++++++++++++++
 qmp-commands.hx           | 24 ++++++++++++++++++++
 6 files changed, 139 insertions(+)
block.c +56 -0
index 8800a6b..cf34b7f 100644
@@ -1983,6 +1983,8 @@ static void bdrv_move_feature_fields(BlockDriverState *bs_dest,
     bs_dest->device_list = bs_src->device_list;
     memcpy(bs_dest->op_blockers, bs_src->op_blockers,
            sizeof(bs_dest->op_blockers));
+
+    bs_dest->wr_watermark_perc  = bs_src->wr_watermark_perc;
 }
 
 /*
@@ -5813,3 +5815,57 @@ void bdrv_flush_io_queue(BlockDriverState *bs)
         bdrv_flush_io_queue(bs->file);
     }
 }
+
+static bool watermark_exceeded(BlockDriverState *bs,
+                               int64_t sector_num,
+                               int nb_sectors)
+{
+
+    if (bs->wr_watermark_perc > 0) {
+        int64_t watermark = (bs->total_sectors) / 100 * bs->wr_watermark_perc;
+        if (sector_num >= watermark) {
+            return true;
+        }
+    }
+    return false;
+}
+
+static int coroutine_fn watermark_before_write_notify(NotifierWithReturn *notifier,
+                                                      void *opaque)
+{
+    BdrvTrackedRequest *req = opaque;
+    int64_t sector_num = req->offset >> BDRV_SECTOR_BITS;
+    int nb_sectors = req->bytes >> BDRV_SECTOR_BITS;
+
+/*  FIXME: needed? */
+    assert((req->offset & (BDRV_SECTOR_SIZE - 1)) == 0);
+    assert((req->bytes & (BDRV_SECTOR_SIZE - 1)) == 0);
+
+    if (watermark_exceeded(req->bs, sector_num, nb_sectors)) {
+        BlockDriverState *bs = req->bs;
+        qapi_event_send_block_watermark(
+            bdrv_get_device_name(bs),
+            sector_num,
+            bs->wr_highest_sector,
+            &error_abort);
+    }
+
+    return 0; /* should always let other notifiers run */
+}
+
+void bdrv_set_watermark_perc(BlockDriverState *bs,
+                             int watermark_perc)
+{
+    NotifierWithReturn before_write = {
+        .notify = watermark_before_write_notify,
+    };
+
+    if (watermark_perc <= 0) {
+        return;
+    }
+
+    if (bs->wr_watermark_perc == 0) {
+        bdrv_add_before_write_notifier(bs, &before_write);
+    }
+    bs->wr_watermark_perc = watermark_perc;
+}
blockdev.c +21 -0
index 48bd9a3..ede21d9 100644
@@ -2546,6 +2546,27 @@ BlockJobInfoList *qmp_query_block_jobs(Error **errp)
     return dummy.next;
 }
 
+void qmp_block_set_watermark(const char *device, int64_t watermark,
+                             Error **errp)
+{
+    BlockDriverState *bs;
+    AioContext *aio_context;
+
+    bs = bdrv_find(device);
+    if (!bs) {
+        error_set(errp, QERR_DEVICE_NOT_FOUND, device);
+        return;
+    }
+
+    aio_context = bdrv_get_aio_context(bs);
+    aio_context_acquire(aio_context);
+
+    bdrv_set_watermark_perc(bs, watermark);
+
+    aio_context_release(aio_context);
+}
+
+
 QemuOptsList qemu_common_drive_opts = {
     .name = "drive",
     .head = QTAILQ_HEAD_INITIALIZER(qemu_common_drive_opts.head),
include/block/block.h +2 -0
index 32d3676..ff92ef9 100644
@@ -588,4 +588,6 @@ void bdrv_io_plug(BlockDriverState *bs);
 void bdrv_io_unplug(BlockDriverState *bs);
 void bdrv_flush_io_queue(BlockDriverState *bs);
 
+void bdrv_set_watermark_perc(BlockDriverState *bs, int watermark_perc);
+
 #endif
include/block/block_int.h +3 -0
index f6c3bef..666ea1d 100644
@@ -393,6 +393,9 @@ struct BlockDriverState {
 
     /* The error object in use for blocking operations on backing_hd */
     Error *backing_blocker;
+
+    /* watermark limit for writes, percentage of sectors */
+    int wr_watermark_perc;
 };
 
 int get_tmp_filename(char *filename, int size);
qapi/block-core.json +33 -0
index e378653..58e3b05 100644
@@ -1643,3 +1643,36 @@
             'len'   : 'int',
             'offset': 'int',
             'speed' : 'int' } }
+
+##
+# @BLOCK_WATERMARK
+#
+# Emitted when a block device reaches or exceeds the watermark limit.
+#
+# @device: device name
+#
+# @sector-num: number of the sector exceeding the threshold
+#
+# @sector-highest: number of the last highest written sector
+#
+# Since: 2.1
+##
+{ 'event': 'BLOCK_WATERMARK',
+  'data': { 'device': 'str', 'sector-num': 'int', 'sector-highest': 'int' } }
+
+##
+# @block_set_watermark
+#
+# Change watermark percentage for a block drive.
+#
+# @device: The name of the device
+#
+# @watermark: high water mark, percentage
+#
+# Returns: Nothing on success
+#          If @device is not a valid block device, DeviceNotFound
+#
+# Since: 2.1
+##
+{ 'command': 'block_set_watermark',
+  'data': { 'device': 'str', 'watermark': 'int' } }
qmp-commands.hx +24 -0
index 4be4765..89fee40 100644
@@ -3755,3 +3755,27 @@ Example:
 <- { "return": {} }
 
 EQMP
+
+    {
+        .name       = "block_set_watermark",
+        .args_type  = "device:B,watermark:l",
+        .mhandler.cmd_new = qmp_marshal_input_block_set_watermark,
+    },
+
+SQMP
+block_set_watermark
+------------
+
+Change the high water mark for a block drive.
+
+Arguments:
+
+- "device": device name (json-string)
+- "watermark": the high water mark in percentage (json-int)
+
+Example:
+
+-> { "execute": "block_set_watermark", "arguments": { "device": "virtio0", "watermark": 75 } }
+<- { "return": {} }
+
+EQMP
Signature · 1 line
1.9.3
#2 lore raw

Re: [Qemu-devel] [PATCH] block: add watermark event

Quoted 13 lines · Francesco Romani

Managing applications, like oVirt (http://www.ovirt.org), make extensive use of thin-provisioned disk images.
In order to let the guest run flawlessly and be not unnecessarily paused, oVirt sets a watermark based on the percentage occupation of the device against the advertised size, and automatically resizes the image once the watermark is reached or exceeded.

In order to detect the mark crossing, the managing application has no choice than aggressively polling the QEMU monitor using the query-blockstats command. This lead to unnecessary system load, and is made even worse under scale: scenarios
with hundreds of VM are becoming not unusual.

Quoted 6 lines

To fix this, this patch adds:
* A new monitor command to set a mark for a given block device.
* A new event to report if a block device usage exceeds the threshold.

This will allow the managing application to drop the polling alltogether and just wait for a watermark crossing event.

s/alltogether/altogether/

Quoted 13 lines
---
 block.c                   | 56 +++++++++++++++++++++++++++++++++++++++++++++++
 blockdev.c                | 21 ++++++++++++++++++
 include/block/block.h     |  2 ++
 include/block/block_int.h |  3 +++
 qapi/block-core.json      | 33 ++++++++++++++++++++++++++++
 qmp-commands.hx           | 24 ++++++++++++++++++++
 6 files changed, 139 insertions(+)

block.c void bdrv_flush_io_queue(BlockDriverState *bs) · line 5865

Quoted 10 lines
block.c +10 -0
+void bdrv_set_watermark_perc(BlockDriverState *bs,
+                             int watermark_perc)
+{
+    NotifierWithReturn before_write = {
+        .notify = watermark_before_write_notify,
+    };
+
+    if (watermark_perc <= 0) {
+        return;
+    }

Shouldn't you uninstall the write_notifier if someone is changing the watermark_perc from non-zero back to zero?

include/block/block_int.h struct BlockDriverState { · line 398

Quoted 8 lines

+++ b/include/block/block_int.h

include/block/block_int.h +3 -0
@@ -393,6 +393,9 @@ struct BlockDriverState {
 
     /* The error object in use for blocking operations on backing_hd */
     Error *backing_blocker;
+
+    /* watermark limit for writes, percentage of sectors */
+    int wr_watermark_perc;

I didn't see any code that ensures that this limit is between 0 and 100; since it is under user control, your code probably misbehaves for values such as 101.

qapi/block-core.json line 1658

Quoted 13 lines
qapi/block-core.json +13 -0
+
+##
+# @BLOCK_WATERMARK
+#
+# Emitted when a block device reaches or exceeds the watermark limit.
+#
+# @device: device name
+#
+# @sector-num: number of the sector exceeding the threshold
+#
+# @sector-highest: number of the last highest written sector
+#
+# Since: 2.1

2.2. You've missed hard freeze for 2.1.

qapi/block-core.json line 1664

Quoted 6 lines
qapi/block-core.json +6 -0
+##
+{ 'event': 'BLOCK_WATERMARK',
+  'data': { 'device': 'str', 'sector-num': 'int', 'sector-highest': 'int' } }
+
+##
+# @block_set_watermark

s/_/-/2 - new commands should use - in their name.

qapi/block-core.json line 1670

Quoted 6 lines
qapi/block-core.json +6 -0
+#
+# Change watermark percentage for a block drive.
+#
+# @device: The name of the device
+#
+# @watermark: high water mark, percentage

Is percentage the right unit? On a 10G disk, that means you only have a granularity down to 100M. Should the watermark limit instead be expressed as a byte value instead of a percentage?

qapi/block-core.json line 1675

Quoted 5 lines
qapi/block-core.json +5 -0
+#
+# Returns: Nothing on success
+#          If @device is not a valid block device, DeviceNotFound
+#
+# Since: 2.1

2.2.

qapi/block-core.json line 1678

Quoted 3 lines
qapi/block-core.json +3 -0
+##
+{ 'command': 'block_set_watermark',
+  'data': { 'device': 'str', 'watermark': 'int' } }

I hate write-only commands. Which query-* command are you modifying to output the current watermark level?

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#3 lore raw

Re: [Qemu-devel] [PATCH] block: add watermark event

In reply to Francesco Romani

block.c void bdrv_flush_io_queue(BlockDriverState *bs) · line 5825

Quoted 12 lines · Francesco Romani
block.c +8 -0
@@ -5813,3 +5815,57 @@ void bdrv_flush_io_queue(BlockDriverState *bs)
         bdrv_flush_io_queue(bs->file);
     }
 }
+
+static bool watermark_exceeded(BlockDriverState *bs,
+                               int64_t sector_num,
+                               int nb_sectors)
+{
+
+    if (bs->wr_watermark_perc > 0) {
+        int64_t watermark = (bs->total_sectors) / 100 * bs->wr_watermark_perc;

bs->total_sectors should not be used directly.

Have you considered making the watermark parameter take sector units instead of a percentage?

I'm not sure whether a precentage makes sense because 25% of a 10GB image is 2.5 GB so a 75% watermark might be reasonable. 25% of a 1 TB image is 250 GB and that's probably not a reasonable watermark.

So let the block-set-watermark caller pass an absolute sector number instead. It keeps things simple for both QEMU and thin provisioning manager.

Quoted 7 lines
block.c +7 -0
+        if (sector_num >= watermark) {
+            return true;
+        }
+    }
+    return false;
+}
+

block.c void bdrv_flush_io_queue(BlockDriverState *bs) · line 5842

Quoted 10 lines
block.c +10 -0
+static int coroutine_fn watermark_before_write_notify(NotifierWithReturn *notifier,
+                                                      void *opaque)
+{
+    BdrvTrackedRequest *req = opaque;
+    int64_t sector_num = req->offset >> BDRV_SECTOR_BITS;
+    int nb_sectors = req->bytes >> BDRV_SECTOR_BITS;
+
+/*  FIXME: needed? */
+    assert((req->offset & (BDRV_SECTOR_SIZE - 1)) == 0);
+    assert((req->bytes & (BDRV_SECTOR_SIZE - 1)) == 0);

Not really needed here. Emulated storage controllers either get requests in block units (i.e. they are automatically aligned) or check them (like virtio-blk).

I guess there's no harm in checking, but I would drop it.

block.c void bdrv_flush_io_queue(BlockDriverState *bs) · line 5850

Quoted 8 lines
block.c +8 -0
+
+    if (watermark_exceeded(req->bs, sector_num, nb_sectors)) {
+        BlockDriverState *bs = req->bs;
+        qapi_event_send_block_watermark(
+            bdrv_get_device_name(bs),
+            sector_num,
+            bs->wr_highest_sector,
+            &error_abort);

How do you prevent flooding events if every write request exceeds the watermark?

Perhaps the watermark should be disabled until block-set-watermark is called again.

Quoted 8 lines
block.c +8 -0
+    }
+
+    return 0; /* should always let other notifiers run */
+}
+
+void bdrv_set_watermark_perc(BlockDriverState *bs,
+                             int watermark_perc)
+{

block.c void bdrv_flush_io_queue(BlockDriverState *bs) · line 5868

Quoted 10 lines
block.c +10 -0
+    NotifierWithReturn before_write = {
+        .notify = watermark_before_write_notify,
+    };
+
+    if (watermark_perc <= 0) {
+        return;
+    }
+
+    if (bs->wr_watermark_perc == 0) {
+        bdrv_add_before_write_notifier(bs, &before_write);

before_write must be a BlockDriverState field so it has the correct lifetime. In this patch before_write is allocated on the stack and will cause invalid memory accesses once we leave this function.

#4 lore raw

Re: [Qemu-devel] [PATCH] block: add watermark event

Am 01.08.2014 um 13:39 hat Stefan Hajnoczi geschrieben:

Quoted 19 lines · Francesco Romani

block.c void bdrv_flush_io_queue(BlockDriverState *bs) · line 5825

Quoted 12 lines
block.c +8 -0
@@ -5813,3 +5815,57 @@ void bdrv_flush_io_queue(BlockDriverState *bs)
         bdrv_flush_io_queue(bs->file);
     }
 }
+
+static bool watermark_exceeded(BlockDriverState *bs,
+                               int64_t sector_num,
+                               int nb_sectors)
+{
+
+    if (bs->wr_watermark_perc > 0) {
+        int64_t watermark = (bs->total_sectors) / 100 * bs->wr_watermark_perc;

bs->total_sectors should not be used directly.

Have you considered making the watermark parameter take sector units instead of a percentage?

Quoted 7 lines

I'm not sure whether a precentage makes sense because 25% of a 10GB image is 2.5 GB so a 75% watermark might be reasonable. 25% of a 1 TB image is 250 GB and that's probably not a reasonable watermark.

So let the block-set-watermark caller pass an absolute sector number instead. It keeps things simple for both QEMU and thin provisioning manager.

No sector numbers in external interfaces, please. These units of 512 bytes are completely arbitrary and don't make any sense. I hope to get rid of BDRV_SECTOR_* eventually even internally.

So for external APIs, please use bytes instead.

Quoted 13 lines
Quoted 13 lines
block.c +13 -0
+        if (sector_num >= watermark) {
+            return true;
+        }
+    }
+    return false;
+}
+
+static int coroutine_fn watermark_before_write_notify(NotifierWithReturn *notifier,
+                                                      void *opaque)
+{
+    BdrvTrackedRequest *req = opaque;
+    int64_t sector_num = req->offset >> BDRV_SECTOR_BITS;
+    int nb_sectors = req->bytes >> BDRV_SECTOR_BITS;
Quoted 8 lines

block.c void bdrv_flush_io_queue(BlockDriverState *bs) · line 5842

Quoted 4 lines
block.c +4 -0
+
+/*  FIXME: needed? */
+    assert((req->offset & (BDRV_SECTOR_SIZE - 1)) == 0);
+    assert((req->bytes & (BDRV_SECTOR_SIZE - 1)) == 0);

Not really needed here. Emulated storage controllers either get requests in block units (i.e. they are automatically aligned) or check them (like virtio-blk).

But we don't only get requests from emulated storage controllers. You cannot reasonably assume anything here (the requests are aligned according to the requirements of the backend, but that may just be 1). Again, do your calculations in a byte granularity to be safe here.

Quoted 11 lines

I guess there's no harm in checking, but I would drop it.

block.c void bdrv_flush_io_queue(BlockDriverState *bs) · line 5850

Quoted 8 lines
block.c +8 -0
+
+    if (watermark_exceeded(req->bs, sector_num, nb_sectors)) {
+        BlockDriverState *bs = req->bs;
+        qapi_event_send_block_watermark(
+            bdrv_get_device_name(bs),
+            sector_num,
+            bs->wr_highest_sector,
+            &error_abort);
Quoted 5 lines

How do you prevent flooding events if every write request exceeds the watermark?

Perhaps the watermark should be disabled until block-set-watermark is called again.

I agree.

Kevin

#5 lore raw

Re: [Qemu-devel] [PATCH] block: add watermark event

Quoted 28 lines · Kevin Wolf

Am 01.08.2014 um 13:39 hat Stefan Hajnoczi geschrieben:

Quoted 19 lines · Francesco Romani

block.c void bdrv_flush_io_queue(BlockDriverState *bs) · line 5825

Quoted 12 lines
block.c +8 -0
@@ -5813,3 +5815,57 @@ void bdrv_flush_io_queue(BlockDriverState *bs)
         bdrv_flush_io_queue(bs->file);
     }
 }
+
+static bool watermark_exceeded(BlockDriverState *bs,
+                               int64_t sector_num,
+                               int nb_sectors)
+{
+
+    if (bs->wr_watermark_perc > 0) {
+        int64_t watermark = (bs->total_sectors) / 100 * bs->wr_watermark_perc;

bs->total_sectors should not be used directly.

Have you considered making the watermark parameter take sector units instead of a percentage?

Quoted 7 lines

I'm not sure whether a precentage makes sense because 25% of a 10GB image is 2.5 GB so a 75% watermark might be reasonable. 25% of a 1 TB image is 250 GB and that's probably not a reasonable watermark.

So let the block-set-watermark caller pass an absolute sector number instead. It keeps things simple for both QEMU and thin provisioning manager.

Quoted 5 lines

No sector numbers in external interfaces, please. These units of 512 bytes are completely arbitrary and don't make any sense. I hope to get rid of BDRV_SECTOR_* eventually even internally.

So for external APIs, please use bytes instead.

I agree and forgot about that. Please use bytes instead of sectors or a percentage.

#6 lore raw

Re: [Qemu-devel] [PATCH] block: add watermark event

----- Original Message -----

Quoted 36 lines

From: "Stefan Hajnoczi" <[email protected]>
To: "Kevin Wolf" <[email protected]>
Cc: [email protected], "Francesco Romani" <[email protected]>, [email protected], [email protected]
Sent: Tuesday, August 5, 2014 3:08:46 PM
Subject: Re: [Qemu-devel] [PATCH] block: add watermark event

Quoted 29 lines · Kevin Wolf

Am 01.08.2014 um 13:39 hat Stefan Hajnoczi geschrieben:

Quoted 27 lines · Francesco Romani
Quoted 12 lines
block.c +8 -0
@@ -5813,3 +5815,57 @@ void bdrv_flush_io_queue(BlockDriverState *bs)
         bdrv_flush_io_queue(bs->file);
     }
 }
+
+static bool watermark_exceeded(BlockDriverState *bs,
+                               int64_t sector_num,
+                               int nb_sectors)
+{
+
+    if (bs->wr_watermark_perc > 0) {
+        int64_t watermark = (bs->total_sectors) / 100 *
Quoted 1 line

bs->wr_watermark_perc;

bs->total_sectors should not be used directly.

Have you considered making the watermark parameter take sector units instead of a percentage?

I'm not sure whether a precentage makes sense because 25% of a 10GB image is 2.5 GB so a 75% watermark might be reasonable. 25% of a 1 TB image is 250 GB and that's probably not a reasonable watermark.

So let the block-set-watermark caller pass an absolute sector number instead. It keeps things simple for both QEMU and thin provisioning manager.

Quoted 9 lines
Quoted 5 lines

No sector numbers in external interfaces, please. These units of 512 bytes are completely arbitrary and don't make any sense. I hope to get rid of BDRV_SECTOR_* eventually even internally.

So for external APIs, please use bytes instead.

I agree and forgot about that. Please use bytes instead of sectors or a percentage.

Thanks everyone for the great feedback received!

I'll post asap a new patch addressing all the comments.

I'll also change the name because 'watermark' may be misleading/wrong jargon :)
(http://en.wikipedia.org/wiki/Watermark)

Bests,

Signature · 4 lines
Francesco Romani
RedHat Engineering Virtualization R & D
Phone: 8261328
IRC: fromani
#7 lore raw

Re: [Qemu-devel] [PATCH] block: add watermark event

Quoted 14 lines · Francesco Romani
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines

So let the block-set-watermark caller pass an absolute sector number instead. It keeps things simple for both QEMU and thin provisioning manager.

Quoted 9 lines
Quoted 5 lines

No sector numbers in external interfaces, please. These units of 512 bytes are completely arbitrary and don't make any sense. I hope to get rid of BDRV_SECTOR_* eventually even internally.

So for external APIs, please use bytes instead.

I agree and forgot about that. Please use bytes instead of sectors or a percentage.

Quoted 6 lines

Thanks everyone for the great feedback received!

I'll post asap a new patch addressing all the comments.

I'll also change the name because 'watermark' may be misleading/wrong jargon :)
(http://en.wikipedia.org/wiki/Watermark)

Is "threshold" a better name than "watermark"?

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#8 lore raw
In reply to Francesco Romani

Sorry, this is actually an RFC; patch was posted separately.

----- Original Message -----

Quoted 62 lines

From: "Francesco Romani" <[email protected]>
To: [email protected]
Cc: [email protected], [email protected], [email protected], [email protected], "Francesco Romani" <[email protected]>
Sent: Tuesday, July 8, 2014 4:49:23 PM
Subject: [PATCH] add watermark reporting for block devices

Hello everyone

I'm one of the oVirt developers (http://www.ovirt.org);
oVirt is a virtualization management application built around qemu/kvm, so it is nice to get in touch :)

We have begun a big scalability improvement effort, aiming to support without problems hundreds of VMs per host, with plans to support thousands in a not so distant future.
In doing so, we are reviewing our usage flows.

One of them is thin-provisioned storage, which is used quite extensively, with block devices (ISCSI for example) and COW images.
When using thin provisioning, oVirt tries hard to hide this fact from the guest OS, and to do so watches closely the usage of the device, and resize it when its usage exceeds a configured threshold (the "high water mark"), in order to avoid the guest OS to get paused for space exhausted.

To do the watching, we poll he devices using libvirt (virDomainGetBlockInfo), which in turn uses query-blockstats. This is suboptimal with just one VM, but with hundereds of them, let alone thousands, it doesn't scale and it is quite a resource hog.

Would be great to have this watermark concept supported into qemu, with a new event to be raised when the limit is crossed.

To track this RFE I filed https://bugs.launchpad.net/qemu/+bug/1338957

Moreover, I had the chance to take a look at the QEMU sources and come up with this tentative patch which I'd also like to submit.

Comments and thoughts very welcome!

Thanks and best regards,

Francesco Romani (1):
block: add watermark event

 block.c                   | 56
 +++++++++++++++++++++++++++++++++++++++++++++++
 blockdev.c                | 21 ++++++++++++++++++
 include/block/block.h     |  2 ++
 include/block/block_int.h |  3 +++
 qapi/block-core.json      | 33 ++++++++++++++++++++++++++++
 qmp-commands.hx           | 24 ++++++++++++++++++++
 6 files changed, 139 insertions(+)

--
1.9.3

Signature · 4 lines
Francesco Romani
RedHat Engineering Virtualization R & D
Phone: 8261328
IRC: fromani