Skip to contentSkip to conversation

block/ssh: Add basic .bdrv_truncate()

PATCH Applied Max Reitz 3 participants 10 messages Feb 14, 2018 – Feb 23, 2018 Block layer

lore
Share & tools
lore

Max Reitz

author
#0 lore raw

cover block/ssh: Add basic .bdrv_truncate()

For (x-)blockdev-create, all protocol drivers that support image creation also need to offer a .bdrv_truncate() implementation that matches in features. A previous series of mine brought gluster's and sheepdog's implementation up to par regarding preallocated truncation; but I forgot about drivers that have a .bdrv_create() but no .bdrv_truncate() at all.

There is only one of these, and that is ssh. Since libssh2 does not seem to know any way of truncating files, we can only support growing files -- but that is what we need for (x-)blockdev-create.

Note that there are also drivers which do not support growing files, namely iscsi and file-posix for host devices (maybe more? I hope not). But these also pretty much ignore the specified size on .bdrv_create() and just use the size of the existing device. They just check that the specified size does not exceed the actual size, so that pretty much matches what their .bdrv_truncate() supports, and we should be fine there.

Max Reitz (3):
block/ssh: Pull ssh_grow_file() from ssh_create() block/ssh: Make ssh_grow_file() blocking block/ssh: Add basic .bdrv_truncate()

 block/ssh.c | 61 +++++++++++++++++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 53 insertions(+), 8 deletions(-)
Signature · 1 line
2.14.3
author
#1 lore raw

PATCH 1/3 block/ssh: Pull ssh_grow_file() from ssh_create()

If we ever want to offer even rudimentary truncation functionality for ssh, we should put the respective code into a reusable function.

---
 block/ssh.c | 30 ++++++++++++++++++++++--------
 1 file changed, 22 insertions(+), 8 deletions(-)
block/ssh.c +22 -8
index b63addcf94..964e55f7fe 100644
@@ -803,6 +803,26 @@ static int ssh_file_open(BlockDriverState *bs, QDict *options, int bdrv_flags,
     return ret;
 }
 
+static int ssh_grow_file(BDRVSSHState *s, int64_t offset, Error **errp)
+{
+    ssize_t ret;
+    char c[1] = { '\0' };
+
+    /* offset must be strictly greater than the current size so we do
+     * not overwrite anything */
+    assert(offset > 0 && offset > s->attrs.filesize);
+
+    libssh2_sftp_seek64(s->sftp_handle, offset - 1);
+    ret = libssh2_sftp_write(s->sftp_handle, c, 1);
+    if (ret < 0) {
+        sftp_error_setg(errp, s, "Failed to grow file");
+        return -EIO;
+    }
+
+    s->attrs.filesize = offset;
+    return 0;
+}
+
 static QemuOptsList ssh_create_opts = {
     .name = "ssh-create-opts",
     .head = QTAILQ_HEAD_INITIALIZER(ssh_create_opts.head),
@@ -822,8 +842,6 @@ static int ssh_create(const char *filename, QemuOpts *opts, Error **errp)
     int64_t total_size = 0;
     QDict *uri_options = NULL;
     BDRVSSHState s;
-    ssize_t r2;
-    char c[1] = { '\0' };
 
     ssh_state_init(&s);
 
@@ -849,14 +867,10 @@ static int ssh_create(const char *filename, QemuOpts *opts, Error **errp)
     }
 
     if (total_size > 0) {
-        libssh2_sftp_seek64(s.sftp_handle, total_size-1);
-        r2 = libssh2_sftp_write(s.sftp_handle, c, 1);
-        if (r2 < 0) {
-            sftp_error_setg(errp, &s, "truncate failed");
-            ret = -EINVAL;
+        ret = ssh_grow_file(&s, total_size, errp);
+        if (ret < 0) {
             goto out;
         }
-        s.attrs.filesize = total_size;
     }
 
     ret = 0;
Signature · 1 line
2.14.3
Reviewed-by on v1 1/3
#2 lore raw

Re: [Qemu-devel] [PATCH 1/3] block/ssh: Pull ssh_grow_file() from ssh_create()

Quoted 11 lines · Max Reitz

If we ever want to offer even rudimentary truncation functionality for ssh, we should put the respective code into a reusable function.

---
  block/ssh.c | 30 ++++++++++++++++++++++--------
  1 file changed, 22 insertions(+), 8 deletions(-)

+++ b/block/ssh.c

block/ssh.c static int ssh_file_open(BlockDriverState *bs, QDict *options, int bdrv_flags, · line 809

Quoted 7 lines
block/ssh.c +4 -0
@@ -803,6 +803,26 @@ static int ssh_file_open(BlockDriverState *bs, QDict *options, int bdrv_flags,
      return ret;
  }
 
+static int ssh_grow_file(BDRVSSHState *s, int64_t offset, Error **errp)
+{
+    ssize_t ret;
+    char c[1] = { '\0' };

Could spell this 'char c[1] = "";', but you just did code motion.

Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org
author
#3 lore raw

PATCH 2/3 block/ssh: Make ssh_grow_file() blocking

In reply to Max Reitz

At runtime (that is, during a future ssh_truncate()), the SSH session is non-blocking. However, ssh_truncate() (or rather, bdrv_truncate() in general) is not a coroutine, so this resize operation needs to block.

For ssh_create(), that is fine, too; the session is never set to non-blocking anyway.

---
 block/ssh.c | 7 +++++++
 1 file changed, 7 insertions(+)
block/ssh.c +7 -0
index 964e55f7fe..ff8576f21e 100644
@@ -803,17 +803,24 @@ static int ssh_file_open(BlockDriverState *bs, QDict *options, int bdrv_flags,
     return ret;
 }
 
+/* Note: This is a blocking operation */
 static int ssh_grow_file(BDRVSSHState *s, int64_t offset, Error **errp)
 {
     ssize_t ret;
     char c[1] = { '\0' };
+    int was_blocking = libssh2_session_get_blocking(s->session);
 
     /* offset must be strictly greater than the current size so we do
      * not overwrite anything */
     assert(offset > 0 && offset > s->attrs.filesize);
 
+    libssh2_session_set_blocking(s->session, 1);
+
     libssh2_sftp_seek64(s->sftp_handle, offset - 1);
     ret = libssh2_sftp_write(s->sftp_handle, c, 1);
+
+    libssh2_session_set_blocking(s->session, was_blocking);
+
     if (ret < 0) {
         sftp_error_setg(errp, s, "Failed to grow file");
         return -EIO;
Signature · 1 line
2.14.3
Reviewed-by on v1 2/3
#4 lore raw

Re: [Qemu-devel] [PATCH 2/3] block/ssh: Make ssh_grow_file() blocking

Quoted 27 lines · Max Reitz

At runtime (that is, during a future ssh_truncate()), the SSH session is non-blocking. However, ssh_truncate() (or rather, bdrv_truncate() in general) is not a coroutine, so this resize operation needs to block.

For ssh_create(), that is fine, too; the session is never set to non-blocking anyway.

---
  block/ssh.c | 7 +++++++
  1 file changed, 7 insertions(+)
block/ssh.c +2 -0
index 964e55f7fe..ff8576f21e 100644
@@ -803,17 +803,24 @@ static int ssh_file_open(BlockDriverState *bs, QDict *options, int bdrv_flags,
      return ret;
  }
 
+/* Note: This is a blocking operation */
  static int ssh_grow_file(BDRVSSHState *s, int64_t offset, Error **errp)
  {
      ssize_t ret;
      char c[1] = { '\0' };
+    int was_blocking = libssh2_session_get_blocking(s->session);
 

block/ssh.c static int ssh_file_open(BlockDriverState *bs, QDict *options, int bdrv_flags, · line 822

Quoted 10 lines
block/ssh.c +4 -0
      /* offset must be strictly greater than the current size so we do
       * not overwrite anything */
      assert(offset > 0 && offset > s->attrs.filesize);
 
+    libssh2_session_set_blocking(s->session, 1);
+
      libssh2_sftp_seek64(s->sftp_handle, offset - 1);
      ret = libssh2_sftp_write(s->sftp_handle, c, 1);
+
+    libssh2_session_set_blocking(s->session, was_blocking);

Is it worth skipping the two libssh2_session_set_blocking() calls if was_blocking is 1? But that's a micro-optimization that probably won't be noticeable, so I'm also fine with unconditional.

Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org
author
#5 lore raw

Re: [Qemu-devel] [PATCH 2/3] block/ssh: Make ssh_grow_file() blocking

Quoted 32 lines · Eric Blake
Quoted 31 lines · Max Reitz

At runtime (that is, during a future ssh_truncate()), the SSH session is non-blocking.  However, ssh_truncate() (or rather, bdrv_truncate() in general) is not a coroutine, so this resize operation needs to block.

For ssh_create(), that is fine, too; the session is never set to non-blocking anyway.

---
  block/ssh.c | 7 +++++++

  1 file changed, 7 insertions(+)
block/ssh.c +0 -0
index 964e55f7fe..ff8576f21e 100644
@@ -803,17 +803,24 @@ static int ssh_file_open(BlockDriverState *bs,
 QDict *options, int bdrv_flags,
       return ret;
   }

  +/* Note: This is a blocking operation */
  static int ssh_grow_file(BDRVSSHState *s, int64_t offset, Error **errp) {
      ssize_t ret;
      char c[1] = { '\0' };
+    int was_blocking = libssh2_session_get_blocking(s->session); /* offset must be strictly greater than the current size so we do
       * not overwrite anything */
      assert(offset > 0 && offset > s->attrs.filesize);
  +    libssh2_session_set_blocking(s->session, 1);
+

Quoted 8 lines
Quoted 4 lines

      libssh2_sftp_seek64(s->sftp_handle, offset - 1); ret = libssh2_sftp_write(s->sftp_handle, c, 1); +
+    libssh2_session_set_blocking(s->session, was_blocking);

Is it worth skipping the two libssh2_session_set_blocking() calls if was_blocking is 1?  But that's a micro-optimization that probably won't be noticeable, so I'm also fine with unconditional.

I was hoping libssh2 is clever enough for that itself. :-)

Quoted 1 line

Thanks!

Max

author
#6 lore raw

PATCH 3/3 block/ssh: Add basic .bdrv_truncate()

In reply to Max Reitz

libssh2 does not seem to offer real truncation support, so we can only grow files -- but that is better than nothing.

---
 block/ssh.c | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)
block/ssh.c +24 -0
index ff8576f21e..c235eec255 100644
@@ -1219,6 +1219,29 @@ static int64_t ssh_getlength(BlockDriverState *bs)
     return length;
 }
 
+static int ssh_truncate(BlockDriverState *bs, int64_t offset,
+                        PreallocMode prealloc, Error **errp)
+{
+    BDRVSSHState *s = bs->opaque;
+
+    if (prealloc != PREALLOC_MODE_OFF) {
+        error_setg(errp, "Unsupported preallocation mode '%s'",
+                   PreallocMode_str(prealloc));
+        return -ENOTSUP;
+    }
+
+    if (offset < s->attrs.filesize) {
+        error_setg(errp, "ssh driver does not support shrinking files");
+        return -ENOTSUP;
+    }
+
+    if (offset == s->attrs.filesize) {
+        return 0;
+    }
+
+    return ssh_grow_file(s, offset, errp);
+}
+
 static BlockDriver bdrv_ssh = {
     .format_name                  = "ssh",
     .protocol_name                = "ssh",
@@ -1231,6 +1254,7 @@ static BlockDriver bdrv_ssh = {
     .bdrv_co_readv                = ssh_co_readv,
     .bdrv_co_writev               = ssh_co_writev,
     .bdrv_getlength               = ssh_getlength,
+    .bdrv_truncate                = ssh_truncate,
     .bdrv_co_flush_to_disk        = ssh_co_flush,
     .create_opts                  = &ssh_create_opts,
 };
Signature · 1 line
2.14.3
Reviewed-by on v1 3/3
#7 lore raw
Quoted 8 lines · Max Reitz

libssh2 does not seem to offer real truncation support, so we can only grow files -- but that is better than nothing.

---
  block/ssh.c | 24 ++++++++++++++++++++++++
  1 file changed, 24 insertions(+)
Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org
Reviewed-by on v1 3/3
#8 lore raw
In reply to Max Reitz

The series looks fine to me:

Rich.

Signature · 4 lines
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
author Applied
#9 lore raw
In reply to Max Reitz
Quoted 20 lines · Max Reitz

For (x-)blockdev-create, all protocol drivers that support image creation also need to offer a .bdrv_truncate() implementation that matches in features. A previous series of mine brought gluster's and sheepdog's implementation up to par regarding preallocated truncation; but I forgot about drivers that have a .bdrv_create() but no .bdrv_truncate() at all.

There is only one of these, and that is ssh. Since libssh2 does not seem to know any way of truncating files, we can only support growing files -- but that is what we need for (x-)blockdev-create.

Note that there are also drivers which do not support growing files, namely iscsi and file-posix for host devices (maybe more? I hope not). But these also pretty much ignore the specified size on .bdrv_create() and just use the size of the existing device. They just check that the specified size does not exceed the actual size, so that pretty much matches what their .bdrv_truncate() supports, and we should be fine there.

Quoted 7 lines

Max Reitz (3):
block/ssh: Pull ssh_grow_file() from ssh_create() block/ssh: Make ssh_grow_file() blocking block/ssh: Add basic .bdrv_truncate()

 block/ssh.c | 61 +++++++++++++++++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 53 insertions(+), 8 deletions(-)

Applied to my block branch.

Max