Skip to contentSkip to conversation

ssh: switch from libssh2 to libssh

PATCH Superseded Pino Toscano 3 participants 8 messages Jun 18, 2019 – Jun 20, 2019 Block layer

v10 of 11 Superseded Latest is v11 →

lore
Share & tools
lore
#0 lore raw

PATCH v10 1/1 ssh: switch from libssh2 to libssh

Rewrite the implementation of the ssh block driver to use libssh instead of libssh2. The libssh library has various advantages over libssh2:
- easier API for authentication (for example for using ssh-agent)
- easier API for known_hosts handling
- supports newer types of keys in known_hosts

Use APIs/features available in libssh 0.8 conditionally, to support older versions (which are not recommended though).

Adjust the tests according to the different error message, and to the newer host keys (ed25519) that are used by default with OpenSSH >= 6.7 and libssh >= 0.7.0.

Adjust the various Docker/Travis scripts to use libssh when available instead of libssh2. The mingw/mxe testing is dropped for now, as there are no packages for it.

---

Changes from v9:
- restored "default" case in the server status switch for libssh < 0.8.0
- print the host key type & fingerprint on mismatch with known_hosts
- improve/fix message for failed socket_set_nodelay()
- reset s->sock properly

Changes from v8:
- use a newer key type in iotest 207
- improve the commit message

Changes from v7:
- #if HAVE_LIBSSH_0_8 -> #ifdef HAVE_LIBSSH_0_8
- ptrdiff_t -> size_t

Changes from v6:
- fixed few checkpatch style issues
- detect libssh 0.8 via symbol detection
- adjust travis/docker test material
- remove dead "default" case in a switch
- use variables for storing MIN() results
- adapt a documentation bit

Changes from v5:
- adapt to newer tracing APIs
- disable ssh compression (mimic what libssh2 does by default)
- use build time checks for libssh 0.8, and use newer APIs directly

Changes from v4:
- fix wrong usages of error_setg/session_error_setg/sftp_error_setg
- fix few return code checks
- remove now-unused parameters in few internal functions
- allow authentication with "none" method
- switch to unsigned int for the port number
- enable TCP_NODELAY on the socket
- fix one reference error message in iotest 207

Changes from v3:
- fix socket cleanup in connect_to_ssh()
- add comments about the socket cleanup
- improve the error reporting (closer to what was with libssh2)
- improve EOF detection on sftp_read()

Changes from v2:
- used again an own fd
- fixed co_yield() implementation

Changes from v1:
- fixed jumbo packets writing
- fixed missing 'err' assignment
- fixed commit message

 .travis.yml                                   |   4 +-
 block/Makefile.objs                           |   6 +-
 block/ssh.c                                   | 665 ++++++++++--------
 block/trace-events                            |  14 +-
 configure                                     |  65 +-
 docs/qemu-block-drivers.texi                  |   2 +-
 .../dockerfiles/debian-win32-cross.docker     |   1 -
 .../dockerfiles/debian-win64-cross.docker     |   1 -
 tests/docker/dockerfiles/fedora.docker        |   4 +-
 tests/docker/dockerfiles/ubuntu.docker        |   2 +-
 tests/docker/dockerfiles/ubuntu1804.docker    |   2 +-
 tests/qemu-iotests/207                        |   4 +-
 tests/qemu-iotests/207.out                    |   2 +-
 13 files changed, 423 insertions(+), 349 deletions(-)
.travis.yml +2 -2
index 08502c0aa2..75f47df3d2 100644
@@ -31,7 +31,7 @@ addons:
       - libseccomp-dev
       - libspice-protocol-dev
       - libspice-server-dev
-      - libssh2-1-dev
+      - libssh-dev
       - liburcu-dev
       - libusb-1.0-0-dev
       - libvte-2.91-dev
@@ -268,7 +268,7 @@ matrix:
             - libseccomp-dev
             - libspice-protocol-dev
             - libspice-server-dev
-            - libssh2-1-dev
+            - libssh-dev
             - liburcu-dev
             - libusb-1.0-0-dev
             - libvte-2.91-dev
block/Makefile.objs +3 -3
index dbd1522722..35f3bca4d9 100644
@@ -31,7 +31,7 @@ block-obj-$(CONFIG_CURL) += curl.o
 block-obj-$(CONFIG_RBD) += rbd.o
 block-obj-$(CONFIG_GLUSTERFS) += gluster.o
 block-obj-$(CONFIG_VXHS) += vxhs.o
-block-obj-$(CONFIG_LIBSSH2) += ssh.o
+block-obj-$(CONFIG_LIBSSH) += ssh.o
 block-obj-y += accounting.o dirty-bitmap.o
 block-obj-y += write-threshold.o
 block-obj-y += backup.o
@@ -52,8 +52,8 @@ rbd.o-libs         := $(RBD_LIBS)
 gluster.o-cflags   := $(GLUSTERFS_CFLAGS)
 gluster.o-libs     := $(GLUSTERFS_LIBS)
 vxhs.o-libs        := $(VXHS_LIBS)
-ssh.o-cflags       := $(LIBSSH2_CFLAGS)
-ssh.o-libs         := $(LIBSSH2_LIBS)
+ssh.o-cflags       := $(LIBSSH_CFLAGS)
+ssh.o-libs         := $(LIBSSH_LIBS)
 block-obj-dmg-bz2-$(CONFIG_BZIP2) += dmg-bz2.o
 block-obj-$(if $(CONFIG_DMG),m,n) += $(block-obj-dmg-bz2-y)
 dmg-bz2.o-libs     := $(BZIP2_LIBS)
block/ssh.c +375 -290

1,065 lines not on this page · raw message

block/trace-events +8 -6
index f6e43ee023..6f198b9b03 100644
@@ -171,19 +171,21 @@ nbd_client_connect_success(const char *export_name) "export '%s'"
 # ssh.c
 ssh_restart_coroutine(void *co) "co=%p"
 ssh_flush(void) "fsync"
-ssh_check_host_key_knownhosts(const char *key) "host key OK: %s"
+ssh_check_host_key_knownhosts(void) "host key OK"
 ssh_connect_to_ssh(char *path, int flags, int mode) "opening file %s flags=0x%x creat_mode=0%o"
 ssh_co_yield(int sock, void *rd_handler, void *wr_handler) "s->sock=%d rd_handler=%p wr_handler=%p"
 ssh_co_yield_back(int sock) "s->sock=%d - back"
 ssh_getlength(int64_t length) "length=%" PRIi64
 ssh_co_create_opts(uint64_t size) "total_size=%" PRIu64
 ssh_read(int64_t offset, size_t size) "offset=%" PRIi64 " size=%zu"
-ssh_read_buf(void *buf, size_t size) "sftp_read buf=%p size=%zu"
-ssh_read_return(ssize_t ret) "sftp_read returned %zd"
+ssh_read_buf(void *buf, size_t size, size_t actual_size) "sftp_read buf=%p size=%zu (actual size=%zu)"
+ssh_read_return(ssize_t ret, int sftp_err) "sftp_read returned %zd (sftp error=%d)"
 ssh_write(int64_t offset, size_t size) "offset=%" PRIi64 " size=%zu"
-ssh_write_buf(void *buf, size_t size) "sftp_write buf=%p size=%zu"
-ssh_write_return(ssize_t ret) "sftp_write returned %zd"
+ssh_write_buf(void *buf, size_t size, size_t actual_size) "sftp_write buf=%p size=%zu (actual size=%zu)"
+ssh_write_return(ssize_t ret, int sftp_err) "sftp_write returned %zd (sftp error=%d)"
 ssh_seek(int64_t offset) "seeking to offset=%" PRIi64
+ssh_auth_methods(int methods) "auth methods=0x%x"
+ssh_server_status(int status) "server status=%d"
 
 # curl.c
 curl_timer_cb(long timeout_ms) "timer callback timeout_ms %ld"
@@ -216,4 +218,4 @@ sheepdog_snapshot_create(const char *sn_name, const char *id) "%s %s"
 sheepdog_snapshot_create_inode(const char *name, uint32_t snap, uint32_t vdi) "s->inode: name %s snap_id 0x%" PRIx32 " vdi 0x%" PRIx32
 
 # ssh.c
-sftp_error(const char *op, const char *ssh_err, int ssh_err_code, unsigned long sftp_err_code) "%s failed: %s (libssh2 error code: %d, sftp error code: %lu)"
+sftp_error(const char *op, const char *ssh_err, int ssh_err_code, int sftp_err_code) "%s failed: %s (libssh error code: %d, sftp error code: %d)"
configure +28 -37

111 lines not on this page · raw message

docs/qemu-block-drivers.texi +1 -1
index da06a9bc83..91ab0eceae 100644
@@ -782,7 +782,7 @@ print a warning when @code{fsync} is not supported:
 
 warning: ssh server @code{ssh.example.com:22} does not support fsync
 
-With sufficiently new versions of libssh2 and OpenSSH, @code{fsync} is
+With sufficiently new versions of libssh and OpenSSH, @code{fsync} is
 supported.
 
 @node disk_images_nvme
tests/docker/dockerfiles/debian-win32-cross.docker +0 -1
index dd021f2df0..0a4970c068 100644
@@ -15,7 +15,6 @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
         mxe-$TARGET-w64-mingw32.shared-curl \
         mxe-$TARGET-w64-mingw32.shared-glib \
         mxe-$TARGET-w64-mingw32.shared-libgcrypt \
-        mxe-$TARGET-w64-mingw32.shared-libssh2 \
         mxe-$TARGET-w64-mingw32.shared-libusb1 \
         mxe-$TARGET-w64-mingw32.shared-lzo \
         mxe-$TARGET-w64-mingw32.shared-nettle \
tests/docker/dockerfiles/debian-win64-cross.docker +0 -1
index 4542bcc821..b27985b1b1 100644
@@ -15,7 +15,6 @@ RUN DEBIAN_FRONTEND=noninteractive eatmydata \
         mxe-$TARGET-w64-mingw32.shared-curl \
         mxe-$TARGET-w64-mingw32.shared-glib \
         mxe-$TARGET-w64-mingw32.shared-libgcrypt \
-        mxe-$TARGET-w64-mingw32.shared-libssh2 \
         mxe-$TARGET-w64-mingw32.shared-libusb1 \
         mxe-$TARGET-w64-mingw32.shared-lzo \
         mxe-$TARGET-w64-mingw32.shared-nettle \
tests/docker/dockerfiles/fedora.docker +1 -3
index 12c460597e..619d1b5656 100644
@@ -35,7 +35,7 @@ ENV PACKAGES \
     libpng-devel \
     librbd-devel \
     libseccomp-devel \
-    libssh2-devel \
+    libssh-devel \
     libubsan \
     libusbx-devel \
     libxml2-devel \
@@ -50,7 +50,6 @@ ENV PACKAGES \
     mingw32-gtk3 \
     mingw32-libjpeg-turbo \
     mingw32-libpng \
-    mingw32-libssh2 \
     mingw32-libtasn1 \
     mingw32-nettle \
     mingw32-pixman \
@@ -64,7 +63,6 @@ ENV PACKAGES \
     mingw64-gtk3 \
     mingw64-libjpeg-turbo \
     mingw64-libpng \
-    mingw64-libssh2 \
     mingw64-libtasn1 \
     mingw64-nettle \
     mingw64-pixman \
tests/docker/dockerfiles/ubuntu.docker +1 -1
index 8d256961f0..d3b72209c8 100644
@@ -53,7 +53,7 @@ ENV PACKAGES flex bison \
     libsnappy-dev \
     libspice-protocol-dev \
     libspice-server-dev \
-    libssh2-1-dev \
+    libssh-dev \
     libusb-1.0-0-dev \
     libusbredirhost-dev \
     libvdeplug-dev \
tests/docker/dockerfiles/ubuntu1804.docker +1 -1
index 2e2900150b..9d80b11500 100644
@@ -40,7 +40,7 @@ ENV PACKAGES flex bison \
     libsnappy-dev \
     libspice-protocol-dev \
     libspice-server-dev \
-    libssh2-1-dev \
+    libssh-dev \
     libusb-1.0-0-dev \
     libusbredirhost-dev \
     libvdeplug-dev \
tests/qemu-iotests/207 +2 -2
index b3816136f7..774eb5f2a9 100755
@@ -111,7 +111,7 @@ with iotests.FilePath('t.img') as disk_path, \
     iotests.img_info_log(remote_path)
 
     md5_key = subprocess.check_output(
-        'ssh-keyscan -t rsa 127.0.0.1 2>/dev/null | grep -v "\\^#" | ' +
+        'ssh-keyscan -t ssh-ed25519 127.0.0.1 2>/dev/null | grep -v "\\^#" | ' +
         'cut -d" " -f3 | base64 -d | md5sum -b | cut -d" " -f1',
         shell=True).rstrip().decode('ascii')
 
@@ -149,7 +149,7 @@ with iotests.FilePath('t.img') as disk_path, \
     iotests.img_info_log(remote_path)
 
     sha1_key = subprocess.check_output(
-        'ssh-keyscan -t rsa 127.0.0.1 2>/dev/null | grep -v "\\^#" | ' +
+        'ssh-keyscan -t ssh-ed25519 127.0.0.1 2>/dev/null | grep -v "\\^#" | ' +
         'cut -d" " -f3 | base64 -d | sha1sum -b | cut -d" " -f1',
         shell=True).rstrip().decode('ascii')
 
tests/qemu-iotests/207.out +1 -1
index ec9823793a..1239d9d648 100644
@@ -68,7 +68,7 @@ virtual size: 4 MiB (4194304 bytes)
 
 {"execute": "blockdev-create", "arguments": {"job-id": "job0", "options": {"driver": "ssh", "location": {"host-key-check": {"mode": "none"}, "path": "/this/is/not/an/existing/path", "server": {"host": "127.0.0.1", "port": "22"}}, "size": 4194304}}}
 {"return": {}}
-Job failed: failed to open remote file '/this/is/not/an/existing/path': Failed opening remote file (libssh2 error code: -31)
+Job failed: failed to open remote file '/this/is/not/an/existing/path': SFTP server: No such file (libssh error code: 1, sftp error code: 2)
 {"execute": "job-dismiss", "arguments": {"id": "job0"}}
 {"return": {}}
 
Signature · 1 line
2.21.0
#1 lore raw
Quoted 87 lines · Pino Toscano

Rewrite the implementation of the ssh block driver to use libssh instead of libssh2. The libssh library has various advantages over libssh2:
- easier API for authentication (for example for using ssh-agent)
- easier API for known_hosts handling
- supports newer types of keys in known_hosts

Use APIs/features available in libssh 0.8 conditionally, to support older versions (which are not recommended though).

Adjust the tests according to the different error message, and to the newer host keys (ed25519) that are used by default with OpenSSH >= 6.7 and libssh >= 0.7.0.

Adjust the various Docker/Travis scripts to use libssh when available instead of libssh2. The mingw/mxe testing is dropped for now, as there are no packages for it.

---

Changes from v9:
- restored "default" case in the server status switch for libssh < 0.8.0
- print the host key type & fingerprint on mismatch with known_hosts
- improve/fix message for failed socket_set_nodelay()
- reset s->sock properly

Changes from v8:
- use a newer key type in iotest 207
- improve the commit message

Changes from v7:
- #if HAVE_LIBSSH_0_8 -> #ifdef HAVE_LIBSSH_0_8
- ptrdiff_t -> size_t

Changes from v6:
- fixed few checkpatch style issues
- detect libssh 0.8 via symbol detection
- adjust travis/docker test material
- remove dead "default" case in a switch
- use variables for storing MIN() results
- adapt a documentation bit

Changes from v5:
- adapt to newer tracing APIs
- disable ssh compression (mimic what libssh2 does by default)
- use build time checks for libssh 0.8, and use newer APIs directly

Changes from v4:
- fix wrong usages of error_setg/session_error_setg/sftp_error_setg
- fix few return code checks
- remove now-unused parameters in few internal functions
- allow authentication with "none" method
- switch to unsigned int for the port number
- enable TCP_NODELAY on the socket
- fix one reference error message in iotest 207

Changes from v3:
- fix socket cleanup in connect_to_ssh()
- add comments about the socket cleanup
- improve the error reporting (closer to what was with libssh2)
- improve EOF detection on sftp_read()

Changes from v2:
- used again an own fd
- fixed co_yield() implementation

Changes from v1:
- fixed jumbo packets writing
- fixed missing 'err' assignment
- fixed commit message

 .travis.yml                                   |   4 +-
 block/Makefile.objs                           |   6 +-
 block/ssh.c                                   | 665 ++++++++++--------
 block/trace-events                            |  14 +-
 configure                                     |  65 +-
 docs/qemu-block-drivers.texi                  |   2 +-
 .../dockerfiles/debian-win32-cross.docker     |   1 -
 .../dockerfiles/debian-win64-cross.docker     |   1 -
 tests/docker/dockerfiles/fedora.docker        |   4 +-
 tests/docker/dockerfiles/ubuntu.docker        |   2 +-
 tests/docker/dockerfiles/ubuntu1804.docker    |   2 +-
 tests/qemu-iotests/207                        |   4 +-
 tests/qemu-iotests/207.out                    |   2 +-
 13 files changed, 423 insertions(+), 349 deletions(-)

[...]

Quoted 4 lines
block/ssh.c +0 -0
index 6da7b9cbfe..644ae8b82c 100644

[...]

Quoted 14 lines
block/ssh.c +14 -0
+    case SSH_SERVER_KNOWN_CHANGED:
+        ret = -EINVAL;
+        r = ssh_get_publickey(s->session, &pubkey);
+        if (r == 0) {
+            r = ssh_get_publickey_hash(pubkey, SSH_PUBLICKEY_HASH_SHA1,
+                                       &server_hash, &server_hash_len);
+            pubkey_type = ssh_key_type(pubkey);
+            ssh_key_free(pubkey);
+        }
+        if (r == 0) {
+            fingerprint = ssh_get_fingerprint_hash(SSH_PUBLICKEY_HASH_SHA1,
+                                                   server_hash,
+                                                   server_hash_len);
+            ssh_clean_pubkey_hash(&server_hash);

block/ssh.c

Quoted 10 lines
block/ssh.c +10 -0
+        }
+        if (fingerprint) {
+            error_setg(errp,
+                       "host key (%s key with fingerprint %s) does not match "
+                       "the one in known_hosts",
+                       ssh_key_type_to_char(pubkey_type), fingerprint);
+            ssh_string_free_char(fingerprint);
+        } else  {
+            error_setg(errp, "host key does not match the one in known_hosts");
+        }

Usually I’d say that more information can’t be bad. But here I don’t see how this additional information is useful. known_hosts contains base64-encoded full public keys. This only prints the SHA-1 digest. The user cannot add that to known_hosts, or easily scan known_hosts to see whether it perhaps belongs to some other hosts (maybe because DHCP scrambled something).

Actually, even if this printed the base64 representation of the full key, the user probably wouldn’t just add that to known_hosts or do anything with it. They’ll debug the problem with other tools.

So I don’t think this new information is really useful...?

(Hmm, I don’t know if this is a response to my “But the specification requires a warning!!1!”, but if so, I was actually not referring to more information but to this:

$ ssh 192.168.0.12
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.

I mean, I was also only half-serious. I should be serious because the libssh specification requires us to print some warning like that, but, well. Yes, I’ll stop mumbling about this stuff now.)

[...]

Quoted 14 lines
tests/qemu-iotests/207 +1 -1
index b3816136f7..774eb5f2a9 100755
@@ -111,7 +111,7 @@ with iotests.FilePath('t.img') as disk_path, \
     iotests.img_info_log(remote_path)
 
     md5_key = subprocess.check_output(
-        'ssh-keyscan -t rsa 127.0.0.1 2>/dev/null | grep -v "\\^#" | ' +
+        'ssh-keyscan -t ssh-ed25519 127.0.0.1 2>/dev/null | grep -v "\\^#" | ' +
         'cut -d" " -f3 | base64 -d | md5sum -b | cut -d" " -f1',
         shell=True).rstrip().decode('ascii')
 

tests/qemu-iotests/207 with iotests.FilePath('t.img') as disk_path, \ · line 154

Quoted 7 lines
tests/qemu-iotests/207 +1 -1
@@ -149,7 +149,7 @@ with iotests.FilePath('t.img') as disk_path, \
     iotests.img_info_log(remote_path)
 
     sha1_key = subprocess.check_output(
-        'ssh-keyscan -t rsa 127.0.0.1 2>/dev/null | grep -v "\\^#" | ' +
+        'ssh-keyscan -t ssh-ed25519 127.0.0.1 2>/dev/null | grep -v "\\^#" | ' +
         'cut -d" " -f3 | base64 -d | sha1sum -b | cut -d" " -f1',
         shell=True).rstrip().decode('ascii')

I’ve attached a diff that makes this test pass for me. Maybe also for you and Philippe.

Max

tests/qemu-iotests/207 +43 -11
index 774eb5f2a9..ec8c1d06f0 100755
@@ -110,12 +110,49 @@ with iotests.FilePath('t.img') as disk_path, \
 
     iotests.img_info_log(remote_path)
 
-    md5_key = subprocess.check_output(
-        'ssh-keyscan -t ssh-ed25519 127.0.0.1 2>/dev/null | grep -v "\\^#" | ' +
-        'cut -d" " -f3 | base64 -d | md5sum -b | cut -d" " -f1',
-        shell=True).rstrip().decode('ascii')
+    keys = subprocess.check_output(
+        'ssh-keyscan 127.0.0.1 2>/dev/null | grep -v "\\^#" | ' +
+        'cut -d" " -f3',
+        shell=True).rstrip().decode('ascii').split('\n')
+
+    # Mappings of base64 representations to digests
+    md5_keys = {}
+    sha1_keys = {}
+
+    for key in keys:
+        md5_keys[key] = subprocess.check_output(
+            'echo %s | base64 -d | md5sum -b | cut -d" " -f1' % key,
+            shell=True).rstrip().decode('ascii')
+
+        sha1_keys[key] = subprocess.check_output(
+            'echo %s | base64 -d | sha1sum -b | cut -d" " -f1' % key,
+            shell=True).rstrip().decode('ascii')
 
     vm.launch()
+
+    # Find correct key first
+    matching_key = None
+    for key in keys:
+        result = vm.qmp('blockdev-add',
+                        driver='ssh', node_name='node0', path=disk_path,
+                        server={
+                             'host': '127.0.0.1',
+                             'port': '22',
+                        }, host_key_check={
+                             'mode': 'hash',
+                             'type': 'md5',
+                             'hash': md5_keys[key],
+                        })
+
+        if 'error' not in result:
+            vm.qmp('blockdev-del', node_name='node0')
+            matching_key = key
+            break
+
+    if matching_key is None:
+        vm.shutdown()
+        iotests.notrun('Did not find a key that fits 127.0.0.1')
+
     blockdev_create(vm, { 'driver': 'ssh',
                           'location': {
                               'path': disk_path,
@@ -140,7 +177,7 @@ with iotests.FilePath('t.img') as disk_path, \
                               'host-key-check': {
                                   'mode': 'hash',
                                   'type': 'md5',
-                                  'hash': md5_key,
+                                  'hash': md5_keys[matching_key],
                               }
                           },
                           'size': 8388608 })
@@ -148,11 +185,6 @@ with iotests.FilePath('t.img') as disk_path, \
 
     iotests.img_info_log(remote_path)
 
-    sha1_key = subprocess.check_output(
-        'ssh-keyscan -t ssh-ed25519 127.0.0.1 2>/dev/null | grep -v "\\^#" | ' +
-        'cut -d" " -f3 | base64 -d | sha1sum -b | cut -d" " -f1',
-        shell=True).rstrip().decode('ascii')
-
     vm.launch()
     blockdev_create(vm, { 'driver': 'ssh',
                           'location': {
@@ -178,7 +210,7 @@ with iotests.FilePath('t.img') as disk_path, \
                               'host-key-check': {
                                   'mode': 'hash',
                                   'type': 'sha1',
-                                  'hash': sha1_key,
+                                  'hash': sha1_keys[matching_key],
                               }
                           },
                           'size': 4194304 })
#2 lore raw
Quoted 123 lines · On Tuesday, 18 June 2019 15:14:30 CEST Max Reitz wrote:
Quoted 87 lines · Pino Toscano

Rewrite the implementation of the ssh block driver to use libssh instead of libssh2. The libssh library has various advantages over libssh2:
- easier API for authentication (for example for using ssh-agent)
- easier API for known_hosts handling
- supports newer types of keys in known_hosts

Use APIs/features available in libssh 0.8 conditionally, to support older versions (which are not recommended though).

Adjust the tests according to the different error message, and to the newer host keys (ed25519) that are used by default with OpenSSH >= 6.7 and libssh >= 0.7.0.

Adjust the various Docker/Travis scripts to use libssh when available instead of libssh2. The mingw/mxe testing is dropped for now, as there are no packages for it.

---

Changes from v9:
- restored "default" case in the server status switch for libssh < 0.8.0
- print the host key type & fingerprint on mismatch with known_hosts
- improve/fix message for failed socket_set_nodelay()
- reset s->sock properly

Changes from v8:
- use a newer key type in iotest 207
- improve the commit message

Changes from v7:
- #if HAVE_LIBSSH_0_8 -> #ifdef HAVE_LIBSSH_0_8
- ptrdiff_t -> size_t

Changes from v6:
- fixed few checkpatch style issues
- detect libssh 0.8 via symbol detection
- adjust travis/docker test material
- remove dead "default" case in a switch
- use variables for storing MIN() results
- adapt a documentation bit

Changes from v5:
- adapt to newer tracing APIs
- disable ssh compression (mimic what libssh2 does by default)
- use build time checks for libssh 0.8, and use newer APIs directly

Changes from v4:
- fix wrong usages of error_setg/session_error_setg/sftp_error_setg
- fix few return code checks
- remove now-unused parameters in few internal functions
- allow authentication with "none" method
- switch to unsigned int for the port number
- enable TCP_NODELAY on the socket
- fix one reference error message in iotest 207

Changes from v3:
- fix socket cleanup in connect_to_ssh()
- add comments about the socket cleanup
- improve the error reporting (closer to what was with libssh2)
- improve EOF detection on sftp_read()

Changes from v2:
- used again an own fd
- fixed co_yield() implementation

Changes from v1:
- fixed jumbo packets writing
- fixed missing 'err' assignment
- fixed commit message

 .travis.yml                                   |   4 +-
 block/Makefile.objs                           |   6 +-
 block/ssh.c                                   | 665 ++++++++++--------
 block/trace-events                            |  14 +-
 configure                                     |  65 +-
 docs/qemu-block-drivers.texi                  |   2 +-
 .../dockerfiles/debian-win32-cross.docker     |   1 -
 .../dockerfiles/debian-win64-cross.docker     |   1 -
 tests/docker/dockerfiles/fedora.docker        |   4 +-
 tests/docker/dockerfiles/ubuntu.docker        |   2 +-
 tests/docker/dockerfiles/ubuntu1804.docker    |   2 +-
 tests/qemu-iotests/207                        |   4 +-
 tests/qemu-iotests/207.out                    |   2 +-
 13 files changed, 423 insertions(+), 349 deletions(-)

[...]

Quoted 4 lines
block/ssh.c +0 -0
index 6da7b9cbfe..644ae8b82c 100644

[...]

Quoted 14 lines
block/ssh.c +14 -0
+    case SSH_SERVER_KNOWN_CHANGED:
+        ret = -EINVAL;
+        r = ssh_get_publickey(s->session, &pubkey);
+        if (r == 0) {
+            r = ssh_get_publickey_hash(pubkey, SSH_PUBLICKEY_HASH_SHA1,
+                                       &server_hash, &server_hash_len);
+            pubkey_type = ssh_key_type(pubkey);
+            ssh_key_free(pubkey);
+        }
+        if (r == 0) {
+            fingerprint = ssh_get_fingerprint_hash(SSH_PUBLICKEY_HASH_SHA1,
+                                                   server_hash,
+                                                   server_hash_len);
+            ssh_clean_pubkey_hash(&server_hash);

block/ssh.c

Quoted 10 lines
block/ssh.c +10 -0
+        }
+        if (fingerprint) {
+            error_setg(errp,
+                       "host key (%s key with fingerprint %s) does not match "
+                       "the one in known_hosts",
+                       ssh_key_type_to_char(pubkey_type), fingerprint);
+            ssh_string_free_char(fingerprint);
+        } else  {
+            error_setg(errp, "host key does not match the one in known_hosts");
+        }
Quoted 3 lines

Usually I’d say that more information can’t be bad. But here I don’t see how this additional information is useful. known_hosts contains base64-encoded full public keys. This only prints the SHA-1 digest.

Note that SHA-1 is printed with libssh < 0.8; with libssh >= 0.8 SHA-256 is used instead.

Quoted 9 lines

The user cannot add that to known_hosts, or easily scan known_hosts to see whether it perhaps belongs to some other hosts (maybe because DHCP scrambled something).

Actually, even if this printed the base64 representation of the full key, the user probably wouldn’t just add that to known_hosts or do anything with it. They’ll debug the problem with other tools.

So I don’t think this new information is really useful...?

When this situation happens with openssh, the big scary message prints three things:
1) the fingerprint of the server
2) the line of the server in the known_hosts file
3) how to remove the keys of the server from known_hosts, i.e. a copy-paste'able ssh-keygen -R host

Here I'm doing only (1). Also, the current libssh2 driver does something else, i.e. print the base64/printable representation of the server key in known_hosts.

Quoted 3 lines

(Hmm, I don’t know if this is a response to my “But the specification requires a warning!!1!”, but if so, I was actually not referring to more information but to this:

You mentioned this few times: can you please point me to this bit? I read few RFCs related to ssh, and I did not find this information...

Quoted 9 lines
$ ssh 192.168.0.12
(quoted diff) +0 -0
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @

(quoted diff) +0 -0
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)! It is also possible that a host key has just been changed.

Quoted 3 lines

I mean, I was also only half-serious. I should be serious because the libssh specification requires us to print some warning like that, but, well. Yes, I’ll stop mumbling about this stuff now.)

To be on the explic side: are you asking to basically put the first 6 lines of the openssh error message (as you quoted them above) as error message in the ssh driver?
As data point (I know it is not a strong argument), I'll mention that the current libssh2-based driver does not do that, and (according to my possibly limited knowledge) there were no problems/complaints about that. Sure, improvements are good, I get that, although I do not see why just changing the underlying implementation of a driver makes an error message for the same situation immediately no more acceptable. I'm perfectly fine in improving this in sequent patches, for example -- as I mentioned, the API for this in libssh is sadly not usable, and it will be usable with the future libssh 0.9.0 (kudos to the libssh guys for the fast fix).

Quoted 19 lines

[...]

Quoted 16 lines
tests/qemu-iotests/207 +1 -1
index b3816136f7..774eb5f2a9 100755
@@ -111,7 +111,7 @@ with iotests.FilePath('t.img') as disk_path, \
     iotests.img_info_log(remote_path)
 
     md5_key = subprocess.check_output(
-        'ssh-keyscan -t rsa 127.0.0.1 2>/dev/null | grep -v "\\^#" | ' +
+        'ssh-keyscan -t ssh-ed25519 127.0.0.1 2>/dev/null | grep -v "\\^#" | ' +
         'cut -d" " -f3 | base64 -d | md5sum -b | cut -d" " -f1',
         shell=True).rstrip().decode('ascii')
 
@@ -149,7 +149,7 @@ with iotests.FilePath('t.img') as disk_path, \
     iotests.img_info_log(remote_path)
 
Quoted 8 lines

tests/qemu-iotests/207 with iotests.FilePath('t.img') as disk_path, \ · line 154

Quoted 5 lines
tests/qemu-iotests/207 +1 -1
     sha1_key = subprocess.check_output(
-        'ssh-keyscan -t rsa 127.0.0.1 2>/dev/null | grep -v "\\^#" | ' +
+        'ssh-keyscan -t ssh-ed25519 127.0.0.1 2>/dev/null | grep -v "\\^#" | ' +
         'cut -d" " -f3 | base64 -d | sha1sum -b | cut -d" " -f1',
         shell=True).rstrip().decode('ascii')

I’ve attached a diff that makes this test pass for me. Maybe also for you and Philippe.

It works for me as well, thanks!
I squashed it locally, and amended the commit message to mention this with credits for you.

Thanks,

Signature · 1 line
Pino Toscano
#3 lore raw
Quoted 148 lines · Pino Toscano
Quoted 123 lines · On Tuesday, 18 June 2019 15:14:30 CEST Max Reitz wrote:
Quoted 87 lines · Pino Toscano

Rewrite the implementation of the ssh block driver to use libssh instead of libssh2. The libssh library has various advantages over libssh2:
- easier API for authentication (for example for using ssh-agent)
- easier API for known_hosts handling
- supports newer types of keys in known_hosts

Use APIs/features available in libssh 0.8 conditionally, to support older versions (which are not recommended though).

Adjust the tests according to the different error message, and to the newer host keys (ed25519) that are used by default with OpenSSH >= 6.7 and libssh >= 0.7.0.

Adjust the various Docker/Travis scripts to use libssh when available instead of libssh2. The mingw/mxe testing is dropped for now, as there are no packages for it.

---

Changes from v9:
- restored "default" case in the server status switch for libssh < 0.8.0
- print the host key type & fingerprint on mismatch with known_hosts
- improve/fix message for failed socket_set_nodelay()
- reset s->sock properly

Changes from v8:
- use a newer key type in iotest 207
- improve the commit message

Changes from v7:
- #if HAVE_LIBSSH_0_8 -> #ifdef HAVE_LIBSSH_0_8
- ptrdiff_t -> size_t

Changes from v6:
- fixed few checkpatch style issues
- detect libssh 0.8 via symbol detection
- adjust travis/docker test material
- remove dead "default" case in a switch
- use variables for storing MIN() results
- adapt a documentation bit

Changes from v5:
- adapt to newer tracing APIs
- disable ssh compression (mimic what libssh2 does by default)
- use build time checks for libssh 0.8, and use newer APIs directly

Changes from v4:
- fix wrong usages of error_setg/session_error_setg/sftp_error_setg
- fix few return code checks
- remove now-unused parameters in few internal functions
- allow authentication with "none" method
- switch to unsigned int for the port number
- enable TCP_NODELAY on the socket
- fix one reference error message in iotest 207

Changes from v3:
- fix socket cleanup in connect_to_ssh()
- add comments about the socket cleanup
- improve the error reporting (closer to what was with libssh2)
- improve EOF detection on sftp_read()

Changes from v2:
- used again an own fd
- fixed co_yield() implementation

Changes from v1:
- fixed jumbo packets writing
- fixed missing 'err' assignment
- fixed commit message

 .travis.yml                                   |   4 +-
 block/Makefile.objs                           |   6 +-
 block/ssh.c                                   | 665 ++++++++++--------
 block/trace-events                            |  14 +-
 configure                                     |  65 +-
 docs/qemu-block-drivers.texi                  |   2 +-
 .../dockerfiles/debian-win32-cross.docker     |   1 -
 .../dockerfiles/debian-win64-cross.docker     |   1 -
 tests/docker/dockerfiles/fedora.docker        |   4 +-
 tests/docker/dockerfiles/ubuntu.docker        |   2 +-
 tests/docker/dockerfiles/ubuntu1804.docker    |   2 +-
 tests/qemu-iotests/207                        |   4 +-
 tests/qemu-iotests/207.out                    |   2 +-
 13 files changed, 423 insertions(+), 349 deletions(-)

[...]

Quoted 4 lines
block/ssh.c +0 -0
index 6da7b9cbfe..644ae8b82c 100644

[...]

Quoted 14 lines
block/ssh.c +14 -0
+    case SSH_SERVER_KNOWN_CHANGED:
+        ret = -EINVAL;
+        r = ssh_get_publickey(s->session, &pubkey);
+        if (r == 0) {
+            r = ssh_get_publickey_hash(pubkey, SSH_PUBLICKEY_HASH_SHA1,
+                                       &server_hash, &server_hash_len);
+            pubkey_type = ssh_key_type(pubkey);
+            ssh_key_free(pubkey);
+        }
+        if (r == 0) {
+            fingerprint = ssh_get_fingerprint_hash(SSH_PUBLICKEY_HASH_SHA1,
+                                                   server_hash,
+                                                   server_hash_len);
+            ssh_clean_pubkey_hash(&server_hash);

block/ssh.c

Quoted 10 lines
block/ssh.c +10 -0
+        }
+        if (fingerprint) {
+            error_setg(errp,
+                       "host key (%s key with fingerprint %s) does not match "
+                       "the one in known_hosts",
+                       ssh_key_type_to_char(pubkey_type), fingerprint);
+            ssh_string_free_char(fingerprint);
+        } else  {
+            error_setg(errp, "host key does not match the one in known_hosts");
+        }
Quoted 3 lines

Usually I’d say that more information can’t be bad. But here I don’t see how this additional information is useful. known_hosts contains base64-encoded full public keys. This only prints the SHA-1 digest.

Note that SHA-1 is printed with libssh < 0.8; with libssh >= 0.8 SHA-256 is used instead.

Quoted 9 lines

The user cannot add that to known_hosts, or easily scan known_hosts to see whether it perhaps belongs to some other hosts (maybe because DHCP scrambled something).

Actually, even if this printed the base64 representation of the full key, the user probably wouldn’t just add that to known_hosts or do anything with it. They’ll debug the problem with other tools.

So I don’t think this new information is really useful...?

When this situation happens with openssh, the big scary message prints three things:
1) the fingerprint of the server
2) the line of the server in the known_hosts file
3) how to remove the keys of the server from known_hosts, i.e. a copy-paste'able ssh-keygen -R host

Quoted 3 lines

Here I'm doing only (1). Also, the current libssh2 driver does something else, i.e. print the base64/printable representation of the server key in known_hosts.

Well, I don’t know whether it’s necessary to reproduce the exact message with all the data it contains. As I said, I think users can and will investigate the exact root of the problem with tools outside of qemu. (Such as openssh’s ssh itself.)

Quoted 6 lines
Quoted 3 lines

(Hmm, I don’t know if this is a response to my “But the specification requires a warning!!1!”, but if so, I was actually not referring to more information but to this:

You mentioned this few times: can you please point me to this bit? I read few RFCs related to ssh, and I did not find this information...

For example:
http://api.libssh.org/master/group__libssh__session.html#gacbc5d04fe66beee863a0c61a93fdf765

Quoted 1 line

SSH_KNOWN_HOSTS_CHANGED: The server key has changed. Either you are under attack or the administrator changed the key. You HAVE to warn the user about a possible attack.

This doesn’t require any specific formatting or data to be given to the user. All it requires is “to warn the user about a possible attack”. That can be as simple as appending “This may be due to a man-in-the-middle attack” to the error message.

Quoted 9 lines
Quoted 9 lines
$ ssh 192.168.0.12
(quoted diff) +0 -0
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @

(quoted diff) +0 -0
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)! It is also possible that a host key has just been changed.

Quoted 7 lines
Quoted 3 lines

I mean, I was also only half-serious. I should be serious because the libssh specification requires us to print some warning like that, but, well. Yes, I’ll stop mumbling about this stuff now.)

To be on the explic side: are you asking to basically put the first 6 lines of the openssh error message (as you quoted them above) as error message in the ssh driver?

God forbid no. I was just making an example of “Here is a warning about a possible attack”. I fully agree with Dan (and probably you) that this is completely unsuitable to qemu’s interface.

Sorry if that came across in another way.

Quoted 10 lines

As data point (I know it is not a strong argument), I'll mention that the current libssh2-based driver does not do that, and (according to my possibly limited knowledge) there were no problems/complaints about that. Sure, improvements are good, I get that, although I do not see why just changing the underlying implementation of a driver makes an error message for the same situation immediately no more acceptable. I'm perfectly fine in improving this in sequent patches, for example -- as I mentioned, the API for this in libssh is sadly not usable, and it will be usable with the future libssh 0.9.0 (kudos to the libssh guys for the fast fix).

Again, I’m only being half-serious. I tried to bring that point across every time I mention this thing.

If you just say “Let’s leave it as it is, I don’t see any benefit”, sure, OK, I’ll accept that.

But so far you didn’t say that, so I kept pestering on.

I would have never mentioned this topic if I hadn’t read the specification while reviewing your patch last year and noticed that it explicitly states “You HAVE to warn the user about a possible attack.”

I never said “Please emit the exact message openssh’s ssh” emits. Well, I never meant to say that. Maybe I accidentally did. Sorry again.

My whole point is: The specification requires libssh users to emit *some* warning. If we want to adhere to the specification, we have to do that. Just one sentence is sufficient.

If we don’t want to adhere to the specification, sure, let’s go. But you’ll have to explicitly say that. I’m not the patch author, I’m but a humble reviewer, and as such I’ll point out all flaws I see. You have to give justifications, not me. O:-)

Quoted 27 lines
Quoted 18 lines

[...]

Quoted 16 lines
tests/qemu-iotests/207 +1 -1
index b3816136f7..774eb5f2a9 100755
@@ -111,7 +111,7 @@ with iotests.FilePath('t.img') as disk_path, \
     iotests.img_info_log(remote_path)
 
     md5_key = subprocess.check_output(
-        'ssh-keyscan -t rsa 127.0.0.1 2>/dev/null | grep -v "\\^#" | ' +
+        'ssh-keyscan -t ssh-ed25519 127.0.0.1 2>/dev/null | grep -v "\\^#" | ' +
         'cut -d" " -f3 | base64 -d | md5sum -b | cut -d" " -f1',
         shell=True).rstrip().decode('ascii')
 
@@ -149,7 +149,7 @@ with iotests.FilePath('t.img') as disk_path, \
     iotests.img_info_log(remote_path)
 
Quoted 8 lines

tests/qemu-iotests/207 with iotests.FilePath('t.img') as disk_path, \ · line 154

Quoted 5 lines
tests/qemu-iotests/207 +1 -1
     sha1_key = subprocess.check_output(
-        'ssh-keyscan -t rsa 127.0.0.1 2>/dev/null | grep -v "\\^#" | ' +
+        'ssh-keyscan -t ssh-ed25519 127.0.0.1 2>/dev/null | grep -v "\\^#" | ' +
         'cut -d" " -f3 | base64 -d | sha1sum -b | cut -d" " -f1',
         shell=True).rstrip().decode('ascii')

I’ve attached a diff that makes this test pass for me. Maybe also for you and Philippe.

Quoted 1 line

It works for me as well, thanks!

Great!

Max

#4 lore raw
Quoted 162 lines · On Thursday, 20 June 2019 14:58:40 CEST Max Reitz wrote:
Quoted 148 lines · Pino Toscano
Quoted 123 lines · On Tuesday, 18 June 2019 15:14:30 CEST Max Reitz wrote:
Quoted 87 lines · Pino Toscano

Rewrite the implementation of the ssh block driver to use libssh instead of libssh2. The libssh library has various advantages over libssh2:
- easier API for authentication (for example for using ssh-agent)
- easier API for known_hosts handling
- supports newer types of keys in known_hosts

Use APIs/features available in libssh 0.8 conditionally, to support older versions (which are not recommended though).

Adjust the tests according to the different error message, and to the newer host keys (ed25519) that are used by default with OpenSSH >= 6.7 and libssh >= 0.7.0.

Adjust the various Docker/Travis scripts to use libssh when available instead of libssh2. The mingw/mxe testing is dropped for now, as there are no packages for it.

---

Changes from v9:
- restored "default" case in the server status switch for libssh < 0.8.0
- print the host key type & fingerprint on mismatch with known_hosts
- improve/fix message for failed socket_set_nodelay()
- reset s->sock properly

Changes from v8:
- use a newer key type in iotest 207
- improve the commit message

Changes from v7:
- #if HAVE_LIBSSH_0_8 -> #ifdef HAVE_LIBSSH_0_8
- ptrdiff_t -> size_t

Changes from v6:
- fixed few checkpatch style issues
- detect libssh 0.8 via symbol detection
- adjust travis/docker test material
- remove dead "default" case in a switch
- use variables for storing MIN() results
- adapt a documentation bit

Changes from v5:
- adapt to newer tracing APIs
- disable ssh compression (mimic what libssh2 does by default)
- use build time checks for libssh 0.8, and use newer APIs directly

Changes from v4:
- fix wrong usages of error_setg/session_error_setg/sftp_error_setg
- fix few return code checks
- remove now-unused parameters in few internal functions
- allow authentication with "none" method
- switch to unsigned int for the port number
- enable TCP_NODELAY on the socket
- fix one reference error message in iotest 207

Changes from v3:
- fix socket cleanup in connect_to_ssh()
- add comments about the socket cleanup
- improve the error reporting (closer to what was with libssh2)
- improve EOF detection on sftp_read()

Changes from v2:
- used again an own fd
- fixed co_yield() implementation

Changes from v1:
- fixed jumbo packets writing
- fixed missing 'err' assignment
- fixed commit message

 .travis.yml                                   |   4 +-
 block/Makefile.objs                           |   6 +-
 block/ssh.c                                   | 665 ++++++++++--------
 block/trace-events                            |  14 +-
 configure                                     |  65 +-
 docs/qemu-block-drivers.texi                  |   2 +-
 .../dockerfiles/debian-win32-cross.docker     |   1 -
 .../dockerfiles/debian-win64-cross.docker     |   1 -
 tests/docker/dockerfiles/fedora.docker        |   4 +-
 tests/docker/dockerfiles/ubuntu.docker        |   2 +-
 tests/docker/dockerfiles/ubuntu1804.docker    |   2 +-
 tests/qemu-iotests/207                        |   4 +-
 tests/qemu-iotests/207.out                    |   2 +-
 13 files changed, 423 insertions(+), 349 deletions(-)

[...]

Quoted 4 lines
block/ssh.c +0 -0
index 6da7b9cbfe..644ae8b82c 100644

[...]

Quoted 14 lines
block/ssh.c +14 -0
+    case SSH_SERVER_KNOWN_CHANGED:
+        ret = -EINVAL;
+        r = ssh_get_publickey(s->session, &pubkey);
+        if (r == 0) {
+            r = ssh_get_publickey_hash(pubkey, SSH_PUBLICKEY_HASH_SHA1,
+                                       &server_hash, &server_hash_len);
+            pubkey_type = ssh_key_type(pubkey);
+            ssh_key_free(pubkey);
+        }
+        if (r == 0) {
+            fingerprint = ssh_get_fingerprint_hash(SSH_PUBLICKEY_HASH_SHA1,
+                                                   server_hash,
+                                                   server_hash_len);
+            ssh_clean_pubkey_hash(&server_hash);

block/ssh.c

Quoted 10 lines
block/ssh.c +10 -0
+        }
+        if (fingerprint) {
+            error_setg(errp,
+                       "host key (%s key with fingerprint %s) does not match "
+                       "the one in known_hosts",
+                       ssh_key_type_to_char(pubkey_type), fingerprint);
+            ssh_string_free_char(fingerprint);
+        } else  {
+            error_setg(errp, "host key does not match the one in known_hosts");
+        }
Quoted 3 lines

Usually I’d say that more information can’t be bad. But here I don’t see how this additional information is useful. known_hosts contains base64-encoded full public keys. This only prints the SHA-1 digest.

Note that SHA-1 is printed with libssh < 0.8; with libssh >= 0.8 SHA-256 is used instead.

Quoted 9 lines

The user cannot add that to known_hosts, or easily scan known_hosts to see whether it perhaps belongs to some other hosts (maybe because DHCP scrambled something).

Actually, even if this printed the base64 representation of the full key, the user probably wouldn’t just add that to known_hosts or do anything with it. They’ll debug the problem with other tools.

So I don’t think this new information is really useful...?

When this situation happens with openssh, the big scary message prints three things:
1) the fingerprint of the server
2) the line of the server in the known_hosts file
3) how to remove the keys of the server from known_hosts, i.e. a copy-paste'able ssh-keygen -R host

Quoted 3 lines

Here I'm doing only (1). Also, the current libssh2 driver does something else, i.e. print the base64/printable representation of the server key in known_hosts.

Well, I don’t know whether it’s necessary to reproduce the exact message with all the data it contains. As I said, I think users can and will investigate the exact root of the problem with tools outside of qemu. (Such as openssh’s ssh itself.)

Quoted 4 lines
Quoted 3 lines

(Hmm, I don’t know if this is a response to my “But the specification requires a warning!!1!”, but if so, I was actually not referring to more information but to this:

Quoted 6 lines
Quoted 2 lines

You mentioned this few times: can you please point me to this bit? I read few RFCs related to ssh, and I did not find this information...

For example:
http://api.libssh.org/master/group__libssh__session.html#gacbc5d04fe66beee863a0c61a93fdf765

Quoted 1 line

SSH_KNOWN_HOSTS_CHANGED: The server key has changed. Either you are under attack or the administrator changed the key. You HAVE to warn the user about a possible attack.

Ah, now I see what you mean! I thought that "libssh specification" was any of the SSH RFCs, and that's why I did not find what you meant. I see you were talking about the libssh API documentation :-) (Never heard the API docs of a library called as "specification" before, TBH.)

Quoted 4 lines

This doesn’t require any specific formatting or data to be given to the user. All it requires is “to warn the user about a possible attack”. That can be as simple as appending “This may be due to a man-in-the-middle attack” to the error message.

Makes sense -- I just asked to the libssh people, and appending "this may be a possible attack" should be enough, especially that this is not a UI message like the one written by the ssh client.

Quoted 17 lines
Quoted 9 lines
Quoted 9 lines
$ ssh 192.168.0.12
(quoted diff) +0 -0
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @

(quoted diff) +0 -0
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)! It is also possible that a host key has just been changed.

Quoted 7 lines
Quoted 3 lines

I mean, I was also only half-serious. I should be serious because the libssh specification requires us to print some warning like that, but, well. Yes, I’ll stop mumbling about this stuff now.)

To be on the explic side: are you asking to basically put the first 6 lines of the openssh error message (as you quoted them above) as error message in the ssh driver?

Quoted 5 lines

God forbid no. I was just making an example of “Here is a warning about a possible attack”. I fully agree with Dan (and probably you) that this is completely unsuitable to qemu’s interface.

Sorry if that came across in another way.

Not a problem. I preferred to ask explicitly to make sure to get it right -- any amount of information shown is fine for me, I want to make sure to follow QEMU best practices (if any).

Signature · 1 line
Pino Toscano
#5 lore raw
Quoted 168 lines · Pino Toscano
Quoted 167 lines · On Thursday, 20 June 2019 14:58:40 CEST Max Reitz wrote:
Quoted 148 lines · Pino Toscano
Quoted 123 lines · On Tuesday, 18 June 2019 15:14:30 CEST Max Reitz wrote:
Quoted 87 lines · Pino Toscano

Rewrite the implementation of the ssh block driver to use libssh instead of libssh2. The libssh library has various advantages over libssh2:
- easier API for authentication (for example for using ssh-agent)
- easier API for known_hosts handling
- supports newer types of keys in known_hosts

Use APIs/features available in libssh 0.8 conditionally, to support older versions (which are not recommended though).

Adjust the tests according to the different error message, and to the newer host keys (ed25519) that are used by default with OpenSSH >= 6.7 and libssh >= 0.7.0.

Adjust the various Docker/Travis scripts to use libssh when available instead of libssh2. The mingw/mxe testing is dropped for now, as there are no packages for it.

---

Changes from v9:
- restored "default" case in the server status switch for libssh < 0.8.0
- print the host key type & fingerprint on mismatch with known_hosts
- improve/fix message for failed socket_set_nodelay()
- reset s->sock properly

Changes from v8:
- use a newer key type in iotest 207
- improve the commit message

Changes from v7:
- #if HAVE_LIBSSH_0_8 -> #ifdef HAVE_LIBSSH_0_8
- ptrdiff_t -> size_t

Changes from v6:
- fixed few checkpatch style issues
- detect libssh 0.8 via symbol detection
- adjust travis/docker test material
- remove dead "default" case in a switch
- use variables for storing MIN() results
- adapt a documentation bit

Changes from v5:
- adapt to newer tracing APIs
- disable ssh compression (mimic what libssh2 does by default)
- use build time checks for libssh 0.8, and use newer APIs directly

Changes from v4:
- fix wrong usages of error_setg/session_error_setg/sftp_error_setg
- fix few return code checks
- remove now-unused parameters in few internal functions
- allow authentication with "none" method
- switch to unsigned int for the port number
- enable TCP_NODELAY on the socket
- fix one reference error message in iotest 207

Changes from v3:
- fix socket cleanup in connect_to_ssh()
- add comments about the socket cleanup
- improve the error reporting (closer to what was with libssh2)
- improve EOF detection on sftp_read()

Changes from v2:
- used again an own fd
- fixed co_yield() implementation

Changes from v1:
- fixed jumbo packets writing
- fixed missing 'err' assignment
- fixed commit message

 .travis.yml                                   |   4 +-
 block/Makefile.objs                           |   6 +-
 block/ssh.c                                   | 665 ++++++++++--------
 block/trace-events                            |  14 +-
 configure                                     |  65 +-
 docs/qemu-block-drivers.texi                  |   2 +-
 .../dockerfiles/debian-win32-cross.docker     |   1 -
 .../dockerfiles/debian-win64-cross.docker     |   1 -
 tests/docker/dockerfiles/fedora.docker        |   4 +-
 tests/docker/dockerfiles/ubuntu.docker        |   2 +-
 tests/docker/dockerfiles/ubuntu1804.docker    |   2 +-
 tests/qemu-iotests/207                        |   4 +-
 tests/qemu-iotests/207.out                    |   2 +-
 13 files changed, 423 insertions(+), 349 deletions(-)

[...]

Quoted 4 lines
block/ssh.c +0 -0
index 6da7b9cbfe..644ae8b82c 100644

[...]

Quoted 14 lines
block/ssh.c +14 -0
+    case SSH_SERVER_KNOWN_CHANGED:
+        ret = -EINVAL;
+        r = ssh_get_publickey(s->session, &pubkey);
+        if (r == 0) {
+            r = ssh_get_publickey_hash(pubkey, SSH_PUBLICKEY_HASH_SHA1,
+                                       &server_hash, &server_hash_len);
+            pubkey_type = ssh_key_type(pubkey);
+            ssh_key_free(pubkey);
+        }
+        if (r == 0) {
+            fingerprint = ssh_get_fingerprint_hash(SSH_PUBLICKEY_HASH_SHA1,
+                                                   server_hash,
+                                                   server_hash_len);
+            ssh_clean_pubkey_hash(&server_hash);

block/ssh.c

Quoted 10 lines
block/ssh.c +10 -0
+        }
+        if (fingerprint) {
+            error_setg(errp,
+                       "host key (%s key with fingerprint %s) does not match "
+                       "the one in known_hosts",
+                       ssh_key_type_to_char(pubkey_type), fingerprint);
+            ssh_string_free_char(fingerprint);
+        } else  {
+            error_setg(errp, "host key does not match the one in known_hosts");
+        }
Quoted 3 lines

Usually I’d say that more information can’t be bad. But here I don’t see how this additional information is useful. known_hosts contains base64-encoded full public keys. This only prints the SHA-1 digest.

Note that SHA-1 is printed with libssh < 0.8; with libssh >= 0.8 SHA-256 is used instead.

Quoted 9 lines

The user cannot add that to known_hosts, or easily scan known_hosts to see whether it perhaps belongs to some other hosts (maybe because DHCP scrambled something).

Actually, even if this printed the base64 representation of the full key, the user probably wouldn’t just add that to known_hosts or do anything with it. They’ll debug the problem with other tools.

So I don’t think this new information is really useful...?

When this situation happens with openssh, the big scary message prints three things:
1) the fingerprint of the server
2) the line of the server in the known_hosts file
3) how to remove the keys of the server from known_hosts, i.e. a copy-paste'able ssh-keygen -R host

Quoted 3 lines

Here I'm doing only (1). Also, the current libssh2 driver does something else, i.e. print the base64/printable representation of the server key in known_hosts.

Well, I don’t know whether it’s necessary to reproduce the exact message with all the data it contains. As I said, I think users can and will investigate the exact root of the problem with tools outside of qemu. (Such as openssh’s ssh itself.)

Quoted 6 lines
Quoted 3 lines

(Hmm, I don’t know if this is a response to my “But the specification requires a warning!!1!”, but if so, I was actually not referring to more information but to this:

You mentioned this few times: can you please point me to this bit? I read few RFCs related to ssh, and I did not find this information...

For example:
http://api.libssh.org/master/group__libssh__session.html#gacbc5d04fe66beee863a0c61a93fdf765

Quoted 7 lines
Quoted 1 line
Quoted 1 line

SSH_KNOWN_HOSTS_CHANGED: The server key has changed. Either you are under attack or the administrator changed the key. You HAVE to warn the user about a possible attack.

Ah, now I see what you mean! I thought that "libssh specification" was any of the SSH RFCs, and that's why I did not find what you meant. I see you were talking about the libssh API documentation :-) (Never heard the API docs of a library called as "specification" before, TBH.)

Ah, sorry. I have no excuse. I just call everything a “spec” that’s telling me what to do. (I should probably stop doing that.)

Quoted 8 lines
Quoted 4 lines

This doesn’t require any specific formatting or data to be given to the user. All it requires is “to warn the user about a possible attack”. That can be as simple as appending “This may be due to a man-in-the-middle attack” to the error message.

Makes sense -- I just asked to the libssh people, and appending "this may be a possible attack" should be enough, especially that this is not a UI message like the one written by the ssh client.

OK, great!

Quoted 21 lines
Quoted 21 lines
Quoted 9 lines
Quoted 9 lines
$ ssh 192.168.0.12
(quoted diff) +0 -0
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @

(quoted diff) +0 -0
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)! It is also possible that a host key has just been changed.

Quoted 7 lines
Quoted 3 lines

I mean, I was also only half-serious. I should be serious because the libssh specification requires us to print some warning like that, but, well. Yes, I’ll stop mumbling about this stuff now.)

To be on the explic side: are you asking to basically put the first 6 lines of the openssh error message (as you quoted them above) as error message in the ssh driver?

God forbid no. I was just making an example of “Here is a warning about a possible attack”. I fully agree with Dan (and probably you) that this is completely unsuitable to qemu’s interface.

Quoted 5 lines
Quoted 1 line

Sorry if that came across in another way.

Not a problem. I preferred to ask explicitly to make sure to get it right -- any amount of information shown is fine for me, I want to make sure to follow QEMU best practices (if any).

I mean, as you’ve seen yourself, there currently is no warning. So it’s not like there is any practice, not to mention a best one...

Max

#6 lore raw
In reply to Max Reitz
Quoted 150 lines · Max Reitz
Quoted 87 lines · Pino Toscano

Rewrite the implementation of the ssh block driver to use libssh instead of libssh2. The libssh library has various advantages over libssh2:
- easier API for authentication (for example for using ssh-agent)
- easier API for known_hosts handling
- supports newer types of keys in known_hosts

Use APIs/features available in libssh 0.8 conditionally, to support older versions (which are not recommended though).

Adjust the tests according to the different error message, and to the newer host keys (ed25519) that are used by default with OpenSSH >= 6.7 and libssh >= 0.7.0.

Adjust the various Docker/Travis scripts to use libssh when available instead of libssh2. The mingw/mxe testing is dropped for now, as there are no packages for it.

---

Changes from v9:
- restored "default" case in the server status switch for libssh < 0.8.0
- print the host key type & fingerprint on mismatch with known_hosts
- improve/fix message for failed socket_set_nodelay()
- reset s->sock properly

Changes from v8:
- use a newer key type in iotest 207
- improve the commit message

Changes from v7:
- #if HAVE_LIBSSH_0_8 -> #ifdef HAVE_LIBSSH_0_8
- ptrdiff_t -> size_t

Changes from v6:
- fixed few checkpatch style issues
- detect libssh 0.8 via symbol detection
- adjust travis/docker test material
- remove dead "default" case in a switch
- use variables for storing MIN() results
- adapt a documentation bit

Changes from v5:
- adapt to newer tracing APIs
- disable ssh compression (mimic what libssh2 does by default)
- use build time checks for libssh 0.8, and use newer APIs directly

Changes from v4:
- fix wrong usages of error_setg/session_error_setg/sftp_error_setg
- fix few return code checks
- remove now-unused parameters in few internal functions
- allow authentication with "none" method
- switch to unsigned int for the port number
- enable TCP_NODELAY on the socket
- fix one reference error message in iotest 207

Changes from v3:
- fix socket cleanup in connect_to_ssh()
- add comments about the socket cleanup
- improve the error reporting (closer to what was with libssh2)
- improve EOF detection on sftp_read()

Changes from v2:
- used again an own fd
- fixed co_yield() implementation

Changes from v1:
- fixed jumbo packets writing
- fixed missing 'err' assignment
- fixed commit message

 .travis.yml                                   |   4 +-
 block/Makefile.objs                           |   6 +-
 block/ssh.c                                   | 665 ++++++++++--------
 block/trace-events                            |  14 +-
 configure                                     |  65 +-
 docs/qemu-block-drivers.texi                  |   2 +-
 .../dockerfiles/debian-win32-cross.docker     |   1 -
 .../dockerfiles/debian-win64-cross.docker     |   1 -
 tests/docker/dockerfiles/fedora.docker        |   4 +-
 tests/docker/dockerfiles/ubuntu.docker        |   2 +-
 tests/docker/dockerfiles/ubuntu1804.docker    |   2 +-
 tests/qemu-iotests/207                        |   4 +-
 tests/qemu-iotests/207.out                    |   2 +-
 13 files changed, 423 insertions(+), 349 deletions(-)

[...]

Quoted 4 lines
block/ssh.c +0 -0
index 6da7b9cbfe..644ae8b82c 100644

[...]

Quoted 14 lines
block/ssh.c +14 -0
+    case SSH_SERVER_KNOWN_CHANGED:
+        ret = -EINVAL;
+        r = ssh_get_publickey(s->session, &pubkey);
+        if (r == 0) {
+            r = ssh_get_publickey_hash(pubkey, SSH_PUBLICKEY_HASH_SHA1,
+                                       &server_hash, &server_hash_len);
+            pubkey_type = ssh_key_type(pubkey);
+            ssh_key_free(pubkey);
+        }
+        if (r == 0) {
+            fingerprint = ssh_get_fingerprint_hash(SSH_PUBLICKEY_HASH_SHA1,
+                                                   server_hash,
+                                                   server_hash_len);
+            ssh_clean_pubkey_hash(&server_hash);

block/ssh.c

Quoted 10 lines
block/ssh.c +10 -0
+        }
+        if (fingerprint) {
+            error_setg(errp,
+                       "host key (%s key with fingerprint %s) does not match "
+                       "the one in known_hosts",
+                       ssh_key_type_to_char(pubkey_type), fingerprint);
+            ssh_string_free_char(fingerprint);
+        } else  {
+            error_setg(errp, "host key does not match the one in known_hosts");
+        }

Usually I’d say that more information can’t be bad. But here I don’t see how this additional information is useful. known_hosts contains base64-encoded full public keys. This only prints the SHA-1 digest. The user cannot add that to known_hosts, or easily scan known_hosts to see whether it perhaps belongs to some other hosts (maybe because DHCP scrambled something).

Actually, even if this printed the base64 representation of the full key, the user probably wouldn’t just add that to known_hosts or do anything with it. They’ll debug the problem with other tools.

So I don’t think this new information is really useful...?

(Hmm, I don’t know if this is a response to my “But the specification requires a warning!!1!”, but if so, I was actually not referring to more information but to this:

$ ssh 192.168.0.12
(quoted diff) +0 -0
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @

(quoted diff) +0 -0
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)! It is also possible that a host key has just been changed.

Quoted 3 lines

I mean, I was also only half-serious. I should be serious because the libssh specification requires us to print some warning like that, but, well. Yes, I’ll stop mumbling about this stuff now.)

Those giant messages are targetted at humans though so I'm not so convinced it is useful for apps managing QEMU. IMHO it is sufficient to just report a simple error that the host ident check failed as the patch does now. It gives enough info to then investigate further to identify the root cause problems.

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#7 lore raw
Quoted 155 lines · Daniel P. Berrangé
Quoted 150 lines · Max Reitz
Quoted 87 lines · Pino Toscano

Rewrite the implementation of the ssh block driver to use libssh instead of libssh2. The libssh library has various advantages over libssh2:
- easier API for authentication (for example for using ssh-agent)
- easier API for known_hosts handling
- supports newer types of keys in known_hosts

Use APIs/features available in libssh 0.8 conditionally, to support older versions (which are not recommended though).

Adjust the tests according to the different error message, and to the newer host keys (ed25519) that are used by default with OpenSSH >= 6.7 and libssh >= 0.7.0.

Adjust the various Docker/Travis scripts to use libssh when available instead of libssh2. The mingw/mxe testing is dropped for now, as there are no packages for it.

---

Changes from v9:
- restored "default" case in the server status switch for libssh < 0.8.0
- print the host key type & fingerprint on mismatch with known_hosts
- improve/fix message for failed socket_set_nodelay()
- reset s->sock properly

Changes from v8:
- use a newer key type in iotest 207
- improve the commit message

Changes from v7:
- #if HAVE_LIBSSH_0_8 -> #ifdef HAVE_LIBSSH_0_8
- ptrdiff_t -> size_t

Changes from v6:
- fixed few checkpatch style issues
- detect libssh 0.8 via symbol detection
- adjust travis/docker test material
- remove dead "default" case in a switch
- use variables for storing MIN() results
- adapt a documentation bit

Changes from v5:
- adapt to newer tracing APIs
- disable ssh compression (mimic what libssh2 does by default)
- use build time checks for libssh 0.8, and use newer APIs directly

Changes from v4:
- fix wrong usages of error_setg/session_error_setg/sftp_error_setg
- fix few return code checks
- remove now-unused parameters in few internal functions
- allow authentication with "none" method
- switch to unsigned int for the port number
- enable TCP_NODELAY on the socket
- fix one reference error message in iotest 207

Changes from v3:
- fix socket cleanup in connect_to_ssh()
- add comments about the socket cleanup
- improve the error reporting (closer to what was with libssh2)
- improve EOF detection on sftp_read()

Changes from v2:
- used again an own fd
- fixed co_yield() implementation

Changes from v1:
- fixed jumbo packets writing
- fixed missing 'err' assignment
- fixed commit message

 .travis.yml                                   |   4 +-
 block/Makefile.objs                           |   6 +-
 block/ssh.c                                   | 665 ++++++++++--------
 block/trace-events                            |  14 +-
 configure                                     |  65 +-
 docs/qemu-block-drivers.texi                  |   2 +-
 .../dockerfiles/debian-win32-cross.docker     |   1 -
 .../dockerfiles/debian-win64-cross.docker     |   1 -
 tests/docker/dockerfiles/fedora.docker        |   4 +-
 tests/docker/dockerfiles/ubuntu.docker        |   2 +-
 tests/docker/dockerfiles/ubuntu1804.docker    |   2 +-
 tests/qemu-iotests/207                        |   4 +-
 tests/qemu-iotests/207.out                    |   2 +-
 13 files changed, 423 insertions(+), 349 deletions(-)

[...]

Quoted 4 lines
block/ssh.c +0 -0
index 6da7b9cbfe..644ae8b82c 100644

[...]

Quoted 14 lines
block/ssh.c +14 -0
+    case SSH_SERVER_KNOWN_CHANGED:
+        ret = -EINVAL;
+        r = ssh_get_publickey(s->session, &pubkey);
+        if (r == 0) {
+            r = ssh_get_publickey_hash(pubkey, SSH_PUBLICKEY_HASH_SHA1,
+                                       &server_hash, &server_hash_len);
+            pubkey_type = ssh_key_type(pubkey);
+            ssh_key_free(pubkey);
+        }
+        if (r == 0) {
+            fingerprint = ssh_get_fingerprint_hash(SSH_PUBLICKEY_HASH_SHA1,
+                                                   server_hash,
+                                                   server_hash_len);
+            ssh_clean_pubkey_hash(&server_hash);

block/ssh.c

Quoted 10 lines
block/ssh.c +10 -0
+        }
+        if (fingerprint) {
+            error_setg(errp,
+                       "host key (%s key with fingerprint %s) does not match "
+                       "the one in known_hosts",
+                       ssh_key_type_to_char(pubkey_type), fingerprint);
+            ssh_string_free_char(fingerprint);
+        } else  {
+            error_setg(errp, "host key does not match the one in known_hosts");
+        }

Usually I’d say that more information can’t be bad. But here I don’t see how this additional information is useful. known_hosts contains base64-encoded full public keys. This only prints the SHA-1 digest. The user cannot add that to known_hosts, or easily scan known_hosts to see whether it perhaps belongs to some other hosts (maybe because DHCP scrambled something).

Actually, even if this printed the base64 representation of the full key, the user probably wouldn’t just add that to known_hosts or do anything with it. They’ll debug the problem with other tools.

So I don’t think this new information is really useful...?

(Hmm, I don’t know if this is a response to my “But the specification requires a warning!!1!”, but if so, I was actually not referring to more information but to this:

$ ssh 192.168.0.12
(quoted diff) +0 -0
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @

(quoted diff) +0 -0
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)! It is also possible that a host key has just been changed.

Quoted 3 lines

I mean, I was also only half-serious. I should be serious because the libssh specification requires us to print some warning like that, but, well. Yes, I’ll stop mumbling about this stuff now.)

Quoted 5 lines

Those giant messages are targetted at humans though so I'm not so convinced it is useful for apps managing QEMU. IMHO it is sufficient to just report a simple error that the host ident check failed as the patch does now. It gives enough info to then investigate further to identify the root cause problems.

I fully agree that that message is absolutely unsuitable for qemu. I didn’t state that, which is my mistake, sorry.

The point I wanted to make is that I’m citing something that doesn’t print any data, but just an attack warning. I’d be content with just a single sentence to that effect (and please not in caps lock), like “It is possible that someone is attempting a man-in-the-middle attack.”

Max