Skip to contentSkip to conversation

Misc VNC fixes / sanity checks

PATCH In review Daniel P. Berrangé 3 participants 12 messages Feb 5, 2018

lore
Share & tools
lore
#0 lore raw

cover Misc VNC fixes / sanity checks

A few VNC changes suggested by Laszlo when reviewing my recent VNC patches

Daniel P. Berrangé (4):
ui: avoid risk of 32-bit int overflow in VNC buffer check
ui: avoid 'local_err' variable shadowing in VNC SASL auth
ui: check VNC audio frequency limit at time of reading from client
ui: extend VNC trottling tracing to SASL codepaths

 ui/vnc-auth-sasl.c | 13 +++++++++++--
 ui/vnc.c           | 26 +++++++++++++++-----------
 2 files changed, 26 insertions(+), 13 deletions(-)
Signature · 1 line
2.14.3
#1 lore raw

PATCH 1/4 ui: avoid risk of 32-bit int overflow in VNC buffer check

For very large framebuffers, it is theoretically possible for the result of 'vs->throttle_output_offset * VNC_THROTTLE_OUTPUT_LIMIT_SCALE' to exceed the size of a 32-bit int. For this to happen in practice, the video RAM would have to be set to a large enough value, which is not likely today. None the less we can be paranoid against future growth by using division instead of multiplication when checking the limits.

---
 ui/vnc.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
ui/vnc.c +2 -2
index 93731accb6..e14e524764 100644
@@ -1572,8 +1572,8 @@ void vnc_write(VncState *vs, const void *data, size_t len)
      * handshake, or from the job thread's VncState clone
      */
     if (vs->throttle_output_offset != 0 &&
-        vs->output.offset > (vs->throttle_output_offset *
-                             VNC_THROTTLE_OUTPUT_LIMIT_SCALE)) {
+        (vs->output.offset / VNC_THROTTLE_OUTPUT_LIMIT_SCALE) >
+        vs->throttle_output_offset) {
         trace_vnc_client_output_limit(vs, vs->ioc, vs->output.offset,
                                       vs->throttle_output_offset);
         vnc_disconnect_start(vs);
Signature · 1 line
2.14.3
Reviewed-by on v1 1/4
#2 lore raw

Re: [Qemu-devel] [PATCH 1/4] ui: avoid risk of 32-bit int overflow in VNC buffer check

Quoted 19 lines · Daniel P. Berrangé

For very large framebuffers, it is theoretically possible for the result of 'vs->throttle_output_offset * VNC_THROTTLE_OUTPUT_LIMIT_SCALE' to exceed the size of a 32-bit int. For this to happen in practice, the video RAM would have to be set to a large enough value, which is not likely today. None the less we can be paranoid against future growth by using division instead of multiplication when checking the limits.

---
 ui/vnc.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
ui/vnc.c +0 -0
index 93731accb6..e14e524764 100644

ui/vnc.c void vnc_write(VncState *vs, const void *data, size_t len) · line 1579

Quoted 10 lines
ui/vnc.c +2 -2
@@ -1572,8 +1572,8 @@ void vnc_write(VncState *vs, const void *data, size_t len)
      * handshake, or from the job thread's VncState clone
      */
     if (vs->throttle_output_offset != 0 &&
-        vs->output.offset > (vs->throttle_output_offset *
-                             VNC_THROTTLE_OUTPUT_LIMIT_SCALE)) {
+        (vs->output.offset / VNC_THROTTLE_OUTPUT_LIMIT_SCALE) >
+        vs->throttle_output_offset) {
         trace_vnc_client_output_limit(vs, vs->ioc, vs->output.offset,
                                       vs->throttle_output_offset);
         vnc_disconnect_start(vs);
Reviewed-by on v1 1/4
#3 lore raw

Re: [Qemu-devel] [PATCH 1/4] ui: avoid risk of 32-bit int overflow in VNC buffer check

In reply to Daniel P. Berrangé
Quoted 9 lines · Daniel P. Berrangé

For very large framebuffers, it is theoretically possible for the result of 'vs->throttle_output_offset * VNC_THROTTLE_OUTPUT_LIMIT_SCALE' to exceed the size of a 32-bit int. For this to happen in practice, the video RAM would have to be set to a large enough value, which is not likely today. None the less we can be paranoid against future growth by using division instead of multiplication when checking the limits.

Quoted 20 lines
---
 ui/vnc.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
ui/vnc.c +2 -2
index 93731accb6..e14e524764 100644
@@ -1572,8 +1572,8 @@ void vnc_write(VncState *vs, const void *data, size_t len)
      * handshake, or from the job thread's VncState clone
      */
     if (vs->throttle_output_offset != 0 &&
-        vs->output.offset > (vs->throttle_output_offset *
-                             VNC_THROTTLE_OUTPUT_LIMIT_SCALE)) {
+        (vs->output.offset / VNC_THROTTLE_OUTPUT_LIMIT_SCALE) >
+        vs->throttle_output_offset) {
         trace_vnc_client_output_limit(vs, vs->ioc, vs->output.offset,
                                       vs->throttle_output_offset);
         vnc_disconnect_start(vs);
#4 lore raw

PATCH 2/4 ui: avoid 'local_err' variable shadowing in VNC SASL auth

In reply to Daniel P. Berrangé

The start_auth_sasl() method declares a 'Error *local_err' variable in an inner if () {...} scope, which shadows a variable of the same name declared at the start of the method. This is confusing for reviewers and may trigger compiler warnings.

---
 ui/vnc-auth-sasl.c | 2 --
 1 file changed, 2 deletions(-)
ui/vnc-auth-sasl.c +0 -2
index fbccca8c8a..8ebd0d3d00 100644
@@ -556,7 +556,6 @@ void start_auth_sasl(VncState *vs)
     /* Inform SASL that we've got an external SSF layer from TLS/x509 */
     if (vs->auth == VNC_AUTH_VENCRYPT &&
         vs->subauth == VNC_AUTH_VENCRYPT_X509SASL) {
-        Error *local_err = NULL;
         int keysize;
         sasl_ssf_t ssf;
 
@@ -565,7 +564,6 @@ void start_auth_sasl(VncState *vs)
         if (keysize < 0) {
             trace_vnc_auth_fail(vs, vs->auth, "cannot TLS get cipher size",
                                 error_get_pretty(local_err));
-            error_free(local_err);
             sasl_dispose(&vs->sasl.conn);
             vs->sasl.conn = NULL;
             goto authabort;
Signature · 1 line
2.14.3
Reviewed-by on v1 2/4
#5 lore raw

Re: [Qemu-devel] [PATCH 2/4] ui: avoid 'local_err' variable shadowing in VNC SASL auth

Quoted 25 lines · Daniel P. Berrangé

The start_auth_sasl() method declares a 'Error *local_err' variable in an inner if () {...} scope, which shadows a variable of the same name declared at the start of the method. This is confusing for reviewers and may trigger compiler warnings.

---
 ui/vnc-auth-sasl.c | 2 --
 1 file changed, 2 deletions(-)
ui/vnc-auth-sasl.c +0 -1
index fbccca8c8a..8ebd0d3d00 100644
@@ -556,7 +556,6 @@ void start_auth_sasl(VncState *vs)
     /* Inform SASL that we've got an external SSF layer from TLS/x509 */
     if (vs->auth == VNC_AUTH_VENCRYPT &&
         vs->subauth == VNC_AUTH_VENCRYPT_X509SASL) {
-        Error *local_err = NULL;
         int keysize;
         sasl_ssf_t ssf;
 

ui/vnc-auth-sasl.c void start_auth_sasl(VncState *vs) · line 569

Quoted 7 lines
ui/vnc-auth-sasl.c +0 -1
@@ -565,7 +564,6 @@ void start_auth_sasl(VncState *vs)
         if (keysize < 0) {
             trace_vnc_auth_fail(vs, vs->auth, "cannot TLS get cipher size",
                                 error_get_pretty(local_err));
-            error_free(local_err);
             sasl_dispose(&vs->sasl.conn);
             vs->sasl.conn = NULL;
             goto authabort;
Reviewed-by on v1 2/4
#6 lore raw

Re: [Qemu-devel] [PATCH 2/4] ui: avoid 'local_err' variable shadowing in VNC SASL auth

In reply to Daniel P. Berrangé
Quoted 7 lines · Daniel P. Berrangé

The start_auth_sasl() method declares a 'Error *local_err' variable in an inner if () {...} scope, which shadows a variable of the same name declared at the start of the method. This is confusing for reviewers and may trigger compiler warnings.

Quoted 25 lines
---
 ui/vnc-auth-sasl.c | 2 --
 1 file changed, 2 deletions(-)
ui/vnc-auth-sasl.c +0 -2
index fbccca8c8a..8ebd0d3d00 100644
@@ -556,7 +556,6 @@ void start_auth_sasl(VncState *vs)
     /* Inform SASL that we've got an external SSF layer from TLS/x509 */
     if (vs->auth == VNC_AUTH_VENCRYPT &&
         vs->subauth == VNC_AUTH_VENCRYPT_X509SASL) {
-        Error *local_err = NULL;
         int keysize;
         sasl_ssf_t ssf;
 
@@ -565,7 +564,6 @@ void start_auth_sasl(VncState *vs)
         if (keysize < 0) {
             trace_vnc_auth_fail(vs, vs->auth, "cannot TLS get cipher size",
                                 error_get_pretty(local_err));
-            error_free(local_err);
             sasl_dispose(&vs->sasl.conn);
             vs->sasl.conn = NULL;
             goto authabort;
#7 lore raw

PATCH 3/4 ui: check VNC audio frequency limit at time of reading from client

In reply to Daniel P. Berrangé

The 'vs->as.freq' value is a signed integer, which is read from an unsigned 32-bit int field on the wire. There is thus a risk of overflow on 32-bit platforms. Move the frequency limit checking to be done at time of read before casting to a signed integer.

---
 ui/vnc.c | 22 +++++++++++++---------
 1 file changed, 13 insertions(+), 9 deletions(-)
ui/vnc.c +13 -9
index e14e524764..79ac9eccde 100644
@@ -982,14 +982,7 @@ static void vnc_update_throttle_offset(VncState *vs)
         vs->client_width * vs->client_height * vs->client_pf.bytes_per_pixel;
 
     if (vs->audio_cap) {
-        int freq = vs->as.freq;
-        /* We don't limit freq when reading settings from client, so
-         * it could be upto MAX_INT in size. 48khz is a sensible
-         * upper bound for trustworthy clients */
         int bps;
-        if (freq > 48000) {
-            freq = 48000;
-        }
         switch (vs->as.fmt) {
         default:
         case  AUD_FMT_U8:
@@ -1005,7 +998,7 @@ static void vnc_update_throttle_offset(VncState *vs)
             bps = 4;
             break;
         }
-        offset += freq * bps * vs->as.nchannels;
+        offset += vs->as.freq * bps * vs->as.nchannels;
     }
 
     /* Put a floor of 1MB on offset, so that if we have a large pending
@@ -2279,6 +2272,7 @@ static int protocol_client_msg(VncState *vs, uint8_t *data, size_t len)
 {
     int i;
     uint16_t limit;
+    uint32_t freq;
     VncDisplay *vd = vs->vd;
 
     if (data[0] > 3) {
@@ -2398,7 +2392,17 @@ static int protocol_client_msg(VncState *vs, uint8_t *data, size_t len)
                     vnc_client_error(vs);
                     break;
                 }
-                vs->as.freq = read_u32(data, 6);
+                freq = read_u32(data, 6);
+                /* No official limit for protocol, but 48khz is a sensible
+                 * upper bound for trustworthy clients, and this limit
+                 * protects calculations involving 'vs->as.freq' later.
+                 */
+                if (freq > 48000) {
+                    VNC_DEBUG("Invalid audio frequency %u > 48000", freq);
+                    vnc_client_error(vs);
+                    break;
+                }
+                vs->as.freq = freq;
                 break;
             default:
                 VNC_DEBUG("Invalid audio message %d\n", read_u8(data, 4));
Signature · 1 line
2.14.3
Reviewed-by on v1 3/4
#8 lore raw

Re: [Qemu-devel] [PATCH 3/4] ui: check VNC audio frequency limit at time of reading from client

Quoted 57 lines · Daniel P. Berrangé

The 'vs->as.freq' value is a signed integer, which is read from an unsigned 32-bit int field on the wire. There is thus a risk of overflow on 32-bit platforms. Move the frequency limit checking to be done at time of read before casting to a signed integer.

---
 ui/vnc.c | 22 +++++++++++++---------
 1 file changed, 13 insertions(+), 9 deletions(-)
ui/vnc.c +6 -9
index e14e524764..79ac9eccde 100644
@@ -982,14 +982,7 @@ static void vnc_update_throttle_offset(VncState *vs)
         vs->client_width * vs->client_height * vs->client_pf.bytes_per_pixel;
 
     if (vs->audio_cap) {
-        int freq = vs->as.freq;
-        /* We don't limit freq when reading settings from client, so
-         * it could be upto MAX_INT in size. 48khz is a sensible
-         * upper bound for trustworthy clients */
         int bps;
-        if (freq > 48000) {
-            freq = 48000;
-        }
         switch (vs->as.fmt) {
         default:
         case  AUD_FMT_U8:
@@ -1005,7 +998,7 @@ static void vnc_update_throttle_offset(VncState *vs)
             bps = 4;
             break;
         }
-        offset += freq * bps * vs->as.nchannels;
+        offset += vs->as.freq * bps * vs->as.nchannels;
     }
 
     /* Put a floor of 1MB on offset, so that if we have a large pending
@@ -2279,6 +2272,7 @@ static int protocol_client_msg(VncState *vs, uint8_t *data, size_t len)
 {
     int i;
     uint16_t limit;
+    uint32_t freq;
     VncDisplay *vd = vs->vd;
 
     if (data[0] > 3) {
@@ -2398,7 +2392,17 @@ static int protocol_client_msg(VncState *vs, uint8_t *data, size_t len)
                     vnc_client_error(vs);
                     break;
                 }
-                vs->as.freq = read_u32(data, 6);
+                freq = read_u32(data, 6);
+                /* No official limit for protocol, but 48khz is a sensible
+                 * upper bound for trustworthy clients, and this limit
+                 * protects calculations involving 'vs->as.freq' later.

ui/vnc.c static int protocol_client_msg(VncState *vs, uint8_t *data, size_t len) · line 2408

Quoted 10 lines
ui/vnc.c +7 -0
+                 */
+                if (freq > 48000) {
+                    VNC_DEBUG("Invalid audio frequency %u > 48000", freq);
+                    vnc_client_error(vs);
+                    break;
+                }
+                vs->as.freq = freq;
                 break;
             default:
                 VNC_DEBUG("Invalid audio message %d\n", read_u8(data, 4));
Reviewed-by on v1 3/4
#9 lore raw

Re: [Qemu-devel] [PATCH 3/4] ui: check VNC audio frequency limit at time of reading from client

In reply to Daniel P. Berrangé
Quoted 7 lines · Daniel P. Berrangé

The 'vs->as.freq' value is a signed integer, which is read from an unsigned 32-bit int field on the wire. There is thus a risk of overflow on 32-bit platforms. Move the frequency limit checking to be done at time of read before casting to a signed integer.

Quoted 60 lines
---
 ui/vnc.c | 22 +++++++++++++---------
 1 file changed, 13 insertions(+), 9 deletions(-)
ui/vnc.c +13 -9
index e14e524764..79ac9eccde 100644
@@ -982,14 +982,7 @@ static void vnc_update_throttle_offset(VncState *vs)
         vs->client_width * vs->client_height * vs->client_pf.bytes_per_pixel;
 
     if (vs->audio_cap) {
-        int freq = vs->as.freq;
-        /* We don't limit freq when reading settings from client, so
-         * it could be upto MAX_INT in size. 48khz is a sensible
-         * upper bound for trustworthy clients */
         int bps;
-        if (freq > 48000) {
-            freq = 48000;
-        }
         switch (vs->as.fmt) {
         default:
         case  AUD_FMT_U8:
@@ -1005,7 +998,7 @@ static void vnc_update_throttle_offset(VncState *vs)
             bps = 4;
             break;
         }
-        offset += freq * bps * vs->as.nchannels;
+        offset += vs->as.freq * bps * vs->as.nchannels;
     }
 
     /* Put a floor of 1MB on offset, so that if we have a large pending
@@ -2279,6 +2272,7 @@ static int protocol_client_msg(VncState *vs, uint8_t *data, size_t len)
 {
     int i;
     uint16_t limit;
+    uint32_t freq;
     VncDisplay *vd = vs->vd;
 
     if (data[0] > 3) {
@@ -2398,7 +2392,17 @@ static int protocol_client_msg(VncState *vs, uint8_t *data, size_t len)
                     vnc_client_error(vs);
                     break;
                 }
-                vs->as.freq = read_u32(data, 6);
+                freq = read_u32(data, 6);
+                /* No official limit for protocol, but 48khz is a sensible
+                 * upper bound for trustworthy clients, and this limit
+                 * protects calculations involving 'vs->as.freq' later.
+                 */
+                if (freq > 48000) {
+                    VNC_DEBUG("Invalid audio frequency %u > 48000", freq);
+                    vnc_client_error(vs);
+                    break;
+                }
+                vs->as.freq = freq;
                 break;
             default:
                 VNC_DEBUG("Invalid audio message %d\n", read_u8(data, 4));
#10 lore raw

PATCH 4/4 ui: extend VNC trottling tracing to SASL codepaths

In reply to Daniel P. Berrangé

In previous commit:

commit 6aa22a29187e1908f5db738d27c64a9efc8d0bfa
Author: Daniel P. Berrange <[email protected]>
Date: Mon Dec 18 19:12:27 2017 +0000

ui: add trace events related to VNC client throttling

trace points related to unthrottling client I/O were missed from the SASL codepaths.

---
 ui/vnc-auth-sasl.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
ui/vnc-auth-sasl.c +11 -0
index 8ebd0d3d00..3751a777a4 100644
@@ -79,12 +79,23 @@ size_t vnc_client_write_sasl(VncState *vs)
 
     vs->sasl.encodedOffset += ret;
     if (vs->sasl.encodedOffset == vs->sasl.encodedLength) {
+        bool throttled = vs->force_update_offset != 0;
+        size_t offset;
         if (vs->sasl.encodedRawLength >= vs->force_update_offset) {
             vs->force_update_offset = 0;
         } else {
             vs->force_update_offset -= vs->sasl.encodedRawLength;
         }
+        if (throttled && vs->force_update_offset == 0) {
+            trace_vnc_client_unthrottle_forced(vs, vs->ioc);
+        }
+        offset = vs->output.offset;
         buffer_advance(&vs->output, vs->sasl.encodedRawLength);
+        if (offset >= vs->throttle_output_offset &&
+            vs->output.offset < vs->throttle_output_offset) {
+            trace_vnc_client_unthrottle_incremental(vs, vs->ioc,
+                                                    vs->output.offset);
+        }
         vs->sasl.encoded = NULL;
         vs->sasl.encodedOffset = vs->sasl.encodedLength = 0;
     }
Signature · 1 line
2.14.3
Reviewed-by on v1 4/4
#11 lore raw

Re: [Qemu-devel] [PATCH 4/4] ui: extend VNC trottling tracing to SASL codepaths

Quoted 36 lines · Daniel P. Berrangé

In previous commit:

commit 6aa22a29187e1908f5db738d27c64a9efc8d0bfa
Author: Daniel P. Berrange <[email protected]>
Date: Mon Dec 18 19:12:27 2017 +0000

ui: add trace events related to VNC client throttling

trace points related to unthrottling client I/O were missed from the SASL codepaths.

---
 ui/vnc-auth-sasl.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
ui/vnc-auth-sasl.c +5 -0
index 8ebd0d3d00..3751a777a4 100644
@@ -79,12 +79,23 @@ size_t vnc_client_write_sasl(VncState *vs)
 
     vs->sasl.encodedOffset += ret;
     if (vs->sasl.encodedOffset == vs->sasl.encodedLength) {
+        bool throttled = vs->force_update_offset != 0;
+        size_t offset;
         if (vs->sasl.encodedRawLength >= vs->force_update_offset) {
             vs->force_update_offset = 0;
         } else {
             vs->force_update_offset -= vs->sasl.encodedRawLength;
         }
+        if (throttled && vs->force_update_offset == 0) {
+            trace_vnc_client_unthrottle_forced(vs, vs->ioc);
+        }

ui/vnc-auth-sasl.c size_t vnc_client_write_sasl(VncState *vs) · line 101

Quoted 10 lines
ui/vnc-auth-sasl.c +6 -0
+        offset = vs->output.offset;
         buffer_advance(&vs->output, vs->sasl.encodedRawLength);
+        if (offset >= vs->throttle_output_offset &&
+            vs->output.offset < vs->throttle_output_offset) {
+            trace_vnc_client_unthrottle_incremental(vs, vs->ioc,
+                                                    vs->output.offset);
+        }
         vs->sasl.encoded = NULL;
         vs->sasl.encodedOffset = vs->sasl.encodedLength = 0;
     }

Thank you!
Laszlo