Skip to contentSkip to conversation

xc_inflate_buffer fix

PATCH In review David Lively 2 participants 3 messages Mar 15, 2006 – Mar 17, 2006

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 xc_inflate_buffer fix

Hi -
I notice some of my initrds are failing to gunzip:
ERROR: inflate failed, sts -5
These same initrds work fine with Ben Thomas' original
patch introducing xc_{linux,hvm}_build_mem. The checked-in version changed some declarations from unsigned char * to char *, breaking the output length computation in xc_inflate_buffer.
This patch assumes you want to keep using the signed char *
for the interface, so it simply inserts a few casts to unsigned. Tested lightly, but the bug is pretty obvious once you notice the signedness change ...

Dave

tools/libxc/xg_private.c +5 -4
diff -r 1d741a415927 tools/libxc/xg_private.c
@@ -77,10 +77,11 @@ char *xc_inflate_buffer(const char *in_b
         return (char *)in_buf;
     }
 
-    out_len = in_buf[in_size-4] +
-        (256 * (in_buf[in_size-3] +
-                (256 * (in_buf[in_size-2] +
-                        (256 * in_buf[in_size-1])))));
+    out_len = (unsigned char)in_buf[in_size-4] +
+        (256 * ((unsigned char)in_buf[in_size-3] +
+                (256 * ((unsigned char)in_buf[in_size-2] +
+                        (256 * (unsigned char)in_buf[in_size-1])))));
+
     bzero(&zStream, sizeof(zStream));
     out_buf = malloc(out_len + 16);        /* Leave a little extra space */
     if ( out_buf == NULL )

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

#1 lore raw

Oops, forgot:

Dave

Quoted 36 lines · David Lively

Hi -
I notice some of my initrds are failing to gunzip:
ERROR: inflate failed, sts -5
These same initrds work fine with Ben Thomas' original
patch introducing xc_{linux,hvm}_build_mem. The checked-in version changed some declarations from unsigned char * to char *, breaking the output length computation in xc_inflate_buffer.
This patch assumes you want to keep using the signed char *
for the interface, so it simply inserts a few casts to unsigned. Tested lightly, but the bug is pretty obvious once you notice the signedness change ...

Dave

------------------------------------------------------------------------

diff -r 1d741a415927 tools/libxc/xg_private.c

tools/libxc/xg_private.c +5 -4
@@ -77,10 +77,11 @@ char *xc_inflate_buffer(const char *in_b
         return (char *)in_buf;
     }
 
-    out_len = in_buf[in_size-4] +
-        (256 * (in_buf[in_size-3] +
-                (256 * (in_buf[in_size-2] +
-                        (256 * in_buf[in_size-1])))));
+    out_len = (unsigned char)in_buf[in_size-4] +
+        (256 * ((unsigned char)in_buf[in_size-3] +
+                (256 * ((unsigned char)in_buf[in_size-2] +
+                        (256 * (unsigned char)in_buf[in_size-1])))));
+
     bzero(&zStream, sizeof(zStream));
     out_buf = malloc(out_len + 16);        /* Leave a little extra space */
     if ( out_buf == NULL )
#2 lore raw
In reply to David Lively
Quoted 10 lines · David Lively

I notice some of my initrds are failing to gunzip:
ERROR: inflate failed, sts -5
These same initrds work fine with Ben Thomas' original
patch introducing xc_{linux,hvm}_build_mem. The checked-in version changed some declarations from unsigned char * to char *, breaking the output length computation in xc_inflate_buffer.
This patch assumes you want to keep using the signed char *
for the interface, so it simply inserts a few casts to unsigned. Tested lightly, but the bug is pretty obvious once you notice the signedness change ...

Yep, my bad. Thanks.

-- Keir