pkt-line: Fix sparse errors and warnings
PATCH 1/1 pkt-line: Fix sparse errors and warnings
Sparse issues the following error and warnings:
pkt-line.c:209:51: warning: Using plain integer as NULL pointer sideband.c:41:52: warning: Using plain integer as NULL pointer daemon.c:615:39: warning: Using plain integer as NULL pointer remote-curl.c:220:75: error: incompatible types for operation (>)
remote-curl.c:220:75: left side has type char *
remote-curl.c:220:75: right side has type int
remote-curl.c:291:53: warning: Using plain integer as NULL pointer remote-curl.c:408:43: warning: Using plain integer as NULL pointer remote-curl.c:562:47: warning: Using plain integer as NULL pointer
All of these complaints "blame" to commit 17243606 ("pkt-line: share buffer/descriptor reading implementation", 20-02-2013).
In order to suppress the warnings, we simply replace the integer constant 0 with NULL.
In order to suppress the error message, we simply remove the "> 0" from the while loop controlling expression.
---
Hi Jeff,
When you next re-roll your 'jk/pkt-line-cleanup' patches, could you please squash this (or something like it) into commit 17243606 ("pkt-line: share buffer/descriptor reading implementation", 20-02-2013).
Please check the resolution of the sparse error, remote-curl.c:220, since I didn't think too deeply about it, making some assumptions ... ;-)
Note: the commit message mentions an strbuf_get_line() function, but that is supposed to be packet_get_line(), right?
In addition, that commit adds the following code as part of function get_packet_data():
+ /* Read up to "size" bytes from our source, whatever it is. */
+ if (src_buf && *src_buf) {
+ ret = size < *src_size ? size : *src_size; + memcpy(dst, *src_buf, ret); + *src_buf += size;
............................^^^^^
+ *src_size -= size;
+ } else {
+
This could lead to the source buffer pointer being incremented past the "one past the end" of the buffer; ie to undefined behaviour. That use of 'size', along with the one on the following line, should be 'ret' no?
ATB,
Ramsay Jones
daemon.c | 2 +- pkt-line.c | 2 +- remote-curl.c | 8 ++++---- sideband.c | 2 +- 4 files changed, 7 insertions(+), 7 deletions(-)
daemon.c +1 -1
@@ -612,7 +612,7 @@ static int execute(void)loginfo ("Connection from %s:%s" , addr, port);alarm (init_timeout ? init_timeout : timeout); - pktlen =packet_read (0 ,NULL ,0 , packet_buffer,sizeof (packet_buffer),0 ); + pktlen =packet_read (0 ,NULL ,NULL , packet_buffer,sizeof (packet_buffer),0 );alarm (0 ); len =strlen (line);
pkt-line.c +1 -1
@@ -206,7 +206,7 @@ static char *packet_read_line_generic(int fd,char *packet_read_line (int fd,int *len_p) { -return packet_read_line_generic (fd,NULL ,0 , len_p); +return packet_read_line_generic (fd,NULL ,NULL , len_p); }char *packet_read_line_buf (char **src,size_t *src_len,int *dst_len)
remote-curl.c +4 -4
@@ -217,7 +217,7 @@ static struct discovery* discover_refs(const char *service, int for_push)* until a packet flush marker. Ignore these now, but * in the future we might start to scan them. */ -while (packet_read_line_buf (&last->buf, &last->len,NULL ) >0 ) +while (packet_read_line_buf (&last->buf, &last->len,NULL )) ; last->proto_git =1 ;
@@ -288,7 +288,7 @@ static size_t rpc_out(void *ptr, size_t eltsize,if (!avail) { rpc->initial_buffer =0 ; - avail =packet_read (rpc->out,NULL ,0 , rpc->buf, rpc->alloc,0 ); + avail =packet_read (rpc->out,NULL ,NULL , rpc->buf, rpc->alloc,0 );if (!avail)return 0 ; rpc->pos =0 ;
@@ -405,7 +405,7 @@ static int post_rpc(struct rpc_state *rpc)break ; } - n =packet_read (rpc->out,0 ,NULL , buf, left,0 ); + n =packet_read (rpc->out,NULL ,NULL , buf, left,0 );if (!n)break ; rpc->len += n;
@@ -559,7 +559,7 @@ static int rpc_service(struct rpc_state *rpc, struct discovery *heads) rpc->hdr_accept =strbuf_detach (&buf,NULL );while (!err) { -int n =packet_read (rpc->out,0 ,NULL , rpc->buf, rpc->alloc,0 ); +int n =packet_read (rpc->out,NULL ,NULL , rpc->buf, rpc->alloc,0 );if (!n)break ; rpc->pos =0 ;
sideband.c +1 -1
@@ -38,7 +38,7 @@ int recv_sideband(const char *me, int in_stream, int out)while (1 ) {int band, len; - len =packet_read (in_stream,NULL ,0 , buf + pf,LARGE_PACKET_MAX ,0 ); + len =packet_read (in_stream,NULL ,NULL , buf + pf,LARGE_PACKET_MAX ,0 );if (len ==0 )break ;if (len <1 ) {
Signature · 1 line
1.8.1
Quoted 12 lines · Ramsay Jones
Sparse issues the following error and warnings:
pkt-line.c:209:51: warning: Using plain integer as NULL pointer sideband.c:41:52: warning: Using plain integer as NULL pointer daemon.c:615:39: warning: Using plain integer as NULL pointer remote-curl.c:220:75: error: incompatible types for operation (>)
remote-curl.c:220:75: left side has type char *
remote-curl.c:220:75: right side has type int
remote-curl.c:291:53: warning: Using plain integer as NULL pointer remote-curl.c:408:43: warning: Using plain integer as NULL pointer remote-curl.c:562:47: warning: Using plain integer as NULL pointer
Quoted 5 lines
All of these complaints "blame" to commit 17243606 ("pkt-line: share buffer/descriptor reading implementation", 20-02-2013).
In order to suppress the warnings, we simply replace the integer constant 0 with NULL.
Thanks for catching. This was just a think-o on my part; we want to pass (NULL, 0) as the (buf, len) pair, but of course we are passing pointers to them, so it is actually (NULL, NULL). And yes, I made sure the function correctly handles both a NULL pointer-to-buf and a NULL pointer-to-pointer-to-buf (and we do not even care about the pointer-to-len unless the buf pointer is valid).
So no bug, but definitely a reasonable cleanup.
Oddly, you seemed to miss the one in connect.c (which my sparse does detect).
Quoted 2 lines
In order to suppress the error message, we simply remove the "> 0" from the while loop controlling expression.
Yeah, that is the right thing to do. The code that's there isn't wrong, but I agree that checking "ptr != NULL" (i.e., just "ptr") is much more sane than "ptr > 0" (and is just a leftover that I missed during refactoring the patch).
Quoted 3 lines
When you next re-roll your 'jk/pkt-line-cleanup' patches, could you please squash this (or something like it) into commit 17243606 ("pkt-line: share buffer/descriptor reading implementation", 20-02-2013).
I don't think we otherwise need a re-roll. Junio hasn't merged the series to next yet, so I've included an updated patch 15 below with your changes (your patch, the missing connect.c fix, the commit message fix, and the size/ret thing below).
Quoted 13 lines
In addition, that commit adds the following code as part of function
get_packet_data():(quoted diff) +5
-0+ /* Read up to "size" bytes from our source, whatever it is. */ + if (src_buf && *src_buf) { + ret = size < *src_size ? size : *src_size; + memcpy(dst, *src_buf, ret); + *src_buf += size;............................^^^^^
(quoted diff) +3
-0+ *src_size -= size; + } else { +
Quoted 3 lines
This could lead to the source buffer pointer being incremented past the "one past the end" of the buffer; ie to undefined behaviour. That use of 'size', along with the one on the following line, should be 'ret' no?
Yeah, thanks for catching. We just die immediately afterwards anyway, and sane systems do not care what kind of junk you put into a pointer unless you dereference. But clearly it is supposed to be "ret".
Thanks for the report. Clearly I should start running "make sparse" more often (the reason I don't is that it produces dozens of complaints about constants in /usr/include/bits/xopen_lim.h, which I could obviously care less about; I should look into suppressing that).
Updated patch 15/19 is below.
-- >8 --
Subject: [PATCH] pkt-line: share buffer/descriptor reading implementation
The packet_read function reads from a descriptor. The packet_get_line function is similar, but reads from an in-memory buffer, and uses a completely separate implementation. This patch teaches the generic packet_read function to accept either source, and we can do away with packet_get_line's implementation.
There are two other differences to account for between the old and new functions. The first is that we used to read into a strbuf, but now read into a fixed size buffer. The only two callers are fine with that, and in fact it simplifies their code, since they can use the same static-buffer interface as the rest of the packet_read_line callers (and we provide a similar convenience wrapper for reading from a buffer rather than a descriptor).
This is technically an externally-visible behavior change in that we used to accept arbitrary sized packets up to 65532 bytes, and now cap out at LARGE_PACKET_MAX, 65520. In practice this doesn't matter, as we use it only for parsing smart-http headers (of which there is exactly one defined, and it is small and fixed-size). And any extension headers would be breaking the protocol to go over LARGE_PACKET_MAX anyway.
The other difference is that packet_get_line would return on error rather than dying. However, both callers of packet_get_line are actually improved by dying.
The first caller does its own error checking, but we can drop that; as a result, we'll actually get more specific reporting about protocol breakage when packet_read dies internally. The only downside is that packet_read will not print the smart-http URL that failed, but that's not a big deal; anybody not debugging can already see the remote's URL already, and anybody debugging would want to run with GIT_CURL_VERBOSE anyway to see way more information.
The second caller, which is just trying to skip past any extra smart-http headers (of which there are none defined, but which we allow to keep room for future expansion), did not error check at all. As a result, it would treat an error just like a flush packet. The resulting mess would generally cause an error later in get_remote_heads, but now we get error reporting much closer to the source of the problem.
--- connect.c | 3 ++- daemon.c | 2 +- pkt-line.c | 76 +++++++++++++++++++++++++++++------------------------------ pkt-line.h | 23 +++++++++++++----- remote-curl.c | 22 ++++++++--------- sideband.c | 2 +- 6 files changed, 69 insertions(+), 59 deletions(-)
connect.c +2 -1
@@ -76,7 +76,8 @@ struct ref **get_remote_heads(int in, struct ref **list,int len, name_len;char *buffer = packet_buffer; - len =packet_read (in, packet_buffer,sizeof (packet_buffer), + len =packet_read (in,NULL ,NULL , + packet_buffer,sizeof (packet_buffer),PACKET_READ_GENTLE_ON_EOF |PACKET_READ_CHOMP_NEWLINE );if (len <0 )
daemon.c +1 -1
@@ -612,7 +612,7 @@ static int execute(void)loginfo ("Connection from %s:%s" , addr, port);alarm (init_timeout ? init_timeout : timeout); - pktlen =packet_read (0 , packet_buffer,sizeof (packet_buffer),0 ); + pktlen =packet_read (0 ,NULL ,NULL , packet_buffer,sizeof (packet_buffer),0 );alarm (0 ); len =strlen (line);
pkt-line.c +38 -38
110 lines not on this page · raw message
pkt-line.h +17 -6
@@ -25,9 +25,16 @@ void packet_buf_write(struct strbuf *buf, const char *fmt, ...) __attribute__((fvoid packet_buf_write (struct strbuf *buf,const char *fmt, ...) __attribute__((format (printf,2 ,3 )));/* -* Read a packetized line from the descriptor into the buffer, which must be at -* least size bytes long. The return value specifies the number of bytes read -* into the buffer. +* Read a packetized line into the buffer, which must be at least size bytes +* long. The return value specifies the number of bytes read into the buffer. +* +* If src_buffer is not NULL (and nor is *src_buffer), it should point to a +* buffer containing the packet data to parse, of at least *src_len bytes. +* After the function returns, src_buf will be incremented and src_len +* decremented by the number of bytes consumed. +* +* If src_buffer (or *src_buffer) is NULL, then data is read from the +* descriptor "fd". * * If options does not contain PACKET_READ_GENTLE_ON_EOF, we will die under any * of the following conditions:
@@ -50,7 +57,8 @@ void packet_buf_write(struct strbuf *buf, const char *fmt, ...) __attribute__((f*/ #define PACKET_READ_GENTLE_ON_EOF (1u<<0)#define PACKET_READ_CHOMP_NEWLINE (1u<<1) -int packet_read (int fd,char *buffer,unsigned size,int options); +int packet_read (int fd,char **src_buffer,size_t *src_len,char + *buffer,unsigned size,int options);/* * Convenience wrapper for packet_read that is not gentle, and sets the
@@ -61,11 +69,14 @@ extern char packet_buffer[LARGE_PACKET_MAX];*/ char *packet_read_line (int fd,int *size); +/* +* Same as packet_read_line, but read from a buf rather than a descriptor; +* see packet_read for details on how src_* is used. +*/ +char *packet_read_line_buf (char **src_buf,size_t *src_len,int *size);#define DEFAULT_PACKET_MAX 1000#define LARGE_PACKET_MAX 65520extern char packet_buffer[LARGE_PACKET_MAX ]; -int packet_get_line (struct strbuf *out,char **src_buf,size_t *src_len); -#endif
remote-curl.c +10 -12
@@ -138,28 +138,26 @@ static struct discovery* discover_refs(const char *service)if (maybe_smart && (5 <= last->len && last->buf[4 ] =='#' ) && !strbuf_cmp (&exp, &type)) { +char *line; +/* * smart HTTP response; validate that the service * pkt-line matches our request. */ -if (packet_get_line (&buffer, &last->buf, &last->len) <=0 ) -die ("%s has invalid packet header" , refs_url); -if (buffer.len && buffer.buf[buffer.len -1 ] =='\n' ) -strbuf_setlen (&buffer, buffer.len -1 ); + line =packet_read_line_buf (&last->buf, &last->len,NULL );strbuf_reset (&exp);strbuf_addf (&exp,"# service=%s" , service); -if (strbuf_cmp (&exp, &buffer)) -die ("invalid server response; got '%s'" , buffer.buf); +if (strcmp (line, exp.buf)) +die ("invalid server response; got '%s'" , line);strbuf_release (&exp);/* The header can include additional metadata lines, up * until a packet flush marker. Ignore these now, but * in the future we might start to scan them. */ -strbuf_reset (&buffer); -while (packet_get_line (&buffer, &last->buf, &last->len) >0 ) -strbuf_reset (&buffer); +while (packet_read_line_buf (&last->buf, &last->len,NULL )) + ; last->proto_git =1 ; }
@@ -308,7 +306,7 @@ static size_t rpc_out(void *ptr, size_t eltsize,if (!avail) { rpc->initial_buffer =0 ; - avail =packet_read (rpc->out, rpc->buf, rpc->alloc,0 ); + avail =packet_read (rpc->out,NULL ,NULL , rpc->buf, rpc->alloc,0 );if (!avail)return 0 ; rpc->pos =0 ;
@@ -425,7 +423,7 @@ static int post_rpc(struct rpc_state *rpc)break ; } - n =packet_read (rpc->out, buf, left,0 ); + n =packet_read (rpc->out,NULL ,NULL , buf, left,0 );if (!n)break ; rpc->len += n;
@@ -579,7 +577,7 @@ static int rpc_service(struct rpc_state *rpc, struct discovery *heads) rpc->hdr_accept =strbuf_detach (&buf,NULL );while (!err) { -int n =packet_read (rpc->out, rpc->buf, rpc->alloc,0 ); +int n =packet_read (rpc->out,NULL ,NULL , rpc->buf, rpc->alloc,0 );if (!n)break ; rpc->pos =0 ;
sideband.c +1 -1
@@ -38,7 +38,7 @@ int recv_sideband(const char *me, int in_stream, int out)while (1 ) {int band, len; - len =packet_read (in_stream, buf + pf,LARGE_PACKET_MAX ,0 ); + len =packet_read (in_stream,NULL ,NULL , buf + pf,LARGE_PACKET_MAX ,0 );if (len ==0 )break ;if (len <1 ) {
Signature · 1 line
1.8.1.4.4.g265d2fa
Quoted 14 lines · Jeff King
Quoted 12 lines · Ramsay Jones
Sparse issues the following error and warnings:
pkt-line.c:209:51: warning: Using plain integer as NULL pointer sideband.c:41:52: warning: Using plain integer as NULL pointer daemon.c:615:39: warning: Using plain integer as NULL pointer remote-curl.c:220:75: error: incompatible types for operation (>)
remote-curl.c:220:75: left side has type char *
remote-curl.c:220:75: right side has type int
remote-curl.c:291:53: warning: Using plain integer as NULL pointer remote-curl.c:408:43: warning: Using plain integer as NULL pointer remote-curl.c:562:47: warning: Using plain integer as NULL pointer
Quoted 8 lines
Quoted 5 lines
All of these complaints "blame" to commit 17243606 ("pkt-line: share buffer/descriptor reading implementation", 20-02-2013).
In order to suppress the warnings, we simply replace the integer constant 0 with NULL.
[...]
Oddly, you seemed to miss the one inconnect.c(which my sparse does detect).
Ah, I saw why as soon as I finished off the rebase: that (NULL, 0) goes away in the very next patch, and you probably ran sparse just on the tip of the topic (via pu). I still think it's worth fixing since we are squashing anyway. Junio, it will give you a trivial conflict on patch 16, but you can just resolve in favor of what patch 16 does. If it's easier, here's the revised patch 16:
-- >8 --
Subject: [PATCH] teach get_remote_heads to read from a memory buffer
Now that we can read packet data from memory as easily as a descriptor, get_remote_heads can take either one as a source. This will allow further refactoring in remote-curl.
--- builtin/fetch-pack.c | 2 +- builtin/send-pack.c | 2 +- cache.h | 4 +++- connect.c | 6 +++--- remote-curl.c | 2 +- transport.c | 6 +++--- 6 files changed, 12 insertions(+), 10 deletions(-)
builtin/fetch-pack.c +1 -1
@@ -125,7 +125,7 @@ int cmd_fetch_pack(int argc, const char **argv, const char *prefix) args.verbose ? CONNECT_VERBOSE : 0); } -get_remote_heads (fd[0 ], &ref,0 ,NULL ); +get_remote_heads (fd[0 ],NULL ,0 , &ref,0 ,NULL ); ref =fetch_pack (&args, fd, conn, ref, dest, &sought, pack_lockfile_ptr);
builtin/send-pack.c +1 -1
@@ -207,7 +207,7 @@ int cmd_send_pack(int argc, const char **argv, const char *prefix)memset (&extra_have,0 ,sizeof (extra_have)); -get_remote_heads (fd[0 ], &remote_refs,REF_NORMAL , &extra_have); +get_remote_heads (fd[0 ],NULL ,0 , &remote_refs,REF_NORMAL , &extra_have);transport_verify_remote_names (nr_refspecs, refspecs);
cache.h +3 -1
@@ -1049,7 +1049,9 @@ struct extra_have_objects {int nr, alloc;unsigned char (*array)[20 ]; }; -extern struct ref **get_remote_heads (int in,struct ref **list,unsigned int flags,struct extra_have_objects *); +extern struct ref **get_remote_heads (int in,char *src_buf,size_t src_len, +struct ref **list,unsigned int flags, +struct extra_have_objects *);extern int server_supports (const char *feature);extern int parse_feature_request (const char *features,const char *feature);extern const char *server_feature_value (const char *feature,int *len_ret);
connect.c +3 -3
@@ -62,8 +62,8 @@ static void die_initial_contact(int got_at_least_one_head)/* * Read all the refs from the other end */ -struct ref **get_remote_heads (int in,struct ref **list, -unsigned int flags, +struct ref **get_remote_heads (int in,char *src_buf,size_t src_len, +struct ref **list,unsigned int flags,struct extra_have_objects *extra_have) {int got_at_least_one_head =0 ;
@@ -76,7 +76,7 @@ struct ref **get_remote_heads(int in, struct ref **list,int len, name_len;char *buffer = packet_buffer; - len =packet_read (in,NULL ,NULL , + len =packet_read (in, &src_buf, &src_len, packet_buffer,sizeof (packet_buffer),PACKET_READ_GENTLE_ON_EOF |PACKET_READ_CHOMP_NEWLINE );
remote-curl.c +1 -1
@@ -192,7 +192,7 @@ static struct ref *parse_git_refs(struct discovery *heads, int for_push)if (start_async (&async))die ("cannot start thread to parse advertised refs" ); -get_remote_heads (async.out, &list, +get_remote_heads (async.out,NULL ,0 , &list, for_push ?REF_NORMAL :0 ,NULL );close (async.out);if (finish_async (&async))
transport.c +3 -3
@@ -507,7 +507,7 @@ static struct ref *get_refs_via_connect(struct transport *transport, int for_pusstruct ref *refs;connect_setup (transport, for_push,0 ); -get_remote_heads (data->fd[0 ], &refs, +get_remote_heads (data->fd[0 ],NULL ,0 , &refs, for_push ?REF_NORMAL :0 , &data->extra_have); data->got_remote_heads =1 ;
@@ -541,7 +541,7 @@ static int fetch_refs_via_pack(struct transport *transport,if (!data->got_remote_heads) {connect_setup (transport,0 ,0 ); -get_remote_heads (data->fd[0 ], &refs_tmp,0 ,NULL ); +get_remote_heads (data->fd[0 ],NULL ,0 , &refs_tmp,0 ,NULL ); data->got_remote_heads =1 ; }
@@ -799,7 +799,7 @@ static int git_transport_push(struct transport *transport, struct ref *remote_restruct ref *tmp_refs;connect_setup (transport,1 ,0 ); -get_remote_heads (data->fd[0 ], &tmp_refs,REF_NORMAL ,NULL ); +get_remote_heads (data->fd[0 ],NULL ,0 , &tmp_refs,REF_NORMAL ,NULL ); data->got_remote_heads =1 ; }
Signature · 1 line
1.8.1.4.4.g265d2fa
Quoted 5 lines · Jeff King
Quoted 2 lines
Oddly, you seemed to miss the one in
connect.c(which my sparse does detect).Ah, I saw why as soon as I finished off the rebase: that (NULL, 0) goes away in the very next patch,...
Yeah, I noticed that myself while replacing 15. The patch in the message I am responding to seems to match byte-for-byte with the result of my rebase, too.
Thanks.
Quoted 21 lines · Jeff King
Quoted 19 lines · Jeff King
Quoted 17 lines · Ramsay Jones
Sparse issues the following error and warnings:
pkt-line.c:209:51: warning: Using plain integer as NULL pointer sideband.c:41:52: warning: Using plain integer as NULL pointer daemon.c:615:39: warning: Using plain integer as NULL pointer remote-curl.c:220:75: error: incompatible types for operation (>)
remote-curl.c:220:75: left side has type char *
remote-curl.c:220:75: right side has type int
remote-curl.c:291:53: warning: Using plain integer as NULL pointer remote-curl.c:408:43: warning: Using plain integer as NULL pointer remote-curl.c:562:47: warning: Using plain integer as NULL pointerAll of these complaints "blame" to commit 17243606 ("pkt-line: share buffer/descriptor reading implementation", 20-02-2013).
In order to suppress the warnings, we simply replace the integer constant 0 with NULL.
Quoted 7 lines
Quoted 3 lines
[...]
Oddly, you seemed to miss the one inconnect.c(which my sparse does detect).Ah, I saw why as soon as I finished off the rebase: that (NULL, 0) goes away in the very next patch, and you probably ran sparse just on the tip of the topic (via pu).
Yes, sorry I should have mentioned that! Ahem, *blush* [Having created and tested the patch (including running "make test") on the tip of pu, I applied the patch directly to commit 17243606 (using git-am). Since it applied cleanly and git-show looked OK, I said to myself "yep, that's OK, send it" ;-) ]
Quoted 4 lines
I still think it's worth fixing since we are
squashing anyway. Junio, it will give you a trivial conflict on patch 16, but you can just resolve in favor of what patch 16 does. If it's easier, here's the revised patch 16:
So, sorry for being a bit sloppy guys!
ATB,
Ramsay Jones
Quoted 4 lines · Jeff King
Thanks for the report. Clearly I should start running "make sparse" more often (the reason I don't is that it produces dozens of complaints about constants in
/usr/include/bits/xopen_lim.h, which I could obviously care less about; I should look into suppressing that).
Hmm, I'm guessing you are on an 64bit system; if so, and you are running the release version of sparse (v0.4.4 Nov 2011), I would clone the sparse repo [1] and build from source. There have been many improvements to the 64bit support since the last release (I hear it was unusable). [Note: I can't confirm that - I'm currently stuck on 32bits!]
ATB,
Ramsay Jones
[1] git://git.kernel.org/pub/scm/devel/sparse/sparse.git (which had been dormant for quite some time, but sprang to life again last week).