Skip to contentSkip to conversation

upload-pack.c: fix a sparse warning

PATCH In review Ramsay Jones 4 participants 11 messages Oct 31, 2020 – Nov 4, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 upload-pack.c: fix a sparse warning

---

Hi Josh,

If you need to re-roll your 'js/trace2-session-id' branch, could you please squash this into the relevant patch (commit edb21a501e, "upload-pack, serve:
log received client trace2 SID", 29-10-2020).

Thanks!

ATB,
Ramsay Jones

 upload-pack.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
upload-pack.c +1 -1
index b7d097bf1b..2996b8083c 100644
@@ -1111,7 +1111,7 @@ static void receive_needs(struct upload_pack_data *data,
 		if (data->allow_filter &&
 		    parse_feature_request(features, "filter"))
 			data->filter_capability_requested = 1;
-		if ((arg = parse_feature_value(features, "trace2-sid", &feature_len, 0)))
+		if ((arg = parse_feature_value(features, "trace2-sid", &feature_len, NULL)))
 		{
 			char *client_sid = xstrndup(arg, feature_len);
 			trace2_data_string("trace2", NULL, "client-sid", client_sid);
Signature · 1 line
2.29.0
#1 lore raw

PATCH 1/1 Makefile: enable -Wsparse-error for DEVELOPER build

With -Wsparse-error, "make sparse" would fail, instead of just giving a warning message.

---
 config.mak.dev | 1 +
 1 file changed, 1 insertion(+)
config.mak.dev +1 -0
index 3126a5364d..022fb58218 100644
@@ -1,5 +1,6 @@
 ifeq ($(filter no-error,$(DEVOPTS)),)
 DEVELOPER_CFLAGS += -Werror
+SPARSE_FLAGS += -Wsparse-error
 endif
 ifneq ($(filter pedantic,$(DEVOPTS)),)
 DEVELOPER_CFLAGS += -pedantic
#2 lore raw

Re: [PATCH] Makefile: enable -Wsparse-error for DEVELOPER build

Hi Junio,

Quoted 14 lines · Junio C Hamano

With -Wsparse-error, "make sparse" would fail, instead of just giving a warning message.

---
 config.mak.dev | 1 +
 1 file changed, 1 insertion(+)
config.mak.dev +0 -0
index 3126a5364d..022fb58218 100644

config.mak.dev line 6

Quoted 6 lines
config.mak.dev +1 -0
@@ -1,5 +1,6 @@
 ifeq ($(filter no-error,$(DEVOPTS)),)
 DEVELOPER_CFLAGS += -Werror
+SPARSE_FLAGS += -Wsparse-error
 endif
 ifneq ($(filter pedantic,$(DEVOPTS)),)
 DEVELOPER_CFLAGS += -pedantic

I certainly wouldn't object to such a patch, but I'm not sure who it would actually help. ;-)

As you may already know, I 'make sparse >sp-out 2>&1' on the master branch, use vim to check for warnings/errors, and then diff the corresponding files for 'next' and 'seen' branches (nsp-out, ssp-out). Similarly, I 'make -k hdr-check >hcout 2>&1' on the master branch, use vim ... etc. Note, however, the use of '-k' on the make invocation to make sure I catch all warnings/errors, since '-Werror' is given with the DEVELOPER variable set. So, I would have to do the same with the 'make sparse' invocation after this patch [1].

Yes, anybody who just does 'make sparse' will notice the failure, so that would be a definite improvement. (How many people run 'make sparse' though?). It probably would help anybody using 'cgcc' to develop, but this doesn't quite work (I just tried this for the first time in ages and was surprised to find it almost works for me, YMMV):

  $ git branch -v | grep '^\*'
  * seen         4141ae2199 Merge branch 'dd/upload-pack-stateless-eof' into seen
  $ make clean >/dev/null
  $ 
  $ make CC=cgcc >sout2 2>&1
  $ ./git version
  git version 2.29.2.389.g4141ae2199
  $ git describe
  v2.29.2-389-g4141ae2199
  $ 
  $ grep 'warn' sout2
  pack-revindex.c:66:23: warning: memset with byte count of 262144
  upload-pack.c:1114:86: warning: Using plain integer as NULL pointer
  $ 

So, this would have failed, because some extra flags ('SP_EXTRA_FLAGS' to be precise) are not being passed to sparse. We can see that:

  $ grep SP_EXTRA_FLAGS Makefile
  SP_EXTRA_FLAGS = -Wno-universal-initializer
  http-push.sp http.sp http-walker.sp remote-curl.sp imap-send.sp: SP_EXTRA_FLAGS += \
  pack-revindex.sp: SP_EXTRA_FLAGS += -Wno-memcpy-max-count
  compat/nedmalloc/nedmalloc.sp: SP_EXTRA_FLAGS += -Wno-non-pointer-null
  		$(SPARSE_FLAGS) $(SP_EXTRA_FLAGS) $<
  $ 

(Note that you can't see -DCURL_DISABLE_TYPECHECK being set for all the libcurl using files). So, either sparse has improved to the point that the 'typecheck' shenanigans used in the libcurl headers don't cause it problems anymore, or the headers have changed sufficiently. (Hmm, that header doesn't seem to have changed much).

I don't build with nedmalloc, but last time I did it was littered with uses of '0' as a NULL pointer (hence the -Wno-non-pointer-null). Given that we will probably not re-import that code, I suppose all of those sparse warnings could be fixed.

Which leaves 'pack-revindex.c'. Unfortunately, I don't see an easy fix for that (-Wno-universal-initializer is now the default, so we don't need that anymore). Well, I suppose anybody could add 'CFLAGS+=-Wno-\
memcpy-max-count' to their config.mak file (this '-W' argument is filtered out by cgcc and not passed to gcc). [EDIT: I just tried this and, yes, this does work! I'm still surprised about curl headers ;-) ]

Note also that my build slowed down by 14% using CC=cgcc, since it is running both sparse and gcc on each source file.

I guess it would be most useful on a CI build, but I don't know what would be involved in setting that up.

ATB,
Ramsay Jones

[1] It took about one month for me to get used to the 'pu'->'seen' change,

    but that was mostly training my fingers to type all of the output
    files that are keyed by the branch name: pout->sout, psp-out->ssp-out,
    psc->ssc, phcout->shcout and ptest-out->stest-out. :D
#3 lore raw

Re: [PATCH] Makefile: enable -Wsparse-error for DEVELOPER build

Quoted 3 lines · Ramsay Jones

Yes, anybody who just does 'make sparse' will notice the failure, so that would be a definite improvement. (How many people run 'make sparse' though?).

At least, I was planning to add one to the number when I wrote the patch, so that I would start using it as part of daily integration build before pushing day's result out.

I can tweak my $(MAKE) command line arguments in the script I use for daily integration builds, so the patch is not strictly needed for me personally.

Quoted 2 lines

I guess it would be most useful on a CI build, but I don't know what would be involved in setting that up.

CI builds already have enough stuff around invocation of "make test" etc., and it would be trivial to pass SPARSE_FLAGS from the command line when adding "make sparse" invocation to one of the scripts in ci/ directory, so from that point of view, this patch is not needed for them, either.

My hope was that it would make it harder for individual contributors who run "make sparse" and fail to notice an error from it.

#4 lore raw

Re: [PATCH] Makefile: enable -Wsparse-error for DEVELOPER build

Quoted 9 lines · Junio C Hamano
Quoted 3 lines · Ramsay Jones

Yes, anybody who just does 'make sparse' will notice the failure, so that would be a definite improvement. (How many people run 'make sparse' though?).

At least, I was planning to add one to the number when I wrote the patch, so that I would start using it as part of daily integration build before pushing day's result out.

Great! :-D

How are you installing sparse? (I am using Linux Mint 20, based on Ubuntu 20.04, whose sparse package version is 0.6.1-2build1 - which is to say too old; you require version 0.6.2 or greater. Building from source, I am currently using v0.6.3-76-gf680124b).

[I seem to remember you saying you used Fedora, which usually has more up-to-date software than most distros].

Quoted 8 lines
Quoted 2 lines

I guess it would be most useful on a CI build, but I don't know what would be involved in setting that up.

CI builds already have enough stuff around invocation of "make test" etc., and it would be trivial to pass SPARSE_FLAGS from the command line when adding "make sparse" invocation to one of the scripts in ci/ directory, so from that point of view, this patch is not needed for them, either.

My concern was more about how the CI system obtains/installs/builds a sufficiently new version of sparse. Otherwise, 'make sparse' won't do very much. ;-) As I said, I don't know what's involved in getting that to work.

ATB,
Ramsay Jones

#5 lore raw

Re: [PATCH] Makefile: enable -Wsparse-error for DEVELOPER build

Quoted 4 lines · Ramsay Jones

How are you installing sparse? (I am using Linux Mint 20, based on Ubuntu 20.04, whose sparse package version is 0.6.1-2build1 - which is to say too old; you require version 0.6.2 or greater. Building from source, I am currently using v0.6.3-76-gf680124b).

Built and installed in ~/$arch/gitstuff/bin which is on my $PATH

[remote "origin"]
url = git://git.kernel.org/pub/scm/devel/sparse/sparse.git

Quoted 4 lines

My concern was more about how the CI system obtains/installs/builds a sufficiently new version of sparse. Otherwise, 'make sparse' won't do very much. ;-) As I said, I don't know what's involved in getting that to work.

Ah, yes, it would be a disaster to trigger false positive by using old version like 0.6.1, so we may end up having to bring our own version in, which is more work X-<.

Thanks.

#6 lore raw

Re: [PATCH] Makefile: enable -Wsparse-error for DEVELOPER build

In reply to Ramsay Jones

Hi Ramsay,

Quoted 8 lines · Ramsay Jones
Quoted 5 lines · Junio C Hamano

CI builds already have enough stuff around invocation of "make test" etc., and it would be trivial to pass SPARSE_FLAGS from the command line when adding "make sparse" invocation to one of the scripts in ci/ directory, so from that point of view, this patch is not needed for them, either.

Quoted 4 lines

My concern was more about how the CI system obtains/installs/builds a sufficiently new version of sparse. Otherwise, 'make sparse' won't do very much. ;-) As I said, I don't know what's involved in getting that to work.

As mentioned in https://github.com/gitgitgadget/git/issues/345, there is a Pipeline that builds sparse packages for Ubuntu, ready for use in our CI
build: https://dev.azure.com/git/git/_build?definitionId=10&_a=summary.
Currently, it is scheduled to run every weekday at 5:00am (I assume that's UTC).

So I went ahead and added more details to the ticket, essentially demonstrating how the sparse package can be downloaded and installed in the CI build in a Bash step:

-- snip --
urlbase=https://dev.azure.com/git/git/_apis/build/builds
id=$(curl "$urlbase?definitions=10&statusFilter=completed&resultFilter=succeeded&\$top=1" |

	json_pp |
	sed -n 's/^         "id" : \([1-9][0-9]*\).*/\1/p')
download_url="$(curl "$urlbase/$id/artifacts" |
	json_pp |
	sed -n '/^      {/{:1;N;/\n      }/b2;b1;:2;/"name" : "sparse"/{s/.*"downloadUrl" : "\([^"]*\).*/\1/p}}')"

curl -Lo sparse.zip "$download_url"
unzip sparse.zip
sudo dpkg -i sparse/*.deb
-- snap --

It is quite likely that the ugly and unwieldy json_pp | sed things could be replaced by elegant jq calls.

Ciao,
Dscho

#7 lore raw

Re: [PATCH] Makefile: enable -Wsparse-error for DEVELOPER build

Quoted 17 lines · Johannes Schindelin

Hi Ramsay,

Quoted 8 lines · Ramsay Jones
Quoted 5 lines · Junio C Hamano

CI builds already have enough stuff around invocation of "make test" etc., and it would be trivial to pass SPARSE_FLAGS from the command line when adding "make sparse" invocation to one of the scripts in ci/ directory, so from that point of view, this patch is not needed for them, either.

Quoted 4 lines

My concern was more about how the CI system obtains/installs/builds a sufficiently new version of sparse. Otherwise, 'make sparse' won't do very much. ;-) As I said, I don't know what's involved in getting that to work.

Quoted 5 lines

As mentioned in https://github.com/gitgitgadget/git/issues/345, there is a Pipeline that builds sparse packages for Ubuntu, ready for use in our CI
build: https://dev.azure.com/git/git/_build?definitionId=10&_a=summary.
Currently, it is scheduled to run every weekday at 5:00am (I assume that's UTC).

Oh-my! Has there been a CI 'make sparse' build since september last year? :-D I missed that! (I couldn't view the azure build summary given above - it just keeps asking for a user/password :( ).

ATB,
Ramsay Jones

#8 lore raw

Re: [PATCH] Makefile: enable -Wsparse-error for DEVELOPER build

Quoted 24 lines · Ramsay Jones
Quoted 17 lines · Johannes Schindelin

Hi Ramsay,

Quoted 8 lines · Ramsay Jones
Quoted 5 lines · Junio C Hamano

CI builds already have enough stuff around invocation of "make test" etc., and it would be trivial to pass SPARSE_FLAGS from the command line when adding "make sparse" invocation to one of the scripts in ci/ directory, so from that point of view, this patch is not needed for them, either.

Quoted 4 lines

My concern was more about how the CI system obtains/installs/builds a sufficiently new version of sparse. Otherwise, 'make sparse' won't do very much. ;-) As I said, I don't know what's involved in getting that to work.

Quoted 5 lines

As mentioned in https://github.com/gitgitgadget/git/issues/345, there is a Pipeline that builds sparse packages for Ubuntu, ready for use in our CI
build: https://dev.azure.com/git/git/_build?definitionId=10&_a=summary.
Currently, it is scheduled to run every weekday at 5:00am (I assume that's UTC).

Quoted 3 lines

Oh-my! Has there been a CI 'make sparse' build since september last year? :-D I missed that! (I couldn't view the azure build summary given above - it just keeps asking for a user/password :( ).

Hmph, is that what Dscho said?

I thought he was just saying packaged versions of latest sparse usable on Ubuntu are regularly available so those who wants to add "make sparse" to our jobs have a place to fetch it from.

#9 lore raw

Re: [PATCH] Makefile: enable -Wsparse-error for DEVELOPER build

Quoted 30 lines · Junio C Hamano
Quoted 24 lines · Ramsay Jones
Quoted 17 lines · Johannes Schindelin

Hi Ramsay,

Quoted 8 lines · Ramsay Jones
Quoted 5 lines · Junio C Hamano

CI builds already have enough stuff around invocation of "make test" etc., and it would be trivial to pass SPARSE_FLAGS from the command line when adding "make sparse" invocation to one of the scripts in ci/ directory, so from that point of view, this patch is not needed for them, either.

Quoted 4 lines

My concern was more about how the CI system obtains/installs/builds a sufficiently new version of sparse. Otherwise, 'make sparse' won't do very much. ;-) As I said, I don't know what's involved in getting that to work.

Quoted 5 lines

As mentioned in https://github.com/gitgitgadget/git/issues/345, there is a Pipeline that builds sparse packages for Ubuntu, ready for use in our CI
build: https://dev.azure.com/git/git/_build?definitionId=10&_a=summary.
Currently, it is scheduled to run every weekday at 5:00am (I assume that's UTC).

Quoted 3 lines

Oh-my! Has there been a CI 'make sparse' build since september last year? :-D I missed that! (I couldn't view the azure build summary given above - it just keeps asking for a user/password :( ).

Quoted 5 lines

Hmph, is that what Dscho said?

I thought he was just saying packaged versions of latest sparse usable on Ubuntu are regularly available so those who wants to add "make sparse" to our jobs have a place to fetch it from.

Ah, yes. Having read that again, I think your interpretation is correct.

Ahem. My mistake. Still, it's good to know that a suitable version of sparse is available should anyone need it.

Thanks!

ATB,
Ramsay Jones

#10 lore raw
In reply to Ramsay Jones
Quoted 6 lines · Ramsay Jones

---

Hi Josh,

Quoted 8 lines

If you need to re-roll your 'js/trace2-session-id' branch, could you please squash this into the relevant patch (commit edb21a501e, "upload-pack, serve:
log received client trace2 SID", 29-10-2020).

Thanks!

ATB,
Ramsay Jones

Fixed in V2, thanks!

Quoted 18 lines
 upload-pack.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
upload-pack.c +1 -1
index b7d097bf1b..2996b8083c 100644
@@ -1111,7 +1111,7 @@ static void receive_needs(struct upload_pack_data *data,
 		if (data->allow_filter &&
 		    parse_feature_request(features, "filter"))
 			data->filter_capability_requested = 1;
-		if ((arg = parse_feature_value(features, "trace2-sid", &feature_len, 0)))
+		if ((arg = parse_feature_value(features, "trace2-sid", &feature_len, NULL)))
 		{
 			char *client_sid = xstrndup(arg, feature_len);
 			trace2_data_string("trace2", NULL, "client-sid", client_sid);
Signature · 1 line
2.29.0