[PATCH] upload-pack.c: fix a sparse warning

[PATCH] upload-pack.c: fix a sparse warning

[Ramsay Jones]

Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
---

Hi Josh,

If you need to re-roll your 'js/trace2-session-id' branch, could you please
squash this into the relevant patch (commit edb21a501e, "upload-pack, serve:
log received client trace2 SID", 29-10-2020).

Thanks!

ATB,
Ramsay Jones
 
 upload-pack.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/upload-pack.c b/upload-pack.c
index b7d097bf1b..2996b8083c 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -1111,7 +1111,7 @@ static void receive_needs(struct upload_pack_data *data,
 		if (data->allow_filter &&
 		    parse_feature_request(features, "filter"))
 			data->filter_capability_requested = 1;
-		if ((arg = parse_feature_value(features, "trace2-sid", &feature_len, 0)))
+		if ((arg = parse_feature_value(features, "trace2-sid", &feature_len, NULL)))
 		{
 			char *client_sid = xstrndup(arg, feature_len);
 			trace2_data_string("trace2", NULL, "client-sid", client_sid);
-- 
2.29.0

[PATCH] Makefile: enable -Wsparse-error for DEVELOPER build

[Junio C Hamano]

With -Wsparse-error, "make sparse" would fail, instead of just
giving a warning message.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
 config.mak.dev | 1 +
 1 file changed, 1 insertion(+)

diff --git c/config.mak.dev w/config.mak.dev
index 3126a5364d..022fb58218 100644
--- c/config.mak.dev
+++ w/config.mak.dev
@@ -1,5 +1,6 @@
 ifeq ($(filter no-error,$(DEVOPTS)),)
 DEVELOPER_CFLAGS += -Werror
+SPARSE_FLAGS += -Wsparse-error
 endif
 ifneq ($(filter pedantic,$(DEVOPTS)),)
 DEVELOPER_CFLAGS += -pedantic

Re: [PATCH] Makefile: enable -Wsparse-error for DEVELOPER build

[Ramsay Jones]

Hi Junio,

On 31/10/2020 22:22, Junio C Hamano wrote:
> With -Wsparse-error, "make sparse" would fail, instead of just
> giving a warning message.
> 
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
>  config.mak.dev | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git c/config.mak.dev w/config.mak.dev
> index 3126a5364d..022fb58218 100644
> --- c/config.mak.dev
> +++ w/config.mak.dev
> @@ -1,5 +1,6 @@
>  ifeq ($(filter no-error,$(DEVOPTS)),)
>  DEVELOPER_CFLAGS += -Werror
> +SPARSE_FLAGS += -Wsparse-error
>  endif
>  ifneq ($(filter pedantic,$(DEVOPTS)),)
>  DEVELOPER_CFLAGS += -pedantic
> 

I certainly wouldn't object to such a patch, but I'm not sure who it
would actually help. ;-)

As you may already know, I 'make sparse >sp-out 2>&1' on the master
branch, use vim to check for warnings/errors, and then diff the
corresponding files for 'next' and 'seen' branches (nsp-out, ssp-out).
Similarly, I 'make -k hdr-check >hcout 2>&1' on the master branch,
use vim ... etc. Note, however, the use of '-k' on the make invocation
to make sure I catch all warnings/errors, since '-Werror' is given with
the DEVELOPER variable set. So, I would have to do the same with the
'make sparse' invocation after this patch [1].

Yes, anybody who just does 'make sparse' will notice the failure, so
that would be a definite improvement. (How many people run 'make sparse'
though?). It probably would help anybody using 'cgcc' to develop, but
this doesn't quite work (I just tried this for the first time in ages
and was surprised to find it almost works for me, YMMV):

  $ git branch -v | grep '^\*'
  * seen         4141ae2199 Merge branch 'dd/upload-pack-stateless-eof' into seen
  $ make clean >/dev/null
  $ 

  $ make CC=cgcc >sout2 2>&1
  $ ./git version
  git version 2.29.2.389.g4141ae2199
  $ git describe
  v2.29.2-389-g4141ae2199
  $ 

  $ grep 'warn' sout2
  pack-revindex.c:66:23: warning: memset with byte count of 262144
  upload-pack.c:1114:86: warning: Using plain integer as NULL pointer
  $ 

So, this would have failed, because some extra flags ('SP_EXTRA_FLAGS' to
be precise) are not being passed to sparse. We can see that:

  $ grep SP_EXTRA_FLAGS Makefile
  SP_EXTRA_FLAGS = -Wno-universal-initializer
  http-push.sp http.sp http-walker.sp remote-curl.sp imap-send.sp: SP_EXTRA_FLAGS += \
  pack-revindex.sp: SP_EXTRA_FLAGS += -Wno-memcpy-max-count
  compat/nedmalloc/nedmalloc.sp: SP_EXTRA_FLAGS += -Wno-non-pointer-null
  		$(SPARSE_FLAGS) $(SP_EXTRA_FLAGS) $<
  $ 

(Note that you can't see -DCURL_DISABLE_TYPECHECK being set for all the
libcurl using files). So, either sparse has improved to the point that
the 'typecheck' shenanigans used in the libcurl headers don't cause it
problems anymore, or the headers have changed sufficiently. (Hmm, that
header doesn't seem to have changed much).

I don't build with nedmalloc, but last time I did it was littered with
uses of '0' as a NULL pointer (hence the -Wno-non-pointer-null). Given
that we will probably not re-import that code, I suppose all of those
sparse warnings could be fixed.

Which leaves 'pack-revindex.c'. Unfortunately, I don't see an easy fix
for that (-Wno-universal-initializer is now the default, so we don't
need that anymore). Well, I suppose anybody could add 'CFLAGS+=-Wno-\
memcpy-max-count' to their config.mak file (this '-W' argument is
filtered out by cgcc and not passed to gcc). [EDIT: I just tried this
and, yes, this does work! I'm still surprised about curl headers ;-) ]

Note also that my build slowed down by 14% using CC=cgcc, since it is
running both sparse and gcc on each source file.

I guess it would be most useful on a CI build, but I don't know what
would be involved in setting that up.

ATB,
Ramsay Jones

[1] It took about one month for me to get used to the 'pu'->'seen' change,
    but that was mostly training my fingers to type all of the output
    files that are keyed by the branch name: pout->sout, psp-out->ssp-out,
    psc->ssc, phcout->shcout and ptest-out->stest-out. :D

Re: [PATCH] Makefile: enable -Wsparse-error for DEVELOPER build

[Junio C Hamano]

Ramsay Jones <ramsay@ramsayjones.plus.com> writes:

> Yes, anybody who just does 'make sparse' will notice the failure, so
> that would be a definite improvement. (How many people run 'make sparse'
> though?).

At least, I was planning to add one to the number when I wrote the
patch, so that I would start using it as part of daily integration
build before pushing day's result out.

I can tweak my $(MAKE) command line arguments in the script I use
for daily integration builds, so the patch is not strictly needed
for me personally.

> I guess it would be most useful on a CI build, but I don't know what
> would be involved in setting that up.

CI builds already have enough stuff around invocation of "make test"
etc., and it would be trivial to pass SPARSE_FLAGS from the command
line when adding "make sparse" invocation to one of the scripts in
ci/ directory, so from that point of view, this patch is not needed
for them, either.

My hope was that it would make it harder for individual contributors
who run "make sparse" and fail to notice an error from it.

Re: [PATCH] upload-pack.c: fix a sparse warning

[Josh Steadmon]

On 2020.10.31 21:04, Ramsay Jones wrote:
> 
> Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
> ---
> 
> Hi Josh,
> 
> If you need to re-roll your 'js/trace2-session-id' branch, could you please
> squash this into the relevant patch (commit edb21a501e, "upload-pack, serve:
> log received client trace2 SID", 29-10-2020).
> 
> Thanks!
> 
> ATB,
> Ramsay Jones

Fixed in V2, thanks!

>  upload-pack.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/upload-pack.c b/upload-pack.c
> index b7d097bf1b..2996b8083c 100644
> --- a/upload-pack.c
> +++ b/upload-pack.c
> @@ -1111,7 +1111,7 @@ static void receive_needs(struct upload_pack_data *data,
>  		if (data->allow_filter &&
>  		    parse_feature_request(features, "filter"))
>  			data->filter_capability_requested = 1;
> -		if ((arg = parse_feature_value(features, "trace2-sid", &feature_len, 0)))
> +		if ((arg = parse_feature_value(features, "trace2-sid", &feature_len, NULL)))
>  		{
>  			char *client_sid = xstrndup(arg, feature_len);
>  			trace2_data_string("trace2", NULL, "client-sid", client_sid);
> -- 
> 2.29.0

Re: [PATCH] Makefile: enable -Wsparse-error for DEVELOPER build

[Ramsay Jones]

On 02/11/2020 18:55, Junio C Hamano wrote:
> Ramsay Jones <ramsay@ramsayjones.plus.com> writes:
> 
>> Yes, anybody who just does 'make sparse' will notice the failure, so
>> that would be a definite improvement. (How many people run 'make sparse'
>> though?).
> 
> At least, I was planning to add one to the number when I wrote the
> patch, so that I would start using it as part of daily integration
> build before pushing day's result out.

Great! :-D

How are you installing sparse? (I am using Linux Mint 20, based on
Ubuntu 20.04, whose sparse package version is 0.6.1-2build1 - which
is to say too old; you require version 0.6.2 or greater. Building
from source, I am currently using v0.6.3-76-gf680124b).

[I seem to remember you saying you used Fedora, which usually has
more up-to-date software than most distros].

>> I guess it would be most useful on a CI build, but I don't know what
>> would be involved in setting that up.
> 
> CI builds already have enough stuff around invocation of "make test"
> etc., and it would be trivial to pass SPARSE_FLAGS from the command
> line when adding "make sparse" invocation to one of the scripts in
> ci/ directory, so from that point of view, this patch is not needed
> for them, either.

My concern was more about how the CI system obtains/installs/builds a
sufficiently new version of sparse. Otherwise, 'make sparse' won't do
very much. ;-)  As I said, I don't know what's involved in getting
that to work.

ATB,
Ramsay Jones

Re: [PATCH] Makefile: enable -Wsparse-error for DEVELOPER build

[Junio C Hamano]

Ramsay Jones <ramsay@ramsayjones.plus.com> writes:

> How are you installing sparse? (I am using Linux Mint 20, based on
> Ubuntu 20.04, whose sparse package version is 0.6.1-2build1 - which
> is to say too old; you require version 0.6.2 or greater. Building
> from source, I am currently using v0.6.3-76-gf680124b).

Built and installed in ~/$arch/gitstuff/bin which is on my $PATH

[remote "origin"]
	url = git://git.kernel.org/pub/scm/devel/sparse/sparse.git

> My concern was more about how the CI system obtains/installs/builds a
> sufficiently new version of sparse. Otherwise, 'make sparse' won't do
> very much. ;-)  As I said, I don't know what's involved in getting
> that to work.

Ah, yes, it would be a disaster to trigger false positive by using
old version like 0.6.1, so we may end up having to bring our own
version in, which is more work X-<.

Thanks.

Re: [PATCH] Makefile: enable -Wsparse-error for DEVELOPER build

[Johannes Schindelin]

Hi Ramsay,

On Tue, 3 Nov 2020, Ramsay Jones wrote:

> On 02/11/2020 18:55, Junio C Hamano wrote:
>
> > CI builds already have enough stuff around invocation of "make test"
> > etc., and it would be trivial to pass SPARSE_FLAGS from the command
> > line when adding "make sparse" invocation to one of the scripts in ci/
> > directory, so from that point of view, this patch is not needed for
> > them, either.
>
> My concern was more about how the CI system obtains/installs/builds a
> sufficiently new version of sparse. Otherwise, 'make sparse' won't do
> very much. ;-)  As I said, I don't know what's involved in getting that
> to work.

As mentioned in https://github.com/gitgitgadget/git/issues/345, there is a
Pipeline that builds sparse packages for Ubuntu, ready for use in our CI
build: https://dev.azure.com/git/git/_build?definitionId=10&_a=summary.
Currently, it is scheduled to run every weekday at 5:00am (I assume that's
UTC).

So I went ahead and added more details to the ticket, essentially
demonstrating how the `sparse` package can be downloaded and installed in
the CI build in a Bash step:

-- snip --
urlbase=https://dev.azure.com/git/git/_apis/build/builds
id=$(curl "$urlbase?definitions=10&statusFilter=completed&resultFilter=succeeded&\$top=1" |
	json_pp |
	sed -n 's/^         "id" : \([1-9][0-9]*\).*/\1/p')
download_url="$(curl "$urlbase/$id/artifacts" |
	json_pp |
	sed -n '/^      {/{:1;N;/\n      }/b2;b1;:2;/"name" : "sparse"/{s/.*"downloadUrl" : "\([^"]*\).*/\1/p}}')"
curl -Lo sparse.zip "$download_url"
unzip sparse.zip
sudo dpkg -i sparse/*.deb
-- snap --

It is quite likely that the ugly and unwieldy `json_pp | sed` things could
be replaced by elegant `jq` calls.

Ciao,
Dscho

Re: [PATCH] Makefile: enable -Wsparse-error for DEVELOPER build

[Ramsay Jones]

On 04/11/2020 13:43, Johannes Schindelin wrote:
> Hi Ramsay,
> 
> On Tue, 3 Nov 2020, Ramsay Jones wrote:
> 
>> On 02/11/2020 18:55, Junio C Hamano wrote:
>>
>>> CI builds already have enough stuff around invocation of "make test"
>>> etc., and it would be trivial to pass SPARSE_FLAGS from the command
>>> line when adding "make sparse" invocation to one of the scripts in ci/
>>> directory, so from that point of view, this patch is not needed for
>>> them, either.
>>
>> My concern was more about how the CI system obtains/installs/builds a
>> sufficiently new version of sparse. Otherwise, 'make sparse' won't do
>> very much. ;-)  As I said, I don't know what's involved in getting that
>> to work.
> 
> As mentioned in https://github.com/gitgitgadget/git/issues/345, there is a
> Pipeline that builds sparse packages for Ubuntu, ready for use in our CI
> build: https://dev.azure.com/git/git/_build?definitionId=10&_a=summary.
> Currently, it is scheduled to run every weekday at 5:00am (I assume that's
> UTC).

Oh-my! Has there been a CI 'make sparse' build since september last year? :-D
I missed that! (I couldn't view the azure build summary given above - it just
keeps asking for a user/password :( ).

ATB,
Ramsay Jones

Re: [PATCH] Makefile: enable -Wsparse-error for DEVELOPER build

[Junio C Hamano]

Ramsay Jones <ramsay@ramsayjones.plus.com> writes:

> On 04/11/2020 13:43, Johannes Schindelin wrote:
>> Hi Ramsay,
>> 
>> On Tue, 3 Nov 2020, Ramsay Jones wrote:
>> 
>>> On 02/11/2020 18:55, Junio C Hamano wrote:
>>>
>>>> CI builds already have enough stuff around invocation of "make test"
>>>> etc., and it would be trivial to pass SPARSE_FLAGS from the command
>>>> line when adding "make sparse" invocation to one of the scripts in ci/
>>>> directory, so from that point of view, this patch is not needed for
>>>> them, either.
>>>
>>> My concern was more about how the CI system obtains/installs/builds a
>>> sufficiently new version of sparse. Otherwise, 'make sparse' won't do
>>> very much. ;-)  As I said, I don't know what's involved in getting that
>>> to work.
>> 
>> As mentioned in https://github.com/gitgitgadget/git/issues/345, there is a
>> Pipeline that builds sparse packages for Ubuntu, ready for use in our CI
>> build: https://dev.azure.com/git/git/_build?definitionId=10&_a=summary.
>> Currently, it is scheduled to run every weekday at 5:00am (I assume that's
>> UTC).
>
> Oh-my! Has there been a CI 'make sparse' build since september last year? :-D
> I missed that! (I couldn't view the azure build summary given above - it just
> keeps asking for a user/password :( ).

Hmph, is that what Dscho said?

I thought he was just saying packaged versions of latest sparse
usable on Ubuntu are regularly available so those who wants to add
"make sparse" to our jobs have a place to fetch it from.

Re: [PATCH] Makefile: enable -Wsparse-error for DEVELOPER build

[Ramsay Jones]

On 04/11/2020 18:11, Junio C Hamano wrote:
> Ramsay Jones <ramsay@ramsayjones.plus.com> writes:
> 
>> On 04/11/2020 13:43, Johannes Schindelin wrote:
>>> Hi Ramsay,
>>>
>>> On Tue, 3 Nov 2020, Ramsay Jones wrote:
>>>
>>>> On 02/11/2020 18:55, Junio C Hamano wrote:
>>>>
>>>>> CI builds already have enough stuff around invocation of "make test"
>>>>> etc., and it would be trivial to pass SPARSE_FLAGS from the command
>>>>> line when adding "make sparse" invocation to one of the scripts in ci/
>>>>> directory, so from that point of view, this patch is not needed for
>>>>> them, either.
>>>>
>>>> My concern was more about how the CI system obtains/installs/builds a
>>>> sufficiently new version of sparse. Otherwise, 'make sparse' won't do
>>>> very much. ;-)  As I said, I don't know what's involved in getting that
>>>> to work.
>>>
>>> As mentioned in https://github.com/gitgitgadget/git/issues/345, there is a
>>> Pipeline that builds sparse packages for Ubuntu, ready for use in our CI
>>> build: https://dev.azure.com/git/git/_build?definitionId=10&_a=summary.
>>> Currently, it is scheduled to run every weekday at 5:00am (I assume that's
>>> UTC).
>>
>> Oh-my! Has there been a CI 'make sparse' build since september last year? :-D
>> I missed that! (I couldn't view the azure build summary given above - it just
>> keeps asking for a user/password :( ).
> 
> Hmph, is that what Dscho said?
> 
> I thought he was just saying packaged versions of latest sparse
> usable on Ubuntu are regularly available so those who wants to add
> "make sparse" to our jobs have a place to fetch it from.

Ah, yes. Having read that again, I think your interpretation is correct.

Ahem. My mistake. Still, it's good to know that a suitable version of
sparse is available should anyone need it.

Thanks!

ATB,
Ramsay Jones