ci: fix windows-build with GCC v12.x
cover ci: fix windows-build with GCC v12.x
A recent update of GCC in Git for Windows' SDK (a subset of which is used in Git's CI/PR builds) broke the build.
These patches address that, and they are based on maint-2.34 (earlier maintenance tracks would have a trivial merge conflict due to 013c7e2b070 (http: drop support for curl < 7.16.0, 2021-07-30) removing support for USE_CURL_MULTI).
Johannes Schindelin (4):
compat/win32/syslog: fix use-after-realloc
nedmalloc: avoid new compile error
http.c: avoid danging pointer to local variable finished dir.c: avoid "exceeds maximum object size" error with GCC v12.x
compat/nedmalloc/nedmalloc.c | 1 - compat/win32/syslog.c | 2 ++ dir.c | 9 +++++++++ http-walker.c | 4 ---- http.c | 15 +++++++-------- http.h | 2 +- 6 files changed, 19 insertions(+), 14 deletions(-)
base-commit: 2f0dde7852b7866bb044926f73334ff3fc30654b
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1238%2Fdscho%2Ffix-win-build-with-gcc-12-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1238/dscho/fix-win-build-with-gcc-12-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1238
Signature · 1 line
gitgitgadget
PATCH 1/4 compat/win32/syslog: fix use-after-realloc
From: Johannes Schindelin <[email protected]>
Git for Windows' SDK recently upgraded to GCC v12.x which points out that the pos variable might be used even after the corresponding memory was realloc()ed and therefore potentially no longer valid.
Since a subset of this SDK is used in Git's CI/PR builds, we need to fix this to continue to be able to benefit from the CI/PR runs.
Note: This bug has been with us since 2a6b149c64f6 (mingw: avoid using strbuf in syslog, 2011-10-06), and while it looks tempting to replace the hand-rolled string manipulation with a strbuf-based one, that commit's message explains why we cannot do that: The syslog() function is called as part of the function in daemon.c which is set as the die() routine, and since strbuf_grow() can call that function if it runs out of memory, this would cause a nasty infinite loop that we do not want to re-introduce.
--- compat/win32/syslog.c | 2 ++ 1 file changed, 2 insertions(+)
compat/win32/syslog.c +2 -0
@@ -43,6 +43,7 @@ void syslog(int priority, const char *fmt, ...)va_end (ap);while ((pos =strstr (str,"%1" )) !=NULL ) { +size_t offset = pos - str;char *oldstr = str; str =realloc (str,st_add (++str_len,1 ));if (!str) {
@@ -50,6 +51,7 @@ void syslog(int priority, const char *fmt, ...)warning_errno ("realloc failed" );return ; } + pos = str + offset;memmove (pos +2 , pos +1 ,strlen (pos)); pos[1 ] =' ' ; }
Signature · 1 line
gitgitgadget
Re: [PATCH 1/4] compat/win32/syslog: fix use-after-realloc
Hi,
compat/win32/syslog.c void syslog(int priority, const char *fmt, ...) · line 48
Quoted 11 lines · Johannes Schindelin via GitGitGadget
compat/win32/syslog.c +1
-0@@ -43,6 +43,7 @@ void syslog(int priority, const char *fmt, ...)va_end (ap);while ((pos =strstr (str,"%1" )) !=NULL ) { +size_t offset = pos - str;char *oldstr = str; str =realloc (str,st_add (++str_len,1 ));
Since it has been raised elsewhere: Why is that ++str_len not turned into an st_add()?
The commit adding that st_add() call (50a6c8efa2b (use st_add and st_mult for allocation size computation, 2016-02-22)) does not really talk about it, but the explanation is simple:
Before this while() loop, we allocate one more than str_len (see compat/win32/syslog.c#L35), and we do that already using st_add(), so that the string and the terminating NUL fit into the allocated memory.
Therefore, the first time we enter the loop, we know that ++str_len is safe.
Now, in this very line, we then increment str_len and then reallocate one more than that, again guarding it via st_add(). So every subsequent iteration will already have checked that ++str_len is safe, too.
By induction (https://en.wikipedia.org/wiki/Mathematical_induction), it follows that this line is safe, and we do not have to change it to a clunkier two-step assignment where we first use st_add() to increment str_len and then use st_add() to allocate enough memory to also fit the trailing NUL.
Now you know,
Dscho
Quoted 13 lines
compat/win32/syslog.c +1
-1if (!str) {@@ -50,6 +51,7 @@ void syslog(int priority, const char *fmt, ...)warning_errno ("realloc failed" );return ; } + pos = str + offset;memmove (pos +2 , pos +1 ,strlen (pos)); pos[1 ] =' ' ; } --gitgitgadget
Re: [PATCH 1/4] compat/win32/syslog: fix use-after-realloc
Quoted 6 lines · Johannes Schindelin
Quoted 1 line
str =
realloc(str, st_add(++str_len, 1));Since it has been raised elsewhere: Why is that
++str_lennot turned into anst_add()?
...
Now you know,
I'd be more worried about a macro looking thing evalutating its parameters more than once, though. But unlike st_addN(), st_add() is an inline function so we do not have to worry about that ;-)
Quoted 5 lines
compat/win32/syslog.c void syslog(int priority, const char *fmt, ...) · line 54
Quoted 5 lines
compat/win32/syslog.c +1
-0@@ -50,6 +51,7 @@ void syslog(int priority, const char *fmt, ...)warning_errno ("realloc failed" );return ; } + pos = str + offset;
The adjustment using ofs is very much straight-forward. Nicely spotted and nicely corrected.
Thanks.
PATCH 2/4 nedmalloc: avoid new compile error
In reply to Johannes Schindelin via GitGitGadgetFrom: Johannes Schindelin <[email protected]>
GCC v12.x complains thusly:
compat/nedmalloc/nedmalloc.c: In function 'DestroyCaches':compat/nedmalloc/nedmalloc.c:326:12: error: the comparison will always
evaluate as 'true' for the address of 'caches' will never be NULL [-Werror=address]
326 | if(p->caches)
| ^compat/nedmalloc/nedmalloc.c:196:22: note: 'caches' declared here
196 | threadcache *caches[THREADCACHEMAXCACHES];
| ^~~~~~
... and it is correct, of course.
--- compat/nedmalloc/nedmalloc.c | 1 - 1 file changed, 1 deletion(-)
compat/nedmalloc/nedmalloc.c +0 -1
@@ -323,7 +323,6 @@ static NOINLINE void RemoveCacheEntries(nedpool *p, threadcache *tc, unsigned in }static void DestroyCaches (nedpool *p)THROWSPEC { -if (p->caches) {threadcache *tc;int n;
Signature · 1 line
gitgitgadget
Re: [PATCH 2/4] nedmalloc: avoid new compile error
Quoted 26 lines · Johannes Schindelin via GitGitGadget
From: Johannes Schindelin <[email protected]>
GCC v12.x complains thusly:
compat/nedmalloc/nedmalloc.c: In function 'DestroyCaches':compat/nedmalloc/nedmalloc.c:326:12: error: the comparison will always
evaluate as 'true' for the address of 'caches' will never be NULL [-Werror=address]
326 |if(p->caches)
| ^compat/nedmalloc/nedmalloc.c:196:22: note: 'caches' declared here
196 | threadcache *caches[THREADCACHEMAXCACHES];
| ^~~~~~... and it is correct, of course.
Signed-off-by Johannes Schindelin <[email protected]>--- compat/nedmalloc/nedmalloc.c | 1 - 1 file changed, 1 deletion(-)compat/nedmalloc/nedmalloc.c +0
-0
compat/nedmalloc/nedmalloc.c static NOINLINE void RemoveCacheEntries(nedpool *p, threadcache *tc, unsigned in · line 328
Quoted 7 lines
compat/nedmalloc/nedmalloc.c +0
-1@@ -323,7 +323,6 @@ static NOINLINE void RemoveCacheEntries(nedpool *p, threadcache *tc, unsigned in }static void DestroyCaches (nedpool *p)THROWSPEC { -if (p->caches) {threadcache *tc;int n;
This seems sensible, I thought "why not submit it to upstream", i.e. see:
https://github.com/ned14/nedmalloc/blob/master/nedmalloc.c#L1298
But that repository was last updated in 2014, I wonder if it's just because nobody's submitted a patch since then, or if it's inactive. Have you tried making Njall Douglas (the nedmalloc author) aware of this issue?
Re: [PATCH 2/4] nedmalloc: avoid new compile error
Am 24.05.22 um 10:00 schrieb Ævar Arnfjörð Bjarmason:
Quoted 10 lines
This seems sensible, I thought "why not submit it to upstream", i.e. see:
https://github.com/ned14/nedmalloc/blob/master/nedmalloc.c#L1298But that repository was last updated in 2014, I wonder if it's just because nobody's submitted a patch since then, or if it's inactive. Have you tried making Njall Douglas (the nedmalloc author) aware of this issue?
https://github.com/ned14/nedmalloc says at the top: "This repository has been archived by the owner. It is now read-only.".
René
Re: [PATCH 2/4] nedmalloc: avoid new compile error
Hi René,
Quoted 6 lines · René Scharfe
Am 24.05.22 um 10:00 schrieb Ævar Arnfjörð Bjarmason:
Quoted 5 lines
This seems sensible, I thought "why not submit it to upstream", i.e. see:
https://github.com/ned14/nedmalloc/blob/master/nedmalloc.c#L1298
Quoted 8 lines
Quoted 5 lines
But that repository was last updated in 2014, I wonder if it's just because nobody's submitted a patch since then, or if it's inactive. Have you tried making Njall Douglas (the nedmalloc author) aware of this issue?
https://github.com/ned14/nedmalloc says at the top: "This repository has been archived by the owner. It is now read-only.".
Indeed, maintenance of nedmalloc has stopped a few years ago (see e.g.
https://github.com/ned14/nedmalloc/issues/20#issuecomment-343432314).
About five years ago I tried to upgrade us to the latest nedmalloc version but ran into a performance regression that I was unable to justify the time to investigate further.
Ciao,
Dscho
Re: [PATCH 2/4] nedmalloc: avoid new compile error
Quoted 16 lines · Johannes Schindelin
Hi René,
Quoted 12 lines · René Scharfe
Am 24.05.22 um 10:00 schrieb Ævar Arnfjörð Bjarmason:
Quoted 10 lines
This seems sensible, I thought "why not submit it to upstream", i.e. see:
https://github.com/ned14/nedmalloc/blob/master/nedmalloc.c#L1298But that repository was last updated in 2014, I wonder if it's just because nobody's submitted a patch since then, or if it's inactive. Have you tried making Njall Douglas (the nedmalloc author) aware of this issue?
Quoted 5 lines
Quoted 2 lines
https://github.com/ned14/nedmalloc says at the top: "This repository has been archived by the owner. It is now read-only.".
Indeed, maintenance of nedmalloc has stopped a few years ago (see e.g.
https://github.com/ned14/nedmalloc/issues/20#issuecomment-343432314).
The author says:
nedmalloc is pretty much EOL. Happy to accept patches, but unwilling to fix.
The "Happy to accept patches" seems to suggest that they're willing to take a PR, just not willing to do spend time on it themselves.
Anyway, I see that we've accumulated quite a few patches on top, and given...
Quoted 3 lines
About five years ago I tried to upgrade us to the latest nedmalloc version but ran into a performance regression that I was unable to justify the time to investigate further.
...perhaps it's not worth it.
Maybe someone should get to updating the readme we carry from it to change the first line from:
nedalloc v1.05 15th June 2008:
To:
Git's (perma-)fork & local hacks on top of nedalloc v1.05 15th June 2008:
Or something :)
PATCH 3/4 http.c: avoid danging pointer to local variable `finished`
In reply to Johannes Schindelin via GitGitGadgetFrom: Johannes Schindelin <[email protected]>
In http.c, the run_active_slot() function allows the given "slot" to make progress by calling step_active_slots() in a loop repeatedly, and the loop is not left until the request held in the slot completes.
Ages ago, we used to use the slot->in_use member to get out of the loop, which misbehaved when the request in "slot" completes (at which time, the result of the request is copied away from the slot, and the in_use member is cleared, making the slot ready to be reused), and the "slot" gets reused to service a different request (at which time, the "slot" becomes in_use again, even though it is for a different request). The loop terminating condition mistakenly thought that the original request has yet to be completed.
Today's code, after baa7b67d (HTTP slot reuse fixes, 2006-03-10) fixed this issue, uses a separate "slot->finished" member that is set in run_active_slot() to point to an on-stack variable, and the code that completes the request in finish_active_slot() clears the on-stack variable via the pointer to signal that the particular request held by the slot has completed. It also clears the in_use member (as before that fix), so that the slot itself can safely be reused for an unrelated request.
One thing that is not quite clean in this arrangement is that, unless the slot gets reused, at which point the finished member is reset to NULL, the member keeps the value of &finished, which becomes a dangling pointer into the stack when run_active_slot() returns.
Let's drop that local variable and introduce a new flag in the slot that is used to indicate that even while the slot is no longer in use, it is still reserved until further notice. It is the responsibility of run_active_slot() to clear that flag once it is done with that slot.
--- http-walker.c | 4 ---- http.c | 15 +++++++-------- http.h | 2 +- 3 files changed, 8 insertions(+), 13 deletions(-)
http-walker.c +0 -4
@@ -225,13 +225,9 @@ static void process_alternates_response(void *callback_data) alt_req->url->buf); active_requests++; slot->in_use =1 ; -if (slot->finished !=NULL ) - (*slot->finished) =0 ;if (!start_active_slot (slot)) { cdata->got_alternates =-1 ; slot->in_use =0 ; -if (slot->finished !=NULL ) - (*slot->finished) =1 ; }return ; }
http.c +7 -8
@@ -197,8 +197,7 @@ static void finish_active_slot(struct active_request_slot *slot)closedown_active_slot (slot);curl_easy_getinfo (slot->curl,CURLINFO_HTTP_CODE , &slot->http_code); -if (slot->finished !=NULL ) - (*slot->finished) =1 ; + slot->in_use =0 ;/* Store slot results so they can be read after the slot is reused */ if (slot->results !=NULL ) {
@@ -1176,13 +1175,14 @@ struct active_request_slot *get_active_slot(void)process_curl_messages (); } -while (slot !=NULL && slot->in_use) +while (slot !=NULL && (slot->in_use || slot->reserved_for_use)) slot = slot->next;if (slot ==NULL ) { newslot =xmalloc (sizeof (*newslot)); newslot->curl =NULL ; newslot->in_use =0 ; + newslot->reserved_for_use =0 ; newslot->next =NULL ; slot = active_queue_head;
@@ -1204,7 +1204,6 @@ struct active_request_slot *get_active_slot(void) active_requests++; slot->in_use =1 ; slot->results =NULL ; - slot->finished =NULL ; slot->callback_data =NULL ; slot->callback_func =NULL ;curl_easy_setopt (slot->curl,CURLOPT_COOKIEFILE , curl_cookie_file);
@@ -1296,7 +1295,7 @@ void fill_active_slots(void) }while (slot !=NULL ) { -if (!slot->in_use && slot->curl !=NULL +if (!slot->in_use && !slot->reserved_for_use && slot->curl && curl_session_count > min_curl_sessions) {curl_easy_cleanup (slot->curl); slot->curl =NULL ;
@@ -1327,10 +1326,9 @@ void run_active_slot(struct active_request_slot *slot)fd_set excfds;int max_fd;struct timeval select_timeout; -int finished =0 ; - slot->finished = &finished; -while (!finished) { + slot->reserved_for_use =1 ; +while (slot->in_use) {step_active_slots ();if (slot->in_use) {
@@ -1367,6 +1365,7 @@ void run_active_slot(struct active_request_slot *slot)select (max_fd+1 , &readfds, &writefds, &excfds, &select_timeout); } } + slot->reserved_for_use =0 ; }static void release_active_slot (struct active_request_slot *slot)
http.h +1 -1
@@ -22,9 +22,9 @@ struct slot_results {struct active_request_slot {CURL *curl;int in_use; +int reserved_for_use;CURLcode curl_result;long http_code; -int *finished;struct slot_results *results;void *callback_data;void (*callback_func)(void *data);
Signature · 1 line
gitgitgadget
Re: [PATCH 3/4] http.c: avoid danging pointer to local variable `finished`
Quoted 8 lines · Johannes Schindelin via GitGitGadget
From: Johannes Schindelin <[email protected]>
[...]
Let's drop that local variable and introduce a new flag in the slot that is used to indicate that even while the slot is no longer in use, it is still reserved until further notice. It is the responsibility ofrun_active_slot()to clear that flag once it is done with that slot.Initial-patch-by Junio C Hamano <[email protected]>
Don't you mean by me?
I.e. https://lore.kernel.org/git/[email protected]/
This seems to be derived from that, or perhaps you just came up with something similar independently. Junio then came up with the smaller
https://lore.kernel.org/git/[email protected]/
Re: [PATCH 3/4] http.c: avoid danging pointer to local variable `finished`
Quoted 9 lines · Ævar Arnfjörð Bjarmason
Quoted 7 lines · Johannes Schindelin via GitGitGadget
From: Johannes Schindelin <[email protected]>
[...]
Let's drop that local variable and introduce a new flag in the slot that is used to indicate that even while the slot is no longer in use, it is still reserved until further notice. It is the responsibility ofrun_active_slot()to clear that flag once it is done with that slot.
Quoted 4 lines
Quoted 1 line
Initial-patch-by Junio C Hamano <[email protected]>Don't you mean by me?
I.e. https://lore.kernel.org/git/[email protected]/
Most likely, but this version is so distant from the "clear slot->finished before leaving run_active_slot()" Dscho and I were recently discussing, that I do not think it can be said to have been derived from that one. This is completely a different patch that makes different changes.
The "clear slot->finished", by the way, is what I think is the right thing to do, especially that the objective is to squelch the false positive warning from a new compiler. If there is a way to annotate the line for the compiler to tell it not to warn about it, that would have been even better.
Quoted 3 lines
This seems to be derived from that, or perhaps you just came up with something similar independently. Junio then came up with the smaller
https://lore.kernel.org/git/[email protected]/
I actually do not think so. Yours is revert of the existing fix the compiler is confused about, and I have a feeling that if the original fix is still relevant, the problem the original fix wanted to address will resurface as a regression.
If I am reading the patch correctly, Dscho's is to avoid [*] reusing a slot while any run_active_slot() is still waiting for its completion. The approach would solve the problem the original fix wanted to solve in a different way. Personally I do not think such a surgery is necessary only to squelch false positives from a new warning compiler, though.
[Footnote]
* I said "is to avoid", not "avoids", because I haven't studied the patch with sufficient degree of carefulness to say for sure, even though I can see that is the intent.
Re: [PATCH 3/4] http.c: avoid danging pointer to local variable `finished`
Hi Junio,
Quoted 5 lines · Junio C Hamano
The "clear
slot->finished", by the way, is what I think is the right thing to do, especially that the objective is to squelch the false positive warning from a new compiler. If there is a way to annotate the line for the compiler to tell it not to warn about it, that would have been even better.
We could do something like this:
-- snip --
http.c +7 -1
@@ -1365,7 +1365,14 @@ void run_active_slot(struct active_request_slot *slot)struct timeval select_timeout;int finished =0 ; +#if __GNUC__ >=12 +#pragma GCC diagnostic push +#pragma GCC diagnostic ignored "-Wdangling-pointer" +#endif slot->finished = &finished; +#if __GNUC__ >=12 +#pragma GCC diagnostic pop +#endif while (!finished) {step_active_slots (); -- snap --
That's quite ugly, though. And what's worse, it is pretty unreadable, too.
Ciao,
Dscho
Re: [PATCH 3/4] http.c: avoid danging pointer to local variable `finished`
Quoted 33 lines · Johannes Schindelin
Hi Junio,
Quoted 5 lines · Junio C Hamano
The "clear
slot->finished", by the way, is what I think is the right thing to do, especially that the objective is to squelch the false positive warning from a new compiler. If there is a way to annotate the line for the compiler to tell it not to warn about it, that would have been even better.We could do something like this:
-- snip --
http.c +7
-1@@ -1365,7 +1365,14 @@ void run_active_slot(struct active_request_slot *slot)struct timeval select_timeout;int finished =0 ; +#if __GNUC__ >=12 +#pragma GCC diagnostic push +#pragma GCC diagnostic ignored "-Wdangling-pointer" +#endif slot->finished = &finished; +#if __GNUC__ >=12 +#pragma GCC diagnostic pop +#endif while (!finished) {step_active_slots (); -- snap --
Quoted 1 line
That's quite ugly, though. And what's worse, it is pretty unreadable, too.
Unfortunately that sort of thing is a logic error as clang, ICC and probably others are on a mission to make __GNUC__ as useless as possible:
https://stackoverflow.com/questions/38499462/how-to-tell-clang-to-stop-pretending-to-be-other-compilers
I think it *might* work in practice though, my local clang claims to be gcc 4, so maybe everyone faking it stops at a low enough version?
But did you spot 9c539d1027d (config.mak.dev: alternative workaround to gcc 12 warning in http.c, 2022-04-15)? We already disable this file-wide in config.mak.dev, but I didn't check if the Windows code was using that (or if you were targeting those without DEVELOPER=1).
We could move that to thake main Makefile, but then we'd have to call detect-compiler there. I have some local patches to do something like that if there's interest (rather, to bootstrap compilation by compiling a C object and getting the macro values, instead of relying on that shellscript).
Re: [PATCH 3/4] http.c: avoid danging pointer to local variable `finished`
In reply to Johannes SchindelinQuoted 11 lines · Johannes Schindelin
Hi Junio,
Quoted 5 lines · Junio C Hamano
The "clear
slot->finished", by the way, is what I think is the right thing to do, especially that the objective is to squelch the false positive warning from a new compiler. If there is a way to annotate the line for the compiler to tell it not to warn about it, that would have been even better.We could do something like this:
Yuck.
Quoted 21 lines
-- snip --
http.c +7
-1@@ -1365,7 +1365,14 @@ void run_active_slot(struct active_request_slot *slot)struct timeval select_timeout;int finished =0 ; +#if __GNUC__ >=12 +#pragma GCC diagnostic push +#pragma GCC diagnostic ignored "-Wdangling-pointer" +#endif slot->finished = &finished; +#if __GNUC__ >=12 +#pragma GCC diagnostic pop +#endif while (!finished) {step_active_slots (); -- snap --
Quoted 1 line
That's quite ugly, though. And what's worse, it is pretty unreadable, too.
Yes, very ugly. Would an unconditional
slot->finished = NULL;
at the end squelch the warning?
Or there is a way to say "we make all warnings into errors with -Werror, but we do not want to turn this dangling-pointer warning to an error, because it has false positives"?
Or we could add "-Wno-dangling-pointer" globally, perhaps.
Re: [PATCH 3/4] http.c: avoid danging pointer to local variable `finished`
Hi Junio,
Quoted 35 lines · Junio C Hamano
Quoted 9 lines · Johannes Schindelin
Quoted 5 lines · Junio C Hamano
The "clear
slot->finished", by the way, is what I think is the right thing to do, especially that the objective is to squelch the false positive warning from a new compiler. If there is a way to annotate the line for the compiler to tell it not to warn about it, that would have been even better.We could do something like this:
Yuck.
Quoted 21 lines
-- snip --
http.c +7
-1@@ -1365,7 +1365,14 @@ void run_active_slot(struct active_request_slot *slot)struct timeval select_timeout;int finished =0 ; +#if __GNUC__ >=12 +#pragma GCC diagnostic push +#pragma GCC diagnostic ignored "-Wdangling-pointer" +#endif slot->finished = &finished; +#if __GNUC__ >=12 +#pragma GCC diagnostic pop +#endif while (!finished) {step_active_slots (); -- snap --
Quoted 7 lines
Quoted 1 line
That's quite ugly, though. And what's worse, it is pretty unreadable, too.
Yes, very ugly. Would an unconditional
slot->finished = NULL;at the end squelch the warning?
No, unfortunately not:
https://github.com/dscho/git/actions/runs/2383492484
As you mentioned elsewhere, the error is clearly about the assignment in the first place, and it does not matter that the function will rectify the situation. It's the correct thing to do for the compiler, too: since slot->finished already has the pointer, and since the active_request_slot struct is declared globally, code outside the current file might squirrel that pointer away for later use.
Quoted 5 lines
Or there is a way to say "we make all warnings into errors with -Werror, but we do not want to turn this dangling-pointer warning to an error, because it has false positives"?
Or we could add "-Wno-dangling-pointer" globally, perhaps.
I'd like to avoid that because it would quite likely hide legitimate issues elsewhere.
It currently seems to be the easiest solution to simply turn the local variable into a heap variable:
-- snip --
http.c +6 -3
@@ -1327,10 +1327,10 @@ void run_active_slot(struct active_request_slot *slot)fd_set excfds;int max_fd;struct timeval select_timeout; -int finished =0 ; +int *finished =xcalloc (1 ,sizeof (int )); - slot->finished = &finished; -while (!finished) { + slot->finished = finished; +while (!*finished) {step_active_slots ();if (slot->in_use) {
@@ -1367,6 +1367,9 @@ void run_active_slot(struct active_request_slot *slot)select (max_fd+1 , &readfds, &writefds, &excfds, &select_timeout); } } +if (slot->finished == finished) + slot->finished =NULL ; +free (finished); }static void release_active_slot (struct active_request_slot *slot)
-- snap --
This pacifies GCC (https://github.com/dscho/git/runs/6589617700) and is the most minimally-intrusive work-around of which I am currently aware.
Ciao,
Dscho
Re: [PATCH 3/4] http.c: avoid danging pointer to local variable `finished`
Quoted 43 lines · Johannes Schindelin
Hi Junio,
Quoted 39 lines · Junio C Hamano
Quoted 9 lines · Johannes Schindelin
Quoted 5 lines · Junio C Hamano
The "clear
slot->finished", by the way, is what I think is the right thing to do, especially that the objective is to squelch the false positive warning from a new compiler. If there is a way to annotate the line for the compiler to tell it not to warn about it, that would have been even better.We could do something like this:
Yuck.
Quoted 21 lines
-- snip --
http.c +7
-1@@ -1365,7 +1365,14 @@ void run_active_slot(struct active_request_slot *slot)struct timeval select_timeout;int finished =0 ; +#if __GNUC__ >=12 +#pragma GCC diagnostic push +#pragma GCC diagnostic ignored "-Wdangling-pointer" +#endif slot->finished = &finished; +#if __GNUC__ >=12 +#pragma GCC diagnostic pop +#endif while (!finished) {step_active_slots (); -- snap --Quoted 1 line
That's quite ugly, though. And what's worse, it is pretty unreadable, too.
Yes, very ugly. Would an unconditional
Quoted 6 lines
Quoted 3 lines
slot->finished = NULL;at the end squelch the warning?
No, unfortunately not:
https://github.com/dscho/git/actions/runs/2383492484
Yes it does. Didn't you just have a PBCAK between writing that test & pushing it? Your
https://github.com/dscho/git/blob/tmp/http.c#L1370-L1371 shows that you didn't make that edit.
This on top of "seen" makes the warning go away:
- if (slot->finished == &finished)
- slot->finished = NULL;
+ slot->finished = NULL;
This is also all covered in the initial thread from back in January, which if you're slowly re-discovering the learnings from there I encourage you to read in more detail... :)
Quoted 6 lines
As you mentioned elsewhere, the error is clearly about the assignment in the first place, and it does not matter that the function will rectify the situation. It's the correct thing to do for the compiler, too: since
slot->finishedalready has the pointer, and since theactive_request_slotstruct is declared globally, code outside the current file might squirrel that pointer away for later use.
Per the above this isn't true, and in some side-thread replies (and in the initial thread) I've linked to the GCC code in question. NULL-ing the slot is sufficient, it doesn't matter that the struct it's in survives the function, just that the pointer isn't exposed.
Quoted 32 lines
Quoted 5 lines
Or there is a way to say "we make all warnings into errors with -Werror, but we do not want to turn this dangling-pointer warning to an error, because it has false positives"?
Or we could add "-Wno-dangling-pointer" globally, perhaps.
I'd like to avoid that because it would quite likely hide legitimate issues elsewhere.
It currently seems to be the easiest solution to simply turn the local variable into a heap variable:
-- snip --
http.c +3
-3@@ -1327,10 +1327,10 @@ void run_active_slot(struct active_request_slot *slot)fd_set excfds;int max_fd;struct timeval select_timeout; -int finished =0 ; +int *finished =xcalloc (1 ,sizeof (int )); - slot->finished = &finished; -while (!finished) { + slot->finished = finished; +while (!*finished) {step_active_slots ();if (slot->in_use) {
http.c void run_active_slot(struct active_request_slot *slot) · line 1373
Quoted 7 lines
http.c +3
-0@@ -1367,6 +1367,9 @@ void run_active_slot(struct active_request_slot *slot)select (max_fd+1 , &readfds, &writefds, &excfds, &select_timeout); } } +if (slot->finished == finished) + slot->finished =NULL ; +free (finished); }
Also, if we were going to tweak this extensively I'd think this slightly larger POC patch would be a better direction, i.e. we don't need a pointer at all, we're just (ab)using it for a tri-state NULL/0/1, why not just use an enum?
I think the "if it's what we just set it to" is just cargo-culting, is there anything to show that run_active_slot() is reentrant? Wouldn't we be better off with a static variable + BUG() that we increment/decremant and panic if it's anything but 0 and 1 if we'd like to add paranoia around that?
http-walker.c +4 -4
@@ -225,13 +225,13 @@ static void process_alternates_response(void *callback_data) alt_req->url->buf); active_requests++; slot->in_use =1 ; -if (slot->finished) - (*slot->finished) =0 ; +if (slot->f !=FIN_UNSET ) + slot->f =FIN_NO ;if (!start_active_slot (slot)) { cdata->got_alternates =-1 ; slot->in_use =0 ; -if (slot->finished) - (*slot->finished) =1 ; +if (slot->f !=FIN_UNSET ) + slot->f =FIN_YES ; }return ; }
http.c +5 -6
@@ -197,8 +197,8 @@ static void finish_active_slot(struct active_request_slot *slot)closedown_active_slot (slot);curl_easy_getinfo (slot->curl,CURLINFO_HTTP_CODE , &slot->http_code); -if (slot->finished) - (*slot->finished) =1 ; +if (slot->f !=FIN_UNSET ) + slot->f =FIN_YES ;/* Store slot results so they can be read after the slot is reused */ if (slot->results) {
@@ -1204,7 +1204,7 @@ struct active_request_slot *get_active_slot(void) active_requests++; slot->in_use =1 ; slot->results =NULL ; - slot->finished =NULL ; + slot->f =FIN_UNSET ; slot->callback_data =NULL ; slot->callback_func =NULL ;curl_easy_setopt (slot->curl,CURLOPT_COOKIEFILE , curl_cookie_file);
@@ -1327,10 +1327,9 @@ void run_active_slot(struct active_request_slot *slot)fd_set excfds;int max_fd;struct timeval select_timeout; -int finished =0 ; - slot->finished = &finished; -while (!finished) { + slot->f =FIN_NO ; +while (slot->f ==FIN_NO ) {step_active_slots ();if (slot->in_use) {
http.h +2 -1
@@ -19,12 +19,13 @@ struct slot_results {long http_connectcode; }; +enum fin {FIN_UNSET ,FIN_NO ,FIN_YES };struct active_request_slot {CURL *curl;int in_use;CURLcode curl_result;long http_code; -int *finished; +enum fin f;struct slot_results *results;void *callback_data;void (*callback_func)(void *data);
PATCH 4/4 dir.c: avoid "exceeds maximum object size" error with GCC v12.x
In reply to Johannes Schindelin via GitGitGadgetFrom: Johannes Schindelin <[email protected]>
Technically, the pointer difference end - start _could_ be negative, and when cast to an (unsigned) size_t that would cause problems. In this instance, the symptom is:
dir.c: In function 'git_url_basename':
dir.c:3087:13: error: 'memchr' specified bound [9223372036854775808, 0]
exceeds maximum object size 9223372036854775807
[-Werror=stringop-overread]
CC ewah/bitmap.o
3087 | if (memchr(start, '/', end - start) == NULL
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~While it is a bit far-fetched to think that end (which is defined as repo + strlen(repo)) and start (which starts at repo and never steps beyond the NUL terminator) could result in such a negative difference, GCC has no way of knowing that.
See also https://gcc.gnu.org/bugzilla//show_bug.cgi?id=85783.
Let's just add a safety check, primarily for GCC's benefit.
--- dir.c | 9 +++++++++ 1 file changed, 9 insertions(+)
dir.c +9 -0
@@ -3076,6 +3076,15 @@ char *git_url_basename(const char *repo, int is_bundle, int is_bare) end--; } +/* +* It should not be possible to overflow `ptrdiff_t` by passing in an +* insanely long URL, but GCC does not know that and will complain +* without this check. +*/ +if (end - start <0 ) +die (_ ("No directory name could be guessed.\n" +"Please specify a directory on the command line" )); +/* * Strip trailing port number if we've got only a * hostname (that is, there is no dir separator but a
Signature · 1 line
gitgitgadget
Re: [PATCH 4/4] dir.c: avoid "exceeds maximum object size" error with GCC v12.x
Quoted 33 lines · Johannes Schindelin via GitGitGadget
From: Johannes Schindelin <[email protected]>
Technically, the pointer difference
end - start_could_ be negative, and when cast to an (unsigned)size_tthat would cause problems. In this instance, the symptom is:
dir.c: In function 'git_url_basename':
dir.c:3087:13: error: 'memchr' specified bound [9223372036854775808, 0]exceeds maximum object size 9223372036854775807 [-Werror=stringop-overread] CC ewah/bitmap.o 3087 | if (memchr(start, '/', end - start) == NULL | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~While it is a bit far-fetched to think that
end(which is defined asrepo + strlen(repo)) andstart(which starts atrepoand never steps beyond the NUL terminator) could result in such a negative difference, GCC has no way of knowing that.See also https://gcc.gnu.org/bugzilla//show_bug.cgi?id=85783.
Let's just add a safety check, primarily for GCC's benefit.
Signed-off-by Johannes Schindelin <[email protected]>--- dir.c | 9 +++++++++ 1 file changed, 9 insertions(+)dir.c +0
-0
dir.c char *git_url_basename(const char *repo, int is_bundle, int is_bare) · line 3085
Quoted 10 lines
dir.c +7
-0@@ -3076,6 +3076,15 @@ char *git_url_basename(const char *repo, int is_bundle, int is_bare) end--; } +/* +* It should not be possible to overflow `ptrdiff_t` by passing in an +* insanely long URL, but GCC does not know that and will complain +* without this check. +*/ +if (end - start <0 ) + die(_("No directory name could be guessed.\n"
This should start with a lower-case letter, see CodingGuidelines.
Quoted 5 lines
dir.c +2
-0+ "Please specify a directory on the command line")); +/* * Strip trailing port number if we've got only a * hostname (that is, there is no dir separator but a
Re: [PATCH 4/4] dir.c: avoid "exceeds maximum object size" error with GCC v12.x
Hi Ævar,
Quoted 39 lines · Ævar Arnfjörð Bjarmason
Quoted 36 lines · Johannes Schindelin via GitGitGadget
From: Johannes Schindelin <[email protected]>
Technically, the pointer difference
end - start_could_ be negative, and when cast to an (unsigned)size_tthat would cause problems. In this instance, the symptom is:
dir.c: In function 'git_url_basename':
dir.c:3087:13: error: 'memchr' specified bound [9223372036854775808, 0]exceeds maximum object size 9223372036854775807 [-Werror=stringop-overread] CC ewah/bitmap.o 3087 | if (memchr(start, '/', end - start) == NULL | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~While it is a bit far-fetched to think that
end(which is defined asrepo + strlen(repo)) andstart(which starts atrepoand never steps beyond the NUL terminator) could result in such a negative difference, GCC has no way of knowing that.See also https://gcc.gnu.org/bugzilla//show_bug.cgi?id=85783.
Let's just add a safety check, primarily for GCC's benefit.
Signed-off-by Johannes Schindelin <[email protected]>--- dir.c | 9 +++++++++ 1 file changed, 9 insertions(+)dir.c +0
-0@@ -3076,6 +3076,15 @@ char *git_url_basename(const char *repo, int is_bundle, int is_bare) end--; }
Quoted 9 lines
dir.c char *git_url_basename(const char *repo, int is_bundle, int is_bare) · line 3085
Quoted 7 lines
dir.c +7
-0+/* +* It should not be possible to overflow `ptrdiff_t` by passing in an +* insanely long URL, but GCC does not know that and will complain +* without this check. +*/ +if (end - start <0 ) + die(_("No directory name could be guessed.\n"This should start with a lower-case letter, see CodingGuidelines.
This message is copied from existing code later in the same function. Since it is a translateable message, I do not want to edit it because that would cause unnecessary work of the translators. Especially given that we do not even expect this message to be shown, ever, but we only add this hunk for GCC's benefit.
Thank you,
Johannes
Quoted 8 lines
Quoted 6 lines
dir.c +2
-0+ "Please specify a directory on the command line")); +/* * Strip trailing port number if we've got only a * hostname (that is, there is no dir separator but a
Re: [PATCH 4/4] dir.c: avoid "exceeds maximum object size" error with GCC v12.x
Quoted 11 lines · Ævar Arnfjörð Bjarmason
Quoted 10 lines · Johannes Schindelin via GitGitGadget
dir.c char *git_url_basename(const char *repo, int is_bundle, int is_bare) · line 3085
Quoted 7 lines
dir.c +7
-0+/* +* It should not be possible to overflow `ptrdiff_t` by passing in an +* insanely long URL, but GCC does not know that and will complain +* without this check. +*/ +if (end - start <0 ) + die(_("No directory name could be guessed.\n"This should start with a lower-case letter, see CodingGuidelines.
Quoted 5 lines
This message is copied from existing code later in the same function. Since it is a translateable message, I do not want to edit it because that would cause unnecessary work of the translators. Especially given that we do not even expect this message to be shown, ever, but we only add this hunk for GCC's benefit.
Perhaps this should be a BUG() statement, then? Without any translation?
Thanks,
-Stolee
Re: [PATCH 4/4] dir.c: avoid "exceeds maximum object size" error with GCC v12.x
Quoted 12 lines · Derrick Stolee
Quoted 11 lines · Ævar Arnfjörð Bjarmason
Quoted 10 lines · Johannes Schindelin via GitGitGadget
dir.c char *git_url_basename(const char *repo, int is_bundle, int is_bare) · line 3085
Quoted 7 lines
dir.c +7
-0+/* +* It should not be possible to overflow `ptrdiff_t` by passing in an +* insanely long URL, but GCC does not know that and will complain +* without this check. +*/ +if (end - start <0 ) + die(_("No directory name could be guessed.\n"This should start with a lower-case letter, see CodingGuidelines.
Quoted 8 lines
Quoted 5 lines
This message is copied from existing code later in the same function. Since it is a translateable message, I do not want to edit it because that would cause unnecessary work of the translators. Especially given that we do not even expect this message to be shown, ever, but we only add this hunk for GCC's benefit.
Perhaps this should be a
BUG()statement, then? Without any translation?
Yeah, both are good. If somehow the caller managed to pass such a long URL then it can be considered a data error at runtime, and not that the user detected a bug in our code, so in that sense die() would be appropriate. It is like xmalloc() running out of memory.
On the other hand, the "should not be possible to overflow" in the comment implicitly assumes that it is impossible to pass insanely long URL to trigger the condition from places we think of offhand, like the command line, where the input is limited to a much shorter string. As "we detected a situation that should not happen unless there is a programming or design bug" is what BUG() means, it is also good here---our assumption that this should not be possible turned out to be faulty, so we noticed a design bug.
I wonder if we can add a separate macro to add more to the documentation value, though. With something like
#define FALSE_WARNING(expression, message) \
do { if (expression) { BUG(message); } while (0)
the above would just become
FALSE_WARNING(end - start < 0, "ptrdiff_t would not overflow here");
without a need for a big comment before it. We might even be able to optimize it out when building with compilers that do not need the workaround.
Quoted 23 lines · Johannes Schindelin via GitGitGadget
A recent update of GCC in Git for Windows' SDK (a subset of which is used in Git's CI/PR builds) broke the build.
These patches address that, and they are based on maint-2.34 (earlier maintenance tracks would have a trivial merge conflict due to
013c7e2b070(http: drop support for curl < 7.16.0, 2021-07-30) removing support forUSE_CURL_MULTI).Johannes Schindelin (4):
compat/win32/syslog: fix use-after-realloc
nedmalloc: avoid new compile error
http.c: avoid danging pointer to local variablefinisheddir.c: avoid "exceeds maximum object size" error with GCC v12.xcompat/nedmalloc/nedmalloc.c | 1 - compat/win32/syslog.c | 2 ++ dir.c | 9 +++++++++ http-walker.c | 4 ---- http.c | 15 +++++++-------- http.h | 2 +- 6 files changed, 19 insertions(+), 14 deletions(-)
Quoted 5 lines
base-commit:
2f0dde7852b7866bb044926f73334ff3fc30654b
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1238%2Fdscho%2Ffix-win-build-with-gcc-12-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1238/dscho/fix-win-build-with-gcc-12-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1238
This looks good to me. I reviewed it earlier in one of our downstream forks.
Jeff