remote-curl: unbreak http.extraHeader with custom allocators

This is one of those bugs that can cost entire days of work.

The symptom of this bug is that git -c http.extraheader push ... will fail frequently in Git for Windows v2.24.0, but not always. When it fails, though, it will cause a segmentation fault in git remote-https while calling http_cleanup() and leave no visual indication of that problem, the only indication of a problem will be the exit code of the caller (in this instance, git push will fail with exit code 1).

In my tests during the pre-release period, I pushed many a time, it probably failed a lot, in the way indicated above, and due to the absence of any error message, I failed to realize that there was a problem in the first place.

Fun side note: this bug haunted me for the best part of yesterday, when I tried to get Git for Windows v2.24.0 out the door. Large parts of Git for Windows' release management are scripted, and that script failed, claiming to have been unsuccessful in pushing the tag v2.24.0.windows.1, just after printing a message to the extent that the tag is already up to date (except in the first attempt, when it reported to have been successful in pushing the tag). My attempts to fix the release script were doomed to fail because the root cause was not a bug in the script, but the bug fixed in this patch.

Johannes Schindelin (1):
remote-curl: unbreak http.extraHeader with custom allocators

 http.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

base-commit: 93b980e58f5624ee4e3b2dc0d0babaa97ef66d19
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-453%2Fdscho%2Ffix-curl-xmalloc-regression-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-453/dscho/fix-curl-xmalloc-regression-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/453

gitgitgadget

From: Johannes Schindelin <[email protected]>

In 93b980e58f5 (http: use xmalloc with cURL, 2019-08-15), we started to ask cURL to use xmalloc(), and if compiled with nedmalloc, that means implicitly a different allocator than the system one.

Which means that all of cURL's allocations and releases now _need_ to use that allocator.

However, the http_options() function used slist_append() to add any configured extra HTTP header(s) _before_ asking cURL to use xmalloc(), and http_cleanup() would release them _afterwards_, i.e. in the presence of custom allocators, cURL would attempt to use the wrong allocator to release the memory.

Let's fix this by moving the initialization _before_ the http_options() function is called.

---
 http.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

index 27aa0a3192..13f50ba158 100644
@@ -1062,6 +1062,9 @@ void http_init(struct remote *remote, const char *url, int proactive_auth)
 	char *normalized_url;
 	struct urlmatch_config config = { STRING_LIST_INIT_DUP };
 
+	if (curl_global_init(CURL_GLOBAL_ALL) != CURLE_OK)
+		die("curl_global_init failed");
+
 	config.section = "http";
 	config.key = NULL;
 	config.collect_fn = http_options;

@@ -1101,9 +1104,6 @@ void http_init(struct remote *remote, const char *url, int proactive_auth)
 	}
 #endif
 
-	if (curl_global_init(CURL_GLOBAL_ALL) != CURLE_OK)
-		die("curl_global_init failed");
-
 	http_proactive_auth = proactive_auth;
 
 	if (remote && remote->http_proxy)

gitgitgadget

Nicely explained.

Another option would be to separate our config mechanism from curl entirely by putting the list of headers into a string_list, and then transforming it later into a curl_slist. I don't think that really buys us much, though. This is all inside http.c, so it's fairly contained. It's not like other random parts of Git are using curl's slist before calling http_init().

I did briefly grep around for other slist users, but they're all what you'd expect: code in http-push.c and remote-curl.c creating header lists while working with an active http request (so well after http_init() has been called).

The patch itself looks good.

-Peff

Hi Peff,

Alas, it _does_ buy us a lot, as I *just* found out (can you imagine how glad I am not to have rushed out another Git for Windows release?). It buys us one less bug: my patch introduces the bug where http.sslbackend can no longer be used, because curl_global_sslset() needs to be called _before_ curl_global_init(), but my patch breaks that because we _need_ to parse the config before we can ask cURL for a specific backend.

Indeed. We cannot use cURL's slist anywhere outside of the cURL-dependent code because we want to keep NO_CURL=Yep working.

Indeed, I came to the same conclusion that Carlo's patch only broke support for http.extraheaders (and only with custom allocators), nothing else.

I will change the patch to avoid using slist early and send another iteration.

Thanks,
Dscho

Thanks both, and especially thanks Dscho for both for a fix and for restraining yourself from the urge to pull trigger too soon ;-)

Will look forward to seeing it.

Thanks.

Hi Junio,

Well, I _will_ trigger the pipeline publishing Git for Windows v2.24.0(2) in a moment, with this fix, but more importantly with a fix for the installer that tries to install some Cygwin-style symlinks, which was broken in non-US locales (most notably in a Japanese setup; apparently there _still_ are developers who did not yet move away from a Japanese locale... at least on Windows).

Ciao,
Dscho

I think the above was a reference to a tangential comment I made in a response to Peff.

Oh, I do not expect Japanese Windows users would "move away from" Japanese locale, ever.

I do however know that an app that supports only legacy encoding is frowned upon these days. They still use and will keep using Japanese menus and messages, and they still write their documents in Japanese and not in US English.

But they store their Japenese documents encoded in UTF-8 on their system that is in Japanese locale.

There are two models of gadgets I am interested in getting, between which one of them that is slightly older supports UTF-8 and MS-Kanji (aka Shift-JIS) while the latest model only supports MS-Kanji. The list price of them are comparable (actually, the latest one lists a bit more), but the latest model is deeply discounted while the other one with UTF-8 is not as much. At shopping sites, user reviews often mention "I've migrated my text to UTF-8 already and going back to Shift JIS in this year is too cumbersome, so I'll skip this latest model".

I am hoping a software update might happen, and will pull the trigger once the latest one starts supporting UTF-8 ;-)

Hi Junio,

Thank you for that interesting note.

Ciao,
Dscho

This is one of those bugs that can cost entire days of work.

The symptom of this bug is that git -c http.extraheader push ... will fail frequently in Git for Windows v2.24.0, but not always. When it fails, though, it will cause a segmentation fault in git remote-https while calling http_cleanup() and leave no visual indication of that problem, the only indication of a problem will be the exit code of the caller (in this instance, git push will fail with exit code 1).

In my tests during the pre-release period, I pushed many a time, it probably failed a lot, in the way indicated above, and due to the absence of any error message, I failed to realize that there was a problem in the first place.

Fun side note: this bug haunted me for the best part of this past Monday, when I tried to get Git for Windows v2.24.0 out the door. Large parts of Git for Windows' release management are scripted, and that script failed, claiming to have been unsuccessful in pushing the tag v2.24.0.windows.1, just after printing a message to the extent that the tag is already up to date (except in the first attempt, when it reported to have been successful in pushing the tag). My attempts to fix the release script were doomed to fail because the root cause was not a bug in the script, but the bug fixed in this patch.

Changes since v1:

* The patch was completely redone: instead of moving the call to curl_global_init() (which would have broken support for http.sslbackend), this iteration instead replaces the usage of cURL's slist in the config handling by using Git's own string_list.

Johannes Schindelin (1):
remote-curl: unbreak http.extraHeader with custom allocators

 http.c | 18 ++++++++----------
 1 file changed, 8 insertions(+), 10 deletions(-)

base-commit: 93b980e58f5624ee4e3b2dc0d0babaa97ef66d19
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-453%2Fdscho%2Ffix-curl-xmalloc-regression-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-453/dscho/fix-curl-xmalloc-regression-v2
Pull-Request: https://github.com/gitgitgadget/git/pull/453

Range-diff vs v1:

1: d47a2aa594 ! 1: 3168ba2c9e remote-curl: unbreak http.extraHeader with custom allocators

     @@ -15,8 +15,18 @@
          presence of custom allocators, cURL would attempt to use the wrong
          allocator to release the memory.
      
     -    Let's fix this by moving the initialization _before_ the
     -    `http_options()` function is called.
     +    A naïve attempt at fixing this would move the call to
     +    `curl_global_init()` _before_ the config is parsed (i.e. before that
     +    call to `slist_append()`).
     +
     +    However, that does work, as we _also_ parse the config setting
     +    `http.sslbackend` and if found, call `curl_global_sslset()` which *must*
     +    be called before `curl_global_init()`, for details see:
     +    https://curl.haxx.se/libcurl/c/curl_global_sslset.html
     +
     +    So let's instead make the config parsing entirely independent from
     +    cURL's data structures. Incidentally, this deletes two more lines than
     +    it introduces, which is nice.
      
          Signed-off-by: Johannes Schindelin <[email protected]>
      
     @@ -24,22 +34,50 @@
       --- a/http.c
       +++ b/http.c
      @@
     - 	char *normalized_url;
     - 	struct urlmatch_config config = { STRING_LIST_INIT_DUP };
     - 
     -+	if (curl_global_init(CURL_GLOBAL_ALL) != CURLE_OK)
     -+		die("curl_global_init failed");
     -+
     - 	config.section = "http";
     - 	config.key = NULL;
     - 	config.collect_fn = http_options;
     + 
     + static struct curl_slist *pragma_header;
     + static struct curl_slist *no_pragma_header;
     +-static struct curl_slist *extra_http_headers;
     ++static struct string_list extra_http_headers = STRING_LIST_INIT_DUP;
     + 
     + static struct active_request_slot *active_queue_head;
     + 
      @@
     + 		if (!value) {
     + 			return config_error_nonbool(var);
     + 		} else if (!*value) {
     +-			curl_slist_free_all(extra_http_headers);
     +-			extra_http_headers = NULL;
     ++			string_list_clear(&extra_http_headers, 0);
     + 		} else {
     +-			extra_http_headers =
     +-				curl_slist_append(extra_http_headers, value);
     ++			string_list_append(&extra_http_headers, value);
     + 		}
     + 		return 0;
       	}
     +@@
       #endif
     + 	curl_global_cleanup();
     + 
     +-	curl_slist_free_all(extra_http_headers);
     +-	extra_http_headers = NULL;
     ++	string_list_clear(&extra_http_headers, 0);
     + 
     + 	curl_slist_free_all(pragma_header);
     + 	pragma_header = NULL;
     +@@
     + 
     + struct curl_slist *http_copy_default_headers(void)
     + {
     +-	struct curl_slist *headers = NULL, *h;
     ++	struct curl_slist *headers = NULL;
     ++	const struct string_list_item *item;
       
     --	if (curl_global_init(CURL_GLOBAL_ALL) != CURLE_OK)
     --		die("curl_global_init failed");
     --
     - 	http_proactive_auth = proactive_auth;
     +-	for (h = extra_http_headers; h; h = h->next)
     +-		headers = curl_slist_append(headers, h->data);
     ++	for_each_string_list_item(item, &extra_http_headers)
     ++		headers = curl_slist_append(headers, item->string);
       
     - 	if (remote && remote->http_proxy)
     + 	return headers;
     + }

gitgitgadget

From: Johannes Schindelin <[email protected]>

In 93b980e58f5 (http: use xmalloc with cURL, 2019-08-15), we started to ask cURL to use xmalloc(), and if compiled with nedmalloc, that means implicitly a different allocator than the system one.

Which means that all of cURL's allocations and releases now _need_ to use that allocator.

However, the http_options() function used slist_append() to add any configured extra HTTP header(s) _before_ asking cURL to use xmalloc(), and http_cleanup() would release them _afterwards_, i.e. in the presence of custom allocators, cURL would attempt to use the wrong allocator to release the memory.

A naïve attempt at fixing this would move the call to curl_global_init() _before_ the config is parsed (i.e. before that call to slist_append()).

However, that does work, as we _also_ parse the config setting http.sslbackend and if found, call curl_global_sslset() which *must* be called before curl_global_init(), for details see:
https://curl.haxx.se/libcurl/c/curl_global_sslset.html

So let's instead make the config parsing entirely independent from cURL's data structures. Incidentally, this deletes two more lines than it introduces, which is nice.

---
 http.c | 18 ++++++++----------
 1 file changed, 8 insertions(+), 10 deletions(-)

index 27aa0a3192..82f493c7fd 100644
@@ -150,7 +150,7 @@ static unsigned long empty_auth_useless =
 
 static struct curl_slist *pragma_header;
 static struct curl_slist *no_pragma_header;
-static struct curl_slist *extra_http_headers;
+static struct string_list extra_http_headers = STRING_LIST_INIT_DUP;
 
 static struct active_request_slot *active_queue_head;
 

@@ -414,11 +414,9 @@ static int http_options(const char *var, const char *value, void *cb)
 		if (!value) {
 			return config_error_nonbool(var);
 		} else if (!*value) {
-			curl_slist_free_all(extra_http_headers);
-			extra_http_headers = NULL;
+			string_list_clear(&extra_http_headers, 0);
 		} else {
-			extra_http_headers =
-				curl_slist_append(extra_http_headers, value);
+			string_list_append(&extra_http_headers, value);
 		}
 		return 0;
 	}

@@ -1199,8 +1197,7 @@ void http_cleanup(void)
 #endif
 	curl_global_cleanup();
 
-	curl_slist_free_all(extra_http_headers);
-	extra_http_headers = NULL;
+	string_list_clear(&extra_http_headers, 0);
 
 	curl_slist_free_all(pragma_header);
 	pragma_header = NULL;

@@ -1624,10 +1621,11 @@ int run_one_slot(struct active_request_slot *slot,
 
 struct curl_slist *http_copy_default_headers(void)
 {
-	struct curl_slist *headers = NULL, *h;
+	struct curl_slist *headers = NULL;
+	const struct string_list_item *item;
 
-	for (h = extra_http_headers; h; h = h->next)
-		headers = curl_slist_append(headers, h->data);
+	for_each_string_list_item(item, &extra_http_headers)
+		headers = curl_slist_append(headers, item->string);
 
 	return headers;
 }

gitgitgadget

Yikes, good catch. It didn't even occur to me that there might be curl things we had to do _before_ calling curl_global_init().

Yes, I actually find the resulting code easier to read. I had feared having to add an extra step to initialize the slist, but it's all handled quite neatly in http_copy_default_headers().

The patch itself looks good to me.

-Peff

"Johannes Schindelin via GitGitGadget" <[email protected]>

s/allocator/de&/; perhaps, even though it is clear enough from the context, so it is probably OK as is.

s/does work/does not work/; presumably?

Yeah, string_list_clear() is more concise than curl_slist_free_all(), and we have already been copying one list to another anyway, so we lucked out ;-)

The patch looked good to me, too.

Hi Junio,

Indeed. Could I ask you to fix up locally, or do you want me to send a new revision of the patch?

Ciao,
Dscho

You can certainly tell me to locally tweak, but you'd need to be more specific when some of my suggestions were "perhaps" (aka "you could do it this way, which may be better, but I do not care too deeply---I am OK with whichever you chooose after you (re)think about it, but I am not choosing for you").

For now, I'll do the "does not work" thing only.

Thanks.

Hi Junio,

Right, I should have been more specific.

Thanks!

FWIW I would like to stick with "custom allocator" because even releasing the memory is the duty of an "allocator", even if that allocator happens to "deallocate" at that point what it has allocated before.

Ciao,
Dscho

My preference is to think of the set of functions we feed curl-global-init-mem as a (allocator, deallocator, reallocator, ...) tuple.

But you are calling that set itself as an allocator that has service functions like (alloc, free, realloc), and as long as that (i.e. how you defined the word "allocator") is clear to the readers that is perfectly OK.

To me as one reader, it was not clear and that was where my comment came from.

"Johannes Schindelin via GitGitGadget" <[email protected]>

This fixes by immediately building on top of the patch that caused the issue, so I should do the same so that any integration branch (either mine or somebody else's) that has the breakage can merge the fix in. Good.

Thanks.