Skip to contentSkip to conversation

remote-curl: use auth for probe_rpc() requests too

PATCH Superseded Aaron Plattner 2 participants 2 messages Jan 14

v4 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

PATCH v4 1/1 remote-curl: use auth for probe_rpc() requests too

If a large request requires post_rpc() to call probe_rpc(), the latter does not use the authorization credentials used for other requests. If this fails with an HTTP 401 error and http_auth.multistage isn't set, then the whole request just fails.

For example, using git-credential-msal [1], the following attempt to clone a large repository fails partway through because the initial request to download the commit history and promisor packs succeeds, but the subsequent request to download the blobs needed to construct the working tree fails with a 401 error and the checkout fails.

(lines removed for brevity)

git clone --filter=blob:none https://secure-server.example/repo
11:03:26.855369 git.c:502 trace: built-in: git clone --filter=blob:none https://secure-server.example/repo
Cloning into 'sw'...
warning: templates not found in /home/aaron/share/git-core/templates

  11:03:26.857169 run-command.c:673       trace: run_command: git remote-https origin https://secure-server.example/repo
  11:03:27.012104 http.c:849              => Send header: GET repo/info/refs?service=git-upload-pack HTTP/1.1
  11:03:27.049243 http.c:849              <= Recv header: HTTP/1.1 401 Unauthorized
  11:03:27.049270 http.c:849              <= Recv header: WWW-Authenticate: Bearer error="invalid_request", error_description="No bearer token found in the request", msal-tenant-id="<tenant>", msal-client-id="<client>"
  11:03:27.053786 run-command.c:673       trace: run_command: 'git credential-msal get'
  11:03:27.952830 http.c:849              => Send header: GET repo/info/refs?service=git-upload-pack HTTP/1.1
  11:03:27.952849 http.c:849              => Send header: Authorization: Bearer <redacted>
  11:03:27.995419 http.c:849              <= Recv header: HTTP/1.1 200 OK
  11:03:28.230039 http.c:890              == Info: Reusing existing https: connection with host secure-server.example
  11:03:28.230208 http.c:849              => Send header: POST repo/git-upload-pack HTTP/1.1
  11:03:28.230216 http.c:849              => Send header: Content-Type: application/x-git-upload-pack-request
  11:03:28.230221 http.c:849              => Send header: Authorization: Bearer <redacted>
  11:03:28.269085 http.c:849              <= Recv header: HTTP/1.1 200 OK
  11:03:28.684163 http.c:890              == Info: Reusing existing https: connection with host secure-server.example
  11:03:28.684379 http.c:849              => Send header: POST repo/git-upload-pack HTTP/1.1
  11:03:28.684391 http.c:849              => Send header: Accept: application/x-git-upload-pack-result
  11:03:28.684393 http.c:849              => Send header: Authorization: Bearer <redacted>
  11:03:28.869546 run-command.c:673       trace: run_command: git index-pack --stdin --fix-thin '--keep=fetch-pack 43856 on dgx-spark' --promisor
  11:06:39.861237 run-command.c:673       trace: run_command: git -c fetch.negotiationAlgorithm=noop fetch origin --no-tags --no-write-fetch-head --recurse-submodules=no --filter=blob:none --stdin
  11:06:39.865981 run-command.c:673       trace: run_command: git remote-https origin https://secure-server.example/repo
  11:06:39.868039 run-command.c:673       trace: run_command: git-remote-https origin https://secure-server.example/repo
  11:07:30.412575 http.c:849              => Send header: GET repo/info/refs?service=git-upload-pack HTTP/1.1
  11:07:30.456285 http.c:849              <= Recv header: HTTP/1.1 401 Unauthorized
  11:07:30.456318 http.c:849              <= Recv header: WWW-Authenticate: Bearer error="invalid_request", error_description="No bearer token found in the request", msal-tenant-id="<tenant>", msal-client-id="<client>"
  11:07:30.456439 run-command.c:673       trace: run_command: 'git credential-cache get'
  11:07:30.461266 http.c:849              => Send header: GET repo/info/refs?service=git-upload-pack HTTP/1.1
  11:07:30.461282 http.c:849              => Send header: Authorization: Bearer <redacted>
  11:07:30.501628 http.c:849              <= Recv header: HTTP/1.1 200 OK
  11:07:34.725262 http.c:849              => Send header: POST repo/git-upload-pack HTTP/1.1
  11:07:34.725279 http.c:849              => Send header: Content-Type: application/x-git-upload-pack-request
  11:07:34.761407 http.c:849              <= Recv header: HTTP/1.1 401 Unauthorized
  11:07:34.761443 http.c:890              == Info: Bearer authentication problem, ignoring.
  11:07:34.761453 http.c:849              <= Recv header: WWW-Authenticate: Bearer error="invalid_request", error_description="No bearer token found in the request", msal-tenant-id="<tenant>", msal-client-id="<client>"
  11:07:34.761509 http.c:890              == Info: The requested URL returned error: 401
  11:07:34.761530 http.c:890              == Info: closing connection #0
  11:07:34.761913 run-command.c:673       trace: run_command: 'git credential-cache erase'
  11:07:34.761927 run-command.c:765       trace: start_command: /bin/sh -c 'git credential-cache erase' 'git credential-cache erase'
  11:07:34.768069 git.c:502               trace: built-in: git credential-cache erase
  11:07:34.768690 run-command.c:673       trace: run_command: 'git credential-msal erase'
  11:07:34.768713 run-command.c:765       trace: start_command: /bin/sh -c 'git credential-msal erase' 'git credential-msal erase'
  11:07:34.772742 git.c:808               trace: exec: git-credential-msal erase
  11:07:34.772783 run-command.c:673       trace: run_command: git-credential-msal erase
  11:07:34.772819 run-command.c:765       trace: start_command: /usr/bin/git-credential-msal erase

error: RPC failed; HTTP 401 curl 22 The requested URL returned error: 401
fatal: unable to write request to remote: Broken pipe
fatal: could not fetch c4fff0229c9be06ecf576356a4d39a8a755b8d81 from promisor remote
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry with 'git restore --source=HEAD :/'

In this case, the HTTP_REAUTH retry logic is not used because the credential helper didn't set the 'continue' flag, so http_auth.multistage is false and handle_curl_result() fails with HTTP_NOAUTH instead.

Fix the immediate problem by including the authorization headers in the probe_rpc() request as well.

Add a test for this scenario:

1. Create a repository with two thousand refs.
2. Clone that into the web root used by t5563-simple-http-auth.sh.
3. Configure http.postBuffer to be very small in order to trigger the probe_rpc() path that fails.
4. Clone using a valid Bearer token.

[1] https://github.com/Binary-Eater/git-credential-msal

---
v4: Style nitpicks, use xargs in case 2000 branches is too many for the command line.

 remote-curl.c               |  1 +
 t/t5563-simple-http-auth.sh | 46 +++++++++++++++++++++++++++++++++++++
 2 files changed, 47 insertions(+)
remote-curl.c +1 -0
index 69f919454a..92e40bb682 100644
@@ -876,6 +876,7 @@ static int probe_rpc(struct rpc_state *rpc, struct slot_results *results)
 
 	headers = curl_slist_append(headers, rpc->hdr_content_type);
 	headers = curl_slist_append(headers, rpc->hdr_accept);
+	headers = http_append_auth_header(&http_auth, headers);
 
 	curl_easy_setopt(slot->curl, CURLOPT_NOBODY, 0L);
 	curl_easy_setopt(slot->curl, CURLOPT_POST, 1L);
t/t5563-simple-http-auth.sh +46 -0

52 lines not on this page · raw message

Signature · 1 line
2.52.0
Tested-by Lucas De Marchi on v4 1/1
#1 lore raw
Quoted 4 lines · Aaron Plattner

[snip]

Unrelated nitpick: It is more common in this project to have your signoff last. If the maintainer applies this patch:

Signed-off-by The Maintainer <>

It looks like The Maintainer added the Tested-by trailer.

At least this is my experience with Acked-by.

Quoted 4 lines

---
v4: Style nitpicks, use xargs in case 2000 branches is too many for the command line.
[snip]