fetch2/wget: limit auth headers on checkstatus redirects
cover fetch2/wget: limit auth headers on checkstatus redirects
FixedHTTPRedirectHandler currently copies request headers when following redirects. The first patch drops Authorization and Cookie when the redirect target has a different origin.
The second patch adds local HTTP server tests covering both same-origin and different-origin redirects.
Anders Heimer (2):
fetch2/wget: limit auth on checkstatus redirects tests/fetch: cover checkstatus redirect auth handling
lib/bb/fetch2/wget.py | 27 +++++++++++++++++-- lib/bb/tests/fetch.py | 62 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 87 insertions(+), 2 deletions(-)
PATCH 1/2 fetch2/wget: limit auth on checkstatus redirects
FixedHTTPRedirectHandler copies request headers when checkstatus() follows a redirect, including Authorization from SRC_URI or .netrc.
Keep same-origin redirects unchanged, but drop Authorization and Cookie for different-origin targets (scheme, host and effective port), following RFC 9110 redirect guidance for resource-specific headers. This only affects the Python checkstatus() path; normal wget downloads are unchanged.
--- lib/bb/fetch2/wget.py | 27 +++++++++++++++++++++++++-- 1 file changed, 25 insertions(+), 2 deletions(-)
lib/bb/fetch2/wget.py +25 -2
@@ -292,6 +292,18 @@ class Wget(FetchMethod): http_error_403 = http_error_405 + def _url_origin(url): + parsed = urllib.parse.urlsplit(url) + scheme = parsed.scheme.lower() + host = parsed.hostname.lower() if parsed.hostname else "" + port = parsed.port + if port is None: + port = {"http": 80, "https": 443}.get(scheme) + return (scheme, host, port) + + def _same_origin(url_a, url_b): + return _url_origin(url_a) == _url_origin(url_b) + class FixedHTTPRedirectHandler(urllib.request.HTTPRedirectHandler): """ urllib2.HTTPRedirectHandler before 3.13 has two flaws:
@@ -305,6 +317,9 @@ class Wget(FetchMethod): Until we depend on Python 3.13 onwards, copy the redirect_request method to fix these issues. + + Additionally, strip sensitive headers (Authorization, Cookie) when + redirecting to a different origin to avoid credential leaks. """ def redirect_request(self, req, fp, code, msg, headers, newurl): m = req.get_method()
@@ -324,8 +339,16 @@ class Wget(FetchMethod): newurl = newurl.replace(' ', '%20') CONTENT_HEADERS = ("content-length", "content-type") - newheaders = {k: v for k, v in req.headers.items() - if k.lower() not in CONTENT_HEADERS} + SENSITIVE_REDIRECT_HEADERS = ("authorization", "cookie") + same_origin = _same_origin(req.get_full_url(), newurl) + newheaders = {} + for k, v in req.headers.items(): + header = k.lower() + if header in CONTENT_HEADERS: + continue + if not same_origin and header in SENSITIVE_REDIRECT_HEADERS: + continue + newheaders[k] = v return urllib.request.Request(newurl, method="HEAD" if m == "HEAD" else "GET", headers=newheaders,
PATCH 2/2 tests/fetch: cover checkstatus redirect auth handling
In reply to Anders HeimerAdd local HTTP server tests for Wget.checkstatus() redirects. They check that Authorization is kept for same-origin redirects and dropped when the target has a different origin.
--- lib/bb/tests/fetch.py | 62 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 62 insertions(+)
lib/bb/tests/fetch.py +62 -0
@@ -7,6 +7,7 @@ # import contextlib +import http.server import shutil import unittest import unittest.mock
@@ -18,6 +19,7 @@ import os import signal import subprocess import tarfile +import threading from bb.fetch2 import URI import bb import bb.utils
@@ -1716,6 +1718,41 @@ class FetchCheckStatusTest(FetcherTest): "ftp://sourceware.org/pub/libffi/libffi-1.20.tar.gz", ] + def _start_checkstatus_server(self): + class CheckStatusHTTPRequestHandler(http.server.BaseHTTPRequestHandler): + def do_HEAD(self): + self.server.requests.append((self.path, dict(self.headers))) + if self.path == "/a" and self.server.redirect_url: + self.send_response(302) + self.send_header("Location", self.server.redirect_url) + self.end_headers() + return + self.send_response(200) + self.end_headers() + + def log_message(self, format_str, *args): + pass + + server = http.server.HTTPServer(("127.0.0.1", 0), CheckStatusHTTPRequestHandler) + server.redirect_url = None + server.requests = [] + thread = threading.Thread(target=server.serve_forever, kwargs={"poll_interval": 0.05}) + thread.daemon = True + thread.start() + + def stop_server(): + server.shutdown() + thread.join() + server.server_close() + + self.addCleanup(stop_server) + return server + + def _checkstatus(self, url): + fetch = bb.fetch2.Fetch([url], self.d) + ud = fetch.ud[url] + return ud.method.checkstatus(fetch, ud, self.d) + @skipIfNoNetwork() def test_wget_checkstatus(self): fetch = bb.fetch2.Fetch(self.test_wget_uris, self.d)
@@ -1743,6 +1780,31 @@ class FetchCheckStatusTest(FetcherTest): connection_cache.close_connections() + def test_wget_checkstatus_same_origin_redirect_keeps_auth(self): + server = self._start_checkstatus_server() + server.redirect_url = "http://127.0.0.1:%s/b" % server.server_port + + url = "http://127.0.0.1:%s/a;user=user;pswd=pass" % server.server_port + self.assertTrue(self._checkstatus(url)) + + self.assertEqual(len(server.requests), 2) + redirected_headers = {k.lower(): v for k, v in server.requests[1][1].items()} + self.assertIn("authorization", redirected_headers) + + def test_wget_checkstatus_different_origin_redirect_drops_auth(self): + origin = self._start_checkstatus_server() + target = self._start_checkstatus_server() + # Same host but different port is a different origin. + origin.redirect_url = "http://127.0.0.1:%s/b" % target.server_port + + url = "http://127.0.0.1:%s/a;user=user;pswd=pass" % origin.server_port + self.assertTrue(self._checkstatus(url)) + + self.assertEqual(len(origin.requests), 1) + self.assertEqual(len(target.requests), 1) + redirected_headers = {k.lower(): v for k, v in target.requests[0][1].items()} + self.assertNotIn("authorization", redirected_headers) + class GitMakeShallowTest(FetcherTest): def setUp(self):
Unusually, this was rapidly merged to master almost immediately after the patches were posted. Is this a sensitive security issue, or what is the rationale?
Alex
Quoted 25 lines · Anders Heimer via lists.openembedded.org
FixedHTTPRedirectHandler currently copies request headers when following redirects. The first patch drops Authorization and Cookie when the redirect target has a different origin.
The second patch adds local HTTP server tests covering both same-origin and different-origin redirects.
Anders Heimer (2):
fetch2/wget: limit auth on checkstatus redirects tests/fetch: cover checkstatus redirect auth handlinglib/bb/fetch2/wget.py | 27 +++++++++++++++++-- lib/bb/tests/fetch.py | 62 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 87 insertions(+), 2 deletions(-)-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#19636): https://lists.openembedded.org/g/bitbake-devel/message/19636
Mute This Topic: https://lists.openembedded.org/mt/119736443/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Quoted 3 lines · Alexander Kanavin via lists.openembedded.org
Unusually, this was rapidly merged to master almost immediately after the patches were posted. Is this a sensitive security issue, or what is the rationale?
I did actually mean to merge it to master-next, test, then merge to master in a bit but there is a security issue in there unfortunately :(.
Basically we could lead auth headers to the wrong servers.
Cheers,
Richard