Skip to contentSkip to conversation

fetch2/wget: limit auth headers on checkstatus redirects

PATCH In review Anders Heimer 3 participants 5 messages Jun 10

lore
Share & tools
lore
#0 lore raw

cover fetch2/wget: limit auth headers on checkstatus redirects

FixedHTTPRedirectHandler currently copies request headers when following redirects. The first patch drops Authorization and Cookie when the redirect target has a different origin.

The second patch adds local HTTP server tests covering both same-origin and different-origin redirects.

Anders Heimer (2):
fetch2/wget: limit auth on checkstatus redirects tests/fetch: cover checkstatus redirect auth handling

 lib/bb/fetch2/wget.py | 27 +++++++++++++++++--
 lib/bb/tests/fetch.py | 62 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 87 insertions(+), 2 deletions(-)
#1 lore raw

PATCH 1/2 fetch2/wget: limit auth on checkstatus redirects

FixedHTTPRedirectHandler copies request headers when checkstatus() follows a redirect, including Authorization from SRC_URI or .netrc.

Keep same-origin redirects unchanged, but drop Authorization and Cookie for different-origin targets (scheme, host and effective port), following RFC 9110 redirect guidance for resource-specific headers. This only affects the Python checkstatus() path; normal wget downloads are unchanged.

---
 lib/bb/fetch2/wget.py | 27 +++++++++++++++++++++++++--
 1 file changed, 25 insertions(+), 2 deletions(-)
lib/bb/fetch2/wget.py +25 -2
index 475f042dd..11b57f2f7 100644
@@ -292,6 +292,18 @@ class Wget(FetchMethod):
             http_error_403 = http_error_405
 
 
+        def _url_origin(url):
+            parsed = urllib.parse.urlsplit(url)
+            scheme = parsed.scheme.lower()
+            host = parsed.hostname.lower() if parsed.hostname else ""
+            port = parsed.port
+            if port is None:
+                port = {"http": 80, "https": 443}.get(scheme)
+            return (scheme, host, port)
+
+        def _same_origin(url_a, url_b):
+            return _url_origin(url_a) == _url_origin(url_b)
+
         class FixedHTTPRedirectHandler(urllib.request.HTTPRedirectHandler):
             """
             urllib2.HTTPRedirectHandler before 3.13 has two flaws:
@@ -305,6 +317,9 @@ class Wget(FetchMethod):
 
             Until we depend on Python 3.13 onwards, copy the redirect_request
             method to fix these issues.
+
+            Additionally, strip sensitive headers (Authorization, Cookie) when
+            redirecting to a different origin to avoid credential leaks.
             """
             def redirect_request(self, req, fp, code, msg, headers, newurl):
                 m = req.get_method()
@@ -324,8 +339,16 @@ class Wget(FetchMethod):
                 newurl = newurl.replace(' ', '%20')
 
                 CONTENT_HEADERS = ("content-length", "content-type")
-                newheaders = {k: v for k, v in req.headers.items()
-                            if k.lower() not in CONTENT_HEADERS}
+                SENSITIVE_REDIRECT_HEADERS = ("authorization", "cookie")
+                same_origin = _same_origin(req.get_full_url(), newurl)
+                newheaders = {}
+                for k, v in req.headers.items():
+                    header = k.lower()
+                    if header in CONTENT_HEADERS:
+                        continue
+                    if not same_origin and header in SENSITIVE_REDIRECT_HEADERS:
+                        continue
+                    newheaders[k] = v
                 return urllib.request.Request(newurl,
                             method="HEAD" if m == "HEAD" else "GET",
                             headers=newheaders,
#2 lore raw

PATCH 2/2 tests/fetch: cover checkstatus redirect auth handling

In reply to Anders Heimer

Add local HTTP server tests for Wget.checkstatus() redirects. They check that Authorization is kept for same-origin redirects and dropped when the target has a different origin.

---
 lib/bb/tests/fetch.py | 62 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 62 insertions(+)
lib/bb/tests/fetch.py +62 -0
index 95cf6c414..d021ad786 100644
@@ -7,6 +7,7 @@
 #
 
 import contextlib
+import http.server
 import shutil
 import unittest
 import unittest.mock
@@ -18,6 +19,7 @@ import os
 import signal
 import subprocess
 import tarfile
+import threading
 from bb.fetch2 import URI
 import bb
 import bb.utils
@@ -1716,6 +1718,41 @@ class FetchCheckStatusTest(FetcherTest):
                       "ftp://sourceware.org/pub/libffi/libffi-1.20.tar.gz",
                       ]
 
+    def _start_checkstatus_server(self):
+        class CheckStatusHTTPRequestHandler(http.server.BaseHTTPRequestHandler):
+            def do_HEAD(self):
+                self.server.requests.append((self.path, dict(self.headers)))
+                if self.path == "/a" and self.server.redirect_url:
+                    self.send_response(302)
+                    self.send_header("Location", self.server.redirect_url)
+                    self.end_headers()
+                    return
+                self.send_response(200)
+                self.end_headers()
+
+            def log_message(self, format_str, *args):
+                pass
+
+        server = http.server.HTTPServer(("127.0.0.1", 0), CheckStatusHTTPRequestHandler)
+        server.redirect_url = None
+        server.requests = []
+        thread = threading.Thread(target=server.serve_forever, kwargs={"poll_interval": 0.05})
+        thread.daemon = True
+        thread.start()
+
+        def stop_server():
+            server.shutdown()
+            thread.join()
+            server.server_close()
+
+        self.addCleanup(stop_server)
+        return server
+
+    def _checkstatus(self, url):
+        fetch = bb.fetch2.Fetch([url], self.d)
+        ud = fetch.ud[url]
+        return ud.method.checkstatus(fetch, ud, self.d)
+
     @skipIfNoNetwork()
     def test_wget_checkstatus(self):
         fetch = bb.fetch2.Fetch(self.test_wget_uris, self.d)
@@ -1743,6 +1780,31 @@ class FetchCheckStatusTest(FetcherTest):
 
         connection_cache.close_connections()
 
+    def test_wget_checkstatus_same_origin_redirect_keeps_auth(self):
+        server = self._start_checkstatus_server()
+        server.redirect_url = "http://127.0.0.1:%s/b" % server.server_port
+
+        url = "http://127.0.0.1:%s/a;user=user;pswd=pass" % server.server_port
+        self.assertTrue(self._checkstatus(url))
+
+        self.assertEqual(len(server.requests), 2)
+        redirected_headers = {k.lower(): v for k, v in server.requests[1][1].items()}
+        self.assertIn("authorization", redirected_headers)
+
+    def test_wget_checkstatus_different_origin_redirect_drops_auth(self):
+        origin = self._start_checkstatus_server()
+        target = self._start_checkstatus_server()
+        # Same host but different port is a different origin.
+        origin.redirect_url = "http://127.0.0.1:%s/b" % target.server_port
+
+        url = "http://127.0.0.1:%s/a;user=user;pswd=pass" % origin.server_port
+        self.assertTrue(self._checkstatus(url))
+
+        self.assertEqual(len(origin.requests), 1)
+        self.assertEqual(len(target.requests), 1)
+        redirected_headers = {k.lower(): v for k, v in target.requests[0][1].items()}
+        self.assertNotIn("authorization", redirected_headers)
+
 
 class GitMakeShallowTest(FetcherTest):
     def setUp(self):
#3 lore raw
In reply to Anders Heimer

Unusually, this was rapidly merged to master almost immediately after the patches were posted. Is this a sensitive security issue, or what is the rationale?

Alex

Quoted 25 lines · Anders Heimer via lists.openembedded.org

FixedHTTPRedirectHandler currently copies request headers when following redirects. The first patch drops Authorization and Cookie when the redirect target has a different origin.

The second patch adds local HTTP server tests covering both same-origin and different-origin redirects.

Anders Heimer (2):
fetch2/wget: limit auth on checkstatus redirects tests/fetch: cover checkstatus redirect auth handling

 lib/bb/fetch2/wget.py | 27 +++++++++++++++++--
 lib/bb/tests/fetch.py | 62 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 87 insertions(+), 2 deletions(-)

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#19636): https://lists.openembedded.org/g/bitbake-devel/message/19636
Mute This Topic: https://lists.openembedded.org/mt/119736443/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#4 lore raw
Quoted 3 lines · Alexander Kanavin via lists.openembedded.org

Unusually, this was rapidly merged to master almost immediately after the patches were posted. Is this a sensitive security issue, or what is the rationale?

I did actually mean to merge it to master-next, test, then merge to master in a bit but there is a security issue in there unfortunately :(.

Basically we could lead auth headers to the wrong servers.

Cheers,

Richard