support/scripts/pkg-stats: check_url_status(): use HEAD requests to limit server load
PATCH 1/1 support/scripts/pkg-stats: check_url_status(): use HEAD requests to limit server load
The gnu.org admins have been blocking the IP address of machines running pkg-stats as the GET requests for the (many) packages with gnu.org URLs are seen as abusive.
The resource body is not used, so use a HTTP HEAD request instead of a GET to limit server load and bandwidth use.
--- support/scripts/pkg-stats | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
support/scripts/pkg-stats +1 -1
@@ -469,7 +469,7 @@ async def check_url_status(session, pkg, npkgs, retry=True, verbose=False): global check_url_count try: - async with session.get(pkg.url) as resp: + async with session.head(pkg.url) as resp: if resp.status >= 400: pkg.status['url'] = ("error", "invalid {}".format(resp.status)) check_url_count += 1
Signature · 6 lines
2.47.3 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Quoted 8 lines · Peter Korsgaard
The gnu.org admins have been blocking the IP address of machines running pkg-stats as the GET requests for the (many) packages with gnu.org URLs are seen as abusive.
The resource body is not used, so use a HTTP HEAD request instead of a GET to limit server load and bandwidth use.
Signed-off-by Peter Korsgaard <[email protected]>
Applied to master, thanks.
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
In reply of:
Quoted 8 lines
The gnu.org admins have been blocking the IP address of machines running pkg-stats as the GET requests for the (many) packages with gnu.org URLs are seen as abusive.
The resource body is not used, so use a HTTP HEAD request instead of a GET to limit server load and bandwidth use.
Signed-off-by Peter Korsgaard <[email protected]>
Applied to 2025.02.x and 2025.08.x. Thanks
Quoted 24 lines
--- support/scripts/pkg-stats | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)support/scripts/pkg-stats +1
-1@@ -469,7 +469,7 @@ async def check_url_status(session, pkg, npkgs, retry=True, verbose=False): global check_url_count try: - async with session.get(pkg.url) as resp: + async with session.head(pkg.url) as resp: if resp.status >= 400: pkg.status['url'] = ("error", "invalid {}".format(resp.status)) check_url_count += 1Signature · 6 lines
2.47.3 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot