Skip to contentSkip to conversation

doc: fix stats page

PATCH In review Andrea Cervesato 3 participants 3 messages Apr 16, 2025

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 doc: fix stats page

From: Andrea Cervesato <[email protected]>

In the last documentation builds we faced syscalls file download error. In particular, git.kernel.org communication started to fail with 403. The reason seems to be realted to the server configuration, that is now requiring a specific User-Agent. Set User-Agent to 'curl/8.6.0' to avoid this issue.

---
Fix the following error during 'make':

WARNING: Can't download syscall_n64.tbl from kernel sources (HTTP Error 403: Forbidden)

---
 doc/conf.py | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
doc/conf.py +12 -2
index 8cc890fcc8d0650aac82adabbbd434c55c7b8ad3..23fe7a1b9742f604a5b97f0db62b29ed9c64c445 100644
@@ -126,8 +126,18 @@ def generate_syscalls_stats(_):
     error = False
     try:
         socket.setdefaulttimeout(3)
-        urllib.request.urlretrieve(
-            f"{syscalls_url}/syscall_n64.tbl", "syscalls.tbl")
+
+        # kernel.org doesn't always allow to download syscalls file, so we need
+        # to emulate a different client in order to download it. Browser
+        # emulation might fail due to captcha request and for this reason we
+        # use the 'curl' command instead
+        req = urllib.request.Request(
+            f"{syscalls_url}/syscall_n64.tbl",
+            headers={'User-Agent': 'curl/8.6.0'})
+
+        with urllib.request.urlopen(req) as response:
+            with open("syscalls.tbl", 'wb') as out_file:
+                out_file.write(response.read())
     except urllib.error.URLError as err:
         error = True
         logger = sphinx.util.logging.getLogger(__name__)

---
base-commit: 8e0d3347944b2044a198ddd69d4be186fcda65ef
change-id: 20250416-fix_docs_stats-460880da49e6

Best regards,

Signature · 5 lines
Andrea Cervesato <[email protected]>


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
Reviewed-by on v1 1/1
#1 lore raw

Hi Andrea,

Quoted 5 lines

In the last documentation builds we faced syscalls file download error. In particular, git.kernel.org communication started to fail with 403. The reason seems to be realted to the server configuration, that is now requiring a specific User-Agent. Set User-Agent to 'curl/8.6.0' to avoid this issue.

Thanks for fixing this! I also noticed this, was in my TODO.
I also prefer to keep using korg if possible than start using github/torvalds/linux (which is not official, might be outdated or also limited similar way for bots).

IMHO this is part of the changes in the korg infrastructure detecting bots. I guess they need to reduce the traffic.

Kind regards,
Petr

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
#2 lore raw

Pushed thanks.

- Andrea

Quoted 19 lines · Petr Vorel

Hi Andrea,

Quoted 5 lines

In the last documentation builds we faced syscalls file download error. In particular, git.kernel.org communication started to fail with 403. The reason seems to be realted to the server configuration, that is now requiring a specific User-Agent. Set User-Agent to 'curl/8.6.0' to avoid this issue.

Thanks for fixing this! I also noticed this, was in my TODO.
I also prefer to keep using korg if possible than start using github/torvalds/linux (which is not official, might be outdated or also limited similar way for bots).

IMHO this is part of the changes in the korg infrastructure detecting bots. I guess they need to reduce the traffic.

Kind regards,
Petr

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp