Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F3FC21F2BAE for ; Wed, 30 Jul 2025 06:41:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753857675; cv=none; b=Ga3MdZlSu1v54pwG0ln60g+44T4352raX8h7X+/VVY7HlCuGgh9ewfYzMmbd7QTkldbW8ztcq0HO4G5ldbSM3eSdHEiVkPS/yV+mHDlLK+JB+R3fZ2NggMbajBzL0Zs1IaH8c6FliYcCJx6ukl3TUxBFqP6xOlrr/tWSZ1Nfs5w= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753857675; c=relaxed/simple; bh=WMNNMlxP0rU0vgmAo0NdTpZmPkb2myvjb2Od1oHkI14=; h=Message-ID:Date:MIME-Version:Subject:To:References:From: In-Reply-To:Content-Type; b=pBkMQpQiu4x49AR3cGUuI/UwCtyM8DTH/mVyKnAJuPAizY+ibgda4tAknykBIR8/DCp1/BTYB5glFS1jEH2uWDSUNHuUa971s8g5rMorKxoXJ1H19+pxlCXLJSfc0OMR2Sm8NpBPOkztzNlTZjO2D/PV0rAhccqwuodXxAB0b+o= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=BzCTS2hW; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="BzCTS2hW" Received: by smtp.kernel.org (Postfix) with ESMTPSA id AFE0CC4CEE7; Wed, 30 Jul 2025 06:41:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1753857674; bh=WMNNMlxP0rU0vgmAo0NdTpZmPkb2myvjb2Od1oHkI14=; h=Date:Reply-To:Subject:To:References:From:In-Reply-To:From; b=BzCTS2hWqcgwrOl5tio9sSQI3UEhdf8Ywukw0aMuyOw9A/zdS7+xc28eXwp1buKB7 Dt+0texmtjeBqlet3gPVSV/BVZl8ST8wXbDXwfRUBzM5QTYMumolAmPLlxiAuyMnYF hJK6QOwZxsGzTF/8cQiuOHujHwJzp73rlbnZgd35xnYAnpVoP3QEcJvFCBWjuOJVam GQex8WKTk/vVZ/gr61kP6U9ZA8h+EXWG9pcmrKg3QpJ9onmzjuEA8/ZlNvGq+dFpnY QRJi2AbTkml5g/QiDCdrCoXmZ+/Uvtd+MUCS4fDUWYq+aSwzBUlv+X/92ZM6oGKtQp 7yoWB7LsS4PLw== Message-ID: <0cc8da70-5146-4960-9e0a-6fe8b34eb05a@kernel.org> Date: Wed, 30 Jul 2025 08:41:12 +0200 Precedence: bulk X-Mailing-List: kdevops@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Reply-To: Daniel Gomez Subject: Re: [PATCH v2 8/9] devconfig: add automatic APT mirror fallback for Debian testing To: Luis Chamberlain , Chuck Lever , Daniel Gomez , kdevops@lists.linux.dev References: <20250730060147.182140-1-mcgrof@kernel.org> <20250730060147.182140-9-mcgrof@kernel.org> Content-Language: en-US From: Daniel Gomez Organization: kernel.org In-Reply-To: <20250730060147.182140-9-mcgrof@kernel.org> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit On 30/07/2025 08.01, Luis Chamberlain wrote: > Debian testing (trixie) VMs can fail to provision when configured APT > mirrors become unavailable or unresponsive. This is particularly common > with local or regional mirrors that may have intermittent connectivity > issues. > > This fix adds automatic mirror health checking specifically for Debian > testing systems. The implementation: > > 1. Extracts the currently configured APT mirror hostname > 2. Tests connectivity to the mirror on port 80 with a 10 second timeout > 3. Falls back to official Debian mirrors if the test fails > 4. Backs up the original sources.list before making changes > 5. Updates the APT cache after switching mirrors > 6. Provides clear user notification about the fallback > > The check only runs on Debian testing systems where devconfig_debian_testing > is set to true, avoiding any impact on stable Debian or other distributions. > > This ensures that Debian testing VMs can successfully provision even when > the initially configured mirror is unavailable, improving reliability for > development and testing workflows. > > Generated-by: Claude AI > Signed-off-by: Luis Chamberlain > --- > .../devconfig/tasks/check-apt-mirrors.yml | 63 +++++++++++++++++++ > playbooks/roles/devconfig/tasks/main.yml | 8 +++ > .../debian-testing-fallback-sources.list | 10 +++ > 3 files changed, 81 insertions(+) > create mode 100644 playbooks/roles/devconfig/tasks/check-apt-mirrors.yml > create mode 100644 playbooks/roles/devconfig/templates/debian-testing-fallback-sources.list > > diff --git a/playbooks/roles/devconfig/tasks/check-apt-mirrors.yml b/playbooks/roles/devconfig/tasks/check-apt-mirrors.yml > new file mode 100644 > index 00000000..02e0c800 > --- /dev/null > +++ b/playbooks/roles/devconfig/tasks/check-apt-mirrors.yml > @@ -0,0 +1,63 @@ > +--- > +# Only run mirror checks for Debian testing (trixie) where mirror issues are common > +- name: Extract current APT mirror hostname > + shell: | > + grep -E "^deb\s+http" /etc/apt/sources.list | head -1 | awk '{print $2}' | sed 's|http://||' | cut -d'/' -f1 > + register: apt_mirror_host > + changed_when: false > + ignore_errors: yes > + > +- name: Check connectivity to current APT mirror > + wait_for: > + host: "{{ apt_mirror_host.stdout }}" > + port: 80 > + timeout: 10 > + register: mirror_connectivity > + ignore_errors: yes > + when: apt_mirror_host.stdout != "" > + > +- name: Display mirror check results > + debug: > + msg: | > + Current APT mirror: {{ apt_mirror_host.stdout | default('Not found') }} > + Mirror connectivity: {{ 'OK' if mirror_connectivity is not failed else 'FAILED' }} > + when: apt_mirror_host.stdout != "" > + > +- name: Fall back to official Debian mirrors if current mirror fails > + block: > + - name: Backup current sources.list > + copy: > + src: /etc/apt/sources.list > + dest: /etc/apt/sources.list.backup > + remote_src: yes > + become: yes > + > + - name: Apply Debian testing fallback sources > + template: > + src: debian-testing-fallback-sources.list > + dest: /etc/apt/sources.list > + owner: root > + group: root > + mode: '0644' > + become: yes > + > + - name: Update APT cache after mirror change > + apt: > + update_cache: yes > + cache_valid_time: 0 > + become: yes > + > + - name: Inform user about mirror fallback > + debug: > + msg: | > + WARNING: The configured APT mirror '{{ apt_mirror_host.stdout }}' is not accessible. > + Falling back to official Debian testing mirrors: > + - deb.debian.org for main packages > + - security.debian.org for security updates > + > + This may result in slower package downloads depending on your location. > + Consider configuring a local mirror for better performance. > + > + when: > + - apt_mirror_host.stdout != "" > + - mirror_connectivity is failed > diff --git a/playbooks/roles/devconfig/tasks/main.yml b/playbooks/roles/devconfig/tasks/main.yml > index 656d5389..ceb0f2e8 100644 > --- a/playbooks/roles/devconfig/tasks/main.yml > +++ b/playbooks/roles/devconfig/tasks/main.yml > @@ -30,6 +30,14 @@ > tags: hostname > > # Distro specific > + > +# Check and fix APT mirrors for Debian testing before installing dependencies > +- name: Check and fix APT mirrors for Debian testing > + include_tasks: check-apt-mirrors.yml > + when: > + - devconfig_debian_testing is defined > + - devconfig_debian_testing | bool > + > - name: Install dependencies > ansible.builtin.include_tasks: install-deps/main.yml > tags: ['vars', 'vars_simple'] > diff --git a/playbooks/roles/devconfig/templates/debian-testing-fallback-sources.list b/playbooks/roles/devconfig/templates/debian-testing-fallback-sources.list > new file mode 100644 > index 00000000..456ed60f > --- /dev/null > +++ b/playbooks/roles/devconfig/templates/debian-testing-fallback-sources.list > @@ -0,0 +1,10 @@ > +deb http://deb.debian.org/debian testing main contrib non-free non-free-firmware > +deb-src http://deb.debian.org/debian testing main contrib non-free non-free-firmware > + > +# Security updates > +deb http://security.debian.org/debian-security testing-security main contrib non-free non-free-firmware > +deb-src http://security.debian.org/debian-security testing-security main contrib non-free non-free-firmware > + > +# Updates (if available for testing) > +deb http://deb.debian.org/debian testing-updates main contrib non-free non-free-firmware > +deb-src http://deb.debian.org/debian testing-updates main contrib non-free non-free-firmware > \ No newline at end of file Debian has switched to a new sources format [1]. They have also provided a manual conversion for users with "apt modernize-sources". So it makes sense to start using the new format directly here. [1] https://wiki.debian.org/SourcesList