Skip to contentSkip to conversation

doc/bitbake-user-manual-ref-variables: extend the MIRRORS documentation

PATCH Superseded Antonin Godard 2 participants 2 messages Jun 1

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 doc/bitbake-user-manual-ref-variables: extend the MIRRORS documentation

The MIRRORS variable wasn't properly documented as Ross highlighted in bug 16297. Give it a proper description detailing its usage and which keywords are available. Also link PREMIRRORS to MIRRORS as they use the same syntax.

[YOCTO #16297]

---
 .../bitbake-user-manual-ref-variables.rst          | 56 ++++++++++++++++++++++
 1 file changed, 56 insertions(+)
doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst +56 -0
index 9824b58ef27..753e9a38f90 100644
@@ -1416,6 +1416,58 @@ overview of their function and contents.
       upstream source, and then locations specified by :term:`MIRRORS` in that
       order.
 
+      This variable has a specific syntax. Each line must contain a regular
+      expression matching the original source URL on the leftmost part and a
+      replacement URL for it on the rightmost part. For example::
+
+         MIRRORS:prepend = " \
+             git://git.openembedded.org/.* git:///mirrors/openembedded/BASENAME \
+         "
+
+      In the above example:
+
+      -  The leftmost part means that any Git repository (``git://`` fetcher)
+         whose URL starts with ``git.openembedded.org/`` will be matched.
+
+      -  The rightmost part is the replacement URL for the original repository,
+         which is hosted locally in the ``/mirrors/openembedded`` directory. It
+         also uses the ``BASENAME`` keyword which is detailed below.
+
+      The rightmost part of each line can contain keywords which are
+      automatically replaced by BitBake. These keywords can contain either a
+      sub-part of the original URL or meta-information about it.
+
+      For what follows, let's take the following URL as an example for the
+      original source (matched by the leftmost part of the line):
+      ``git://git.openembedded.org/project/repo.git``.
+
+      A replacement for this URL can be created using the following keywords:
+
+      -  ``TYPE``: the URL scheme or fetcher type. For the example above: ``git``.
+      -  ``HOST``: the URL host. For the example above: ``git.openembedded.org``.
+      -  ``PATH``: the URL path. For the example above: ``/project/repo.git``.
+      -  ``BASENAME``: the URL basename. For the example above: ``repo.git``.
+      -  ``MIRRORNAME``: a translation of the host and path with common
+         characters substituted to dots (``.``). For the example above:
+         ``git.openembedded.org.project.repo.git``.
+
+      The rightmost part of the line can also be a bare URL that uses no special
+      keywords (usually ``https://`` or ``file://``). For example::
+
+         MIRRORS:prepend = " \
+             git://git.openembedded.org/.* file:///mirrors/openembedded/ \
+         "
+
+      In the above example, the fetcher will try to find a mirror tarball in the
+      ``/mirrors/openembedded`` directory. The name of this tarball should follow
+      the name of the tarball created by BitBake (for Git repositories,
+      the :term:`BB_GENERATE_MIRROR_TARBALLS` variable must be set to "1" to
+      generate them).
+
+      If we take ``git://git.openembedded.org/project/repo.git`` as an example,
+      the tarball would have to be named
+      ``git2_git.openembedded.org.project.repo.git.tar.gz``.
+
    :term:`OVERRIDES`
       A colon-separated list that BitBake uses to control what variables are
       overridden after BitBake parses recipes and configuration files.
@@ -1544,6 +1596,10 @@ overview of their function and contents.
       use ``file://`` URLs to point to local directories or network shares as
       well.
 
+      The syntax of the :term:`PREMIRRORS` variable is the same as
+      :term:`MIRRORS`. See the definition of the :term:`MIRRORS` for more
+      details on it.
+
    :term:`PROVIDES`
       A list of aliases by which a particular recipe can be known. By
       default, a recipe's own :term:`PN` is implicitly already in its

---
base-commit: e9a06f79d9ec767c9d95470be78b006d6fd0d59c
change-id: 20260601-clarify-mirrors-syntax-e3abbf7c0e49

#1 lore raw

Hi Antonin,

Quoted 5 lines · Antonin Godard via lists.yoctoproject.org

The MIRRORS variable wasn't properly documented as Ross highlighted in bug 16297. Give it a proper description detailing its usage and which keywords are available. Also link PREMIRRORS to MIRRORS as they use the same syntax.

Quick drive-by comment.

Quoted 12 lines

[YOCTO #16297]

---
  .../bitbake-user-manual-ref-variables.rst          | 56 ++++++++++++++++++++++
  1 file changed, 56 insertions(+)
doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst +0 -0
index 9824b58ef27..753e9a38f90 100644

doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst overview of their function and contents. · line 1422

Quoted 7 lines
doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst +4 -0
@@ -1416,6 +1416,58 @@ overview of their function and contents.
        upstream source, and then locations specified by :term:`MIRRORS` in that
        order.
 
+      This variable has a specific syntax. Each line must contain a regular
+      expression matching the original source URL on the leftmost part and a
+      replacement URL for it on the rightmost part. For example::
+

Leftmost and rightmost are confusing here. This is a space separated list which is expected to have an even number of elements as it contains a list of pairs (themselves separated by a space).

Note that the code handles newlines in MIRROR (see mirror_from_string in lib/bb/fetch2/__init__.py).

doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst overview of their function and contents. · line 1435

Quoted 13 lines
doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst +13 -0
+         MIRRORS:prepend = " \
+             git://git.openembedded.org/.* git:///mirrors/openembedded/BASENAME \
+         "
+
+      In the above example:
+
+      -  The leftmost part means that any Git repository (``git://`` fetcher)
+         whose URL starts with ``git.openembedded.org/`` will be matched.
+
+      -  The rightmost part is the replacement URL for the original repository,
+         which is hosted locally in the ``/mirrors/openembedded`` directory. It
+         also uses the ``BASENAME`` keyword which is detailed below.
+

This is mostly fine as it visually is leftmost/rightmost verbatim.

Quoted 39 lines
doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst +35 -0
+      The rightmost part of each line can contain keywords which are
+      automatically replaced by BitBake. These keywords can contain either a
+      sub-part of the original URL or meta-information about it.
+
+      For what follows, let's take the following URL as an example for the
+      original source (matched by the leftmost part of the line):
+      ``git://git.openembedded.org/project/repo.git``.
+
+      A replacement for this URL can be created using the following keywords:
+
+      -  ``TYPE``: the URL scheme or fetcher type. For the example above: ``git``.
+      -  ``HOST``: the URL host. For the example above: ``git.openembedded.org``.
+      -  ``PATH``: the URL path. For the example above: ``/project/repo.git``.
+      -  ``BASENAME``: the URL basename. For the example above: ``repo.git``.
+      -  ``MIRRORNAME``: a translation of the host and path with common
+         characters substituted to dots (``.``). For the example above:
+         ``git.openembedded.org.project.repo.git``.
+
+      The rightmost part of the line can also be a bare URL that uses no special
+      keywords (usually ``https://`` or ``file://``). For example::
+
+         MIRRORS:prepend = " \
+             git://git.openembedded.org/.* file:///mirrors/openembedded/ \
+         "
+
+      In the above example, the fetcher will try to find a mirror tarball in the
+      ``/mirrors/openembedded`` directory. The name of this tarball should follow
+      the name of the tarball created by BitBake (for Git repositories,
+      the :term:`BB_GENERATE_MIRROR_TARBALLS` variable must be set to "1" to
+      generate them).
+
+      If we take ``git://git.openembedded.org/project/repo.git`` as an example,
+      the tarball would have to be named
+      ``git2_git.openembedded.org.project.repo.git.tar.gz``.
+
     :term:`OVERRIDES`
        A colon-separated list that BitBake uses to control what variables are
        overridden after BitBake parses recipes and configuration files.

doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst overview of their function and contents. · line 1602

Quoted 7 lines
doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst +4 -0
@@ -1544,6 +1596,10 @@ overview of their function and contents.
        use ``file://`` URLs to point to local directories or network shares as
        well.
 
+      The syntax of the :term:`PREMIRRORS` variable is the same as
+      :term:`MIRRORS`. See the definition of the :term:`MIRRORS` for more
+      details on it.
+

s/on it//

It's implied.

Cheers,
Quentin