Skip to contentSkip to conversation

doc: document fixed revisions override in bitbake-setup manual

PATCH In review Alexander Kanavin 3 participants 10 messages Jan 9 – Jan 15

lore
Share & tools
lore
#0 lore raw

PATCH 1/5 doc: document fixed revisions override in bitbake-setup manual

From: Alexander Kanavin <[email protected]>

---
 .../bitbake-user-manual-environment-setup.rst | 48 +++++++++++++++++++
 1 file changed, 48 insertions(+)
doc/bitbake-user-manual/bitbake-user-manual-environment-setup.rst +48 -0
index ec1bdeecd..fcffab812 100644
@@ -938,3 +938,51 @@ Here is an example file that overrides the branch of the BitBake repository to
 -  The ``sources`` section contains the same options as the ``sources`` option
    of a :term:`Generic Configuration` file. See the
    :ref:`ref-bbsetup-section-config-reference` section for more information.
+
+.. _ref-bbsetup-fixed-revisions:
+
+Fixed source revisions
+======================
+
+:term:`Generic Configuration` can set source revisions in ``rev`` to a tag or a branch.
+Bitbake-setup will make sure the actual revision will match the tag or branch when performing
+initializations or updates, and will capture the revisions in a :ref:`_ref-bbsetup-source-overrides`
+file.
+
+This file is named ``sources-fixed-revisions.json`` and is available in ``layers/``
+under the :term:`Setup` directory (as well as under ``config/`` for backwards compatibility).
+It can be used to keep record of what was checked out when using configurations that do not
+specify exact revisions.
+
+Also, as it is an override file, it can be combined with the original configurations
+to initialize a setup in a reproducible way that guarantees an exact, never-changing
+set of revisions, by using the ``--source-overrides`` option of the
+:ref:`ref-bbsetup-command-init` command.
+
+For example if the original configuration had specified only a master branch
+for a source::
+
+     "bitbake": {
+         "git-remote": {
+             "remotes": {
+                 "origin": {
+                     "uri": "https://git.openembedded.org/bitbake"
+                 }
+             },
+             "branch": "master",
+             "rev": "master"
+     }
+
+
+the override will contain the exact revisions::
+
+     "bitbake": {
+         "git-remote": {
+             "remotes": {
+                 "origin": {
+                     "uri": "https://git.openembedded.org/bitbake"
+                 }
+             },
+             "branch": "master",
+             "rev": "720df1a53452983c1c832f624490e255cf389204"
+     }
Signature · 1 line
2.47.3
#1 lore raw

PATCH 2/5 bitbake-setup: ensure paths with timestamps in them are unique

From: Alexander Kanavin <[email protected]>

There have been instances where directories with timestamps in them are created within the same second, and their paths clash as the timestamp in them is the same for both, as the timestamp is accurate to 1 second.

This adds a check for the directory existence and then adds an ever-increasing counter until there's a path that isn't yet created.

[YOCTO #16121]

---
 bin/bitbake-setup | 18 ++++++++++++++----
 1 file changed, 14 insertions(+), 4 deletions(-)
bin/bitbake-setup +14 -4
index 06255b112..4ccf47e87 100755
@@ -93,6 +93,17 @@ def _write_layer_list(dest, repodirs):
     with open(layers_f, 'w') as f:
         json.dump({"version":"1.0","layers":layers}, f, sort_keys=True, indent=4)
 
+def add_unique_timestamp_to_path(path):
+    timestamp = time.strftime("%Y%m%d%H%M%S")
+    path_unique = "{}.{}".format(path, timestamp)
+    if os.path.exists(path_unique):
+        import itertools
+        for i in itertools.count(start=1):
+            path_unique = "{}.{}.{}".format(path, timestamp, i)
+            if not os.path.exists(path_unique):
+                break
+    return path_unique
+
 def checkout_layers(layers, layerdir, d):
     def _checkout_git_remote(r_remote, repodir, layers_fixed_revisions):
         rev = r_remote['rev']
@@ -242,9 +253,8 @@ def setup_bitbake_build(bitbake_config, layerdir, setupdir, thisdir, update_bb_c
         raise Exception("Cannot complete setting up a bitbake build directory from OpenEmbedded template '{}' as oe-setup-build was not found in any layers; please use oe-init-build-env manually.".format(template))
 
     bitbake_confdir = os.path.join(bitbake_builddir, 'conf')
-    timestamp = time.strftime("%Y%m%d%H%M%S")
-    backup_bitbake_confdir = os.path.join(bitbake_builddir, 'conf-backup.{}'.format(timestamp))
-    upstream_bitbake_confdir = os.path.join(bitbake_builddir, 'conf-upstream.{}'.format(timestamp))
+    backup_bitbake_confdir = add_unique_timestamp_to_path(os.path.join(bitbake_builddir, 'conf-backup'))
+    upstream_bitbake_confdir = add_unique_timestamp_to_path(os.path.join(bitbake_builddir, 'conf-upstream'))
 
     if os.path.exists(bitbake_confdir):
         os.rename(bitbake_confdir, backup_bitbake_confdir)
@@ -760,7 +770,7 @@ def install_buildtools(top_dir, settings, args, d):
         shutil.rmtree(buildtools_install_dir)
 
     install_buildtools = os.path.join(args.setup_dir, 'layers/oe-scripts/install-buildtools')
-    buildtools_download_dir = os.path.join(args.setup_dir, 'buildtools-downloads/{}'.format(time.strftime("%Y%m%d%H%M%S")))
+    buildtools_download_dir = add_unique_timestamp_to_path(os.path.join(args.setup_dir, 'buildtools-downloads/buildtools'))
     logger.plain("Buildtools archive is downloaded into {} and its content installed into {}".format(buildtools_download_dir, buildtools_install_dir))
     subprocess.check_call("{} -d {} --downloads-directory {}".format(install_buildtools, buildtools_install_dir, buildtools_download_dir), shell=True)
 
Signature · 1 line
2.47.3
#2 lore raw

Re: [bitbake-devel] [PATCH 2/5] bitbake-setup: ensure paths with timestamps in them are unique

Hi Alex,

Quoted 11 lines · Alexander Kanavin via lists.openembedded.org

From: Alexander Kanavin <[email protected]>

There have been instances where directories with timestamps in them are created within the same second, and their paths clash as the timestamp in them is the same for both, as the timestamp is accurate to 1 second.

This adds a check for the directory existence and then adds an ever-increasing counter until there's a path that isn't yet created.

That's just making the race window smaller but it still exists.

Quoted 16 lines

[YOCTO #16121]

---
  bin/bitbake-setup | 18 ++++++++++++++----
  1 file changed, 14 insertions(+), 4 deletions(-)
bin/bitbake-setup +1 -0
index 06255b112..4ccf47e87 100755
@@ -93,6 +93,17 @@ def _write_layer_list(dest, repodirs):
      with open(layers_f, 'w') as f:
          json.dump({"version":"1.0","layers":layers}, f, sort_keys=True, indent=4)
 
+def add_unique_timestamp_to_path(path):

bin/bitbake-setup def _write_layer_list(dest, repodirs): · line 106

Quoted 10 lines
bin/bitbake-setup +10 -0
+    timestamp = time.strftime("%Y%m%d%H%M%S")
+    path_unique = "{}.{}".format(path, timestamp)
+    if os.path.exists(path_unique):
+        import itertools
+        for i in itertools.count(start=1):
+            path_unique = "{}.{}.{}".format(path, timestamp, i)
+            if not os.path.exists(path_unique):
+                break
+    return path_unique
+

Let's say you have two instances of bitbake-setup running at the same time and entering this function.

They may both very well exit with the same path_unique as you only check for the presence of the directory and not create it, so they both will find .10 doesn't exist yet and decide to have this suffix.

The logic should be something like (untested) instead:

now = time.strftime("%Y%m%d%H%M%S")
for i in itertools.count(start=0):

    suffix = '' if i == 0 else f'.{i}'
    newpath = f'{path}.{now}{suffix}'
    try:
        os.mkdir(f'{newpath}')
    except FileExistsError:
        continue
    else:
        return newpath

Cheers,
Quentin

#3 lore raw

Re: [bitbake-devel] [PATCH 2/5] bitbake-setup: ensure paths with timestamps in them are unique

Quoted 2 lines · Quentin Schulz

Let's say you have two instances of bitbake-setup running at the same time and entering this function.

This patch is not fixing a race between parallel invocations. It fixes two *consecutive* invocations of bitbake-setup that happen within the same second. The commit message should be clearer, yes, but it's also already merged.

Parallel invocations of bitbake-setup that handle the same setup would step on each other in many different ways otherwise, and it's simply not supported at all.

Alex

#4 lore raw

PATCH 3/5 bitbake-setup: allow empty commits in configuration history repo

In reply to Alexander Kanavin

From: Alexander Kanavin <[email protected]>

This can happen when a user has checked out a different commit in a component under layers/. 'bitbake-setup update' will reset to the commit in the configuration, but there will be no changes to it, and so no difference in the config history repo to be committed.

---
 bin/bitbake-setup | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
bin/bitbake-setup +1 -1
index 4ccf47e87..93c11bd09 100755
@@ -81,7 +81,7 @@ def write_sources_fixed_revisions(config_dir, layer_dir, config_data):
 
 def commit_config(config_dir):
     bb.process.run("git -C {} add .".format(config_dir))
-    bb.process.run("git -C {} commit --no-verify -a -m 'Configuration at {}'".format(config_dir, time.asctime()))
+    bb.process.run("git -C {} commit --allow-empty --no-verify -a -m 'Configuration at {}'".format(config_dir, time.asctime()))
 
 def _write_layer_list(dest, repodirs):
     layers = []
Signature · 1 line
2.47.3
#5 lore raw

PATCH 4/5 bitbake-setup: correct several scenarios in layer updates

In reply to Alexander Kanavin

From: Alexander Kanavin <[email protected]>

There were untested scenarios in layer updates which did not work properly.

The most serious of them is updating a remote layer which had user-made local modifications (e.g. newly made local commits or edited files without a commit). Bitbake-setup relies on git fetcher's unpack operation which simply wipes it all out. Yes, the user should've used -L option to symlink a local clone from elsewhere; this advice doesn't help when their work is gone.

To address this, a git hook is added that prevents making commits in checkouts under layers/ that are managed by bitbake-setup. The hook directs users to clone the layer they'd like to modify separately and then use 'init -L' option.

(note: later on this workflow can be streamlined by converting parts of an existing setup to local sources, or perhaps ability to 'detach' parts of layers/ from bitbake-setup's modifications, so that it's not necessary to do a full re-init).

This still does not cover local modifications without making commits, so there's also a function that checks if such modifications have been
made, and a code that moves the layer into a backup location if the function returns true. I considered asking the user what to do (similar to bitbake config handling), but this would've resulted in a half-updated, possibly broken layer set which ultimately just would be more confusing.

The local modification check is a part of a broader newly added logic block that handles already existing sources, and in particular addresses yet another
issue: when a local source replaces a git remote or another local source, this scenario failed with 'file exists' errors (thanks to Anibal Limon for spotting that). In this case the original symlink is removed first. If the original is a git checkout, it is backed up if there are local modifications, or simply removed otherwise.

---
 bin/bitbake-setup | 47 ++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 44 insertions(+), 3 deletions(-)
bin/bitbake-setup +44 -3
index 93c11bd09..abe7614c8 100755
@@ -104,7 +104,7 @@ def add_unique_timestamp_to_path(path):
                 break
     return path_unique
 
-def checkout_layers(layers, layerdir, d):
+def checkout_layers(layers, confdir, layerdir, d):
     def _checkout_git_remote(r_remote, repodir, layers_fixed_revisions):
         rev = r_remote['rev']
         branch = r_remote.get('branch', None)
@@ -128,6 +128,31 @@ def checkout_layers(layers, layerdir, d):
         logger.plain("Making a symbolic link {} pointing to {}".format(dst, src))
         os.symlink(src, dst)
 
+    def _has_local_modifications(r_name, r_path):
+        fixed_revisions = json.load(open(os.path.join(confdir, "sources-fixed-revisions.json")))
+        rev = fixed_revisions['sources'][r_name]['git-remote']['rev']
+        status = bb.process.run('git -C {} status --porcelain'.format(r_path))[0]
+        if status:
+            return True
+        diff = bb.process.run('git -C {} diff {}'.format(r_path, rev))[0]
+        if diff:
+            return True
+        return False
+
+    def _restrict_commits(r_name, r_path):
+        hook_path = os.path.join(r_path, '.git', 'hooks', 'pre-commit')
+        restrict_hook = """#!/bin/sh
+echo "This repository is managed by bitbake-setup, and making commits is restricted.
+If you wish to make local modifications, clone it separately, and re-initialize using
+bitbake-setup init -L {} /path/to/repo/checkout"
+exit 1
+""".format(r_name)
+        with open(hook_path, 'w') as f:
+            f.write(restrict_hook)
+        import stat
+        st = os.stat(hook_path)
+        os.chmod(hook_path, st.st_mode | stat.S_IEXEC)
+
     layers_fixed_revisions = copy.deepcopy(layers)
     repodirs = []
     oesetupbuild = None
@@ -141,10 +166,26 @@ def checkout_layers(layers, layerdir, d):
         r_local = r_data.get('local')
         if r_remote and r_local:
             raise Exception("Source {} contains both git-remote and local properties.".format(r_name))
+
+        repodir_path = os.path.join(layerdir, repodir)
+        if os.path.lexists(repodir_path):
+            if os.path.islink(repodir_path):
+                os.remove(repodir_path)
+            elif _has_local_modifications(r_name, repodir_path):
+                backup_path = add_unique_timestamp_to_path(repodir_path + '-backup')
+                logger.warning("""Source {} in {} contains local modifications. Renaming to {} to preserve them.
+For local development work it is recommended to clone the needed layers separately and re-initialize using -L option:
+bitbake-setup init -L {} /path/to/repo/checkout""".format(
+                    r_name, repodir_path, backup_path, r_name))
+                os.rename(repodir_path, backup_path)
+            else:
+                shutil.rmtree(repodir_path)
+
         if r_remote:
             _checkout_git_remote(r_remote, repodir, layers_fixed_revisions)
+            _restrict_commits(r_name, repodir_path)
         if r_local:
-            _symlink_local(os.path.expanduser(r_local["path"]), os.path.join(layerdir,repodir))
+            _symlink_local(os.path.expanduser(r_local["path"]), repodir_path)
 
         if os.path.exists(os.path.join(layerdir, repodir, 'scripts/oe-setup-build')):
             oesetupbuild = os.path.join(layerdir, repodir, 'scripts/oe-setup-build')
@@ -354,7 +395,7 @@ def merge_overrides_into_sources(sources, overrides):
 
 def update_build(config, confdir, setupdir, layerdir, d, update_bb_conf="prompt"):
     layer_config = merge_overrides_into_sources(config["data"]["sources"], config["source-overrides"]["sources"])
-    sources_fixed_revisions = checkout_layers(layer_config, layerdir, d)
+    sources_fixed_revisions = checkout_layers(layer_config, confdir, layerdir, d)
     bitbake_config = config["bitbake-config"]
     thisdir = os.path.dirname(config["path"]) if config["type"] == 'local' else None
     setup_bitbake_build(bitbake_config, layerdir, setupdir, thisdir, update_bb_conf)
Signature · 1 line
2.47.3
#6 lore raw

PATCH 5/5 bitbake-setup: add tests for update scenarios fixed in the previous commit

In reply to Alexander Kanavin

From: Alexander Kanavin <[email protected]>

This adds tests for the issues fixed in the previous commits:
- transitions in sources from remote to local, from local to local and from local to remote
- ensuring that local commits in sources are restricted
- ensuring that source backups do happen when that is expected.

Add a helper function that adds a test configuration with a local source in it.

---
 lib/bb/tests/setup.py | 68 ++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 67 insertions(+), 1 deletion(-)
lib/bb/tests/setup.py +67 -1
index 438dc0cd8..834d09854 100644
@@ -175,6 +175,16 @@ print("BBPATH is {{}}".format(os.environ["BBPATH"]))
 """ % (self.testrepopath, branch, rev)
         return self._add_json_config_to_registry_helper(name, sources)
 
+    def add_local_json_config_to_registry(self, name, path):
+        sources = """
+        "test-repo": {
+            "local": {
+                "path": "%s"
+            }
+        }
+""" % (path)
+        return self._add_json_config_to_registry_helper(name, sources)
+
     def add_file_to_testrepo(self, name, content, script=False):
         fullname = os.path.join(self.testrepopath, name)
         os.makedirs(os.path.join(self.testrepopath, os.path.dirname(name)), exist_ok=True)
@@ -413,7 +423,6 @@ print("BBPATH is {{}}".format(os.environ["BBPATH"]))
             sums_after = _conf_chksum(f"{setuppath}/build/conf")
             self.assertEqual(sums_before, sums_after)
 
-        # check source overrides, local sources provided with symlinks, and custom setup dir name
         def _check_local_sources(custom_setup_dir):
             custom_setup_path = os.path.join(self.tempdir, 'bitbake-builds', custom_setup_dir)
             custom_layer_path = os.path.join(custom_setup_path, 'layers', 'test-repo')
@@ -421,6 +430,63 @@ print("BBPATH is {{}}".format(os.environ["BBPATH"]))
             self.assertEqual(self.testrepopath, os.path.realpath(custom_layer_path))
             self.config_is_unchanged(custom_setup_path)
 
+        # Change the configuration to refer to a local source, then to another local source, then back to a git remote
+        # Run status/update after each change and verify that nothing breaks
+        c = 'gadget'
+        setuppath = self.get_setup_path('test-config-1', c)
+        self.config_is_unchanged(setuppath)
+
+        json_1 = self.add_local_json_config_to_registry('test-config-1.conf.json', self.testrepopath)
+        os.environ['BBPATH'] = os.path.join(setuppath, 'build')
+        out = self.runbbsetup("update --update-bb-conf='yes'")
+        _check_local_sources(setuppath)
+
+        prev_path = self.testrepopath
+        self.testrepopath = prev_path + "-2"
+        self.git("clone {} {}".format(prev_path, self.testrepopath), cwd=self.tempdir)
+        json_1 = self.add_local_json_config_to_registry('test-config-1.conf.json', self.testrepopath)
+        os.environ['BBPATH'] = os.path.join(setuppath, 'build')
+        out = self.runbbsetup("update --update-bb-conf='yes'")
+        _check_local_sources(setuppath)
+
+        self.testrepopath = prev_path
+        json_1 = self.add_json_config_to_registry('test-config-1.conf.json', branch, branch)
+        os.environ['BBPATH'] = os.path.join(setuppath, 'build')
+        out = self.runbbsetup("update --update-bb-conf='yes'")
+        self.check_setupdir_files(setuppath, test_file_content)
+
+        # Also check that there are no layer backups up to this point, then make a change that should
+        # result in a layer backup, and check that it does happen.
+        def _check_layer_backups(layer_path, expected_backups):
+            files = os.listdir(layer_path)
+            backups = len([f for f in files if 'backup' in f])
+            self.assertEqual(backups, expected_backups, msg = "Expected {} layer backups, got {}, directory listing: {}".format(expected_backups, backups, files))
+
+        layers_path = os.path.join(setuppath, 'layers')
+        layer_path = os.path.join(layers_path, 'test-repo')
+        _check_layer_backups(layers_path, 0)
+
+        ## edit a file without making a commit
+        with open(os.path.join(layer_path, 'local-modification'), 'w') as f:
+            f.write('locally-modified\n')
+        test_file_content = "modified-again\n"
+        self.add_file_to_testrepo('test-file', test_file_content)
+        os.environ['BBPATH'] = os.path.join(setuppath, 'build')
+        out = self.runbbsetup("update --update-bb-conf='yes'")
+        _check_layer_backups(layers_path, 1)
+
+        ## edit a file and try to make a commit; this should be rejected
+        with open(os.path.join(layer_path, 'local-modification'), 'w') as f:
+            f.write('locally-modified-again\n')
+        self.git('add .', cwd=layer_path)
+        with self.assertRaisesRegex(bb.process.ExecutionError, "making commits is restricted"):
+            self.git('commit -m "Adding a local modification"', cwd=layer_path)
+        test_file_content = "modified-again-and-again\n"
+        self.add_file_to_testrepo('test-file', test_file_content)
+        out = self.runbbsetup("update --update-bb-conf='yes'")
+        _check_layer_backups(layers_path, 2)
+
+        # check source overrides, local sources provided with symlinks, and custom setup dir name
         source_override_content = """
 {
     "sources": {
Signature · 1 line
2.47.3
#7 lore raw
In reply to Alexander Kanavin

Hi Alex,

Quoted 12 lines · Alexander Kanavin via lists.openembedded.org

From: Alexander Kanavin <[email protected]>

---
  .../bitbake-user-manual-environment-setup.rst | 48 +++++++++++++++++++
  1 file changed, 48 insertions(+)
doc/bitbake-user-manual/bitbake-user-manual-environment-setup.rst +0 -0
index ec1bdeecd..fcffab812 100644

doc/bitbake-user-manual/bitbake-user-manual-environment-setup.rst Here is an example file that overrides the branch of the BitBake repository to · line 943

Quoted 6 lines
doc/bitbake-user-manual/bitbake-user-manual-environment-setup.rst +3 -0
@@ -938,3 +938,51 @@ Here is an example file that overrides the branch of the BitBake repository to
  -  The ``sources`` section contains the same options as the ``sources`` option
     of a :term:`Generic Configuration` file. See the
     :ref:`ref-bbsetup-section-config-reference` section for more information.
+
+.. _ref-bbsetup-fixed-revisions:
+

We have autosectionlabel sphinx extension enabled, so this shouldn't be necessary. I know Antonin's got some opinion on when to use manual labels vs autolabels so up to him to decide. Would be nice to have some sort of rule documented somewhere :)

doc/bitbake-user-manual/bitbake-user-manual-environment-setup.rst Here is an example file that overrides the branch of the BitBake repository to · line 947

Quoted 4 lines
doc/bitbake-user-manual/bitbake-user-manual-environment-setup.rst +4 -0
+Fixed source revisions
+======================
+
+:term:`Generic Configuration` can set source revisions in ``rev`` to a tag or a branch.

Will the tag be refetched every time or once it's available locally, if the tag is moved upstream it's not changed locally?

doc/bitbake-user-manual/bitbake-user-manual-environment-setup.rst Here is an example file that overrides the branch of the BitBake repository to · line 948

Quoted 1 line
doc/bitbake-user-manual/bitbake-user-manual-environment-setup.rst +1 -0
+Bitbake-setup will make sure the actual revision will match the tag or branch when performing

Do not capitalize bitbake-setup, it is a tool that is named like that. Maybe highlight it as such as well:

``bitbake-setup``

Quoted 5 lines
doc/bitbake-user-manual/bitbake-user-manual-environment-setup.rst +5 -0
+initializations or updates, and will capture the revisions in a :ref:`_ref-bbsetup-source-overrides`
+file.> +
+This file is named ``sources-fixed-revisions.json`` and is available in ``layers/``
+under the :term:`Setup` directory (as well as under ``config/`` for backwards compatibility).
+It can be used to keep record of what was checked out when using configurations that do not

doc/bitbake-user-manual/bitbake-user-manual-environment-setup.rst Here is an example file that overrides the branch of the BitBake repository to · line 964

Quoted 10 lines
doc/bitbake-user-manual/bitbake-user-manual-environment-setup.rst +10 -0
+specify exact revisions.
+
+Also, as it is an override file, it can be combined with the original configurations
+to initialize a setup in a reproducible way that guarantees an exact, never-changing
+set of revisions, by using the ``--source-overrides`` option of the
+:ref:`ref-bbsetup-command-init` command.
+
+For example if the original configuration had specified only a master branch
+for a source::
+

This is JSON I believe? So probably:

.. code-block:: json

instead?

Quoted 4 lines
doc/bitbake-user-manual/bitbake-user-manual-environment-setup.rst +4 -0
+     "bitbake": {
+         "git-remote": {
+             "remotes": {
+                 "origin": {

doc/bitbake-user-manual/bitbake-user-manual-environment-setup.rst Here is an example file that overrides the branch of the BitBake repository to · line 978

Quoted 10 lines
doc/bitbake-user-manual/bitbake-user-manual-environment-setup.rst +10 -0
+                     "uri": "https://git.openembedded.org/bitbake"
+                 }
+             },
+             "branch": "master",
+             "rev": "master"
+     }
+
+
+the override will contain the exact revisions::
+

Ditto.

Cheers,
Quentin

#8 lore raw
Quoted 7 lines · Quentin Schulz

doc/bitbake-user-manual/bitbake-user-manual-environment-setup.rst Here is an example file that overrides the branch of the BitBake repository to · line 947

Quoted 4 lines
doc/bitbake-user-manual/bitbake-user-manual-environment-setup.rst +4 -0
+Fixed source revisions
+======================
+
+:term:`Generic Configuration` can set source revisions in ``rev`` to a tag or a branch.

Will the tag be refetched every time or once it's available locally, if the tag is moved upstream it's not changed locally?

I have not verified that, and there's no specific test for it, but I think tags will be refetched, same way as branch references will also be refetched if there are new commits in them (for that there is a test).

This patch has already merged, so if you can do the fixups in a followup I'd appreciate (you can probably go over the entire document and check for consistency).

Alex

#9 lore raw
In reply to Quentin Schulz

Hi,

Quoted 10 lines · Quentin Schulz via lists.openembedded.org

Hi Alex,

Quoted 7 lines · Alexander Kanavin via lists.openembedded.org

From: Alexander Kanavin <[email protected]>

---
  .../bitbake-user-manual-environment-setup.rst | 48 +++++++++++++++++++
  1 file changed, 48 insertions(+)
Quoted 16 lines

doc/bitbake-user-manual/bitbake-user-manual-environment-setup.rst Here is an example file that overrides the branch of the BitBake repository to · line 943

Quoted 11 lines
doc/bitbake-user-manual/bitbake-user-manual-environment-setup.rst +3 -0
index ec1bdeecd..fcffab812 100644
@@ -938,3 +938,51 @@ Here is an example file that overrides the branch of the BitBake repository to
  -  The ``sources`` section contains the same options as the ``sources`` option
     of a :term:`Generic Configuration` file. See the
     :ref:`ref-bbsetup-section-config-reference` section for more information.
+
+.. _ref-bbsetup-fixed-revisions:
+

We have autosectionlabel sphinx extension enabled, so this shouldn't be necessary. I know Antonin's got some opinion on when to use manual labels vs autolabels so up to him to decide. Would be nice to have some sort of rule documented somewhere :)

I don't think having a custom ref prevents from using autosectionlabels, so I'm fine either way.

[...]

Quoted 12 lines

doc/bitbake-user-manual/bitbake-user-manual-environment-setup.rst Here is an example file that overrides the branch of the BitBake repository to · line 964

Quoted 9 lines
doc/bitbake-user-manual/bitbake-user-manual-environment-setup.rst +9 -0
+
+Also, as it is an override file, it can be combined with the original configurations
+to initialize a setup in a reproducible way that guarantees an exact, never-changing
+set of revisions, by using the ``--source-overrides`` option of the
+:ref:`ref-bbsetup-command-init` command.
+
+For example if the original configuration had specified only a master branch
+for a source::
+

This is JSON I believe? So probably:

Quoted 1 line

.. code-block:: json

Careful with that I think Sphinx is not happy when the underlying JSON is not valid JSON (missing comma or unmatched curly braces etc.)

Antonin

Signature · 3 lines
Antonin Godard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com