Package bitbake-setup for PyPI
v2 cover Package bitbake-setup for PyPI
Thanks for the feedback from the 1st series. I included all requested changes:
* Moved packaging files from top-level to contrib/pypi
* Created package-bitbake-setup.py to create a new workspace to build the package and be compliant with PEP 517
* Changed the title of patch #4 to start with bitbake-setup:
* Converted setup.cfg to pyproject.toml
* Remove old license classifiers
* Made AND operator uppercase for license entry
* Set the minimum Python requirement to 3.9
* Created a custom README.md and BUILD.md for bitbake-setup
* Changed the URL to https://git.openembedded.org/bitbake/
* Added support for nox to automate the packaging including linting, building, and testing
There was a question about the wildcards in the includes section. In this context they are not used for filename globbing and instead denote to include any Python sub-modules as well. See the Python documentation for details:
https://setupttools.pypa.io/en/latest/userguide/package_discovery.html
The bitbake-setup script is no longer being turned into a module in the git repository. Instead it is being done by the script. The rationale for doing this is that it:
* ensures the module works reliably on all platforms
* avoids shebang and path issues
* places nicely with wheels, venvs, and zipapps
* easier to test
I did make an attempt to ensure that pyproject.toml and noxfile.py are compatible with older tools. Building the wheel using the pyprojet.toml should be supported with pip >=19.
Wheels built newer versions of pip should remain backwards compatible with pip when installing on older systems.
Outstanding:
* Determine a release strategy and version scheme
* Document steps for publishing the module to PyPI
To run the automated tests all that should be required is to install the python3-nox package or nox Python module. Then run
nox -s lint
nox -s build
nox -s tests
Rob Woolley (10):
bitbake-setup: Add the conditional script stanza
bitbake-setup: Add pypi packaging
bitbake-setup: Remove extraneous variable from str.format()
bitbake-setup: Sort and format imports
bitbake-setup: Resolve unused loop control variables
bitbake-setup: Fix ambiguous variable names
bitbake-setup: Remove unused stdout variable
bitbake-setup: Replace f-string without placeholders
bitbake-setup: Remove unused Namespace import
bitbake-setup: Remove unused parser_settings_list
.gitignore | 1 + bin/bitbake-setup | 76 ++++++++++----------- contrib/pypi/BUILD.md | 57 ++++++++++++++++ contrib/pypi/LICENSE | 9 +++ contrib/pypi/README.md | 42 ++++++++++++ contrib/pypi/noxfile.py | 30 +++++++++ contrib/pypi/package-bitbake-setup.py | 93 +++++++++++++++++++++++++ contrib/pypi/pyproject.toml | 97 +++++++++++++++++++++++++++ contrib/pypi/tests/test_install.py | 17 +++++ 9 files changed, 383 insertions(+), 39 deletions(-) create mode 100644 contrib/pypi/BUILD.md create mode 100644 contrib/pypi/LICENSE create mode 100644 contrib/pypi/README.md create mode 100644 contrib/pypi/noxfile.py create mode 100755 contrib/pypi/package-bitbake-setup.py create mode 100644 contrib/pypi/pyproject.toml create mode 100644 contrib/pypi/tests/test_install.py
Signature · 1 line
2.49.0
PATCH v2 1/10 bitbake-setup: Add the conditional script stanza
This ensures that the main function is only executed if the module is executed in the top-level environment.
--- bin/bitbake-setup | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
bin/bitbake-setup +2 -1
@@ -1106,4 +1106,5 @@ def main(): from argparse import Namespace parser.print_help() -main() +if __name__ == '__main__': + main()
Signature · 1 line
2.49.0
PATCH v2 2/10 bitbake-setup: Add pypi packaging
In reply to Rob Woolley--- .gitignore | 1 + contrib/pypi/BUILD.md | 57 ++++++++++++++++ contrib/pypi/LICENSE | 9 +++ contrib/pypi/README.md | 42 ++++++++++++ contrib/pypi/noxfile.py | 30 +++++++++ contrib/pypi/package-bitbake-setup.py | 93 +++++++++++++++++++++++++ contrib/pypi/pyproject.toml | 97 +++++++++++++++++++++++++++ contrib/pypi/tests/test_install.py | 17 +++++ 8 files changed, 346 insertions(+) create mode 100644 contrib/pypi/BUILD.md create mode 100644 contrib/pypi/LICENSE create mode 100644 contrib/pypi/README.md create mode 100644 contrib/pypi/noxfile.py create mode 100755 contrib/pypi/package-bitbake-setup.py create mode 100644 contrib/pypi/pyproject.toml create mode 100644 contrib/pypi/tests/test_install.py
.gitignore +1 -0
@@ -17,3 +17,4 @@ lib/toaster/contrib/tts/backlog.txt lib/toaster/contrib/tts/log/* lib/toaster/contrib/tts/.cache/* lib/bb/tests/runqueue-tests/bitbake-cookerdaemon.log +packaging_workspace/
contrib/pypi/BUILD.md new file +57 -0
58 lines not on this page · raw message
contrib/pypi/LICENSE new file +9 -0
@@ -0,0 +1,9 @@ +bitbake-setup is licensed under the GNU General Public License version 2.0. See +LICENSE.GPL-2.0-only for further details. + +Individual files contain the following style tags instead of the full license text: + + SPDX-License-Identifier: GPL-2.0-only + +This enables machine processing of license information based on the SPDX +License Identifiers that are here available: http://spdx.org/licenses/
contrib/pypi/README.md new file +42 -0
43 lines not on this page · raw message
contrib/pypi/noxfile.py new file +30 -0
@@ -0,0 +1,30 @@ +#!/usr/bin/env python3 + +import glob +import nox +from pathlib import Path + +nox.options.sessions = ["lint", "tests", "build"] + [email protected] +def lint(session): + session.install("ruff", "mypy") + session.run("ruff", "check", "src/bitbake_setup") + # Ignore mypy for now as there are too many errors in imported modules + # session.run("mypy", "src/bitbake_setup") + [email protected] +def tests(session): + session.install(".[test]") + session.run("pytest") + [email protected] +def build(session): + session.install("build", "twine") + session.run("python", "-m", "build") + + # Verify the wheel installs cleanly + wheels = list(Path("dist").glob("*.whl")) + session.install(str(wheels[0])) + session.run("bitbake-setup", "--help") + session.run("bitbake-setup", "list")
contrib/pypi/package-bitbake-setup.py new file +93 -0
93 lines not on this page · raw message
contrib/pypi/pyproject.toml new file +97 -0
98 lines not on this page · raw message
contrib/pypi/tests/test_install.py new file +17 -0
@@ -0,0 +1,17 @@ +import subprocess +import sys + +def test_imports(): + import bb + import bs4 + import ply + import bitbake_setup + +def test_console_script(): + result = subprocess.run( + ["bitbake-setup", "--help"], + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + text=True, + ) + assert result.returncode == 0
Signature · 1 line
2.49.0
Re: [bitbake-devel] [PATCH v2 02/10] bitbake-setup: Add pypi packaging
Thanks! This does look much cleaner, my only concern is that there are no tests. Is it possible to produce the .whl such that the steps and the output are contained entirely in some temporary location, or (less good) bitbake repo itself, e.g. nothing gets written in $HOME? I see that contrib/pypi/package-bitbake-setup.py takes a -d parameter, so perhaps that could be used?
I'd really like to extend lib/bb/tests/setup.py with a new class (e.g. PypiTest) that does it.
Alex
Quoted 432 lines · Rob Woolley via lists.openembedded.org
Signed-off-by Rob Woolley <[email protected]>--- .gitignore | 1 + contrib/pypi/BUILD.md | 57 ++++++++++++++++ contrib/pypi/LICENSE | 9 +++ contrib/pypi/README.md | 42 ++++++++++++ contrib/pypi/noxfile.py | 30 +++++++++ contrib/pypi/package-bitbake-setup.py | 93 +++++++++++++++++++++++++ contrib/pypi/pyproject.toml | 97 +++++++++++++++++++++++++++ contrib/pypi/tests/test_install.py | 17 +++++ 8 files changed, 346 insertions(+) create mode 100644 contrib/pypi/BUILD.md create mode 100644 contrib/pypi/LICENSE create mode 100644 contrib/pypi/README.md create mode 100644 contrib/pypi/noxfile.py create mode 100755 contrib/pypi/package-bitbake-setup.py create mode 100644 contrib/pypi/pyproject.toml create mode 100644 contrib/pypi/tests/test_install.py.gitignore +1
-0@@ -17,3 +17,4 @@ lib/toaster/contrib/tts/backlog.txt lib/toaster/contrib/tts/log/* lib/toaster/contrib/tts/.cache/* lib/bb/tests/runqueue-tests/bitbake-cookerdaemon.log +packaging_workspace/contrib/pypi/BUILD.md new file +57
-0@@ -0,0 +1,57 @@ +# Development Instructions + +## Requirements + +- Python >= 3.9 +- pip >= 19 (for installation) +- git (for SCM-based versioning) + +## Automated testing + +The steps to lint, build, and test the bitbake-setup pypi packaging have been automated with the nox tool. This tool automatically creates a Python virtual environment for you. + +To run the linters do: +```bash +nox -s lint +``` + +To run the build do: +```bash +nox -s build +``` + +To run the tests do: +```bash +nox -s tests +``` + +## Manual Steps + +### Building + +To install the development tools manually run: +```bash +python3 -m pip install -e '.[dev]' +``` + +To build a wheel (.whl) then use: +```bash +python3 -m build +``` + +This produces a wheel (.whl) file in the dist directory. This may be installed using pip. + +### Testing + +```bash +python3 -m pip install -e '.[test]' +pytest +``` + +### Linting + +```bash +python3 -m pip install -e '.[lint]' + +ruff check src/bitbake_setup +```contrib/pypi/LICENSE new file +9
-0@@ -0,0 +1,9 @@ +bitbake-setup is licensed under the GNU General Public License version 2.0. See +LICENSE.GPL-2.0-only for further details. + +Individual files contain the following style tags instead of the full license text: + + SPDX-License-Identifier: GPL-2.0-only + +This enables machine processing of license information based on the SPDX +License Identifiers that are here available: http://spdx.org/licenses/contrib/pypi/README.md new file +42
-0@@ -0,0 +1,42 @@ +# bitbake-setup + +This package provides the `bitbake-setup` command and the Python modules +required to support BitBake setup and configuration. + +## Usage + +Instructions on uses of bitbake-setup can be found in +[Setting Up the Environment With bitbake-setup](https://docs.yoctoproject.org/bitbake/dev/bitbake-user-manual/bitbake-user-manual-environment-setup.html) from the Yocto Project manual. + +List the available configurations; +```bash +bitbake-setup list +``` + +Show the help for the bitbake-setup init subcommand: +```bash +bitbake-setup init --help +usage: bitbake-setup init [-h] [--non-interactive] [--source-overrides SOURCE_OVERRIDES] [--setup-dir-name SETUP_DIR_NAME] [--skip-selection SKIP_SELECTION] [-L SOURCE_NAME PATH] + [config ...] + +positional arguments: + config path/URL/id to a configuration file (use 'list' command to get available ids), followed by configuration options. Bitbake-setup will ask to choose from available + choices if command line doesn't completely specify them. + +options: + -h, --help show this help message and exit + --non-interactive Do not ask to interactively choose from available options; if bitbake-setup cannot make a decision it will stop with a failure. + --source-overrides SOURCE_OVERRIDES + Override sources information (repositories/revisions) with values from a local json file. + --setup-dir-name SETUP_DIR_NAME + A custom setup directory name under the top directory. + --skip-selection SKIP_SELECTION + Do not select and set an option/fragment from available choices; the resulting bitbake configuration may be incomplete. + -L SOURCE_NAME PATH, --use-local-source SOURCE_NAME PATH + Symlink local source into a build, instead of getting it as prescribed by a configuration (useful for local development). +``` + +To initialize a workspace for the Poky reference distro using the development branch (ie. "master") of OpenEmbedded: +```bash +bitbake-setup init poky-master +```contrib/pypi/noxfile.py new file +30
-0@@ -0,0 +1,30 @@ +#!/usr/bin/env python3 + +import glob +import nox +from pathlib import Path + +nox.options.sessions = ["lint", "tests", "build"] + [email protected] +def lint(session): + session.install("ruff", "mypy") + session.run("ruff", "check", "src/bitbake_setup") + # Ignore mypy for now as there are too many errors in imported modules + # session.run("mypy", "src/bitbake_setup") + [email protected] +def tests(session): + session.install(".[test]") + session.run("pytest") + [email protected] +def build(session): + session.install("build", "twine") + session.run("python", "-m", "build") + + # Verify the wheel installs cleanly + wheels = list(Path("dist").glob("*.whl")) + session.install(str(wheels[0])) + session.run("bitbake-setup", "--help") + session.run("bitbake-setup", "list")contrib/pypi/package-bitbake-setup.py new file +93
-0@@ -0,0 +1,93 @@ +#!/usr/bin/env python3 + +import argparse +import logging +import os +import shutil +from pathlib import Path + +def create_packaging_workspace(directory): + # Create the directory for packaging workspace + if len(directory or "") > 0: + workspace_dir = Path(directory) + else: + # This script is located in contrib/pypi/package-bitbake-setup.py + workspace_dir = Path(__file__).parents[2] / "packaging_workspace" + + if not workspace_dir.exists(): + logging.debug(f"Created packaging workspace at: {workspace_dir}") + workspace_dir.mkdir(exist_ok=True) + else: + logging.debug(f"Packaging workspace already exists at: {workspace_dir}") + + # Copy packaging files to the workspace + files_to_copy = [ + "contrib/pypi/pyproject.toml", + "contrib/pypi/README.md", + "contrib/pypi/LICENSE", + "contrib/pypi/noxfile.py", + "LICENSE.MIT", + "LICENSE.GPL-2.0-only" + ] + + for file in files_to_copy: + src_path = Path(__file__).parents[2] / file + dest_path = workspace_dir / Path(file).name + + if src_path.is_dir(): + shutil.copytree(src_path, dest_path, dirs_exist_ok=True) + logging.debug(f"Copied directory: {src_path} to {dest_path}") + else: + shutil.copy2(src_path, dest_path) + logging.debug(f"Copied file: {src_path} to {dest_path}") + + + # Copy necessary modules to the workspace + modules_to_copy = [ + "lib/bb", + "lib/bs4", + "lib/ply" + ] + + for module in modules_to_copy: + src_path = Path(__file__).parents[2] / module + dest_path = workspace_dir / "src" / Path(module).name + + if src_path.is_dir(): + shutil.copytree(src_path, dest_path, dirs_exist_ok=True) + logging.debug(f"Copied module directory: {src_path} to {dest_path}") + else: + shutil.copy2(src_path, dest_path) + logging.debug(f"Copied module file: {src_path} to {dest_path}") + + # Create bitbake_setup module + bitbake_setup_dir = Path(workspace_dir / "src" / "bitbake_setup") + bitbake_setup_dir.mkdir(exist_ok=True) + Path(bitbake_setup_dir / "__init__.py").touch() + shutil.copy2(Path(__file__).parents[2] / "bin" / "bitbake-setup", str(bitbake_setup_dir / "__main__.py")) + + # Create codegen module + codegen_dir = Path(workspace_dir / "src" / "codegen") + codegen_dir.mkdir(exist_ok=True) + Path(codegen_dir / "__init__.py").touch() + shutil.copy2(Path(__file__).parents[2] / "lib" / "codegen.py", str(codegen_dir / "__main__.py")) + + # Copy tests + tests_dir = Path(workspace_dir / "tests") + tests_dir.mkdir(exist_ok=True) + shutil.copytree(Path(__file__).parents[2] / "contrib" / "pypi" / "tests", str(tests_dir), dirs_exist_ok=True) + + +if __name__ == "__main__": + logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s') + + parser = argparse.ArgumentParser(description='Package bitbake-setup for PyPI') + parser.add_argument('-v', '--verbose', action='store_true', help='increase output verbosity.') + parser.add_argument('-d', '--directory', type=str, help='specify the directory to create the packaging workspace in.') + + args = parser.parse_args(); + + if args.verbose: + logging.getLogger().setLevel(logging.DEBUG) + + create_packaging_workspace(args.directory);contrib/pypi/pyproject.toml new file +97
-0@@ -0,0 +1,97 @@ +[build-system] +requires = [ + "setuptools>=64", + "setuptools_scm[toml]>=8", +] +build-backend = "setuptools.build_meta" + +[project] +name = "bitbake-setup" +dynamic = ["version"] +description = "bitbake-setup" +readme = "README.md" +requires-python = ">=3.9" +license = "GPL-2.0-only AND MIT" +authors = [ + { name = "OpenEmbedded BitBake Developers", email = "[email protected]" } +] +classifiers = [ + "Programming Language :: Python :: 3", + "Operating System :: POSIX :: Linux", +] + +dependencies = [ + # BitBake is mostly self-contained +] + +[project.optional-dependencies] +test = [ + "pytest>=7", +] + +lint = [ + "ruff>=0.3", + "mypy>=1.8", +] + +dev = [ + "nox>=2022.11.21", + "pytest>=7", + "ruff>=0.3", + "mypy>=1.8", + "build", + "twine", +] + +[project.scripts] +bitbake-setup = "bitbake_setup.__main__:main" + +[project.urls] +Homepage = "https://git.openembedded.org/bitbake/" +Documentation = "https://docs.yoctoproject.org/bitbake/dev/bitbake-user-manual/" +Repository = "https://git.openembedded.org/bitbake/" + +[tool.mypy] +python_version = "3.9" +warn_unused_configs = true +ignore_missing_imports = true +no_implicit_optional = true +check_untyped_defs = false + +[tool.ruff] +target-version = "py39" +line-length = 88 +src = ["src"] + +[tool.ruff.lint] +select = [ + "E", # pycodestyle + "F", # pyflakes + "B", # flake8-bugbear + "I", # import sorting +] +ignore = [ + "E501", # line length (handled by formatter) +] + +[tool.ruff.format] +quote-style = "double" +indent-style = "space" + +[tool.setuptools] +package-dir = { "" = "src" } +zip-safe = false +include-package-data = true + +[tool.setuptools.packages.find] +where = ["src"] +include = [ + "bb*", + "bitbake_setup", + "bs4*", + "codegen", + "ply" +] + +[tool.setuptools_scm] +fallback_version = "0.0.0"contrib/pypi/tests/test_install.py new file +17
-1@@ -0,0 +1,17 @@ +import subprocess +import sys + +def test_imports(): + import bb + import bs4 + import ply + import bitbake_setup + +def test_console_script(): + result = subprocess.run( + ["bitbake-setup", "--help"], + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + text=True, + ) + assert result.returncode == 0 --2.49.0
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18891): https://lists.openembedded.org/g/bitbake-devel/message/18891
Mute This Topic: https://lists.openembedded.org/mt/117490846/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [bitbake-devel] [PATCH v2 02/10] bitbake-setup: Add pypi packaging
Hi Alex,
Sure, I can look into extending the lib/bb/tests/setup.py.
There are system tests using pytest and nox. The 2 pytests confirm that all the imports succeed and that bitbake-setup --help returns 0.
Nox runs the pytests as well as bitbake-setup --help and bitbake-setup list.
Should I replace the nox tests with pytest or create new tests?
Regards,
Rob
Get Outlook for Android<https://aka.ms/AAb9ysg>
________________________________
From: Alexander Kanavin <[email protected]>
Sent: Wednesday, January 28, 2026 7:26:46 AM
To: Woolley, Rob <[email protected]>
Cc: [email protected] <[email protected]>
Subject: Re: [bitbake-devel] [PATCH v2 02/10] bitbake-setup: Add pypi packaging
CAUTION: This email comes from a non Wind River email account!
Do not click links or open attachments unless you recognize the sender and know the content is safe.
Thanks! This does look much cleaner, my only concern is that there are no tests. Is it possible to produce the .whl such that the steps and the output are contained entirely in some temporary location, or (less good) bitbake repo itself, e.g. nothing gets written in $HOME? I see that contrib/pypi/package-bitbake-setup.py takes a -d parameter, so perhaps that could be used?
I'd really like to extend lib/bb/tests/setup.py with a new class (e.g. PypiTest) that does it.
Alex
Quoted 432 lines · Rob Woolley via lists.openembedded.org
Signed-off-by Rob Woolley <[email protected]>--- .gitignore | 1 + contrib/pypi/BUILD.md | 57 ++++++++++++++++ contrib/pypi/LICENSE | 9 +++ contrib/pypi/README.md | 42 ++++++++++++ contrib/pypi/noxfile.py | 30 +++++++++ contrib/pypi/package-bitbake-setup.py | 93 +++++++++++++++++++++++++ contrib/pypi/pyproject.toml | 97 +++++++++++++++++++++++++++ contrib/pypi/tests/test_install.py | 17 +++++ 8 files changed, 346 insertions(+) create mode 100644 contrib/pypi/BUILD.md create mode 100644 contrib/pypi/LICENSE create mode 100644 contrib/pypi/README.md create mode 100644 contrib/pypi/noxfile.py create mode 100755 contrib/pypi/package-bitbake-setup.py create mode 100644 contrib/pypi/pyproject.toml create mode 100644 contrib/pypi/tests/test_install.py.gitignore +1
-0@@ -17,3 +17,4 @@ lib/toaster/contrib/tts/backlog.txt lib/toaster/contrib/tts/log/* lib/toaster/contrib/tts/.cache/* lib/bb/tests/runqueue-tests/bitbake-cookerdaemon.log +packaging_workspace/contrib/pypi/BUILD.md new file +57
-0@@ -0,0 +1,57 @@ +# Development Instructions + +## Requirements + +- Python >= 3.9 +- pip >= 19 (for installation) +- git (for SCM-based versioning) + +## Automated testing + +The steps to lint, build, and test the bitbake-setup pypi packaging have been automated with the nox tool. This tool automatically creates a Python virtual environment for you. + +To run the linters do: +```bash +nox -s lint +``` + +To run the build do: +```bash +nox -s build +``` + +To run the tests do: +```bash +nox -s tests +``` + +## Manual Steps + +### Building + +To install the development tools manually run: +```bash +python3 -m pip install -e '.[dev]' +``` + +To build a wheel (.whl) then use: +```bash +python3 -m build +``` + +This produces a wheel (.whl) file in the dist directory. This may be installed using pip. + +### Testing + +```bash +python3 -m pip install -e '.[test]' +pytest +``` + +### Linting + +```bash +python3 -m pip install -e '.[lint]' + +ruff check src/bitbake_setup +```contrib/pypi/LICENSE new file +9
-0@@ -0,0 +1,9 @@ +bitbake-setup is licensed under the GNU General Public License version 2.0. See +LICENSE.GPL-2.0-only for further details. + +Individual files contain the following style tags instead of the full license text: + + SPDX-License-Identifier: GPL-2.0-only + +This enables machine processing of license information based on the SPDX +License Identifiers that are here available: http://spdx.org/licenses/contrib/pypi/README.md new file +42
-0@@ -0,0 +1,42 @@ +# bitbake-setup + +This package provides the `bitbake-setup` command and the Python modules +required to support BitBake setup and configuration. + +## Usage + +Instructions on uses of bitbake-setup can be found in +[Setting Up the Environment With bitbake-setup](https://docs.yoctoproject.org/bitbake/dev/bitbake-user-manual/bitbake-user-manual-environment-setup.html) from the Yocto Project manual. + +List the available configurations; +```bash +bitbake-setup list +``` + +Show the help for the bitbake-setup init subcommand: +```bash +bitbake-setup init --help +usage: bitbake-setup init [-h] [--non-interactive] [--source-overrides SOURCE_OVERRIDES] [--setup-dir-name SETUP_DIR_NAME] [--skip-selection SKIP_SELECTION] [-L SOURCE_NAME PATH] + [config ...] + +positional arguments: + config path/URL/id to a configuration file (use 'list' command to get available ids), followed by configuration options. Bitbake-setup will ask to choose from available + choices if command line doesn't completely specify them. + +options: + -h, --help show this help message and exit + --non-interactive Do not ask to interactively choose from available options; if bitbake-setup cannot make a decision it will stop with a failure. + --source-overrides SOURCE_OVERRIDES + Override sources information (repositories/revisions) with values from a local json file. + --setup-dir-name SETUP_DIR_NAME + A custom setup directory name under the top directory. + --skip-selection SKIP_SELECTION + Do not select and set an option/fragment from available choices; the resulting bitbake configuration may be incomplete. + -L SOURCE_NAME PATH, --use-local-source SOURCE_NAME PATH + Symlink local source into a build, instead of getting it as prescribed by a configuration (useful for local development). +``` + +To initialize a workspace for the Poky reference distro using the development branch (ie. "master") of OpenEmbedded: +```bash +bitbake-setup init poky-master +```contrib/pypi/noxfile.py new file +30
-0@@ -0,0 +1,30 @@ +#!/usr/bin/env python3 + +import glob +import nox +from pathlib import Path + +nox.options.sessions = ["lint", "tests", "build"] + [email protected] +def lint(session): + session.install("ruff", "mypy") + session.run("ruff", "check", "src/bitbake_setup") + # Ignore mypy for now as there are too many errors in imported modules + # session.run("mypy", "src/bitbake_setup") + [email protected] +def tests(session): + session.install(".[test]") + session.run("pytest") + [email protected] +def build(session): + session.install("build", "twine") + session.run("python", "-m", "build") + + # Verify the wheel installs cleanly + wheels = list(Path("dist").glob("*.whl")) + session.install(str(wheels[0])) + session.run("bitbake-setup", "--help") + session.run("bitbake-setup", "list")contrib/pypi/package-bitbake-setup.py new file +93
-0@@ -0,0 +1,93 @@ +#!/usr/bin/env python3 + +import argparse +import logging +import os +import shutil +from pathlib import Path + +def create_packaging_workspace(directory): + # Create the directory for packaging workspace + if len(directory or "") > 0: + workspace_dir = Path(directory) + else: + # This script is located in contrib/pypi/package-bitbake-setup.py + workspace_dir = Path(__file__).parents[2] / "packaging_workspace" + + if not workspace_dir.exists(): + logging.debug(f"Created packaging workspace at: {workspace_dir}") + workspace_dir.mkdir(exist_ok=True) + else: + logging.debug(f"Packaging workspace already exists at: {workspace_dir}") + + # Copy packaging files to the workspace + files_to_copy = [ + "contrib/pypi/pyproject.toml", + "contrib/pypi/README.md", + "contrib/pypi/LICENSE", + "contrib/pypi/noxfile.py", + "LICENSE.MIT", + "LICENSE.GPL-2.0-only" + ] + + for file in files_to_copy: + src_path = Path(__file__).parents[2] / file + dest_path = workspace_dir / Path(file).name + + if src_path.is_dir(): + shutil.copytree(src_path, dest_path, dirs_exist_ok=True) + logging.debug(f"Copied directory: {src_path} to {dest_path}") + else: + shutil.copy2(src_path, dest_path) + logging.debug(f"Copied file: {src_path} to {dest_path}") + + + # Copy necessary modules to the workspace + modules_to_copy = [ + "lib/bb", + "lib/bs4", + "lib/ply" + ] + + for module in modules_to_copy: + src_path = Path(__file__).parents[2] / module + dest_path = workspace_dir / "src" / Path(module).name + + if src_path.is_dir(): + shutil.copytree(src_path, dest_path, dirs_exist_ok=True) + logging.debug(f"Copied module directory: {src_path} to {dest_path}") + else: + shutil.copy2(src_path, dest_path) + logging.debug(f"Copied module file: {src_path} to {dest_path}") + + # Create bitbake_setup module + bitbake_setup_dir = Path(workspace_dir / "src" / "bitbake_setup") + bitbake_setup_dir.mkdir(exist_ok=True) + Path(bitbake_setup_dir / "__init__.py").touch() + shutil.copy2(Path(__file__).parents[2] / "bin" / "bitbake-setup", str(bitbake_setup_dir / "__main__.py")) + + # Create codegen module + codegen_dir = Path(workspace_dir / "src" / "codegen") + codegen_dir.mkdir(exist_ok=True) + Path(codegen_dir / "__init__.py").touch() + shutil.copy2(Path(__file__).parents[2] / "lib" / "codegen.py", str(codegen_dir / "__main__.py")) + + # Copy tests + tests_dir = Path(workspace_dir / "tests") + tests_dir.mkdir(exist_ok=True) + shutil.copytree(Path(__file__).parents[2] / "contrib" / "pypi" / "tests", str(tests_dir), dirs_exist_ok=True) + + +if __name__ == "__main__": + logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s') + + parser = argparse.ArgumentParser(description='Package bitbake-setup for PyPI') + parser.add_argument('-v', '--verbose', action='store_true', help='increase output verbosity.') + parser.add_argument('-d', '--directory', type=str, help='specify the directory to create the packaging workspace in.') + + args = parser.parse_args(); + + if args.verbose: + logging.getLogger().setLevel(logging.DEBUG) + + create_packaging_workspace(args.directory);contrib/pypi/pyproject.toml new file +97
-0@@ -0,0 +1,97 @@ +[build-system] +requires = [ + "setuptools>=64", + "setuptools_scm[toml]>=8", +] +build-backend = "setuptools.build_meta" + +[project] +name = "bitbake-setup" +dynamic = ["version"] +description = "bitbake-setup" +readme = "README.md" +requires-python = ">=3.9" +license = "GPL-2.0-only AND MIT" +authors = [ + { name = "OpenEmbedded BitBake Developers", email = "[email protected]" } +] +classifiers = [ + "Programming Language :: Python :: 3", + "Operating System :: POSIX :: Linux", +] + +dependencies = [ + # BitBake is mostly self-contained +] + +[project.optional-dependencies] +test = [ + "pytest>=7", +] + +lint = [ + "ruff>=0.3", + "mypy>=1.8", +] + +dev = [ + "nox>=2022.11.21", + "pytest>=7", + "ruff>=0.3", + "mypy>=1.8", + "build", + "twine", +] + +[project.scripts] +bitbake-setup = "bitbake_setup.__main__:main" + +[project.urls] +Homepage = "https://git.openembedded.org/bitbake/" +Documentation = "https://docs.yoctoproject.org/bitbake/dev/bitbake-user-manual/" +Repository = "https://git.openembedded.org/bitbake/" + +[tool.mypy] +python_version = "3.9" +warn_unused_configs = true +ignore_missing_imports = true +no_implicit_optional = true +check_untyped_defs = false + +[tool.ruff] +target-version = "py39" +line-length = 88 +src = ["src"] + +[tool.ruff.lint] +select = [ + "E", # pycodestyle + "F", # pyflakes + "B", # flake8-bugbear + "I", # import sorting +] +ignore = [ + "E501", # line length (handled by formatter) +] + +[tool.ruff.format] +quote-style = "double" +indent-style = "space" + +[tool.setuptools] +package-dir = { "" = "src" } +zip-safe = false +include-package-data = true + +[tool.setuptools.packages.find] +where = ["src"] +include = [ + "bb*", + "bitbake_setup", + "bs4*", + "codegen", + "ply" +] + +[tool.setuptools_scm] +fallback_version = "0.0.0"contrib/pypi/tests/test_install.py new file +17
-1@@ -0,0 +1,17 @@ +import subprocess +import sys + +def test_imports(): + import bb + import bs4 + import ply + import bitbake_setup + +def test_console_script(): + result = subprocess.run( + ["bitbake-setup", "--help"], + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + text=True, + ) + assert result.returncode == 0 --2.49.0
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18891): https://lists.openembedded.org/g/bitbake-devel/message/18891
Mute This Topic: https://lists.openembedded.org/mt/117490846/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [bitbake-devel] [PATCH v2 02/10] bitbake-setup: Add pypi packaging
Quoted 7 lines · Rob
Sure, I can look into extending the
lib/bb/tests/setup.py.There are system tests using pytest and nox. The 2 pytests confirm that all the imports succeed and that bitbake-setup --help returns 0.
Nox runs the pytests as well as bitbake-setup --help and bitbake-setup list.
Should I replace the nox tests with pytest or create new tests?
This is up to you. You can experiment and try different approaches. I think lib/bb/tests/setup.py is itself pytest based. It's a part of bitbake-selftest that you can run like this:
bin/bitbake-selftest -v bb.tests.setup
(you can further specify what you want to run in that suite, as existing tests for bitbake-setup itself take significant time, and you don't need to run them).
Alex
Re: [bitbake-devel] [PATCH v2 02/10] bitbake-setup: Add pypi packaging
In reply to Rob WoolleyHi,
On Tue Jan 27, 2026 at 5:04 PM CET, Rob Woolley via lists.openembedded.org wrote:
[...]
contrib/pypi/README.md line 4
Quoted 10 lines
contrib/pypi/README.md new file +4
-0@@ -0,0 +1,42 @@ +# bitbake-setup + +This package provides the `bitbake-setup` command and the Python modules +required to support BitBake setup and configuration.
s/BitBake setup/bitbake-setup/
contrib/pypi/README.md line 9
Quoted 5 lines
contrib/pypi/README.md new file +5
-0+ +## Usage + +Instructions on uses of bitbake-setup can be found in +[Setting Up the Environment With bitbake-setup](https://docs.yoctoproject.org/bitbake/dev/bitbake-user-manual/bitbake-user-manual-environment-setup.html) from the Yocto Project manual.
s/Yocto Project Manual/BitBake Manual/
contrib/pypi/README.md line 16
Quoted 7 lines
contrib/pypi/README.md new file +7
-0+ +List the available configurations; +```bash +bitbake-setup list +``` + +Show the help for the bitbake-setup init subcommand:
s/bitbake-setup/bitbake-setup/
Quoted 11 lines
contrib/pypi/README.md new file +11
-0+```bash +bitbake-setup init --help +usage: bitbake-setup init [-h] [--non-interactive] [--source-overrides SOURCE_OVERRIDES] [--setup-dir-name SETUP_DIR_NAME] [--skip-selection SKIP_SELECTION] [-L SOURCE_NAME PATH] + [config ...] + +positional arguments: + config path/URL/id to a configuration file (use 'list' command to get available ids), followed by configuration options. Bitbake-setup will ask to choose from available + choices if command line doesn't completely specify them. + +options: + -h, --help show this help message and exit
contrib/pypi/README.md line 37
Quoted 10 lines
contrib/pypi/README.md new file +10
-0+ --non-interactive Do not ask to interactively choose from available options; if bitbake-setup cannot make a decision it will stop with a failure. + --source-overrides SOURCE_OVERRIDES + Override sources information (repositories/revisions) with values from a local json file. + --setup-dir-name SETUP_DIR_NAME + A custom setup directory name under the top directory. + --skip-selection SKIP_SELECTION + Do not select and set an option/fragment from available choices; the resulting bitbake configuration may be incomplete. + -L SOURCE_NAME PATH, --use-local-source SOURCE_NAME PATH + Symlink local source into a build, instead of getting it as prescribed by a configuration (useful for local development). +```
I would avoid pasting a command output here, it's hard to maintain properly. Just leave bitbake-setup init --help is enough.
[...]
Quoted 8 lines
contrib/pypi/pyproject.toml new file +2
-0@@ -0,0 +1,97 @@ +[build-system] +requires = [
contrib/pypi/pyproject.toml line 12
Quoted 10 lines
contrib/pypi/pyproject.toml new file +10
-0+ "setuptools>=64", + "setuptools_scm[toml]>=8", +] +build-backend = "setuptools.build_meta" + +[project] +name = "bitbake-setup" +dynamic = ["version"] +description = "bitbake-setup" +readme = "README.md"
Wondering if it would be possible to include the doc/bitbake-user-manual/bitbake-user-manual-environment-setup.rst file here instead of a custom README?
We will likely forget about the above README.md file, and that's one more file to maintain. If the README needs to be next to the other file, maybe create a symlink to bitbake-user-manual-environment-setup.rst?
Quoted 29 lines
contrib/pypi/pyproject.toml new file +29
-0+requires-python = ">=3.9" +license = "GPL-2.0-only AND MIT" +authors = [ + { name = "OpenEmbedded BitBake Developers", email = "[email protected]" } +] +classifiers = [ + "Programming Language :: Python :: 3", + "Operating System :: POSIX :: Linux", +] + +dependencies = [ + # BitBake is mostly self-contained +] + +[project.optional-dependencies] +test = [ + "pytest>=7", +] + +lint = [ + "ruff>=0.3", + "mypy>=1.8", +] + +dev = [ + "nox>=2022.11.21", + "pytest>=7", + "ruff>=0.3", + "mypy>=1.8",
contrib/pypi/pyproject.toml line 51
Quoted 10 lines
contrib/pypi/pyproject.toml new file +10
-0+ "build", + "twine", +] + +[project.scripts] +bitbake-setup = "bitbake_setup.__main__:main" + +[project.urls] +Homepage = "https://git.openembedded.org/bitbake/" +Documentation = "https://docs.yoctoproject.org/bitbake/dev/bitbake-user-manual/"
Not sure we want to link to the development version of the documentation here. You could instead link to the latest released documentation:
https://docs.yoctoproject.org/bitbake/bitbake-user-manual/
Antonin
Signature · 3 lines
Antonin Godard, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
Re: [bitbake-devel] [PATCH v2 02/10] bitbake-setup: Add pypi packaging
Hi Antonin,
Thanks for the feedback. Sorry for the delay. I am about to push the changes Alex asked for so I am revisiing your comments.
Quoted 5 lines · Antonin Godard via lists.openembedded.org
Hi,
On Tue Jan 27, 2026 at 5:04 PM CET, Rob Woolley via lists.openembedded.org wrote:
[...]
Quoted 13 lines
contrib/pypi/README.md line 4
Quoted 10 lines
contrib/pypi/README.md new file +4
-0@@ -0,0 +1,42 @@ +# bitbake-setup + +This package provides the `bitbake-setup` command and the Python modules +required to support BitBake setup and configuration.s/BitBake setup/
bitbake-setup/
I don't believe this change is correct as the bitbake-setup module supports the setup and configuration of BitBake.
Otherwise we are saying that bitbake-setup supports setting up and configuring bitbake-setup.
Quoted 10 lines
contrib/pypi/README.md
Quoted 5 lines
contrib/pypi/README.md new file +5
-0+ +## Usage + +Instructions on uses of bitbake-setup can be found in +[Setting Up the Environment With bitbake-setup](https://docs.yoctoproject.org/bitbake/dev/bitbake-user-manual/bitbake-user-manual-environment-setup.html)
from the Yocto Project manual.s/Yocto Project Manual/BitBake Manual/
Thanks for pointing this out.
Quoted 11 lines
contrib/pypi/README.md line 16
Quoted 8 lines
contrib/pypi/README.md new file +7
-0+ +List the available configurations; +```bash +bitbake-setup list +``` + +Show the help for the bitbake-setup init subcommand:s/bitbake-setup/
bitbake-setup/
Thanks.
Quoted 29 lines
contrib/pypi/README.md
Quoted 4 lines
contrib/pypi/README.md new file +3
-0+```bash +bitbake-setup init --help +usage: bitbake-setup init [-h] [--non-interactive] [--source-overrides
SOURCE_OVERRIDES] [--setup-dir-nameSETUP_DIR_NAME] [--skip-selectionSKIP_SELECTION] [-LSOURCE_NAMEPATH]contrib/pypi/README.md
Quoted 4 lines
contrib/pypi/README.md new file +4
-0+ [config ...] + +positional arguments: + config path/URL/id to a configuration file (use 'list'command to get available ids), followed by configuration options. Bitbake-setup will ask to choose from available
contrib/pypi/README.md line 24
Quoted 1 line
contrib/pypi/README.md new file +1
-0+ choices if command line doesn't completelyspecify them.
contrib/pypi/README.md
Quoted 4 lines
contrib/pypi/README.md new file +4
-0+ +options: + -h, --help show this help message and exit + --non-interactive Do not ask to interactively choose fromavailable options; if bitbake-setup cannot make a decision it will stop with a failure.
contrib/pypi/README.md line 30
Quoted 2 lines
contrib/pypi/README.md new file +2
-0+ --source-overrides SOURCE_OVERRIDES + Override sources information(repositories/revisions) with values from a local json file.
contrib/pypi/README.md line 32
Quoted 2 lines
contrib/pypi/README.md new file +2
-0+ --setup-dir-name SETUP_DIR_NAME + A custom setup directory name under the topdirectory.
contrib/pypi/README.md line 34
Quoted 2 lines
contrib/pypi/README.md new file +2
-0+ --skip-selection SKIP_SELECTION + Do not select and set an option/fragment fromavailable choices; the resulting bitbake configuration may be incomplete.
Quoted 9 lines
contrib/pypi/README.md line 36
Quoted 2 lines
contrib/pypi/README.md new file +2
-0+ -L SOURCE_NAME PATH, --use-local-source SOURCE_NAME PATH + Symlink local source into a build, instead ofgetting it as prescribed by a configuration (useful for local development).
contrib/pypi/README.md line 37
Quoted 1 line
contrib/pypi/README.md new file +1
-0+```I would avoid pasting a command output here, it's hard to maintain properly.
Just leavebitbake-setup init --helpis enough.
Thanks.
Quoted 25 lines
[...]
Quoted 8 lines
contrib/pypi/pyproject.toml new file +2
-0@@ -0,0 +1,97 @@ +[build-system] +requires = [contrib/pypi/pyproject.toml line 12
Quoted 10 lines
contrib/pypi/pyproject.toml new file +10
-0+ "setuptools>=64", + "setuptools_scm[toml]>=8", +] +build-backend = "setuptools.build_meta" + +[project] +name = "bitbake-setup" +dynamic = ["version"] +description = "bitbake-setup" +readme = "README.md"Wondering if it would be possible to include the
doc/bitbake-user-manual/bitbake-user-manual-environment-setup.rstfile here instead of a custom README?
Quoted 6 lines
We will likely forget about the above README.md file, and that's one more file
to maintain. If the README needs to be next to the other file, maybe create a
symlink tobitbake-user-manual-environment-setup.rst?
That is fine by me. However, we will need to ensure that the .rst is validated and appropriate for PyPI:
https://packaging.python.org/en/latest/guides/making-a-pypi-friendly-readme/
When I look at that page now it doesn't seem tailored for PyPI:
https://github.com/openembedded/bitbake/blob/master/doc/bitbake-user-manual/bitbake-user-manual-environment-setup.rst
It seems to be > 25 pages long which seems too long. Can we create a condensed version with the basics and a link to the formal documentation?
Quoted 42 lines
contrib/pypi/pyproject.toml
Quoted 4 lines
contrib/pypi/pyproject.toml new file +4
-0+requires-python = ">=3.9" +license = "GPL-2.0-only AND MIT" +authors = [ + { name = "OpenEmbedded BitBake Developers", email = "Quoted 25 lines
contrib/pypi/pyproject.toml new file +25
-0+] +classifiers = [ + "Programming Language :: Python :: 3", + "Operating System :: POSIX :: Linux", +] + +dependencies = [ + # BitBake is mostly self-contained +] + +[project.optional-dependencies] +test = [ + "pytest>=7", +] + +lint = [ + "ruff>=0.3", + "mypy>=1.8", +] + +dev = [ + "nox>=2022.11.21", + "pytest>=7", + "ruff>=0.3", + "mypy>=1.8",contrib/pypi/pyproject.toml
Quoted 10 lines
contrib/pypi/pyproject.toml new file +10
-0+ "build", + "twine", +] + +[project.scripts] +bitbake-setup = "bitbake_setup.__main__:main" + +[project.urls] +Homepage = "https://git.openembedded.org/bitbake/" +Documentation = "https://docs.yoctoproject.org/bitbake/dev/bitbake-user-manual/"
Quoted 4 lines
Not sure we want to link to the development version of the documentation here.
You could instead link to the latest released documentation:
https://docs.yoctoproject.org/bitbake/bitbake-user-manual/
Thanks, good suggestion.
Quoted 20 lines
Antonin
--
Antonin Godard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#18919):
https://lists.openembedded.org/g/bitbake-devel/message/18919
Mute This Topic: https://lists.openembedded.org/mt/117490846/556952
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [bitbake-devel] [PATCH v2 02/10] bitbake-setup: Add pypi packaging
Hi,
Quoted 27 lines · Rob Woolley
Hi Antonin,
Thanks for the feedback. Sorry for the delay. I am about to push the changes Alex asked for so I am revisiing your comments.
Quoted 5 lines · Antonin Godard via lists.openembedded.org
Hi,
On Tue Jan 27, 2026 at 5:04 PM CET, Rob Woolley via lists.openembedded.org wrote:
[...]Quoted 13 lines
contrib/pypi/README.md line 4
Quoted 10 lines
contrib/pypi/README.md new file +4
-0@@ -0,0 +1,42 @@ +# bitbake-setup + +This package provides the `bitbake-setup` command and the Python modules +required to support BitBake setup and configuration.s/BitBake setup/
bitbake-setup/
Quoted 5 lines
I don't believe this change is correct as the bitbake-setup module supports the setup and configuration of BitBake.
Otherwise we are saying that bitbake-setup supports setting up and configuring bitbake-setup.
You are right, sorry for the noise
[...]
Quoted 16 lines
Quoted 10 lines
Wondering if it would be possible to include the
doc/bitbake-user-manual/bitbake-user-manual-environment-setup.rstfile here instead of a custom README?We will likely forget about the above README.md file, and that's one more file
to maintain. If the README needs to be next to the other file, maybe create a
symlink tobitbake-user-manual-environment-setup.rst?That is fine by me. However, we will need to ensure that the .rst is validated and appropriate for PyPI:
https://packaging.python.org/en/latest/guides/making-a-pypi-friendly-readme/
Quoted 5 lines
When I look at that page now it doesn't seem tailored for PyPI:
https://github.com/openembedded/bitbake/blob/master/doc/bitbake-user-manual/bitbake-user-manual-environment-setup.rstIt seems to be > 25 pages long which seems too long. Can we create a condensed version with the basics and a link to the formal documentation?
Yes, if the rst document does not conform with pypi, I would be very brief and link to the original documentation. We may do this differently in the future, but for now I think this is enough
(perhaps the "quick start" section of bitbake-user-manual-environment-setup.rst could be shared between pypi's docs and the official doc, with some rST includes
- but let's postpone for later).
Thanks!
Antonin
PATCH 1/1 bitbake-setup: Remove unused parser_settings_list
In reply to Rob WoolleyThe ruff lint tool detected an used variable:
F841 Local variable parser_settings_list is assigned to but never used
The list_settings function gets invoked through the parent command's settings_func function. This means that creating this variable is unnecessary.
--- bin/bitbake-setup | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
bin/bitbake-setup +1 -2
@@ -1026,8 +1026,7 @@ def main(): subparser_settings = parser_settings.add_subparsers(dest="subcommand", required=True, help="The action to perform on the settings file") - parser_settings_list = subparser_settings.add_parser('list', - help="List all settings with their values") + subparser_settings.add_parser('list', help="List all settings with their values") parser_settings_set = subparser_settings.add_parser('set', parents=[parser_settings_arg_global], help="In a Section, set a setting to a certain value")
Signature · 1 line
2.49.0
PATCH v2 3/10 bitbake-setup: Remove extraneous variable from str.format()
In reply to Rob WoolleyThe invocation of str.format() supplied an extra parameter that was not being referenced in the string.
--- bin/bitbake-setup | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
bin/bitbake-setup +1 -1
@@ -1093,7 +1093,7 @@ def main(): list_configs(all_settings, args) return - logger.info('Bitbake-setup is using {} as top directory.'.format(top_dir, global_settings_path(args))) + logger.info('Bitbake-setup is using {} as top directory.'.format(top_dir)) if args.func == init_config: init_config(top_dir, all_settings, args)
Signature · 1 line
2.49.0
PATCH v2 4/10 bitbake-setup: Sort and format imports
In reply to Rob WoolleyThe ruff linting tool reported 2 problems:
I001 Import block is un-sorted or un-formatted E402 Module level import not at top of file
--- bin/bitbake-setup | 25 ++++++++++++------------- 1 file changed, 12 insertions(+), 13 deletions(-)
bin/bitbake-setup +12 -13
@@ -4,29 +4,28 @@ # SPDX-License-Identifier: GPL-2.0-only # -import logging -import os -import sys import argparse -import json -import shutil -import time import configparser +import copy import datetime +import functools import glob -import subprocess -import copy -import textwrap +import json +import logging +import os +import shutil import signal -import functools import string - -bindir = os.path.abspath(os.path.dirname(__file__)) -sys.path[0:0] = [os.path.join(os.path.dirname(bindir), 'lib')] +import subprocess +import sys +import textwrap +import time import bb.msg import bb.process +bindir = os.path.abspath(os.path.dirname(__file__)) +sys.path[0:0] = [os.path.join(os.path.dirname(bindir), 'lib')] logger = bb.msg.logger_create('bitbake-setup', sys.stdout) # These settings can only be set in the global context
Signature · 1 line
2.49.0
PATCH v2 5/10 bitbake-setup: Resolve unused loop control variables
In reply to Rob WoolleyThe ruff lint tool reported loop control variables that were not being used:
B007 Loop control variable dirs not used within loop body
Adding underscore as a prefix helps indicate that it is not being used inside the loop.
--- bin/bitbake-setup | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
bin/bitbake-setup +3 -3
@@ -94,7 +94,7 @@ def commit_config(config_dir): def _write_layer_list(dest, repodirs): layers = [] for r in repodirs: - for root, dirs, files in os.walk(os.path.join(dest,r)): + for root, _dirs, files in os.walk(os.path.join(dest,r)): if os.path.basename(root) == 'conf' and 'layer.conf' in files: layers.append(os.path.relpath(os.path.dirname(root), dest)) layers_f = os.path.join(dest, ".oe-layers.json")
@@ -388,7 +388,7 @@ The bitbake configuration files (local.conf, bblayers.conf and more) can be foun logger.plain("The bitbake configuration files (local.conf, bblayers.conf and more) can be found in\n {}/conf\n".format(bitbake_builddir)) def get_registry_config(registry_path, id): - for root, dirs, files in os.walk(registry_path): + for root, _dirs, files in os.walk(registry_path): for f in files: if f.endswith('.conf.json') and id == get_config_name(f): return os.path.join(root, f)
@@ -766,7 +766,7 @@ def has_expired(expiry_date): def list_registry(registry_path, with_expired): json_data = {} - for root, dirs, files in os.walk(registry_path): + for root, _dirs, files in os.walk(registry_path): for f in files: if f.endswith('.conf.json'): config_name = get_config_name(f)
Signature · 1 line
2.49.0
PATCH v2 6/10 bitbake-setup: Fix ambiguous variable names
In reply to Rob WoolleyThe ruff lint tool detected use of ambiguous variables that were named with a single letter:
E741 Ambiguous variable name: l
E741 Ambiguous variable name: f
This replaces the variables with a descriptive variable to add clarity.
--- bin/bitbake-setup | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-)
bin/bitbake-setup +16 -16
@@ -204,11 +204,11 @@ bitbake-setup init -L {} /path/to/repo/checkout""".format( if oesetupbuild: links = {'setup-build': oesetupbuild, 'oe-scripts': os.path.dirname(oesetupbuild), 'oe-init-build-env-dir': oeinitbuildenvdir} - for l,t in links.items(): - symlink = os.path.join(layerdir, l) + for link,item in links.items(): + symlink = os.path.join(layerdir, link) if os.path.lexists(symlink): os.remove(symlink) - os.symlink(os.path.relpath(t,layerdir),symlink) + os.symlink(os.path.relpath(item,layerdir),symlink) return layers_fixed_revisions
@@ -217,19 +217,19 @@ def setup_bitbake_build(bitbake_config, layerdir, setupdir, thisdir, update_bb_c os.makedirs(build_conf_dir) layers_s = [] - for l in layers: - l = os.path.join(layerdir, l) - layers_s.append(" {} \\".format(l)) + for layer in layers: + layer = os.path.join(layerdir, layer) + layers_s.append(" {} \\".format(layer)) - for l in filerelative_layers: + for layer in filerelative_layers: if thisdir: - l = os.path.join(thisdir, l) + layer = os.path.join(thisdir, layer) else: raise Exception("Configuration is using bb-layers-file-relative to specify " \ "a layer path relative to itself. This can be done only " \ "when the configuration is specified by its path on local " \ "disk, not when it's in a registry or is fetched over http.") - layers_s.append(" {} \\".format(l)) + layers_s.append(" {} \\".format(layer)) layers_s = "\n".join(layers_s) bblayers_conf = """BBLAYERS ?= " \\
@@ -389,9 +389,9 @@ The bitbake configuration files (local.conf, bblayers.conf and more) can be foun def get_registry_config(registry_path, id): for root, _dirs, files in os.walk(registry_path): - for f in files: - if f.endswith('.conf.json') and id == get_config_name(f): - return os.path.join(root, f) + for file in files: + if file.endswith('.conf.json') and id == get_config_name(file): + return os.path.join(root, file) raise Exception("Unable to find {} in available configurations; use 'list' sub-command to see what is available".format(id)) def merge_overrides_into_sources(sources, overrides):
@@ -767,10 +767,10 @@ def list_registry(registry_path, with_expired): json_data = {} for root, _dirs, files in os.walk(registry_path): - for f in files: - if f.endswith('.conf.json'): - config_name = get_config_name(f) - config_data = json.load(open(os.path.join(root, f))) + for file in files: + if file.endswith('.conf.json'): + config_name = get_config_name(file) + config_data = json.load(open(os.path.join(root, file))) config_desc = config_data["description"] expiry_date = config_data.get("expires", None) if expiry_date:
Signature · 1 line
2.49.0
PATCH v2 7/10 bitbake-setup: Remove unused stdout variable
In reply to Rob WoolleyThe ruff lint tool reported that stdout was never used:
F841 Local variable stdout is assigned to but never used
--- bin/bitbake-setup | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
bin/bitbake-setup +1 -1
@@ -583,7 +583,7 @@ def init_config(top_dir, settings, args): create_siteconf(top_dir, args.non_interactive, settings) d = init_bb_cache(top_dir, settings, args) - stdout = sys.stdout + def handle_task_progress(event, d): rate = event.rate if event.rate else '' progress = event.progress if event.progress > 0 else 0
Signature · 1 line
2.49.0
Re: [bitbake-devel] [PATCH v2 07/10] bitbake-setup: Remove unused stdout variable
Quoted 15 lines · Rob Woolley via lists.openembedded.org
The ruff lint tool reported that stdout was never used:
F841 Local variable
stdoutis assigned to but never usedSigned-off-by Rob Woolley <[email protected]>--- bin/bitbake-setup | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)bin/bitbake-setup +0
-0
bin/bitbake-setup def init_config(top_dir, settings, args): · line 589
Quoted 8 lines
bin/bitbake-setup +1
-1@@ -583,7 +583,7 @@ def init_config(top_dir, settings, args): create_siteconf(top_dir, args.non_interactive, settings) d = init_bb_cache(top_dir, settings, args) - stdout = sys.stdout + def handle_task_progress(event, d): rate = event.rate if event.rate else '' progress = event.progress if event.progress > 0 else 0
The next line is
print("{}% {} ".format(progress, rate),
file=stdout, end='\r')
So stdout is used. We can't simply supply sys.stdout, as do_fetch does tricks with it to redirect the logs into a file (otherwise they would clutter the console).
Alex
Re: [bitbake-devel] [PATCH v2 07/10] bitbake-setup: Remove unused stdout variable
Hi Alex
That's cool. We can drop the patch.
I checked upstream and don't see the line you mentioned on the master branch: https://github.com/openembedded/bitbake/blob/master/bin%2Fbitbake-setup#L583
Should I be looking at a development repo or -next branch? I want to make sure to rebase any future patches against that.
Cheers,
Rob
Get Outlook for Android<https://aka.ms/AAb9ysg>
________________________________
From: Alexander Kanavin <[email protected]>
Sent: Wednesday, January 28, 2026 7:14:45 AM
To: Woolley, Rob <[email protected]>
Cc: [email protected] <[email protected]>
Subject: Re: [bitbake-devel] [PATCH v2 07/10] bitbake-setup: Remove unused stdout variable
CAUTION: This email comes from a non Wind River email account!
Do not click links or open attachments unless you recognize the sender and know the content is safe.
Quoted 15 lines · Rob Woolley via lists.openembedded.org
The ruff lint tool reported that stdout was never used:
F841 Local variable
stdoutis assigned to but never usedSigned-off-by Rob Woolley <[email protected]>--- bin/bitbake-setup | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)bin/bitbake-setup +0
-0
bin/bitbake-setup def init_config(top_dir, settings, args): · line 589
Quoted 8 lines
bin/bitbake-setup +1
-1@@ -583,7 +583,7 @@ def init_config(top_dir, settings, args): create_siteconf(top_dir, args.non_interactive, settings) d = init_bb_cache(top_dir, settings, args) - stdout = sys.stdout + def handle_task_progress(event, d): rate = event.rate if event.rate else '' progress = event.progress if event.progress > 0 else 0
The next line is
print("{}% {} ".format(progress, rate),
file=stdout, end='\r')
So stdout is used. We can't simply supply sys.stdout, as do_fetch does tricks with it to redirect the logs into a file (otherwise they would clutter the console).
Alex
Re: [bitbake-devel] [PATCH v2 07/10] bitbake-setup: Remove unused stdout variable
Quoted 5 lines · Rob
That's cool. We can drop the patch.
I checked upstream and don't see the line you mentioned on the master branch: https://github.com/openembedded/bitbake/blob/master/bin%2Fbitbake-setup#L583
Should I be looking at a development repo or -next branch? I want to make sure to rebase any future patches against that.
Ah, I was looking at my own outdated code. The patch is okay.
Alex
PATCH v2 8/10 bitbake-setup: Replace f-string without placeholders
In reply to Rob WoolleyThe ruff lint tool reported an f-string that didn't include any placeholders:
F541 f-string without any placeholders
--- bin/bitbake-setup | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
bin/bitbake-setup +1 -1
@@ -957,7 +957,7 @@ def merge_settings(builtin_settings, global_settings, topdir_settings, cmdline_s return all_settings def sigint_handler(sig, frame, func, top_dir): - logger.plain(f'\nShutting down...') + logger.plain('\nShutting down...') if isinstance(top_dir, str) and os.path.exists(top_dir): if func in [init_config, build_update]: logger.warning(f'{top_dir} may contain an incomplete setup!')
Signature · 1 line
2.49.0
PATCH v2 9/10 bitbake-setup: Remove unused Namespace import
In reply to Rob WoolleyThe ruff lint tool reported that Namespace was not used:
F401 argparse.Namespace imported but unused
--- bin/bitbake-setup | 1 - 1 file changed, 1 deletion(-)
bin/bitbake-setup +0 -1
@@ -1102,7 +1102,6 @@ def main(): save_bb_cache() else: - from argparse import Namespace parser.print_help() if __name__ == '__main__':
Signature · 1 line
2.49.0
PATCH v2 10/10 bitbake-setup: Remove unused parser_settings_list
In reply to Rob WoolleyThe ruff lint tool detected an used variable:
F841 Local variable parser_settings_list is assigned to but never used
The list_settings function gets invoked through the parent command's settings_func function. This means that creating this variable is unnecessary.
--- bin/bitbake-setup | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
bin/bitbake-setup +1 -2
@@ -1026,8 +1026,7 @@ def main(): subparser_settings = parser_settings.add_subparsers(dest="subcommand", required=True, help="The action to perform on the settings file") - parser_settings_list = subparser_settings.add_parser('list', - help="List all settings with their values") + subparser_settings.add_parser('list', help="List all settings with their values") parser_settings_set = subparser_settings.add_parser('set', parents=[parser_settings_arg_global], help="In a Section, set a setting to a certain value")
Signature · 1 line
2.49.0
Hello Rob,
can you send the latest patchset for bitbake-setup in pypi please?
Alex
Quoted 90 lines · Rob Woolley via lists.openembedded.org
Thanks for the feedback from the 1st series. I included all requested changes:
* Moved packaging files from top-level to contrib/pypi
* Createdpackage-bitbake-setup.pyto create a new workspace to build the package and be compliant with PEP 517
* Changed the title of patch #4 to start with bitbake-setup:
* Converted setup.cfg to pyproject.toml
* Remove old license classifiers
* Made AND operator uppercase for license entry
* Set the minimum Python requirement to 3.9
* Created a custom README.md and BUILD.md for bitbake-setup
* Changed the URL to https://git.openembedded.org/bitbake/
* Added support for nox to automate the packaging including linting, building, and testingThere was a question about the wildcards in the includes section. In this context they are not used for filename globbing and instead denote to include any Python sub-modules as well. See the Python documentation for details:
https://setupttools.pypa.io/en/latest/userguide/package_discovery.htmlThe bitbake-setup script is no longer being turned into a module in the git repository. Instead it is being done by the script. The rationale for doing this is that it:
* ensures the module works reliably on all platforms
* avoids shebang and path issues
* places nicely with wheels, venvs, and zipapps
* easier to testI did make an attempt to ensure that pyproject.toml and
noxfile.pyare compatible with older tools. Building the wheel using the pyprojet.toml should be supported with pip >=19.Wheels built newer versions of pip should remain backwards compatible with pip when installing on older systems.
Outstanding:
* Determine a release strategy and version scheme
* Document steps for publishing the module to PyPITo run the automated tests all that should be required is to install the python3-nox package or nox Python module. Then run
nox -s lint
nox -s build
nox -s testsRob Woolley (10):
bitbake-setup: Add the conditional script stanza
bitbake-setup: Add pypi packaging
bitbake-setup: Remove extraneous variable fromstr.format()
bitbake-setup: Sort and format imports
bitbake-setup: Resolve unused loop control variables
bitbake-setup: Fix ambiguous variable names
bitbake-setup: Remove unused stdout variable
bitbake-setup: Replace f-string without placeholders
bitbake-setup: Remove unused Namespace import
bitbake-setup: Remove unusedparser_settings_list.gitignore | 1 + bin/bitbake-setup | 76 ++++++++++----------- contrib/pypi/BUILD.md | 57 ++++++++++++++++ contrib/pypi/LICENSE | 9 +++ contrib/pypi/README.md | 42 ++++++++++++ contrib/pypi/noxfile.py | 30 +++++++++ contrib/pypi/package-bitbake-setup.py | 93 +++++++++++++++++++++++++ contrib/pypi/pyproject.toml | 97 +++++++++++++++++++++++++++ contrib/pypi/tests/test_install.py | 17 +++++ 9 files changed, 383 insertions(+), 39 deletions(-) create mode 100644 contrib/pypi/BUILD.md create mode 100644 contrib/pypi/LICENSE create mode 100644 contrib/pypi/README.md create mode 100644 contrib/pypi/noxfile.py create mode 100755 contrib/pypi/package-bitbake-setup.py create mode 100644 contrib/pypi/pyproject.toml create mode 100644 contrib/pypi/tests/test_install.py--
2.49.0-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18889): https://lists.openembedded.org/g/bitbake-devel/message/18889
Mute This Topic: https://lists.openembedded.org/mt/117490844/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Hello Rob,
can you send the latest patchset for bitbake-setup in pypi please?
Alex
Quoted 90 lines · Rob Woolley via lists.openembedded.org
Thanks for the feedback from the 1st series. I included all requested changes:
* Moved packaging files from top-level to contrib/pypi
* Createdpackage-bitbake-setup.pyto create a new workspace to build the package and be compliant with PEP 517
* Changed the title of patch #4 to start with bitbake-setup:
* Converted setup.cfg to pyproject.toml
* Remove old license classifiers
* Made AND operator uppercase for license entry
* Set the minimum Python requirement to 3.9
* Created a custom README.md and BUILD.md for bitbake-setup
* Changed the URL to https://git.openembedded.org/bitbake/
* Added support for nox to automate the packaging including linting, building, and testingThere was a question about the wildcards in the includes section. In this context they are not used for filename globbing and instead denote to include any Python sub-modules as well. See the Python documentation for details:
https://setupttools.pypa.io/en/latest/userguide/package_discovery.htmlThe bitbake-setup script is no longer being turned into a module in the git repository. Instead it is being done by the script. The rationale for doing this is that it:
* ensures the module works reliably on all platforms
* avoids shebang and path issues
* places nicely with wheels, venvs, and zipapps
* easier to testI did make an attempt to ensure that pyproject.toml and
noxfile.pyare compatible with older tools. Building the wheel using the pyprojet.toml should be supported with pip >=19.Wheels built newer versions of pip should remain backwards compatible with pip when installing on older systems.
Outstanding:
* Determine a release strategy and version scheme
* Document steps for publishing the module to PyPITo run the automated tests all that should be required is to install the python3-nox package or nox Python module. Then run
nox -s lint
nox -s build
nox -s testsRob Woolley (10):
bitbake-setup: Add the conditional script stanza
bitbake-setup: Add pypi packaging
bitbake-setup: Remove extraneous variable fromstr.format()
bitbake-setup: Sort and format imports
bitbake-setup: Resolve unused loop control variables
bitbake-setup: Fix ambiguous variable names
bitbake-setup: Remove unused stdout variable
bitbake-setup: Replace f-string without placeholders
bitbake-setup: Remove unused Namespace import
bitbake-setup: Remove unusedparser_settings_list.gitignore | 1 + bin/bitbake-setup | 76 ++++++++++----------- contrib/pypi/BUILD.md | 57 ++++++++++++++++ contrib/pypi/LICENSE | 9 +++ contrib/pypi/README.md | 42 ++++++++++++ contrib/pypi/noxfile.py | 30 +++++++++ contrib/pypi/package-bitbake-setup.py | 93 +++++++++++++++++++++++++ contrib/pypi/pyproject.toml | 97 +++++++++++++++++++++++++++ contrib/pypi/tests/test_install.py | 17 +++++ 9 files changed, 383 insertions(+), 39 deletions(-) create mode 100644 contrib/pypi/BUILD.md create mode 100644 contrib/pypi/LICENSE create mode 100644 contrib/pypi/README.md create mode 100644 contrib/pypi/noxfile.py create mode 100755 contrib/pypi/package-bitbake-setup.py create mode 100644 contrib/pypi/pyproject.toml create mode 100644 contrib/pypi/tests/test_install.py--
2.49.0-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18889): https://lists.openembedded.org/g/bitbake-devel/message/18889
Mute This Topic: https://lists.openembedded.org/mt/117490844/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Hi Alex,
Thanks for reaching out. Just wanted to acknowledge that I saw your request.
I have my latest changes here:
https://github.com/robwoolley/bitbake/tree/pypi-version-six
I am still fighting with one last bug related to differences in relative imports in scripts versus modules. Hope to resolve that soon and send out the latest patchset.
Regards,
Rob
Quoted 101 lines · Alexander Kanavin
Hello Rob,
can you send the latest patchset for bitbake-setup in pypi please?
Alex
Quoted 78 lines · Rob Woolley via lists.openembedded.org
Thanks for the feedback from the 1st series. I included all requested changes:
* Moved packaging files from top-level to contrib/pypi
* Createdpackage-bitbake-setup.pyto create a new workspace to build the package and be compliant with PEP 517
* Changed the title of patch #4 to start with bitbake-setup:
* Converted setup.cfg to pyproject.toml
* Remove old license classifiers
* Made AND operator uppercase for license entry
* Set the minimum Python requirement to 3.9
* Created a custom README.md and BUILD.md for bitbake-setup
* Changed the URL to https://git.openembedded.org/bitbake/
* Added support for nox to automate the packaging including linting, building, and testingThere was a question about the wildcards in the includes section. In this context they are not used for filename globbing and instead denote to include any Python sub-modules as well. See the Python documentation for details:
https://setupttools.pypa.io/en/latest/userguide/package_discovery.htmlThe bitbake-setup script is no longer being turned into a module in the git repository. Instead it is being done by the script. The rationale for doing this is that it:
* ensures the module works reliably on all platforms
* avoids shebang and path issues
* places nicely with wheels, venvs, and zipapps
* easier to testI did make an attempt to ensure that pyproject.toml and
noxfile.pyare compatible with older tools. Building the wheel using the pyprojet.toml should be supported with pip >=19.Wheels built newer versions of pip should remain backwards compatible with pip when installing on older systems.
Outstanding:
* Determine a release strategy and version scheme
* Document steps for publishing the module to PyPITo run the automated tests all that should be required is to install the python3-nox package or nox Python module. Then run
nox -s lint
nox -s build
nox -s testsRob Woolley (10):
bitbake-setup: Add the conditional script stanza
bitbake-setup: Add pypi packaging
bitbake-setup: Remove extraneous variable fromstr.format()
bitbake-setup: Sort and format imports
bitbake-setup: Resolve unused loop control variables
bitbake-setup: Fix ambiguous variable names
bitbake-setup: Remove unused stdout variable
bitbake-setup: Replace f-string without placeholders
bitbake-setup: Remove unused Namespace import
bitbake-setup: Remove unusedparser_settings_list.gitignore | 1 + bin/bitbake-setup | 76 ++++++++++----------- contrib/pypi/BUILD.md | 57 ++++++++++++++++ contrib/pypi/LICENSE | 9 +++ contrib/pypi/README.md | 42 ++++++++++++ contrib/pypi/noxfile.py | 30 +++++++++ contrib/pypi/package-bitbake-setup.py | 93 +++++++++++++++++++++++++ contrib/pypi/pyproject.toml | 97 +++++++++++++++++++++++++++ contrib/pypi/tests/test_install.py | 17 +++++ 9 files changed, 383 insertions(+), 39 deletions(-) create mode 100644 contrib/pypi/BUILD.md create mode 100644 contrib/pypi/LICENSE create mode 100644 contrib/pypi/README.md create mode 100644 contrib/pypi/noxfile.py create mode 100755 contrib/pypi/package-bitbake-setup.py create mode 100644 contrib/pypi/pyproject.toml create mode 100644 contrib/pypi/tests/test_install.pyQuoted 7 lines
--
2.49.0-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#18889):https://lists.openembedded.org/g/bitbake-devel/message/18889
Quoted 3 lines
Mute This Topic: https://lists.openembedded.org/mt/117490844/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [Quoted 2 lines
-=-=-=-=-=-=-=-=-=-=-=-