Devtool/Recipetool: adding pypi support
cover Devtool/Recipetool: adding pypi support
Hello all,
This series adds basic support for the pypi class in devtool/recipetool.
The idea is to be able to detect that a package is available on pypi and in that case inherits from pypi class. This helps to produce cleaner recipes for pypi packages.
To do this, I am adding a new optional callback "process_url" that plugin can register to implement some logic on matching url.
By implementing this callback in create_buildsys_python we can match on pypi URLs directly or on release tarballs hosted on "files.pythonhosted.org".
To create a recipe taking advantage of the pypi class we can use one of the following new syntax:
* recipetool create https://pypi.org/project/<package>
* recipetool create https://pypi.org/project/<package>/<version>
* recipetool create https://pypi.org/project/<package> --version <version>
or the old syntax:
* recipetool create https://files.pythonhosted.org/packages/<...>
If the URL points to a github URL or a release tarball not coming from "files.pythonhosted.org", the created recipe is the same as before. One can also use the newly added "--no-pypi" switch to NOT inherit from pypi class on matching URL, to keep legacy behaviour.
This series also contains some bug fixes I found during my testing.
Pushed my dev branch here: https://git.yoctoproject.org/poky-contrib/log/?h=jstephan/devtool-add-pypi-support
Cheers
Julien
Julien Stephan (7):
bitbake: utils: remove spaces on empty lines
recipetool: create_buildsys_python.py: initialize metadata
recipetool: create: add trailing newlines
recipetool: create: add new optional process_url callback for plugins
recipetool: create_buildsys_python: add pypi support oeqa/selftest/recipetool: remove spaces on empty lines oeqa/selftest/recipetool/devtool: add test for pypi class
bitbake/lib/bb/utils.py | 16 +-- meta/lib/oeqa/selftest/cases/devtool.py | 4 +- meta/lib/oeqa/selftest/cases/recipetool.py | 114 +++++++++++++----- scripts/lib/devtool/standard.py | 3 + scripts/lib/recipetool/create.py | 63 ++++++---- .../lib/recipetool/create_buildsys_python.py | 73 +++++++++++ 6 files changed, 208 insertions(+), 65 deletions(-)
Signature · 1 line
2.42.0
PATCH 1/7 bitbake: utils: remove spaces on empty lines
--- bitbake/lib/bb/utils.py | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)
bitbake/lib/bb/utils.py +8 -8
@@ -50,7 +50,7 @@ def clean_context(): def get_context(): return _context - + def set_context(ctx): _context = ctx
@@ -212,8 +212,8 @@ def explode_dep_versions2(s, *, sort=True): inversion = True # This list is based on behavior and supported comparisons from deb, opkg and rpm. # - # Even though =<, <<, ==, !=, =>, and >> may not be supported, - # we list each possibly valid item. + # Even though =<, <<, ==, !=, =>, and >> may not be supported, + # we list each possibly valid item. # The build system is responsible for validation of what it supports. if i.startswith(('<=', '=<', '<<', '==', '!=', '>=', '=>', '>>')): lastcmp = i[0:2]
@@ -347,7 +347,7 @@ def _print_exception(t, value, tb, realfile, text, context): exception = traceback.format_exception_only(t, value) error.append('Error executing a python function in %s:\n' % realfile) - # Strip 'us' from the stack (better_exec call) unless that was where the + # Strip 'us' from the stack (better_exec call) unless that was where the # error came from if tb.tb_next is not None: tb = tb.tb_next
@@ -746,9 +746,9 @@ def prunedir(topdir, ionice=False): # but thats possibly insane and suffixes is probably going to be small # def prune_suffix(var, suffixes, d): - """ + """ See if var ends with any of the suffixes listed and - remove it if found + remove it if found """ for suffix in suffixes: if suffix and var.endswith(suffix):
@@ -1001,9 +1001,9 @@ def umask(new_mask): os.umask(current_mask) def to_boolean(string, default=None): - """ + """ Check input string and return boolean value True/False/None - depending upon the checks + depending upon the checks """ if not string: return default
Signature · 1 line
2.42.0
Patchtest results for [PATCH 1/7] bitbake: utils: remove spaces on empty lines
Thank you for your submission. Patchtest identified one or more issues with the patch. Please see the log below for more information:
---
Testing patch /home/patchtest/share/mboxes/1-7-bitbake-utils-remove-spaces-on-empty-lines.patch
FAIL: test commit message presence: Please include a commit message on your patch explaining the change (test_mbox.TestMbox.test_commit_message_presence)
PASS: pretest pylint (test_python_pylint.PyLint.pretest_pylint)
PASS: test Signed-off-by presence (test_mbox.TestMbox.test_signed_off_by_presence)
PASS: test author valid (test_mbox.TestMbox.test_author_valid)
PASS: test max line length (test_metadata.TestMetadata.test_max_line_length)
PASS: test mbox format (test_mbox.TestMbox.test_mbox_format)
PASS: test non-AUH upgrade (test_mbox.TestMbox.test_non_auh_upgrade)
PASS: test pylint (test_python_pylint.PyLint.test_pylint)
PASS: test shortlog format (test_mbox.TestMbox.test_shortlog_format)
PASS: test shortlog length (test_mbox.TestMbox.test_shortlog_length)
SKIP: pretest src uri left files: No modified recipes, skipping pretest (test_metadata.TestMetadata.pretest_src_uri_left_files)
SKIP: test CVE tag format: No new CVE patches introduced (test_patch.TestPatch.test_cve_tag_format)
SKIP: test Signed-off-by presence: No new CVE patches introduced (test_patch.TestPatch.test_signed_off_by_presence)
SKIP: test Upstream-Status presence: No new CVE patches introduced (test_patch.TestPatch.test_upstream_status_presence_format)
SKIP: test bugzilla entry format: No bug ID found (test_mbox.TestMbox.test_bugzilla_entry_format)
SKIP: test lic files chksum modified not mentioned: No modified recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_modified_not_mentioned)
SKIP: test lic files chksum presence: No added recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_presence)
SKIP: test license presence: No added recipes, skipping test (test_metadata.TestMetadata.test_license_presence)
SKIP: test series merge on head: Merge test is disabled for now (test_mbox.TestMbox.test_series_merge_on_head)
SKIP: test src uri left files: No modified recipes, skipping pretest (test_metadata.TestMetadata.test_src_uri_left_files)
SKIP: test summary presence: No added recipes, skipping test (test_metadata.TestMetadata.test_summary_presence)
SKIP: test target mailing list: Series merged, no reason to check other mailing lists (test_mbox.TestMbox.test_target_mailing_list)
---
Please address the issues identified and
submit a new revision of the patch, or alternatively, reply to this email with an explanation of why the patch should be accepted. If you believe these results are due to an error in patchtest, please submit a bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' category under 'Yocto Project Subprojects'). For more information on specific failures, see: https://wiki.yoctoproject.org/wiki/Patchtest. Thank you!
PATCH 2/7 recipetool: create_buildsys_python.py: initialize metadata
In reply to Julien StephanIn the case pyproject.toml doesn't contains metadatas, the metadata variable is not initialized and the plugin throws an error and falls back to another plugin, which is not the desired behaviour. So just ignore metadata if we don't have them
--- scripts/lib/recipetool/create_buildsys_python.py | 1 + 1 file changed, 1 insertion(+)
scripts/lib/recipetool/create_buildsys_python.py +1 -0
@@ -726,6 +726,7 @@ class PythonPyprojectTomlRecipeHandler(PythonRecipeHandler): def process(self, srctree, classes, lines_before, lines_after, handled, extravalues): info = {} + metadata = {} if 'buildsystem' in handled: return False
Signature · 1 line
2.42.0
PATCH 3/7 recipetool: create: add trailing newlines
In reply to Julien Stephancreate_recipe() function relies on oe.recipeutils.patch_recipe_lines() which relies on bb.utils.edit_metadata(). edit_metada expect lines to have trailing newlines, so add it to each line before calling patch_recipe_lines, otherwise edit_metadata will not be able to squash blank line if there are two consecutive blanks after a removal
--- scripts/lib/recipetool/create.py | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)
scripts/lib/recipetool/create.py +5 -3
@@ -873,8 +873,10 @@ def create_recipe(args): outlines.append('') outlines.extend(lines_after) + outlines = [ line.rstrip('\n') +"\n" for line in outlines] + if extravalues: - _, outlines = oe.recipeutils.patch_recipe_lines(outlines, extravalues, trailing_newline=False) + _, outlines = oe.recipeutils.patch_recipe_lines(outlines, extravalues, trailing_newline=True) if args.extract_to: scriptutils.git_convert_standalone_clone(srctree)
@@ -890,7 +892,7 @@ def create_recipe(args): log_info_cond('Source extracted to %s' % args.extract_to, args.devtool) if outfile == '-': - sys.stdout.write('\n'.join(outlines) + '\n') + sys.stdout.write(''.join(outlines) + '\n') else: with open(outfile, 'w') as f: lastline = None
@@ -898,7 +900,7 @@ def create_recipe(args): if not lastline and not line: # Skip extra blank lines continue - f.write('%s\n' % line) + f.write('%s' % line) lastline = line log_info_cond('Recipe %s has been created; further editing may be required to make it fully functional' % outfile, args.devtool) tinfoil.modified_files()
Signature · 1 line
2.42.0
PATCH 4/7 recipetool: create: add new optional process_url callback for plugins
In reply to Julien StephanAdd a new process_url callback that plugins can optionally implement if they which to handle url.
Plugins can implement this callback for example, to:
* transform the url
* add special variables using extravalues
* add extra classes
* ...
If a plugin handles the url, it must append 'url' to the handled list and must return the fetchuri
No functional changes expected for plugins non implementing this optional callback
--- scripts/lib/recipetool/create.py | 54 +++++++++++++++++++------------- 1 file changed, 32 insertions(+), 22 deletions(-)
scripts/lib/recipetool/create.py +32 -22
@@ -423,6 +423,36 @@ def create_recipe(args): storeTagName = '' pv_srcpv = False + handled = [] + classes = [] + + # Find all plugins that want to register handlers + logger.debug('Loading recipe handlers') + raw_handlers = [] + for plugin in plugins: + if hasattr(plugin, 'register_recipe_handlers'): + plugin.register_recipe_handlers(raw_handlers) + # Sort handlers by priority + handlers = [] + for i, handler in enumerate(raw_handlers): + if isinstance(handler, tuple): + handlers.append((handler[0], handler[1], i)) + else: + handlers.append((handler, 0, i)) + handlers.sort(key=lambda item: (item[1], -item[2]), reverse=True) + for handler, priority, _ in handlers: + logger.debug('Handler: %s (priority %d)' % (handler.__class__.__name__, priority)) + setattr(handler, '_devtool', args.devtool) + handlers = [item[0] for item in handlers] + + fetchuri = None + for handler in handlers: + if hasattr(handler, 'process_url'): + ret = handler.process_url(args, classes, handled, extravalues) + if 'url' in handled and ret: + fetchuri = ret + break + if os.path.isfile(source): source = 'file://%s' % os.path.abspath(source)
@@ -431,7 +461,8 @@ def create_recipe(args): if re.match(r'https?://github.com/[^/]+/[^/]+/archive/.+(\.tar\..*|\.zip)$', source): logger.warning('github archive files are not guaranteed to be stable and may be re-generated over time. If the latter occurs, the checksums will likely change and the recipe will fail at do_fetch. It is recommended that you point to an actual commit or tag in the repository instead (using the repository URL in conjunction with the -S/--srcrev option).') # Fetch a URL - fetchuri = reformat_git_uri(urldefrag(source)[0]) + if not fetchuri: + fetchuri = reformat_git_uri(urldefrag(source)[0]) if args.binary: # Assume the archive contains the directory structure verbatim # so we need to extract to a subdirectory
@@ -638,8 +669,6 @@ def create_recipe(args): # We'll come back and replace this later in handle_license_vars() lines_before.append('##LICENSE_PLACEHOLDER##') - handled = [] - classes = [] # FIXME This is kind of a hack, we probably ought to be using bitbake to do this pn = None
@@ -718,25 +747,6 @@ def create_recipe(args): if args.npm_dev: extravalues['NPM_INSTALL_DEV'] = 1 - # Find all plugins that want to register handlers - logger.debug('Loading recipe handlers') - raw_handlers = [] - for plugin in plugins: - if hasattr(plugin, 'register_recipe_handlers'): - plugin.register_recipe_handlers(raw_handlers) - # Sort handlers by priority - handlers = [] - for i, handler in enumerate(raw_handlers): - if isinstance(handler, tuple): - handlers.append((handler[0], handler[1], i)) - else: - handlers.append((handler, 0, i)) - handlers.sort(key=lambda item: (item[1], -item[2]), reverse=True) - for handler, priority, _ in handlers: - logger.debug('Handler: %s (priority %d)' % (handler.__class__.__name__, priority)) - setattr(handler, '_devtool', args.devtool) - handlers = [item[0] for item in handlers] - # Apply the handlers if args.binary: classes.append('bin_package')
Signature · 1 line
2.42.0
PATCH 5/7 recipetool: create_buildsys_python: add pypi support
In reply to Julien StephanToday, we can use devtool/recipetool to create recipes for python projects using the github url or the direct realse tarball of the project, but the create_buildsys_python plugin doesn't support the pypi class, since we cannot know from the extracted source if the package is available on pypi or not.
By implementing the new optional process_url callback, we can detect that the url is a pypi one (i.e 'https://pypi.org/project/<package>') and retrieve the release tarball location.
Also detect if the url points to a release tarball hosted on "files.pythonhosted.iorg" (i.e https://files.pythonhosted.org/packages/...)
In both cases, adds the pypi class, remove 'S' and 'SRC_URIxxx' variables from the created recipe as they will be handled by the pypi class and add the PYPI_PACKAGE variable
This helps to produce cleaner recipes when package is hosted on pypi.
If the url points to a github url or a release tarball not coming from "files.pythonhosted.org" website, the created recipe is the same as before. One can also use the newly added "--no-pypi" switch to devtool/recipetool to NOT inherit from pypi class on matching url, to keep legacy behaviour.
To create a recipe for a pypi package, one can now use one of the new following syntax (using recipetool create / devtool add):
* recipetool create https://pypi.org/project/<package>
* recipetool create https://pypi.org/project/<package>/<version>
* recipetool create https://pypi.org/project/<package> --version <version>
or the old syntax:
* recipetool create https://files.pythonhosted.org/packages/<...>
--- scripts/lib/devtool/standard.py | 3 + scripts/lib/recipetool/create.py | 1 + .../lib/recipetool/create_buildsys_python.py | 72 +++++++++++++++++++ 3 files changed, 76 insertions(+)
scripts/lib/devtool/standard.py +3 -0
@@ -147,6 +147,8 @@ def add(args, config, basepath, workspace): extracmdopts += ' -a' if args.npm_dev: extracmdopts += ' --npm-dev' + if args.no_pypi: + extracmdopts += ' --no-pypi' if args.mirrors: extracmdopts += ' --mirrors' if args.srcrev:
@@ -2260,6 +2262,7 @@ def register_commands(subparsers, context): group.add_argument('--no-same-dir', help='Force build in a separate build directory', action="store_true") parser_add.add_argument('--fetch', '-f', help='Fetch the specified URI and extract it to create the source tree (deprecated - pass as positional argument instead)', metavar='URI') parser_add.add_argument('--npm-dev', help='For npm, also fetch devDependencies', action="store_true") + parser_add.add_argument('--no-pypi', help='Do not inherit pypi class', action="store_true") parser_add.add_argument('--version', '-V', help='Version to use within recipe (PV)') parser_add.add_argument('--no-git', '-g', help='If fetching source, do not set up source tree as a git repository', action="store_true") group = parser_add.add_mutually_exclusive_group()
scripts/lib/recipetool/create.py +1 -0
@@ -1413,6 +1413,7 @@ def register_commands(subparsers): parser_create.add_argument('-B', '--srcbranch', help='Branch in source repository if fetching from an SCM such as git (default master)') parser_create.add_argument('--keep-temp', action="store_true", help='Keep temporary directory (for debugging)') parser_create.add_argument('--npm-dev', action="store_true", help='For npm, also fetch devDependencies') + parser_create.add_argument('--no-pypi', action="store_true", help='Do not inherit pypi class') parser_create.add_argument('--devtool', action="store_true", help=argparse.SUPPRESS) parser_create.add_argument('--mirrors', action="store_true", help='Enable PREMIRRORS and MIRRORS for source tree fetching (disabled by default).') parser_create.set_defaults(func=create_recipe)
scripts/lib/recipetool/create_buildsys_python.py +72 -0
85 lines not on this page · raw message
Signature · 1 line
2.42.0
PATCH 6/7 oeqa/selftest/recipetool: remove spaces on empty lines
In reply to Julien Stephan--- meta/lib/oeqa/selftest/cases/recipetool.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/lib/oeqa/selftest/cases/recipetool.py +1 -1
@@ -853,7 +853,7 @@ class RecipetoolTests(RecipetoolBase): self._test_recipe_contents(deps_require_file, checkvars, []) - + def _copy_file_with_cleanup(self, srcfile, basedstdir, *paths): dstdir = basedstdir self.assertTrue(os.path.exists(dstdir))
Signature · 1 line
2.42.0
Patchtest results for [PATCH 6/7] oeqa/selftest/recipetool: remove spaces on empty lines
Thank you for your submission. Patchtest identified one or more issues with the patch. Please see the log below for more information:
---
Testing patch /home/patchtest/share/mboxes/6-7-oeqa-selftest-recipetool-remove-spaces-on-empty-lines.patch
FAIL: test commit message presence: Please include a commit message on your patch explaining the change (test_mbox.TestMbox.test_commit_message_presence)
PASS: pretest pylint (test_python_pylint.PyLint.pretest_pylint)
PASS: test Signed-off-by presence (test_mbox.TestMbox.test_signed_off_by_presence)
PASS: test author valid (test_mbox.TestMbox.test_author_valid)
PASS: test max line length (test_metadata.TestMetadata.test_max_line_length)
PASS: test mbox format (test_mbox.TestMbox.test_mbox_format)
PASS: test non-AUH upgrade (test_mbox.TestMbox.test_non_auh_upgrade)
PASS: test pylint (test_python_pylint.PyLint.test_pylint)
PASS: test shortlog format (test_mbox.TestMbox.test_shortlog_format)
PASS: test shortlog length (test_mbox.TestMbox.test_shortlog_length)
SKIP: pretest src uri left files: No modified recipes, skipping pretest (test_metadata.TestMetadata.pretest_src_uri_left_files)
SKIP: test CVE tag format: No new CVE patches introduced (test_patch.TestPatch.test_cve_tag_format)
SKIP: test Signed-off-by presence: No new CVE patches introduced (test_patch.TestPatch.test_signed_off_by_presence)
SKIP: test Upstream-Status presence: No new CVE patches introduced (test_patch.TestPatch.test_upstream_status_presence_format)
SKIP: test bugzilla entry format: No bug ID found (test_mbox.TestMbox.test_bugzilla_entry_format)
SKIP: test lic files chksum modified not mentioned: No modified recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_modified_not_mentioned)
SKIP: test lic files chksum presence: No added recipes, skipping test (test_metadata.TestMetadata.test_lic_files_chksum_presence)
SKIP: test license presence: No added recipes, skipping test (test_metadata.TestMetadata.test_license_presence)
SKIP: test series merge on head: Merge test is disabled for now (test_mbox.TestMbox.test_series_merge_on_head)
SKIP: test src uri left files: No modified recipes, skipping pretest (test_metadata.TestMetadata.test_src_uri_left_files)
SKIP: test summary presence: No added recipes, skipping test (test_metadata.TestMetadata.test_summary_presence)
SKIP: test target mailing list: Series merged, no reason to check other mailing lists (test_mbox.TestMbox.test_target_mailing_list)
---
Please address the issues identified and
submit a new revision of the patch, or alternatively, reply to this email with an explanation of why the patch should be accepted. If you believe these results are due to an error in patchtest, please submit a bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' category under 'Yocto Project Subprojects'). For more information on specific failures, see: https://wiki.yoctoproject.org/wiki/Patchtest. Thank you!
PATCH 7/7 oeqa/selftest/recipetool/devtool: add test for pypi class
In reply to Julien Stephanrecipetool now supports the pypi class and python recipes can by created using the new following syntax:
* recipetool create https://pypi.org/project/<package>
* recipetool create https://pypi.org/project/<package>/<version>
* recipetool create https://pypi.org/project/<package> --version <version>
or the old syntax:
* recipetool create https://files.pythonhosted.org/packages/<...>
So add tests for the new syntax and modify old tests
--- meta/lib/oeqa/selftest/cases/devtool.py | 4 +- meta/lib/oeqa/selftest/cases/recipetool.py | 112 +++++++++++++++------ 2 files changed, 85 insertions(+), 31 deletions(-)
meta/lib/oeqa/selftest/cases/devtool.py +2 -2
@@ -543,7 +543,7 @@ class DevtoolAddTests(DevtoolBase): self.track_for_cleanup(self.workspacedir) self.add_command_to_tearDown('bitbake -c cleansstate %s' % testrecipe) self.add_command_to_tearDown('bitbake-layers remove-layer */workspace') - result = runCmd('devtool add %s %s -f %s' % (testrecipe, srcdir, url)) + result = runCmd('devtool add --no-pypi %s %s -f %s' % (testrecipe, srcdir, url)) self.assertExists(os.path.join(self.workspacedir, 'conf', 'layer.conf'), 'Workspace directory not created. %s' % result.output) self.assertTrue(os.path.isfile(os.path.join(srcdir, 'setup.py')), 'Unable to find setup.py in source directory') self.assertTrue(os.path.isdir(os.path.join(srcdir, '.git')), 'git repository for external source tree was not created')
@@ -562,7 +562,7 @@ class DevtoolAddTests(DevtoolBase): result = runCmd('devtool reset -n %s' % testrecipe) shutil.rmtree(srcdir) fakever = '1.9' - result = runCmd('devtool add %s %s -f %s -V %s' % (testrecipe, srcdir, url, fakever)) + result = runCmd('devtool add --no-pypi %s %s -f %s -V %s' % (testrecipe, srcdir, url, fakever)) self.assertTrue(os.path.isfile(os.path.join(srcdir, 'setup.py')), 'Unable to find setup.py in source directory') # Test devtool status result = runCmd('devtool status')
meta/lib/oeqa/selftest/cases/recipetool.py +83 -29
154 lines not on this page · raw message
Signature · 1 line
2.42.0
Hello,
The tests seem to be failing on the AB:
https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/6065/steps/14/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/6062/steps/15/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/6013/steps/14/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/6078/steps/14/logs/stdio
Quoted 67 lines · Julien Stephan
Hello all,
This series adds basic support for the pypi class in devtool/recipetool.
The idea is to be able to detect that a package is available on pypi and in that case inherits from pypi class. This helps to produce cleaner recipes for pypi packages.
To do this, I am adding a new optional callback "
process_url" that plugin can register to implement some logic on matching url.By implementing this callback in
create_buildsys_pythonwe can match on pypi URLs directly or on release tarballs hosted on "files.pythonhosted.org".To create a recipe taking advantage of the pypi class we can use one of the following new syntax:
* recipetool create https://pypi.org/project/<package>
* recipetool create https://pypi.org/project/<package>/<version>
* recipetool create https://pypi.org/project/<package> --version <version>or the old syntax:
* recipetool create https://files.pythonhosted.org/packages/<...>
If the URL points to a github URL or a release tarball not coming from "files.pythonhosted.org", the created recipe is the same as before. One can also use the newly added "--no-pypi" switch to NOT inherit from pypi class on matching URL, to keep legacy behaviour.
This series also contains some bug fixes I found during my testing.
Pushed my dev branch here: https://git.yoctoproject.org/poky-contrib/log/?h=jstephan/devtool-add-pypi-support
Cheers
JulienJulien Stephan (7):
bitbake: utils: remove spaces on empty lines
recipetool:create_buildsys_python.py: initialize metadata
recipetool: create: add trailing newlines
recipetool: create: add new optionalprocess_urlcallback for plugins
recipetool:create_buildsys_python: add pypi support oeqa/selftest/recipetool: remove spaces on empty lines oeqa/selftest/recipetool/devtool: add test for pypi classbitbake/lib/bb/utils.py | 16 +-- meta/lib/oeqa/selftest/cases/devtool.py | 4 +- meta/lib/oeqa/selftest/cases/recipetool.py | 114 +++++++++++++----- scripts/lib/devtool/standard.py | 3 + scripts/lib/recipetool/create.py | 63 ++++++---- .../lib/recipetool/create_buildsys_python.py | 73 +++++++++++ 6 files changed, 208 insertions(+), 65 deletions(-)Signature · 11 lines
2.42.0 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#190836): https://lists.openembedded.org/g/openembedded-core/message/190836 Mute This Topic: https://lists.openembedded.org/mt/102645283/3617179 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
Signature · 3 lines
Alexandre Belloni, co-owner and COO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 8 lines · Alexandre Belloni via lists.openembedded.org
Hello,
The tests seem to be failing on the AB:
https://autobuilder.yoctoproject.org/typhoon/#/builders/86/builds/6065/steps/14/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/6062/steps/15/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/6013/steps/14/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/6078/steps/14/logs/stdio
Actually this seems to have been caused by something else.
Quoted 79 lines · Julien Stephan
Quoted 61 lines
Hello all,
This series adds basic support for the pypi class in devtool/recipetool.
The idea is to be able to detect that a package is available on pypi and in that case inherits from pypi class. This helps to produce cleaner recipes for pypi packages.
To do this, I am adding a new optional callback "
process_url" that plugin can register to implement some logic on matching url.By implementing this callback in
create_buildsys_pythonwe can match on pypi URLs directly or on release tarballs hosted on "files.pythonhosted.org".To create a recipe taking advantage of the pypi class we can use one of the following new syntax:
* recipetool create https://pypi.org/project/<package>
* recipetool create https://pypi.org/project/<package>/<version>
* recipetool create https://pypi.org/project/<package> --version <version>or the old syntax:
* recipetool create https://files.pythonhosted.org/packages/<...>
If the URL points to a github URL or a release tarball not coming from "files.pythonhosted.org", the created recipe is the same as before. One can also use the newly added "--no-pypi" switch to NOT inherit from pypi class on matching URL, to keep legacy behaviour.
This series also contains some bug fixes I found during my testing.
Pushed my dev branch here: https://git.yoctoproject.org/poky-contrib/log/?h=jstephan/devtool-add-pypi-support
Cheers
JulienJulien Stephan (7):
bitbake: utils: remove spaces on empty lines
recipetool:create_buildsys_python.py: initialize metadata
recipetool: create: add trailing newlines
recipetool: create: add new optionalprocess_urlcallback for plugins
recipetool:create_buildsys_python: add pypi support oeqa/selftest/recipetool: remove spaces on empty lines oeqa/selftest/recipetool/devtool: add test for pypi classbitbake/lib/bb/utils.py | 16 +-- meta/lib/oeqa/selftest/cases/devtool.py | 4 +- meta/lib/oeqa/selftest/cases/recipetool.py | 114 +++++++++++++----- scripts/lib/devtool/standard.py | 3 + scripts/lib/recipetool/create.py | 63 ++++++---- .../lib/recipetool/create_buildsys_python.py | 73 +++++++++++ 6 files changed, 208 insertions(+), 65 deletions(-)Signature · 1 line
2.42.0Signature · 12 lines
Alexandre Belloni, co-owner and COO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#190862): https://lists.openembedded.org/g/openembedded-core/message/190862 Mute This Topic: https://lists.openembedded.org/mt/102645283/3617179 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
Signature · 3 lines
Alexandre Belloni, co-owner and COO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com