master.cfg: Replace pkg_resources with importlib.metadata
RFC 1/1 master.cfg: Replace pkg_resources with importlib.metadata
pkg_resources (from setuptools) is deprecated. Replace the fake Distribution/EntryPoint/working_set manipulation with importlib.metadata equivalents:
- Intercept importlib.metadata.version() for the local plugin, since buildbot's Application() calls it at import time before the plugin is registered.
- Use importlib.metadata.EntryPoint directly instead of the old pkg_resources.EntryPoint.parse() plus fake Distribution approach.
- Remove the now-unused os import.
---
This has been working for me on my local AB cluster, where the controller is running on a Debian 13 AMD Ryzen system.
master.cfg | 61 +++++++++++++++++++++++++++--------------------------- 1 file changed, 31 insertions(+), 30 deletions(-)
master.cfg +31 -30
@@ -1,46 +1,47 @@ # -*- python -*- # ex: set filetype=python: -import os import importlib -import pkg_resources - -from buildbot.plugins import * -from buildbot.plugins import db -from yoctoabb import builders, config, schedulers, workers, services, www +import importlib.metadata +from importlib.metadata import EntryPoint # -# Prepare to feel ill +# Buildbot discovers www plugins via entry points. Since the yocto_console_view +# plugin lives in this repo rather than being installed as a package, we need +# two workarounds: # -# Buildbot uses plugins from pkg_resources, i.e. things which are installed -# For the Yocto autobuilder, we want the plugin from this repo/directory. +# 1. Buildbot's Application() calls importlib.metadata.version() at import +# time, so we intercept that to return a version for our local plugin +# before any yoctoabb imports occur. # -# Firstly we therefore have to create a dummy pkg_resources entry/distribution -# which we add into the global working set. Ugly. It gets worse. +# 2. The get_plugins('www') call from www/service.py happens before this +# master.cfg file is parsed, so we create an EntryPoint directly and +# register it into buildbot's internal plugin DB. # -# The get_plugins('www') call from www/service.py happens before this -# master.cfg file is parsed, which means our plugin won't be found. There -# is no API to rescan for plugins. We therefore so some horrible internal -# API monkey patching to add ourselves into the internal list. -# Create a fake distribution to insert into the global working_set -us = os.path.dirname(os.path.realpath(__file__)) -d = pkg_resources.Distribution(us + "/yocto_console_view", metadata=None, project_name="yocto_console_view", version="0.0.1") +# Handle importlib.metadata.version() for our uninstalled local plugin. +# This must happen before importing yoctoabb, since that triggers +# Application().__init__ which calls importlib.metadata.version(). +_orig_metadata_version = importlib.metadata.version +def _local_metadata_version(name): + if name == 'yoctoabb.yocto_console_view.yocto_console_view': + return '0.0.1' + return _orig_metadata_version(name) +importlib.metadata.version = _local_metadata_version -# Create the fake entry point definition -ep = pkg_resources.EntryPoint.parse("yocto_console_view = yoctoabb.yocto_console_view.yocto_console_view:ep", dist=d) - -# Add the mapping to the fake EntryPoint -d._ep_map = {'buildbot.www': {'yocto_console_view': ep}} +from buildbot.plugins import * +from buildbot.plugins import db +from yoctoabb import builders, config, schedulers, workers, services, www -# Add the fake distribution to the global working_set -pkg_resources.working_set.add(d) +ep_entry = EntryPoint( + name='yocto_console_view', + value='yoctoabb.yocto_console_view.yocto_console_view:ep', + group='buildbot.www' +) -# Now monkey patch us into buildbot's plugins DB -for entry in pkg_resources.iter_entry_points('buildbot.www'): - if entry.name == "yocto_console_view": - plugindb = db._DB._namespaces['www'] - plugindb._real_tree.add(entry.name, db._PluginEntry('buildbot.www', entry, plugindb._load_entry)) +# Register the plugin directly into buildbot's plugin DB +plugindb = db._DB._namespaces['www'] +plugindb._real_tree.add(ep_entry.name, db._PluginEntry('buildbot.www', ep_entry, plugindb._load_entry)) # supports re-loading configuration with buildbot sighup, meaning we can change
Signature · 1 line
2.50.1 (Apple Git-155)
Quoted 11 lines · Tim Orling via lists.yoctoproject.org
pkg_resources(from setuptools) is deprecated. Replace the fake Distribution/EntryPoint/working_set manipulation with importlib.metadata equivalents:- Intercept
importlib.metadata.version()for the local plugin, since buildbot'sApplication()calls it at import time before the plugin is registered.
- Use importlib.metadata.EntryPoint directly instead of the oldpkg_resources.EntryPoint.parse()plus fake Distribution approach.
- Remove the now-unused os import.
Quoted 4 lines
Signed-off-by Tim Orling <[email protected]>---
This has been working for me on my local AB cluster, where the controller is running on a Debian 13 AMD Ryzen system.
Nice!
I think Michael gave up on the local install and is building/running things a bit differently. I did find this approach handy for local development and did try and get it working again but failed and couldn't spend more time on it.
I'm happy to merge this and replace what is there as it has to be an improvement, thanks!
Cheers,
Richard