Skip to contentSkip to conversation

wic: pluginbase workaround on invalid module filenames

PATCH In review Anibal Limon 5 participants 13 messages Jun 11, 2025 – Jun 13, 2025

lore
Share & tools
lore
#0 lore raw

PATCH 1/3 wic: pluginbase workaround on invalid module filenames

Apply a workaround on invalid identifiers replace '-' to '_' to load the module with valid identifier.

Let the user know (warning) about plugin doesn't has a valid module identifier to encourage fix, it gives time before rename oe-core wic plugins and drop workaround.

---
 scripts/lib/wic/pluginbase.py | 25 +++++++++++++++++++++----
 1 file changed, 21 insertions(+), 4 deletions(-)
scripts/lib/wic/pluginbase.py +21 -4
index b64568339b..ec515229e7 100644
@@ -8,6 +8,7 @@
 __all__ = ['ImagerPlugin', 'SourcePlugin']
 
 import os
+import sys
 import logging
 import types
 
@@ -20,7 +21,8 @@ from wic.misc import get_bitbake_var
 
 PLUGIN_TYPES = ["imager", "source"]
 
-SCRIPTS_PLUGIN_DIR = ["scripts/lib/wic/plugins", "lib/wic/plugins"]
+SCRIPTS_PLUGIN_WIC_DIR = "wic/plugins"
+SCRIPTS_PLUGIN_BASE_DIR = ["scripts/lib", "lib"]
 
 logger = logging.getLogger('wic')
 
@@ -40,7 +42,9 @@ class PluginMgr:
             cls._plugin_dirs = [os.path.join(os.path.dirname(__file__), 'plugins')]
             layers = get_bitbake_var("BBLAYERS") or ''
             for layer_path in layers.split():
-                for script_plugin_dir in SCRIPTS_PLUGIN_DIR:
+                for script_plugin_base_dir in SCRIPTS_PLUGIN_BASE_DIR:
+                    script_plugin_dir = os.path.join(script_plugin_base_dir,
+                                                     SCRIPTS_PLUGIN_WIC_DIR);
                     path = os.path.join(layer_path, script_plugin_dir)
                     path = os.path.abspath(os.path.expanduser(path))
                     if path not in cls._plugin_dirs and os.path.isdir(path):
@@ -53,13 +57,26 @@ class PluginMgr:
                 if os.path.isdir(ppath):
                     for fname in os.listdir(ppath):
                         if fname.endswith('.py'):
-                            mname = fname[:-3]
                             mpath = os.path.join(ppath, fname)
+                            # If module doesn't have a valid identifier let the
+                            # user know and apply a workaround replacing '-' 
+                            # with '_' to load module with valid identifier.
+                            mname = fname[:-3]
+                            if not mname.isidentifier():
+                                logger.warn("invalid identifier at plugin module"\
+                                            " %s, please fix it.", mpath)
+                                mname = mname.replace("-", "_")
+
                             logger.debug("loading plugin module %s", mpath)
-                            spec = importlib.util.spec_from_file_location(mname, mpath)
+                            spec = importlib.util.spec_from_file_location(
+                                     mname, mpath)
                             module = importlib.util.module_from_spec(spec)
                             spec.loader.exec_module(module)
 
+                            cname = os.path.join(SCRIPTS_PLUGIN_WIC_DIR,
+                                                 ptype, mname).replace('/', '.')
+                            sys.modules[cname] = module
+
         return PLUGINS.get(ptype)
 
 class PluginMeta(type):
Signature · 1 line
2.39.5
#1 lore raw

PATCH 2/3 wic: pluginbase ensure layer order when load plugins

To support extensions on wic plugins, the load order needs to be grauntee matching BBLAYERS variable.

Fix cases when try to import a plugin from another layer, example of the case to fix,

```
Traceback (most recent call last):
  File "/workspaces/ls/linux/layers/openembedded-core/scripts/wic",line
547, in <module>
    sys.exit(main(sys.argv[1:]))
             ^^^^^^^^^^^^^^^^^^
...
  File "/workspaces/ls/linux/layers/openembedded-core/scripts/lib/wic/
engine.py", line 137, in list_source_plugins
    plugins = PluginMgr.get_plugins('source')
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/ls/linux/layers/openembedded-core/scripts/lib/wic/
pluginbase.py", line 73, in get_plugins
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/workspaces/ls/linux/limonsoftware/bsp/scripts/lib/wic/plugins/
source/bootimg_rpi_autoboot_partition.py", line 1, in <module>
    from wic.plugins.source.bootimg_partition import BootimgPartitionPlugin
ModuleNotFoundError: No module named 'wic.plugins.source.bootimg_partition'
```
---
 scripts/lib/wic/pluginbase.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/lib/wic/pluginbase.py +1 -1
index ec515229e7..d2229c91a1 100644
@@ -48,7 +48,7 @@ class PluginMgr:
                     path = os.path.join(layer_path, script_plugin_dir)
                     path = os.path.abspath(os.path.expanduser(path))
                     if path not in cls._plugin_dirs and os.path.isdir(path):
-                        cls._plugin_dirs.insert(0, path)
+                        cls._plugin_dirs.append(path)
 
         if ptype not in PLUGINS:
             # load all ptype plugins
Signature · 1 line
2.39.5
#2 lore raw

PATCH 3/3 wic: plugins source bootimage/isoimage rename to have valid indentifier

In reply to Anibal Limon

Python has rules on valid identifiers and not support modules with '-' on the filename, change to '_';

---
 .../source/{bootimg-biosplusefi.py => bootimg_biosplusefi.py}     | 0
 scripts/lib/wic/plugins/source/{bootimg-efi.py => bootimg_efi.py} | 0
 .../plugins/source/{bootimg-partition.py => bootimg_partition.py} | 0
 .../wic/plugins/source/{bootimg-pcbios.py => bootimg_pcbios.py}   | 0
 .../source/{isoimage-isohybrid.py => isoimage_isohybrid.py}       | 0
 5 files changed, 0 insertions(+), 0 deletions(-)
 rename scripts/lib/wic/plugins/source/{bootimg-biosplusefi.py => bootimg_biosplusefi.py} (100%)
 rename scripts/lib/wic/plugins/source/{bootimg-efi.py => bootimg_efi.py} (100%)
 rename scripts/lib/wic/plugins/source/{bootimg-partition.py => bootimg_partition.py} (100%)
 rename scripts/lib/wic/plugins/source/{bootimg-pcbios.py => bootimg_pcbios.py} (100%)
 rename scripts/lib/wic/plugins/source/{isoimage-isohybrid.py => isoimage_isohybrid.py} (100%)
scripts/lib/wic/plugins/source/bootimg_biosplusefi.py renamed from scripts/lib/wic/plugins/source/bootimg-biosplusefi.py +0 -0
similarity index 100%
rename from scripts/lib/wic/plugins/source/bootimg-biosplusefi.py
rename to scripts/lib/wic/plugins/source/bootimg_biosplusefi.py
scripts/lib/wic/plugins/source/bootimg_efi.py renamed from scripts/lib/wic/plugins/source/bootimg-efi.py +0 -0
similarity index 100%
rename from scripts/lib/wic/plugins/source/bootimg-efi.py
rename to scripts/lib/wic/plugins/source/bootimg_efi.py
scripts/lib/wic/plugins/source/bootimg_partition.py renamed from scripts/lib/wic/plugins/source/bootimg-partition.py +0 -0
similarity index 100%
rename from scripts/lib/wic/plugins/source/bootimg-partition.py
rename to scripts/lib/wic/plugins/source/bootimg_partition.py
scripts/lib/wic/plugins/source/bootimg_pcbios.py renamed from scripts/lib/wic/plugins/source/bootimg-pcbios.py +0 -0
similarity index 100%
rename from scripts/lib/wic/plugins/source/bootimg-pcbios.py
rename to scripts/lib/wic/plugins/source/bootimg_pcbios.py
scripts/lib/wic/plugins/source/isoimage_isohybrid.py renamed from scripts/lib/wic/plugins/source/isoimage-isohybrid.py +0 -0
similarity index 100%
rename from scripts/lib/wic/plugins/source/isoimage-isohybrid.py
rename to scripts/lib/wic/plugins/source/isoimage_isohybrid.py
Signature · 1 line
2.39.5
#3 lore raw

Re: [PATCH 3/3] wic: plugins source bootimage/isoimage rename to have valid indentifier

This patch is the previous one, only sent to note is part of the series and can
be integrated later after layers fix plugin filenames.

Cheers!,
Anibal

Quoted 51 lines · Anibal Limon

Python has rules on valid identifiers and not support modules with '-' on the filename, change to '_';

---
 .../source/{bootimg-biosplusefi.py => bootimg_biosplusefi.py}     | 0
 scripts/lib/wic/plugins/source/{bootimg-efi.py => bootimg_efi.py} | 0
 .../plugins/source/{bootimg-partition.py => bootimg_partition.py} | 0
 .../wic/plugins/source/{bootimg-pcbios.py => bootimg_pcbios.py}   | 0
 .../source/{isoimage-isohybrid.py => isoimage_isohybrid.py}       | 0
 5 files changed, 0 insertions(+), 0 deletions(-)
 rename scripts/lib/wic/plugins/source/{bootimg-biosplusefi.py =>
bootimg_biosplusefi.py} (100%)
 rename scripts/lib/wic/plugins/source/{bootimg-efi.py => bootimg_efi.py}
(100%)
 rename scripts/lib/wic/plugins/source/{bootimg-partition.py =>
bootimg_partition.py} (100%)
 rename scripts/lib/wic/plugins/source/{bootimg-pcbios.py =>
bootimg_pcbios.py} (100%)
 rename scripts/lib/wic/plugins/source/{isoimage-isohybrid.py =>

isoimage_isohybrid.py} (100%)

scripts/lib/wic/plugins/source/bootimg_biosplusefi.py renamed from scripts/lib/wic/plugins/source/bootimg-biosplusefi.py +0 -0
similarity index 100%
rename from scripts/lib/wic/plugins/source/bootimg-biosplusefi.py
rename to scripts/lib/wic/plugins/source/bootimg_biosplusefi.py
scripts/lib/wic/plugins/source/bootimg_efi.py renamed from scripts/lib/wic/plugins/source/bootimg-efi.py +0 -0
similarity index 100%
rename from scripts/lib/wic/plugins/source/bootimg-efi.py
rename to scripts/lib/wic/plugins/source/bootimg_efi.py
scripts/lib/wic/plugins/source/bootimg_partition.py renamed from scripts/lib/wic/plugins/source/bootimg-partition.py +0 -0
similarity index 100%
rename from scripts/lib/wic/plugins/source/bootimg-partition.py
rename to scripts/lib/wic/plugins/source/bootimg_partition.py
scripts/lib/wic/plugins/source/bootimg_pcbios.py renamed from scripts/lib/wic/plugins/source/bootimg-pcbios.py +0 -0
similarity index 100%
rename from scripts/lib/wic/plugins/source/bootimg-pcbios.py
rename to scripts/lib/wic/plugins/source/bootimg_pcbios.py
scripts/lib/wic/plugins/source/isoimage_isohybrid.py renamed from scripts/lib/wic/plugins/source/isoimage-isohybrid.py +0 -1
similarity index 100%
rename from scripts/lib/wic/plugins/source/isoimage-isohybrid.py
rename to scripts/lib/wic/plugins/source/isoimage_isohybrid.py
--

2.39.5

#4 lore raw

Re: [PATCH 3/3] wic: plugins source bootimage/isoimage rename to have valid indentifier

Quoted 2 lines · Anibal Limon

This patch is the previous one, only sent to note is part of the series and can be integrated later after layers fix plugin filenames.

We don’t need to make everyone change their wic files though:

1) change wic to warn if the plugins have invalid filenames
2) rename plugins to be valid
3) when parsing wic files, transform the plugin name with s/-/_. This means we don’t need to alter any existing wic files.

You have 1 and 2 already. 3 is a quick .replace() call.

Cheers,
Ross

#5 lore raw

Re: [PATCH 3/3] wic: plugins source bootimage/isoimage rename to have valid indentifier

Quoted 5 lines · Ross Burton
Quoted 1 line · Anibal Limon

This patch is the previous one, only sent to note is part of the series

and can

Quoted 1 line

be integrated later after layers fix plugin filenames.

Quoted 8 lines

We don’t need to make everyone change their wic files though:

1) change wic to warn if the plugins have invalid filenames
2) rename plugins to be valid
3) when parsing wic files, transform the plugin name with s/-/_. This means we don’t need to alter any existing wic files.

You have 1 and 2 already. 3 is a quick .replace() call.

Yeah ack, we do not need 3 at this stage only give time to fix later.

Cheers,
Anibal

Quoted 4 lines

Cheers,
Ross

#6 lore raw

Re: [OE-core] [PATCH 3/3] wic: plugins source bootimage/isoimage rename to have valid indentifier

Quoted 14 lines · Anibal Limon via lists.openembedded.org
Quoted 12 lines · Ross Burton
Quoted 1 line · Anibal Limon

This patch is the previous one, only sent to note is part of the series

and can

Quoted 1 line

be integrated later after layers fix plugin filenames.

We don’t need to make everyone change their wic files though:

1) change wic to warn if the plugins have invalid filenames
2) rename plugins to be valid
3) when parsing wic files, transform the plugin name with s/-/_. This means we don’t need to alter any existing wic files.

Quoted 4 lines
Quoted 1 line

You have 1 and 2 already. 3 is a quick .replace() call.

Yeah ack, we do not need 3 at this stage only give time to fix later.

I assume this patch is referring to (roughly):
https://packaging.python.org/en/latest/discussions/distribution-package-vs-import-package/#how-do-distribution-package-names-and-import-package-names-compare
?

The name normalization section provides an example:
https://packaging.python.org/en/latest/specifications/name-normalization/#name-normalization

#7 lore raw
In reply to Anibal Limon
Quoted 9 lines · Anibal Limon

Apply a workaround on invalid identifiers replace '-' to '_' to load the module with valid identifier.

Let the user know (warning) about plugin doesn't has a valid module identifier to encourage fix, it gives time before rename oe-core wic plugins and drop workaround.

---

Hi Anibal,

Thanks for the new series.

I believe we have oe oe-selftest errors on x86, while creating image:

2025-06-12 09:33:17,228 - oe-selftest - INFO - overlayfs.OverlayFSEtcRunTimeTests.test_lower_layer_access (subunit.RemotedTestCase)
2025-06-12 09:33:17,228 - oe-selftest - INFO -  ... FAIL
...
|   File "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/scripts/lib/wic/plugins/source/bootimg_biosplusefi.py", line 159, in do_configure_partition
|     cls.__instanciateSubClasses()
|     ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
|   File "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/scripts/lib/wic/plugins/source/bootimg_biosplusefi.py", line 108, in __instanciateSubClasses
|     loader.exec_module(mod)
|     ~~~~~~~~~~~~~~~~~~^^^^^
|   File "<frozen importlib._bootstrap_external>", line 1022, in exec_module
|   File "<frozen importlib._bootstrap_external>", line 1159, in get_code
|   File "<frozen importlib._bootstrap_external>", line 1217, in get_data
| FileNotFoundError: [Errno 2] No such file or directory: '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/scripts/lib/wic/plugins/source/bootimg-pcbios.py'
| WARNING: exit code 1 from a shell command.

https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/1750
https://autobuilder.yoctoproject.org/valkyrie/#/builders/48/builds/1689

Can you have a look at this issue?

Signature · 3 lines
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#8 lore raw

Hi Mathieu,

Quoted 37 lines · Mathieu Dubois-Briand
Quoted 9 lines · Anibal Limon

Apply a workaround on invalid identifiers replace '-' to '_' to load the module with valid identifier.

Let the user know (warning) about plugin doesn't has a valid module identifier to encourage fix, it gives time before rename oe-core wic plugins and drop workaround.

---

Hi Anibal,

Thanks for the new series.

I believe we have oe oe-selftest errors on x86, while creating image:

2025-06-12 09:33:17,228 - oe-selftest - INFO -
overlayfs.OverlayFSEtcRunTimeTests.test_lower_layer_access
(subunit.RemotedTestCase)
2025-06-12 09:33:17,228 - oe-selftest - INFO -  ... FAIL

...
| File
"/srv/pokybuild/yocto-worker/oe-selftest-debian/build/scripts/lib/wic/plugins/source/bootimg_biosplusefi.py", line 159, in do_configure_partition

|     cls.__instanciateSubClasses()
|     ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
|   File
"/srv/pokybuild/yocto-worker/oe-selftest-debian/build/scripts/lib/wic/plugins/source/bootimg_biosplusefi.py",
line 108, in __instanciateSubClasses
|     loader.exec_module(mod)
|     ~~~~~~~~~~~~~~~~~~^^^^^
|   File "<frozen importlib._bootstrap_external>", line 1022, in
exec_module
|   File "<frozen importlib._bootstrap_external>", line 1159, in get_code
|   File "<frozen importlib._bootstrap_external>", line 1217, in get_data
| FileNotFoundError: [Errno 2] No such file or directory:
Quoted 8 lines

'/srv/pokybuild/yocto-worker/oe-selftest-debian/build/scripts/lib/wic/plugins/source/bootimg-pcbios.py' | WARNING: exit code 1 from a shell command.

https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/1750
https://autobuilder.yoctoproject.org/valkyrie/#/builders/48/builds/1689

Can you have a look at this issue?

I saw RP is fixing the renaming issues, applying patches upon, to see if it is possible to land the renames.
If not, only the initial two patches of the series will be applied.

Thanks!,
Anibal

Quoted 7 lines

--
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

#9 lore raw
In reply to Anibal Limon
Quoted 6 lines · Anibal Limon via lists.openembedded.org

Apply a workaround on invalid identifiers replace '-' to '_' to load the module with valid identifier.

Let the user know (warning) about plugin doesn't has a valid module identifier to encourage fix, it gives time before rename oe-core wic plugins and drop workaround.

As far as I'm aware, everything in wic currently works, despite these bad names. I'm curious to find out what your use-case is that prompted this patch. Are you hoping to use these plugins with another tool?

Quoted 42 lines
---
 scripts/lib/wic/pluginbase.py | 25 +++++++++++++++++++++----
 1 file changed, 21 insertions(+), 4 deletions(-)
scripts/lib/wic/pluginbase.py +6 -2
index b64568339b..ec515229e7 100644
@@ -8,6 +8,7 @@
 __all__ = ['ImagerPlugin', 'SourcePlugin']
 
 import os
+import sys
 import logging
 import types
 
@@ -20,7 +21,8 @@ from wic.misc import get_bitbake_var
 
 PLUGIN_TYPES = ["imager", "source"]
 
-SCRIPTS_PLUGIN_DIR = ["scripts/lib/wic/plugins", "lib/wic/plugins"]
+SCRIPTS_PLUGIN_WIC_DIR = "wic/plugins"
+SCRIPTS_PLUGIN_BASE_DIR = ["scripts/lib", "lib"]
 
 logger = logging.getLogger('wic')
 
@@ -40,7 +42,9 @@ class PluginMgr:
             cls._plugin_dirs = [os.path.join(os.path.dirname(__file__), 'plugins')]
             layers = get_bitbake_var("BBLAYERS") or ''
             for layer_path in layers.split():
-                for script_plugin_dir in SCRIPTS_PLUGIN_DIR:
+                for script_plugin_base_dir in SCRIPTS_PLUGIN_BASE_DIR:
+                    script_plugin_dir = os.path.join(script_plugin_base_dir,
+                                                     SCRIPTS_PLUGIN_WIC_DIR);
                     path = os.path.join(layer_path, script_plugin_dir)
                     path = os.path.abspath(os.path.expanduser(path))
                     if path not in cls._plugin_dirs and os.path.isdir(path):
@@ -53,13 +57,26 @@ class PluginMgr:
                 if os.path.isdir(ppath):
                     for fname in os.listdir(ppath):
                         if fname.endswith('.py'):

scripts/lib/wic/pluginbase.py class PluginMgr: · line 68

Quoted 10 lines
scripts/lib/wic/pluginbase.py +8 -1
-                            mname = fname[:-3]
                             mpath = os.path.join(ppath, fname)
+                            # If module doesn't have a valid identifier let the
+                            # user know and apply a workaround replacing '-' 
+                            # with '_' to load module with valid identifier.
+                            mname = fname[:-3]
+                            if not mname.isidentifier():
+                                logger.warn("invalid identifier at plugin module"\
+                                            " %s, please fix it.", mpath)
+                                mname = mname.replace("-", "_")

There are several ways that a module name could be incorrect in addition to using dashes (upper vs lower case, periods, etc), and I believe runs of dashes/dots/underscores are to be replaced with a single underscore:
https://packaging.python.org/en/latest/specifications/name-normalization/#name-normalization

So if we do move forward with this, maybe it should be expanded to include all the cases.

Quoted 28 lines
scripts/lib/wic/pluginbase.py +7 -1
+
                             logger.debug("loading plugin module %s", mpath)
-                            spec = importlib.util.spec_from_file_location(mname, mpath)
+                            spec = importlib.util.spec_from_file_location(
+                                     mname, mpath)
                             module = importlib.util.module_from_spec(spec)
                             spec.loader.exec_module(module)
 
+                            cname = os.path.join(SCRIPTS_PLUGIN_WIC_DIR,
+                                                 ptype, mname).replace('/', '.')
+                            sys.modules[cname] = module
+
         return PLUGINS.get(ptype)
 
 class PluginMeta(type):
Signature · 11 lines
2.39.5



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#218458): https://lists.openembedded.org/g/openembedded-core/message/218458
Mute This Topic: https://lists.openembedded.org/mt/113590717/900817
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
#10 lore raw
Quoted 5 lines · Trevor Woerner
Quoted 3 lines · Anibal Limon via lists.openembedded.org

Apply a workaround on invalid identifiers replace '-' to '_' to load the module with valid identifier.

Quoted 9 lines
Quoted 3 lines

Let the user know (warning) about plugin doesn't has a valid module identifier to encourage fix, it gives time before rename oe-core wic plugins and drop workaround.

As far as I'm aware, everything in wic currently works, despite these bad names. I'm curious to find out what your use-case is that prompted this patch.
Are you hoping to use these plugins with another tool?

Is to enable extending wic plugins in other layers, see for example meta-virtualization
code to enable import, removed after this fix.

https://lists.yoctoproject.org/g/meta-virtualization/message/9240?p=%2C%2C%2C20%2C0%2C0%2C0%3A%3Acreated%2C%2Cwic%2C20%2C2%2C0%2C112618356

Regards!,
Anibal

Quoted 111 lines

Review

Quoted 7 lines
---
 scripts/lib/wic/pluginbase.py | 25 +++++++++++++++++++++----
 1 file changed, 21 insertions(+), 4 deletions(-)
(quoted diff) +0 -0

b/scripts/lib/wic/pluginbase.py

Quoted 22 lines

index b64568339b..ec515229e7 100644

scripts/lib/wic/pluginbase.py +3 -1
@@ -8,6 +8,7 @@
 __all__ = ['ImagerPlugin', 'SourcePlugin']
 
 import os
+import sys
 import logging
 import types
 
@@ -20,7 +21,8 @@ from wic.misc import get_bitbake_var
 
 PLUGIN_TYPES = ["imager", "source"]
 
-SCRIPTS_PLUGIN_DIR = ["scripts/lib/wic/plugins", "lib/wic/plugins"]
+SCRIPTS_PLUGIN_WIC_DIR = "wic/plugins"
+SCRIPTS_PLUGIN_BASE_DIR = ["scripts/lib", "lib"]
 
 logger = logging.getLogger('wic')
 

scripts/lib/wic/pluginbase.py class PluginMgr: · line 42

Quoted 1 line
scripts/lib/wic/pluginbase.py +0 -0
@@ -40,7 +42,9 @@ class PluginMgr:
             cls._plugin_dirs = [os.path.join(os.path.dirname(__file__),

'plugins')]

scripts/lib/wic/pluginbase.py class PluginMgr:

Quoted 5 lines
scripts/lib/wic/pluginbase.py +2 -1
             layers = get_bitbake_var("BBLAYERS") or ''
             for layer_path in layers.split():
-                for script_plugin_dir in SCRIPTS_PLUGIN_DIR:
+                for script_plugin_base_dir in SCRIPTS_PLUGIN_BASE_DIR:
+                    script_plugin_dir =

os.path.join(script_plugin_base_dir,

scripts/lib/wic/pluginbase.py class PluginMgr: · line 47

Quoted 1 line
scripts/lib/wic/pluginbase.py +1 -0
+

SCRIPTS_PLUGIN_WIC_DIR);

scripts/lib/wic/pluginbase.py class PluginMgr:

Quoted 3 lines
scripts/lib/wic/pluginbase.py +0 -0
                     path = os.path.join(layer_path, script_plugin_dir)
                     path = os.path.abspath(os.path.expanduser(path))
                     if path not in cls._plugin_dirs and

os.path.isdir(path):

scripts/lib/wic/pluginbase.py class PluginMgr: · line 61

Quoted 7 lines
scripts/lib/wic/pluginbase.py +1 -1
@@ -53,13 +57,26 @@ class PluginMgr:
                 if os.path.isdir(ppath):
                     for fname in os.listdir(ppath):
                         if fname.endswith('.py'):
-                            mname = fname[:-3]
                             mpath = os.path.join(ppath, fname)
+                            # If module doesn't have a valid identifier

let the

scripts/lib/wic/pluginbase.py class PluginMgr: · line 62

Quoted 1 line
scripts/lib/wic/pluginbase.py +1 -0
+                            # user know and apply a workaround

replacing '-'

scripts/lib/wic/pluginbase.py class PluginMgr: · line 63

Quoted 1 line
scripts/lib/wic/pluginbase.py +1 -0
+                            # with '_' to load module with valid

identifier.

scripts/lib/wic/pluginbase.py class PluginMgr:

Quoted 3 lines
scripts/lib/wic/pluginbase.py +3 -0
+                            mname = fname[:-3]
+                            if not mname.isidentifier():
+                                logger.warn("invalid identifier at

plugin module"\

scripts/lib/wic/pluginbase.py class PluginMgr: · line 67

Quoted 1 line
scripts/lib/wic/pluginbase.py +1 -0
+                                            " %s, please fix it.",

mpath)

scripts/lib/wic/pluginbase.py class PluginMgr: · line 68

Quoted 1 line
scripts/lib/wic/pluginbase.py +1 -0
+                                mname = mname.replace("-", "_")

There are several ways that a module name could be incorrect in addition to using dashes (upper vs lower case, periods, etc), and I believe runs of dashes/dots/underscores are to be replaced with a single underscore:

https://packaging.python.org/en/latest/specifications/name-normalization/#name-normalization

So if we do move forward with this, maybe it should be expanded to include all
the cases.

scripts/lib/wic/pluginbase.py class PluginMgr: · line 70

Quoted 2 lines
scripts/lib/wic/pluginbase.py +1 -0
+
                             logger.debug("loading plugin module %s",

mpath)

scripts/lib/wic/pluginbase.py class PluginMgr: · line 59

Quoted 1 line
scripts/lib/wic/pluginbase.py +0 -1
-                            spec =

importlib.util.spec_from_file_location(mname, mpath)

scripts/lib/wic/pluginbase.py class PluginMgr: · line 71

Quoted 1 line
scripts/lib/wic/pluginbase.py +1 -0
+                            spec =

importlib.util.spec_from_file_location(

scripts/lib/wic/pluginbase.py class PluginMgr: · line 73

Quoted 2 lines
scripts/lib/wic/pluginbase.py +1 -0
+                                     mname, mpath)
                             module =

importlib.util.module_from_spec(spec)

scripts/lib/wic/pluginbase.py class PluginMgr:

Quoted 4 lines
scripts/lib/wic/pluginbase.py +2 -0
                             spec.loader.exec_module(module)
 
+                            cname = os.path.join(SCRIPTS_PLUGIN_WIC_DIR,
+                                                 ptype,

mname).replace('/', '.')

Quoted 10 lines
scripts/lib/wic/pluginbase.py +2 -1
+                            sys.modules[cname] = module
+
         return PLUGINS.get(ptype)
 
 class PluginMeta(type):
--

2.39.5

Quoted 3 lines

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#218458):

https://lists.openembedded.org/g/openembedded-core/message/218458

Quoted 3 lines

Mute This Topic: https://lists.openembedded.org/mt/113590717/900817
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [

[email protected]]

Quoted 2 lines

-=-=-=-=-=-=-=-=-=-=-=-

#11 lore raw
In reply to Anibal Limon
Quoted 11 lines · Anibal Limon

Apply a workaround on invalid identifiers replace '-' to '_' to load the module with valid identifier.

Let the user know (warning) about plugin doesn't has a valid module identifier to encourage fix, it gives time before rename oe-core wic plugins and drop workaround.

---
 scripts/lib/wic/pluginbase.py | 25 +++++++++++++++++++++----

 1 file changed, 21 insertions(+), 4 deletions(-)

I think there was a bit of a misunderstanding about what Ross/I were suggesting. I've sent out an alternative patch showing what I was thinking of, basically just renaming the plugin names passed in into the correct format.

We could put warnings in the code about old naming too, I've not done that but perhaps we should, maybe as a function alongside get_plugins().

Cheers,

Richard

#12 lore raw
Quoted 13 lines · Richard Purdie
Quoted 11 lines · Anibal Limon

Apply a workaround on invalid identifiers replace '-' to '_' to load the module with valid identifier.

Let the user know (warning) about plugin doesn't has a valid module identifier to encourage fix, it gives time before rename oe-core wic plugins and drop workaround.

---
 scripts/lib/wic/pluginbase.py | 25 +++++++++++++++++++++----
 1 file changed, 21 insertions(+), 4 deletions(-)
Quoted 5 lines

I think there was a bit of a misunderstanding about what Ross/I were suggesting. I've sent out an alternative patch showing what I was thinking of, basically just renaming the plugin names passed in into the correct format.

Hey, so yeah looks fine to me, better patch then!, :).
I will give a look and do my test case for extension.

Thanks!,
Anibal

Quoted 9 lines

We could put warnings in the code about old naming too, I've not done that but perhaps we should, maybe as a function alongside get_plugins().

Cheers,

Richard