Skip to contentSkip to conversation

image-bootfiles.bbclass: new class, copy boot files to /boot on rfs

PATCH In review Marcus Folkesson 3 participants 9 messages May 21, 2024 – May 23, 2024

lore
Share & tools
lore
#0 lore raw

PATCH 1/2 image-bootfiles.bbclass: new class, copy boot files to /boot on rfs

image-bootfiles class copy files listed IMAGE_BOOT_FILES to the /boot directory of the root filesystem.

This is useful when there is no explicit boot partition but all boot files should instead reside inside the root filesystem.

---
 meta/classes/image-bootfiles.bbclass | 69 ++++++++++++++++++++++++++++
 1 file changed, 69 insertions(+)
 create mode 100644 meta/classes/image-bootfiles.bbclass
meta/classes/image-bootfiles.bbclass new file +69 -0
new file mode 100644
index 0000000000..850e14e4bb
@@ -0,0 +1,69 @@
+#
+# Writes IMAGE_BOOT_FILES to the /boot directory
+#
+# Copyright (C) 2024 Marcus Folkesson
+# Author: Marcus Folkesson <[email protected]>
+#
+# Licensed under the MIT license, see COPYING.MIT for details
+# Inspired of bootimg-partition.py
+#
+# Usage: add INHERIT += "image-bootfiles" to your conf file
+#
+
+def bootfiles_populate(d):
+    import re
+    from glob import glob
+    import shutil
+
+    boot_files = d.getVar("IMAGE_BOOT_FILES")
+    deploy_image_dir = d.getVar("DEPLOY_DIR_IMAGE")
+    boot_dir = d.getVar("IMAGE_ROOTFS") + "/boot"
+    install_files = []
+
+    if boot_files is None:
+        return
+
+    # list of tuples (src_name, dst_name)
+    deploy_files = []
+    for src_entry in re.findall(r'[\w;\-\./\*]+', boot_files):
+        if ';' in src_entry:
+            dst_entry = tuple(src_entry.split(';'))
+            if not dst_entry[0] or not dst_entry[1]:
+                raise bb.parse.SkipRecipe('Malformed boot file entry: %s' % src_entry)
+        else:
+            dst_entry = (src_entry, src_entry)
+
+        deploy_files.append(dst_entry)
+
+    for deploy_entry in deploy_files:
+        src, dst = deploy_entry
+        if '*' in src:
+            # by default install files under their basename
+            entry_name_fn = os.path.basename
+            if dst != src:
+                # unless a target name was given, then treat name
+                # as a directory and append a basename
+                entry_name_fn = lambda name: \
+                                os.path.join(dst,
+                                             os.path.basename(name))
+
+
+                srcs = glob(os.path.join(deploy_image_dir, src))
+                for entry in srcs:
+                    src = os.path.relpath(entry, deploy_mage_dir)
+                    entry_dst_name = entry_name_fn(entry)
+                    install_files.append((src, entry_dst_name))
+            else:
+                install_files.append((src, dst))
+
+    for entry in install_files:
+        src, dst = entry
+        image_src = os.path.join(deploy_image_dir, src)
+        image_dst = os.path.join(boot_dir, dst)
+        shutil.copyfile(image_src, image_dst)
+
+python bootfiles () {
+   bootfiles_populate(d),
+}
+
+IMAGE_PREPROCESS_COMMAND += "bootfiles;"
Signature · 1 line
2.44.0
#1 lore raw

PATCH 2/2 ref-manual: classes: add new image-bootfiles class

Describe the newly introduced image-bootfiles class.

---
 documentation/ref-manual/classes.rst | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
documentation/ref-manual/classes.rst +14 -0
index 9520d0bf7c..53b3697bee 100644
@@ -1169,6 +1169,20 @@ Yocto Project Overview and Concepts Manual.
 
 .. _ref-classes-image-buildinfo:
 
+``image-bootfiles``
+===================
+
+The :ref:`ref-classes-image-buildinfo` class copies over files listed
+in :ref:`IMAGE_BOOT_FILES` to the /boot directory of the root filesystem.
+
+This can be useful if no separate boot partition is used but all boot files
+should be included into the rootfs image.
+
+:ref:`IMAGE_BOOT_FILES` is the same space-separated list of files used
+by the ``bootimg-partition`` source plugin to populate the boot partition.
+
+.. _ref-classes-image_types:
+
 ``image-buildinfo``
 ===================
 
Signature · 1 line
2.44.0
#2 lore raw

Re: [OE-core] [PATCH 2/2] ref-manual: classes: add new image-bootfiles class

Hi Markus,

Quoted 14 lines · Marcus Folkesson via lists.openembedded.org

[You don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]

Describe the newly introduced image-bootfiles class.

---
  documentation/ref-manual/classes.rst | 14 ++++++++++++++
  1 file changed, 14 insertions(+)
documentation/ref-manual/classes.rst +0 -0
index 9520d0bf7c..53b3697bee 100644

documentation/ref-manual/classes.rst Yocto Project Overview and Concepts Manual. · line 1171

Quoted 3 lines
documentation/ref-manual/classes.rst +0 -0
@@ -1169,6 +1169,20 @@ Yocto Project Overview and Concepts Manual.
 
  .. _ref-classes-image-buildinfo:
 

This isn't the appropriate name for the reflink, we should ave

.. _ref-classes-image-bootfiles:

instead, to match the actual name of the class.

documentation/ref-manual/classes.rst Yocto Project Overview and Concepts Manual. · line 1175

Quoted 4 lines
documentation/ref-manual/classes.rst +4 -0
+``image-bootfiles``
+===================
+
+The :ref:`ref-classes-image-buildinfo` class copies over files listed

That's not the appropriate name, it should be ref-classes-image-bootfiles (it currently renders properly because the reflink is incorrect).

documentation/ref-manual/classes.rst Yocto Project Overview and Concepts Manual. · line 1177

Quoted 2 lines
documentation/ref-manual/classes.rst +2 -0
+in :ref:`IMAGE_BOOT_FILES` to the /boot directory of the root filesystem.
+

Tick quote /boot:
/boot
it;'s not meant to be understood as an English word.

documentation/ref-manual/classes.rst Yocto Project Overview and Concepts Manual. · line 1178

Quoted 1 line
documentation/ref-manual/classes.rst +1 -0
+This can be useful if no separate boot partition is used but all boot files

Replace "all" with "some", it depends on the content of IMAGE_BOOT_FILES, so we cannot say "all" here, it is misleading.

documentation/ref-manual/classes.rst Yocto Project Overview and Concepts Manual. · line 1183

Quoted 5 lines
documentation/ref-manual/classes.rst +5 -0
+should be included into the rootfs image.
+
+:ref:`IMAGE_BOOT_FILES` is the same space-separated list of files used
+by the ``bootimg-partition`` source plugin to populate the boot partition.
+

I would not mention this here, rather edit the variables glossary entry for IMAGE_BOOT_FILES to mention that this is also used by image-bootfiles. Also, it seems bootimg-efi plugin uses it, according to the glossary entry.

documentation/ref-manual/classes.rst Yocto Project Overview and Concepts Manual. · line 1184

Quoted 1 line
documentation/ref-manual/classes.rst +1 -0
+.. _ref-classes-image_types:

This isn't the appropriate name for the reflink, we should have

.. _ref-classes-image-buildinfo:

instead.

Cheers,
Quentin

#3 lore raw

Re: [OE-core] [PATCH 2/2] ref-manual: classes: add new image-bootfiles class

Hi Quentin,

Quoted 59 lines · Quentin Schulz

Hi Markus,

Quoted 14 lines · Marcus Folkesson via lists.openembedded.org

[You don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]

Describe the newly introduced image-bootfiles class.

---
  documentation/ref-manual/classes.rst | 14 ++++++++++++++
  1 file changed, 14 insertions(+)
documentation/ref-manual/classes.rst +0 -0
index 9520d0bf7c..53b3697bee 100644

documentation/ref-manual/classes.rst Yocto Project Overview and Concepts Manual. · line 1171

Quoted 3 lines
documentation/ref-manual/classes.rst +0 -0
@@ -1169,6 +1169,20 @@ Yocto Project Overview and Concepts Manual.
 
  .. _ref-classes-image-buildinfo:
 

This isn't the appropriate name for the reflink, we should ave

.. _ref-classes-image-bootfiles:

instead, to match the actual name of the class.

documentation/ref-manual/classes.rst Yocto Project Overview and Concepts Manual. · line 1175

Quoted 4 lines
documentation/ref-manual/classes.rst +4 -0
+``image-bootfiles``
+===================
+
+The :ref:`ref-classes-image-buildinfo` class copies over files listed

That's not the appropriate name, it should be ref-classes-image-bootfiles (it currently renders properly because the reflink is incorrect).

documentation/ref-manual/classes.rst Yocto Project Overview and Concepts Manual. · line 1177

Quoted 2 lines
documentation/ref-manual/classes.rst +2 -0
+in :ref:`IMAGE_BOOT_FILES` to the /boot directory of the root filesystem.
+

Tick quote /boot:
/boot
it;'s not meant to be understood as an English word.

documentation/ref-manual/classes.rst Yocto Project Overview and Concepts Manual. · line 1178

Quoted 1 line
documentation/ref-manual/classes.rst +1 -0
+This can be useful if no separate boot partition is used but all boot files

Replace "all" with "some", it depends on the content of IMAGE_BOOT_FILES, so we cannot say "all" here, it is misleading.

documentation/ref-manual/classes.rst Yocto Project Overview and Concepts Manual. · line 1183

Quoted 5 lines
documentation/ref-manual/classes.rst +5 -0
+should be included into the rootfs image.
+
+:ref:`IMAGE_BOOT_FILES` is the same space-separated list of files used
+by the ``bootimg-partition`` source plugin to populate the boot partition.
+

I would not mention this here, rather edit the variables glossary entry for IMAGE_BOOT_FILES to mention that this is also used by image-bootfiles. Also, it seems bootimg-efi plugin uses it, according to the glossary entry.

documentation/ref-manual/classes.rst Yocto Project Overview and Concepts Manual. · line 1184

Quoted 1 line
documentation/ref-manual/classes.rst +1 -0
+.. _ref-classes-image_types:
Quoted 8 lines

This isn't the appropriate name for the reflink, we should have

.. _ref-classes-image-buildinfo:

instead.

Cheers,
Quentin

Thank you for all the comments, I agree with them all and will update the patch accordingly.

FYI, I will drop the documentation patch from this series and resend it once the class is ready for merging.
It was suggested by the doc mailing list (had to resend it as I was not subscribed, so it is not part of this thread).

Best regards,
Marcus Folkesson

#4 lore raw
In reply to Marcus Folkesson

Hi Markus,

Quoted 21 lines · Marcus Folkesson via lists.openembedded.org

[You don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]

image-bootfiles class copy files listed IMAGE_BOOT_FILES to the /boot directory of the root filesystem.

This is useful when there is no explicit boot partition but all boot files should instead reside inside the root filesystem.

---
  meta/classes/image-bootfiles.bbclass | 69 ++++++++++++++++++++++++++++
  1 file changed, 69 insertions(+)

create mode 100644 meta/classes/image-bootfiles.bbclass

meta/classes/image-bootfiles.bbclass new file +1 -0
new file mode 100644
index 0000000000..850e14e4bb
@@ -0,0 +1,69 @@
+#

meta/classes/image-bootfiles.bbclass line 11

Quoted 10 lines
meta/classes/image-bootfiles.bbclass new file +10 -0
+# Writes IMAGE_BOOT_FILES to the /boot directory
+#
+# Copyright (C) 2024 Marcus Folkesson
+# Author: Marcus Folkesson <[email protected]>
+#
+# Licensed under the MIT license, see COPYING.MIT for details
+# Inspired of bootimg-partition.py
+#
+# Usage: add INHERIT += "image-bootfiles" to your conf file
+#

Since it's in meta/classes, I assume we can also inherit image-bootfiles in an image recipe?

Quoted 7 lines
meta/classes/image-bootfiles.bbclass new file +7 -0
+
+def bootfiles_populate(d):
+    import re
+    from glob import glob
+    import shutil
+
+    boot_files = d.getVar("IMAGE_BOOT_FILES")

meta/classes/image-bootfiles.bbclass line 28

Quoted 10 lines
meta/classes/image-bootfiles.bbclass new file +10 -0
+    deploy_image_dir = d.getVar("DEPLOY_DIR_IMAGE")
+    boot_dir = d.getVar("IMAGE_ROOTFS") + "/boot"
+    install_files = []
+
+    if boot_files is None:
+        return
+
+    # list of tuples (src_name, dst_name)
+    deploy_files = []
+    for src_entry in re.findall(r'[\w;\-\./\*]+', boot_files):

Isn't this a more complex boot_files.split()? What do we gain from using this regex? If we have malformed IMAGE_BOOT_FILES, this would actually introduce some logic error:

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

s="plop\\plip;plep plop;plap"
re.findall(r'[\w;\-\./\*]+', s)

['plop', 'plip;plep', 'plop;plap']

I'm pretty sure we don't want that. Instead, I would suggest to use split() and iterate over that, split(';'), check for each if there's 1 or 2 entries, add one if it's one.

Quoted 26 lines
meta/classes/image-bootfiles.bbclass new file +26 -0
+        if ';' in src_entry:
+            dst_entry = tuple(src_entry.split(';'))
+            if not dst_entry[0] or not dst_entry[1]:
+                raise bb.parse.SkipRecipe('Malformed boot file entry: %s' % src_entry)
+        else:
+            dst_entry = (src_entry, src_entry)
+
+        deploy_files.append(dst_entry)
+
+    for deploy_entry in deploy_files:
+        src, dst = deploy_entry
+        if '*' in src:
+            # by default install files under their basename
+            entry_name_fn = os.path.basename
+            if dst != src:
+                # unless a target name was given, then treat name
+                # as a directory and append a basename
+                entry_name_fn = lambda name: \
+                                os.path.join(dst,
+                                             os.path.basename(name))
+
+
+                srcs = glob(os.path.join(deploy_image_dir, src))
+                for entry in srcs:
+                    src = os.path.relpath(entry, deploy_mage_dir)
+                    entry_dst_name = entry_name_fn(entry)

meta/classes/image-bootfiles.bbclass line 64

Quoted 10 lines
meta/classes/image-bootfiles.bbclass new file +10 -0
+                    install_files.append((src, entry_dst_name))
+            else:
+                install_files.append((src, dst))
+
+    for entry in install_files:
+        src, dst = entry
+        image_src = os.path.join(deploy_image_dir, src)
+        image_dst = os.path.join(boot_dir, dst)
+        shutil.copyfile(image_src, image_dst)
+

We literally iterate three times over the same files, I don't see a reason for not merging those three for loops into one?

Also, I now realize that this was heavily inspired from what's in scripts/lib/wic/plugins/source/bootimg-partition.py, would have been nice to say so ;)

meta/classes/image-bootfiles.bbclass line 66

Quoted 2 lines
meta/classes/image-bootfiles.bbclass new file +2 -0
+python bootfiles () {
+   bootfiles_populate(d),

Not entirely convinced we should have a comma after bootfiles_populate here?

More general question:
1) how does this work if one has a boot partition created by wic (via the plugin bootimg-partition?
2) shouldn't we factor out this code here and bootimg-partition's? or make bootimg-partition somehow call bootfiles IMAGE_PREPROCESS_COMMAND to avoid code duplication?

Cheers,
Quentin

#5 lore raw

I also have to repeat what I said in docs@ list: why do we need a special mechanism for placing files into /boot? If /boot is not a special partition, then it's just a folder in / and can be populated from packages like any other folder.

Alex

Quoted 136 lines · Quentin Schulz via lists.yoctoproject.org

Hi Markus,

Quoted 21 lines · Marcus Folkesson via lists.openembedded.org

[You don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]

image-bootfiles class copy files listed IMAGE_BOOT_FILES to the /boot directory of the root filesystem.

This is useful when there is no explicit boot partition but all boot files should instead reside inside the root filesystem.

---
  meta/classes/image-bootfiles.bbclass | 69 ++++++++++++++++++++++++++++
  1 file changed, 69 insertions(+)

create mode 100644 meta/classes/image-bootfiles.bbclass

meta/classes/image-bootfiles.bbclass new file +1 -0
new file mode 100644
index 0000000000..850e14e4bb
@@ -0,0 +1,69 @@
+#

meta/classes/image-bootfiles.bbclass line 11

Quoted 10 lines
meta/classes/image-bootfiles.bbclass new file +10 -0
+# Writes IMAGE_BOOT_FILES to the /boot directory
+#
+# Copyright (C) 2024 Marcus Folkesson
+# Author: Marcus Folkesson <[email protected]>
+#
+# Licensed under the MIT license, see COPYING.MIT for details
+# Inspired of bootimg-partition.py
+#
+# Usage: add INHERIT += "image-bootfiles" to your conf file
+#

Since it's in meta/classes, I assume we can also inherit image-bootfiles in an image recipe?

Quoted 7 lines
meta/classes/image-bootfiles.bbclass new file +7 -0
+
+def bootfiles_populate(d):
+    import re
+    from glob import glob
+    import shutil
+
+    boot_files = d.getVar("IMAGE_BOOT_FILES")

meta/classes/image-bootfiles.bbclass line 28

Quoted 10 lines
meta/classes/image-bootfiles.bbclass new file +10 -0
+    deploy_image_dir = d.getVar("DEPLOY_DIR_IMAGE")
+    boot_dir = d.getVar("IMAGE_ROOTFS") + "/boot"
+    install_files = []
+
+    if boot_files is None:
+        return
+
+    # list of tuples (src_name, dst_name)
+    deploy_files = []
+    for src_entry in re.findall(r'[\w;\-\./\*]+', boot_files):

Isn't this a more complex boot_files.split()? What do we gain from using this regex? If we have malformed IMAGE_BOOT_FILES, this would actually introduce some logic error:

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

s="plop\\plip;plep plop;plap"
re.findall(r'[\w;\-\./\*]+', s)

['plop', 'plip;plep', 'plop;plap']

I'm pretty sure we don't want that. Instead, I would suggest to use split() and iterate over that, split(';'), check for each if there's 1 or 2 entries, add one if it's one.

Quoted 26 lines
meta/classes/image-bootfiles.bbclass new file +26 -0
+        if ';' in src_entry:
+            dst_entry = tuple(src_entry.split(';'))
+            if not dst_entry[0] or not dst_entry[1]:
+                raise bb.parse.SkipRecipe('Malformed boot file entry: %s' % src_entry)
+        else:
+            dst_entry = (src_entry, src_entry)
+
+        deploy_files.append(dst_entry)
+
+    for deploy_entry in deploy_files:
+        src, dst = deploy_entry
+        if '*' in src:
+            # by default install files under their basename
+            entry_name_fn = os.path.basename
+            if dst != src:
+                # unless a target name was given, then treat name
+                # as a directory and append a basename
+                entry_name_fn = lambda name: \
+                                os.path.join(dst,
+                                             os.path.basename(name))
+
+
+                srcs = glob(os.path.join(deploy_image_dir, src))
+                for entry in srcs:
+                    src = os.path.relpath(entry, deploy_mage_dir)
+                    entry_dst_name = entry_name_fn(entry)

meta/classes/image-bootfiles.bbclass line 64

Quoted 10 lines
meta/classes/image-bootfiles.bbclass new file +10 -0
+                    install_files.append((src, entry_dst_name))
+            else:
+                install_files.append((src, dst))
+
+    for entry in install_files:
+        src, dst = entry
+        image_src = os.path.join(deploy_image_dir, src)
+        image_dst = os.path.join(boot_dir, dst)
+        shutil.copyfile(image_src, image_dst)
+

We literally iterate three times over the same files, I don't see a reason for not merging those three for loops into one?

Also, I now realize that this was heavily inspired from what's in scripts/lib/wic/plugins/source/bootimg-partition.py, would have been nice to say so ;)

meta/classes/image-bootfiles.bbclass line 66

Quoted 2 lines
meta/classes/image-bootfiles.bbclass new file +2 -0
+python bootfiles () {
+   bootfiles_populate(d),

Not entirely convinced we should have a comma after bootfiles_populate here?

More general question:
1) how does this work if one has a boot partition created by wic (via the plugin bootimg-partition?
2) shouldn't we factor out this code here and bootimg-partition's? or make bootimg-partition somehow call bootfiles IMAGE_PREPROCESS_COMMAND to avoid code duplication?

Cheers,
Quentin

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#5236): https://lists.yoctoproject.org/g/docs/message/5236
Mute This Topic: https://lists.yoctoproject.org/mt/106262122/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.yoctoproject.org/g/docs/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#6 lore raw

Hi Alex,

Quoted 5 lines · Alexander Kanavin

I also have to repeat what I said in docs@ list: why do we need a special mechanism for placing files into /boot? If /boot is not a special partition, then it's just a folder in / and can be populated from packages like any other folder.

There's renaming logic here. Which provides much more flexibility than a package would. You could have a different name for each image recipe for example, instead of having to use a distro, packageconfig, bbappend to modify it. Is it a valid use-case we want to support /me shrugs, be we already do support it in wic, so maybe that ship has sailed already.

Cheers,
Quentin

#7 lore raw
In reply to Quentin Schulz

Hello Quentin,

First, thank you for your feedback.

Quoted 29 lines · Quentin Schulz

Hi Markus,

Quoted 26 lines · Marcus Folkesson via lists.openembedded.org

[You don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]

image-bootfiles class copy files listed IMAGE_BOOT_FILES to the /boot directory of the root filesystem.

This is useful when there is no explicit boot partition but all boot files should instead reside inside the root filesystem.

---
  meta/classes/image-bootfiles.bbclass | 69 ++++++++++++++++++++++++++++
  1 file changed, 69 insertions(+)

create mode 100644 meta/classes/image-bootfiles.bbclass

meta/classes/image-bootfiles.bbclass new file +6 -0
new file mode 100644
index 0000000000..850e14e4bb
@@ -0,0 +1,69 @@
+#
+# Writes IMAGE_BOOT_FILES to the /boot directory
+#
+# Copyright (C) 2024 Marcus Folkesson
+# Author: Marcus Folkesson <[email protected]>
+#
Quoted 8 lines

meta/classes/image-bootfiles.bbclass line 11

Quoted 5 lines
meta/classes/image-bootfiles.bbclass new file +5 -0
+# Licensed under the MIT license, see COPYING.MIT for details
+# Inspired of bootimg-partition.py
+#
+# Usage: add INHERIT += "image-bootfiles" to your conf file
+#

Since it's in meta/classes, I assume we can also inherit image-bootfiles in an image recipe?

Yes, that is how I do it right now.

Quoted 14 lines
Quoted 14 lines
meta/classes/image-bootfiles.bbclass new file +13 -0
+
+def bootfiles_populate(d):
+    import re
+    from glob import glob
+    import shutil
+
+    boot_files = d.getVar("IMAGE_BOOT_FILES")
+    deploy_image_dir = d.getVar("DEPLOY_DIR_IMAGE")
+    boot_dir = d.getVar("IMAGE_ROOTFS") + "/boot"
+    install_files = []
+
+    if boot_files is None:
+        return
Quoted 8 lines

meta/classes/image-bootfiles.bbclass line 28

Quoted 4 lines
meta/classes/image-bootfiles.bbclass new file +4 -0
+
+    # list of tuples (src_name, dst_name)
+    deploy_files = []
+    for src_entry in re.findall(r'[\w;\-\./\*]+', boot_files):

Isn't this a more complex boot_files.split()? What do we gain from using this regex? If we have malformed IMAGE_BOOT_FILES, this would actually introduce some logic error:

Probably it is. The code is heavily based on the bootimg_partition [1] wic plugin, and this regex comes from there.
The rationale to keep it was that it is probably well-tested already and I want to keep the same behavior as bootimg_partition as I want it to be interchangable.

Quoted 8 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

s="plop\\plip;plep plop;plap"
re.findall(r'[\w;\-\./\*]+', s)

['plop', 'plip;plep', 'plop;plap']

I'm pretty sure we don't want that. Instead, I would suggest to use split() and iterate over that, split(';'), check for each if there's 1 or 2 entries, add one if it's one.

Looks better to me. Thanks

Quoted 32 lines
Quoted 32 lines
meta/classes/image-bootfiles.bbclass new file +31 -0
+        if ';' in src_entry:
+            dst_entry = tuple(src_entry.split(';'))
+            if not dst_entry[0] or not dst_entry[1]:
+                raise bb.parse.SkipRecipe('Malformed boot file entry: %s' % src_entry)
+        else:
+            dst_entry = (src_entry, src_entry)
+
+        deploy_files.append(dst_entry)
+
+    for deploy_entry in deploy_files:
+        src, dst = deploy_entry
+        if '*' in src:
+            # by default install files under their basename
+            entry_name_fn = os.path.basename
+            if dst != src:
+                # unless a target name was given, then treat name
+                # as a directory and append a basename
+                entry_name_fn = lambda name: \
+                                os.path.join(dst,
+                                             os.path.basename(name))
+
+
+                srcs = glob(os.path.join(deploy_image_dir, src))
+                for entry in srcs:
+                    src = os.path.relpath(entry, deploy_mage_dir)
+                    entry_dst_name = entry_name_fn(entry)
+                    install_files.append((src, entry_dst_name))
+            else:
+                install_files.append((src, dst))
+
+    for entry in install_files:
Quoted 8 lines

meta/classes/image-bootfiles.bbclass line 64

Quoted 5 lines
meta/classes/image-bootfiles.bbclass new file +5 -0
+        src, dst = entry
+        image_src = os.path.join(deploy_image_dir, src)
+        image_dst = os.path.join(boot_dir, dst)
+        shutil.copyfile(image_src, image_dst)
+

We literally iterate three times over the same files, I don't see a reason for not merging those three for loops into one?

Yes, that is probably better.

Quoted 4 lines

Also, I now realize that this was heavily inspired from what's in scripts/lib/wic/plugins/source/bootimg-partition.py, would have been nice to say so ;)

Yes it is, I have this text in the header

+#
+# Licensed under the MIT license, see COPYING.MIT for details +# Inspired of bootimg-partition.py

Maybe I should make it more clear?

Quoted 9 lines

meta/classes/image-bootfiles.bbclass line 66

Quoted 3 lines
meta/classes/image-bootfiles.bbclass new file +2 -0
+python bootfiles () {
+   bootfiles_populate(d),

Not entirely convinced we should have a comma after bootfiles_populate here?

More general question:
1) how does this work if one has a boot partition created by wic (via the plugin bootimg-partition?

The images will be copied to both the partition and /boot.

Quoted 3 lines

2) shouldn't we factor out this code here and bootimg-partition's? or make bootimg-partition somehow call bootfiles IMAGE_PREPROCESS_COMMAND to avoid code duplication?

I would love that. Not sure where to but it though, I'm still learning the structure of poky.

I'm preparing a v2 with an indentation error and that the installation directory is configurable so that it does not *have* to be /boot.
But I will probably wait one or two more daysto give people a chance to
give their comments.

Quoted 3 lines

Cheers,
Quentin

[1] https://github.com/yoctoproject/poky/blob/master/scripts/lib/wic/plugins/source/bootimg-partition.py#L71

Best regards,
Marcus Folkesson

#8 lore raw

Hi Marcus,

(First, sorry for the typo I made in your name)

Quoted 43 lines · Marcus Folkesson

Hello Quentin,

First, thank you for your feedback.

Quoted 29 lines · Quentin Schulz

Hi Markus,

Quoted 26 lines · Marcus Folkesson via lists.openembedded.org

[You don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]

image-bootfiles class copy files listed IMAGE_BOOT_FILES to the /boot directory of the root filesystem.

This is useful when there is no explicit boot partition but all boot files should instead reside inside the root filesystem.

---
   meta/classes/image-bootfiles.bbclass | 69 ++++++++++++++++++++++++++++
   1 file changed, 69 insertions(+)

create mode 100644 meta/classes/image-bootfiles.bbclass

meta/classes/image-bootfiles.bbclass new file +6 -0
new file mode 100644
index 0000000000..850e14e4bb
@@ -0,0 +1,69 @@
+#
+# Writes IMAGE_BOOT_FILES to the /boot directory
+#
+# Copyright (C) 2024 Marcus Folkesson
+# Author: Marcus Folkesson <[email protected]>
+#
Quoted 8 lines

meta/classes/image-bootfiles.bbclass line 11

Quoted 5 lines
meta/classes/image-bootfiles.bbclass new file +5 -0
+# Licensed under the MIT license, see COPYING.MIT for details
+# Inspired of bootimg-partition.py
+#
+# Usage: add INHERIT += "image-bootfiles" to your conf file
+#

Since it's in meta/classes, I assume we can also inherit image-bootfiles in an image recipe?

Quoted 2 lines

Yes, that is how I do it right now.

Would be nice to add this to the comment as well then, so people know their options?

[...]

Quoted 41 lines
Quoted 32 lines
Quoted 32 lines
meta/classes/image-bootfiles.bbclass new file +31 -0
+        if ';' in src_entry:
+            dst_entry = tuple(src_entry.split(';'))
+            if not dst_entry[0] or not dst_entry[1]:
+                raise bb.parse.SkipRecipe('Malformed boot file entry: %s' % src_entry)
+        else:
+            dst_entry = (src_entry, src_entry)
+
+        deploy_files.append(dst_entry)
+
+    for deploy_entry in deploy_files:
+        src, dst = deploy_entry
+        if '*' in src:
+            # by default install files under their basename
+            entry_name_fn = os.path.basename
+            if dst != src:
+                # unless a target name was given, then treat name
+                # as a directory and append a basename
+                entry_name_fn = lambda name: \
+                                os.path.join(dst,
+                                             os.path.basename(name))
+
+
+                srcs = glob(os.path.join(deploy_image_dir, src))
+                for entry in srcs:
+                    src = os.path.relpath(entry, deploy_mage_dir)
+                    entry_dst_name = entry_name_fn(entry)
+                    install_files.append((src, entry_dst_name))
+            else:
+                install_files.append((src, dst))
+
+    for entry in install_files:
Quoted 8 lines

meta/classes/image-bootfiles.bbclass line 64

Quoted 5 lines
meta/classes/image-bootfiles.bbclass new file +5 -0
+        src, dst = entry
+        image_src = os.path.join(deploy_image_dir, src)
+        image_dst = os.path.join(boot_dir, dst)
+        shutil.copyfile(image_src, image_dst)
+

We literally iterate three times over the same files, I don't see a reason for not merging those three for loops into one?

Quoted 2 lines

Yes, that is probably better.

The code already exists, it's probably robust thanks to the test of time, so maybe not worth refactoring until someone complains about it. I assume we shouldn't have hundreds/thousands of entries in that variable so the cost is probably fine.

Quoted 11 lines
Quoted 4 lines

Also, I now realize that this was heavily inspired from what's in scripts/lib/wic/plugins/source/bootimg-partition.py, would have been nice to say so ;)

Yes it is, I have this text in the header

meta/classes/image-bootfiles.bbclass new file +3 -0
+#
+# Licensed under the MIT license, see COPYING.MIT for details
+# Inspired of bootimg-partition.py
Quoted 2 lines

Maybe I should make it more clear?

No, I should rather learn to not skip whatever looks like a license to me at the top of the file :) Sorry for the misplaced complaint.

Quoted 10 lines
Quoted 9 lines

meta/classes/image-bootfiles.bbclass line 66

Quoted 3 lines
meta/classes/image-bootfiles.bbclass new file +2 -0
+python bootfiles () {
+   bootfiles_populate(d),

Not entirely convinced we should have a comma after bootfiles_populate here?

More general question:
1) how does this work if one has a boot partition created by wic (via the plugin bootimg-partition?

Quoted 9 lines

The images will be copied to both the partition and /boot.

Quoted 3 lines

2) shouldn't we factor out this code here and bootimg-partition's? or make bootimg-partition somehow call bootfiles IMAGE_PREPROCESS_COMMAND to avoid code duplication?

I would love that. Not sure where to but it though, I'm still learning the structure of poky.

This probably should be another python lib or something like that, maybe in meta/lib/ ? It's a bit trickier since the wic plugins are not really any part of a recipe, rather pure python stuff, so a bit different than the usual way of plugging stuff in it the proepr way?

Quoted 5 lines

I'm preparing a v2 with an indentation error and that the installation directory is configurable so that it does not *have* to be /boot.
But I will probably wait one or two more daysto give people a chance to
give their comments.

Sounds good.

Cheers,
Quentin